User testing — a good servant but a bad master

Design - 21st September 2019
By Pixelfield team

There are few things as popular among digital agencies and developers these days as user testing. Don’t get me wrong — I absolutely believe in exposing our ideas and gathering feedback as often as possible. That being said, I have seen some projects being ruined by user testing. We have to be careful about how we ask the questions and how we interpret the answers. Here are a few stories of user testing going wrong. 

Wish is the father of the though

Or in this case, it was a father of all the question posed to the users of one of our clients. When rebuilding the website, the client has an innovative website concept and design in mind. That was in an evident contrary to their current website; simple, a bit old-school, but working. 

Cool animations and an unorthodox menu is something that might seem nice, but can easily distract the user from performing the desired actions. The client decided to test the new UX design and had only positive feedback. The launch was a disaster with almost all KPIs plummeting to the ground. They approached us shortly after launching the new website, wanting us to fix the crucial performance. 

Where was the problem? The wording of the questions. Biased and enthusiastic researchers made almost all the subject to applaud the new website concept. The website was live only for a few weeks, but it cost the business an incredible amount of resources and leads.

Wow, this is cool!

Here is another scenario with a similar outcome. We helped to develop an app, while the concept and design were done by another agency. When we saw a chatbot for buying cars, we’ve had serious doubt that this is going to work. We were told that this is heavily supported by user research. We did our job, and the bot was launched, generating almost zero conversions from heavy traffic. The project was shut down after a month or so. 

Why did that happen? Because people thought that chatbots are cool. No doubt they are. However, are you ready to buy a car just because a chatbot told you so? I am not (although, this might change in time). Be careful when testing something that people think is really cool. They might incline to praise your idea even if they would never use the product in the end.

Veni, Vidi, Vici

Sometimes you just have to be brave if you really believe in your idea. Even if the users don’t understand. There always is a vast difference between the final product and the prototype. Also, sometimes, the users might not understand what you are trying to achieve. 

There are many examples of companies cancelling their products, based on negative feedback from the market sometimes because they were too early, sometimes because they didn’t quite finish what they have started. Or just because they were not brave enough to educate the consumers. The frustration of seeing your product succeeding several years later is enormous. Don’t let that happen to you! 

Even with this experience, I believe that user testing is crucial for every project. Just be careful before you act on it. If you want to do the user testing right, or app development agency London is here to help.

Written by
Pixelfield team
Related posts
Step-by-Step Prioritization for Startups: Build Your Roadmap With the PriX Method
Design | Development | Resources - 2nd April 2020
By Marek Hasa
How to Use AR In Campaigns: The Ultimate Guide to Augmented Reality Marketing
Design | Development | Marketing | Unsorted - 23rd March 2020
By Marek Hasa