You’re Probably Wrong, but That’s Okay

If you look at the master list of ‘things that everyone hates’ (trust me, there is one), you’ll find ‘being wrong’ near the top of the list. People hate being wrong. We humans have managed to circumnavigate the awkward feeling of being wrong by coming up with all sorts of ways to prove we’re right — from strategically entertaining only the evidence that supports our claims, to flat-out ignoring anything that says otherwise. It’s a concept we are all familiar with, but few of us care to either define or give thought to it. It’s called confirmation bias, and it can be especially dangerous when employed by a User Experience Designer.

User Experience Designers are the judges, juries, and executioners in an industry where the usability of a product is of the utmost importance. They hold the keys for gathering and crunching the user data that forms the basis of a product’s development. So what happens when they’re wrong? What happens when a new product’s relationship with its users is based on a biased assumption?

Bad things happen.

Walmart’s $2 Billion Blunder

My personal favorite case study is a $2 billion blunder by the retail giant Walmart. Walmart, like any responsible retail institution, routinely runs surveys to find ways to improve their customer experience. In 2009, survey results suggested that their customers felt overwhelmed by the cluttered aisles and the disordered nature of their stores. In response, Walmart devised a plan to reduce inventory, lower aisle end caps, and shorten the height of shelves to provide a more open, streamlined shopping experience. All this would (supposedly) increase customer satisfaction and therefore (supposedly) increase sales.

Here’s where the Confirmation Bias comes. Walmart cooked up their solution, then released another survey asking customers if they liked it. But they did all this without testing their solution. Instead of examining the impact on customer experience and thoroughly testing it to rule out the possibility of being wrong, Walmart opted to ignore that possibility. What resulted? Wait for it — Walmart sales went down. In-store sales went down $1.85 billion to be exact. All because a group of User Experience Designers got a few nods towards their solution, patted themselves on the back in their echo chamber of ‘yes!’ and ‘congratulations!’ but neglected to test against one of the fundamental pillars of UX philosophy: what people say they want may not actually be what they want.

Test, Test and Test Again

So what can all of us UX people learn from this? Test, test, and test again. Swallow your pride and try your best to prove yourself wrong. Only when you’ve exhausted all of the possible ways a solution can cause more harm than good can you safely say that you’ve found your solution.

Anyone can fall victim to confirmation bias. What makes us effective is our willingness to thoroughly test before we implement our solutions.