The Problem With Online Reviews

I often take my coffee at a place with a 3 stars rating on Google, but it serves one of the best coffees in the city. The reviews come from two people who had once a bad experience. And Google shows to everyone that it has 3/5 stars without saying that it is one the best places to have a non-Starbucks, coffee.

Some companies keep manufacturing certain products only because 10 years ago they received so many 5 stars on Amazon, that it’s almost impossible to retire the product.

My Uber score is 4.7, and the only “bad thing” I do is the lack of conversation with my driver. I always say “Hello”, “Thank you”, “Goodbye” and leave some tips. So apparently for not talking you still get a low score.

For drivers, sometimes all you need is a grumpy passenger who is not allowed to smoke and will give you a one star for that. And then you have to work hard for a month to recover your initial score. Only because somebody wasn’t allowed to do what he wants.

The hysteresis effect

A relative of mine put her villa on Airbnb. And when you are new in a service business, you tend to accept anyone, only because you need ratings. Her first customers were a group of youngsters, and after renting the place, they wanted to throw a party. She did not allow them, fearing they will destroy the house. The group wasn’t satisfied and left a bad, 2 stars, review. She couldn’t rent the house to anyone else anymore. She had to rename the place.

In her case, if she had left the stars, almost nobody would have rented her house. This is an effect called hysteresis — the dependence of the state of a system on its history. It’s the same way banks work when giving loans, and it’s the same way schools operate and so on. We judge something based on a metric that characterises past actions or events. And what we see now is a glimpse of the future which everybody will accept.

“Leaving a feedback/review — we reduced a thought process, that should take some time and effort, to basic numbers that mean nothing. The episode of Black Mirror were everybody rates each other with stars has all the chances to become true. Actually, it already did.”

There are moments when we should realise that leaving a review to is not what everyone should be able to do. Why? Because we are irrational and emotional animals.

For example, apparently, some people constantly give 4 stars to their Uber drivers even if the service was spectacular. Reason? They leave 5 stars only if, I will quote: “I will give him 5 stars only when the driver would save my life from a threatening situation, otherwise why would he deserve it?”.

“We expect a rational feedback or review from irrational beings. I somehow find it irrational.”

Should I ask customers to write reviews for my business?

On Yelp.com, in the Q&A section, for the question “Should I ask customers to write reviews for my business?” the site says:

Probably not. It’s a slippery slope between the customer who is so delighted by her experience that she takes it upon herself to write a glowing review and the customer who is “encouraged” to write a favorable review in exchange for a special discount. And let’s be candid: most business owners are only going to solicit reviews from their happy customers, not the unhappy ones. Over time, these self-selected reviews create intrinsic bias in the business listing — a bias that savvy consumers can smell from a mile away. Don’t be surprised, then, if your solicited reviews get filtered by Yelp’s automated review filter.

Customer is not always right

I always say that to build a great product or service, watching people’s behaviour is more revealing than listening to what they have to say. See what they do and not what they say. There’s also the same problem in the social sciences with questionnaires. They are usually answered aspirationally. For example: “are you a generous person?” And people will say “Of course I am!”. No matter if you put as an option to answer yes/no or choose from 1 to 10. This is a human tendency to overestimate one’s importance or opinion.

“You know the old adage that the customer’s always right?” he said. “Well, I kind of think that the opposite is true. The customer is rarely right.” — Charlie Trotter, Source: NY Times

Why do we still use reviews then?

If so many things are wrong with current review systems, why do we still use them? It’s a form of communication, and we need it as social proof when buying, reading or consuming something. But unfortunately, it is an undeveloped form of communication which contains some human flaws.

On the other side, we still use the 5-star rating system because people are lazy and it’s always easy to go with this type of review system. “Everybody does it, why wouldn’t I do it too?” or a better way to say it “Facebook, Google and Amazon does it. I don’t think such a big company can be wrong”. And this reminds me of a story:

“A policemen sees a drunk searching for something under a streetlight and asks what the drunk man has lost. He says he lost his keys and they both look under the streetlight together. After a few minutes the policeman asks if he is sure he lost them here, and the drunk replies “no”. The policeman asks “Then why are you searching here?” and the drunk replies “This is where the light is.””

This story has a two way meaning, of course. But, you can realise how the power of social proof can influence your decision making, without you ever giving it a thought.

Another reason why we use such a simplistic review system is because it’s quantifiable and offers some sort of logic. “What shall we think about our own product or service?” For example, if your average score for your product is 3 stars out of 5, it means there is something to be changed and it’s an average product.

But are the numbers so reliable? In math they are. But when you are trying to translate human emotions — I don’t think so.

Should we trust the numbers?

A great example here is Facebook’s Messenger on a smartphone. Everytime you finish a phone call, you are asked with their annoying survey “How was the phone call?”. I would always rate it with 1 star, because, first of all it’s Facebook, and then because that annoying question pops up every damn time after a phone call.

But what real insights does it give to you when someone rates the phone call with 1 star and does not leave a feedback? And even if the person leaves a feedback, 90% of those reviews are basic human emotions. Emotions, that come from a specific context they had a phone call in with many factors happening. And we translate that into a number? A number which has a different meaning from person to person?

“We translate human emotions into raw data, like numbers and star reviews and expect to derive some logical answers from that on how we can improve our products. To me it looks like there’s more rationality in the human emotions than those numbers.”

This is a broader problem

But now as I sit and think, it has to do more with us humans than with a review system. We are encouraged everywhere to share our own opinion, everyone asks for it. A shop may send you an email and ask how was your experience, are you satisfied with their product or not. At the airport, you are asked how was the line service, and so on.

Nowadays, everyone gets a chance to finally have some meaning in their lives by giving their two cents about something. And with that, we forget that sometimes it’s better to shut up, even if something did not meet your delusional expectations when buying a $7 cup of coffee.

Now, should we do something about it?

Can we improve the review systems? We could, but how far can we go until we hit another human problem? And here we should look at the issue of not fixing the review systems, but rather playing around human flaws. So there is just a little bit less of lousy stuff that ruins products, businesses or peoples careers. If we can work around this and create a system that can encourage better behaviour, then we can impact lives in a better way.

On the other side, why would you want such simplistic review systems (like a 5 star) if there are so many downsides? Speed and less friction. Something that we crave for in the Internet world. We are trying to maximise one and minimise the other. We use these type of systems because they are easy and nothing else. But is it the best way? Are there alternatives?

Don’t make the reviews so simplistic

Giving a rating from 1–5 without an explanation is a bad UX, because this may delude future customers or users about the value of that service. Giving a review and enforcing the user to leave a well-thought comment is a different subject. Some companies offer a review system with many levels of feedback (ex.Adidas). Where you can choose the level of comfort of a shoe, material quality, design, and describing each part of the product. On the other hand, others don’t allow you to post something unless you answer all the questions that can give a broader picture to other users.

Once I was working for an e-commerce shop, and I was having a discussion with a customer about his shopping experience. He gave 3 stars to a product because the delivery guy was rude to him. Meanwhile, the delivery company has nothing to do with the shop itself. And of course, you can’t blame him. But it’s our job to make it harder for people to leave delusional feedbacks and encourage more well-thought and constructive criticism.

“But in the end, it all will always come down to what you really want for your product — better reviews or more reviews?”