This page is part of a global project to create a better online reviews system. If you want to know more or give your feedback, write at [email protected] and we’ll grab a beer ;)
Studies have shown that people use a Bayesian approach $^1$ when checking average ratings: they also consider the number of reviews. Bayesian principles suggest that when confronted with two highly desirable options (both with high average ratings), the option with more reviews is perceived as the better choice. Researchers have developed models to calculate a “modified average rating,” a value that incorporates both the ratings and the number of ratings, representing the perceived score. These “strategies of selection” change depending on whether there are few reviews or many reviews.
Take this example from Amazon. The product on the left has a rating of 4.5, and the product on the right has a rating of 4.6. Which would you choose?
In any case, the number of reviews alone is a signal for the viability of a business because it’s perceived as the number of purchases, which can be misleading because the review rate might be low (fera.ai’s review rate of marketplace products is 8.3% on average- and that’s a high estimation).
Specifically, consumers use the number of reviews to evaluate if the product is popular $^2$, and the total number of reviews a seller accumulates to assume if the seller has generated large sales $^3$. This counts in their choice: a study from Salsify found that consumers expect at least 112 reviews to consider a product a valid option on Amazon $^4$.
This is because people follow a basic herd mentality: if enough people bought the product before, it proves that it’s a good option.
This explains why businesses push so hard to get reviews, even risking bad ratings: the more reviews, the greater the consideration (perceived number of purchases) and the higher the perceived average rating (Bayesian approach).
Additionally, a low number of reviews can appear suspicious because there’s a higher chance that those reviews are fake or biased, possibly left by friends or individuals who didn’t actually try the product. As a result, people tend not to trust the average rating in such cases.
Two clear downsides emerge from this:
<aside> 💡 Exploration
Display the number of purchases separately from the number of reviews to clarify that these indicators are different. No review can sometimes be a bad review (cf. “Feeling Bad Judging People”), and showing both KPIs could provide additional useful information to potential customers: a product with few ratings despite many purchases might be suspicious.
A proposition of redesign for Airbnb, with the number of hosted stays distinct to the number of reviews.
Show the number of positive recommendations instead of the number of reviews. Following the logic above, this is the actual information users expect: how many people in the set of purchasers recommended it, compared to other products?
Another proposition of redesign for Airbnb, with the possibility for guests to mark their stay as exceptional.
</aside>
$^1$ “Reaching for the star ratings: A Bayesian-inspired account of how people use consumer ratings”, Janine Christin Hoffart, Sebastian Olschewski, and Jörg Rieskamp, 2019.
$^2$ “Harnessing the Influence of Social Proof in Online Shopping: The Effect of Electronic Word-of-Mouth on Sales of Digital Microproducts”, Ambee and Bui, 2011.
$^3$ “Do online reviews matter? — An empirical investigation of panel data”, Duan, Gu and Whinston, 2008.