The reviews system is one of the things that makes Amazon good. Thousands of crowd-sourced, supposedly unbiased reviews take the guesswork out of buying sight unseen.
But a new study is shedding light on the dark underbelly of incentivized Amazon reviews. Those are reviews where the reviewer has received an item for free, which according to the results of the study, introduce bias into the system. A lot of bias.
DON’T MISS: You can now disable iOS 10’s most annoying new feature… but I don’t want to
The team at ReviewMeta analyzed seven million Amazon reviews to try and work out if receiving a product for free impacts the eventual score. The sample size is huge, although the methodology is perhaps questionable: the researchers used an algorithm to look for “reviews containing language that would indicate the reviewer received the item for free or at a discount.”
Assuming that the methodology is sound, the results are unsurprisingly damning. Incentivized reviews (which accounted for a whopping 30% of all reviews) were, on average, 0.38 stars higher. That doesn’t sound like a big difference, but because of the star inflation on Amazon reviews, it’s enough to bump a product from a mid-range review average to top five percent.
It gets worse for critical reviews. On average, incentivized reviewers were four times less likely to leave a negative review than people who actually paid for their products.
The study also found that incentivized reviews are also rising sharply. In 2014, there were barely any, but they’re up to over half of all reviews now. And, of course, that’s just the reviews that are disclosed.
