Amazon reviews have generally been a good source of data on cheap products, but the last two years has seen the rise of something sinister: incentivized reviews. People receive products at a discount or for free in return for reviewing them, and while there’s no money changing hands for positive reviews, it’s still a problem.

Review Meta is a website that has been trying to analyze Amazon reviews to work out if incentivized reviews are a problem. According to its analysis, giving freebies does skew the reviews towards five stars, in some cases dramatically.

DON’T MISS: iPhone 7 camera review: So good, you don’t need the 7 Plus

We covered Review Meta’s analysis of incentivized reviews when it first came out, but the video is still worth watching. In particular, the dive into the murky world of incentivized review websites gives some idea of why incentivized reviews can push ratings into five-star territory, even if there’s no direct payment for providing good reviews.

As before, Review Meta’s methodology isn’t necessarily perfect. The site told me how it relies on analyzing the text of reviews to see if it is incentivized:

“The methodology involves checking each sentence of a review to see if they use several of the following strings in close proximity:
free, discount, promo, review, feedback, opinion, exchange, leave, in return, receive, purchase, provided, honest, unbiased… + a few more.
We group these words, assign different weights based on where in the sentence each one is, and combine some other factors such as sentence length to make a final determination.”
It’s not a perfect system, and it throws up a false positive about 1 in 200 times, according to the site. That’s still good though, and better than just taking every review at face value.
Although Amazon has acknowledged the problem of paid-for reviews in the past, even suing to take down sites that sell five-star reviews, it doesn’t seem to think incentivized reviews are such a problem.
Amazon’s terms of service allow for incentivized reviews, and last time we wrote about Review Meta, an Amazon representative reached out to me, and described Review Meta’s methodology as “very questionable.” Amazon did not provide any further details.
In response to a request for comment on this video, Amazon said that it is “working on some significant changes that will be rolled-out shortly.”
View Comments