Possibly the hottest topic for sellers—and the area that is seeing the wildest changes right now—the Amazon review policy. We’ve identified six major review changes over the last couple of months. Some of them have had devastating effects on brands.
Without any formal statement or direction from Amazon on policy or structural changes, we in the seller communities are (classically) left in the dark wondering what’s going on. Some of us are asking what we’ve done wrong, and all of us asking ourselves what’s the best way to move forward.
As always, the Amazon seller forums and Facebook groups are running wild with speculation and questions as to what’s going on with the latest review changes, and few truly have a grasp on the magnitude of some of these changes.
If you’re at all familiar with Viral Launch, you know we are a data-driven company. And you know that we love to answer questions and solve problems with vast amounts of data (I mean how else? 😉 )!
To find out what is happening with reviews, we used the advanced data aggregation of our product and market research tools, Market Intelligence and Product Discovery. These product trackers gather historical and real-time data on well over 100,000,000 Amazon products. So we were able to uncover some jaw-dropping stats. You’ve got to see!
Change 1 of 6: Amazon Removing Reviews or The Great Amazon Review Abduction of 2018
Some of you have likely heard of or experienced a brand or product where Amazon was removing reviews over the last month or two. You—like everyone else—were probably asking yourself questions like, “How widespread is this?” and “Have other sellers lost as many reviews as me?”
We had the same questions. So our engineers started crunching the numbers. And here’s a glimpse of just how widespread Amazon’s latest review removal was. We should note that we are still working through the data to try to make it as clean as possible. There are a lot of moving parts when it comes to reviews, including changes and discrepancies in variations.
For each top-level category in the Amazon US market, we looked at the top 10,000 best selling ASINs with at least 90 days of history. From that set of products, we looked at how many of the listings lost reviews, which listings lost the most reviews, and how many reviews the category lost as a whole.
Here’s a summary of the report showing just how huge and far-reaching this review abduction is:
Reviews Removed Between May 1 and May 31
Top 5 Categories with Highest Number of Reviews Removed from May 1 – May 31:
Cell Phones & Accessories: 324,423 reviews removed!
Health & Household: 305,777
Grocery & Gourmet Food: 204,553
Beauty & Personal Care: 193,242
Tools & Home Improvement: 177,375
Top 5 Products Hit With Review Removal:
Etekcity Digital Scale (B073W64CY8): 13,999
Pansonite VR Glasses (B07516YQ59): 10,202
Bark Solution Dog Collar Training System (B01HSH194S): 6,690
Fox Print Princess Castle Play Tent (B0120XRWLE): 1,710
Amazon Basics Stainless Steel Kitchen Scale (B06X9NQ8GX): 1,472
Top 5 Most Affected Categories by Percent of Listings Affected:
Cell Phones & Accessories: 21.08% of the top 10,000 listings had reviews removed
Camera & Photo: 19.43%
Video Games: 19.05%
Health & Household: 17.66%
Sports & Outdoors: 17.39%
The category that lost the most reviews was Cell Phones & Accessories with 2,108 of the top 10,000 listings losing 324,423 reviews. That’s the total number of reviews that we saw Amazon removing between May 1st and 31st. When accounting for the reviews that sellers regained (organically and otherwise), the category still came out with a net loss of 185,302 reviews.
The single listing that lost the most reviews in the Cell Phones & Accessories category was this Pansonite 3D VR Glasses. The listing lost 10,202 reviews taking them down from 10,339 reviews to just 65 reviews at its lowest!
The category that was hit second hardest was Health & Household with 1,766 of the top 10,000 listings losing 305,777 reviews. Accounting for reviews gained and re-gained, the net loss for the category was 177,324 reviews. Of the 10,000 Heath & Household listings we analyzed, the single listing that lost the most reviews was this Etekcity Digital Scale (B073W64CY8) which lost 13,999 reviews.
You can see the spikes and drops in their review count using Market Intelligence. It’s crazy to look at the review graphs for some of these listings that lost a ton of reviews. You can see where Amazon removed the majority of their reviews and that they quickly got them back. Then Amazon removed them again, and the seller recovered the reviews again. This Etekcity Scale is an excellent example!
These graphs show you just how crazy things are right now. You can see these sellers fighting for the life of their listings. Losing that many reviews changes a product’s star rating, drastically affects the product’s perceived popularity, probably forfeits Best Seller and Amazon’s Choice badges, and just, in general, destroys a listing’s credibility, which we know affects sales.
Change 2 of 6: Reviews Being Blocked
*The specifics of this section seem to currently be in a state of change, so depending on when you are reading this, it is possible that changes may have come about. Some of the tests were limited in scope as we wanted to leave reviews in an ethical fashion. The concepts and context are what’s most important.
From our perspective, this was the scariest change of the six we’re listing here.
To summarize, Amazon seemed to have put proverbial speed limits in place when it comes to review acquisition velocity. For the month of May and into June, it seemed as though the Amazon review policy was limiting the number of reviews a product could obtain in a day. If you exceeded the threshold/speed limit, the product would be punished with a review block for some period of time ranging 3 to 7 days.
Two very interesting facts from this finding:
1) unverified and verified reviews were treated separately
2) the review limit was a static number.
Let me explain.
After a bit of testing and observing, we noticed that receiving 3 unverified reviews in a day would cause Amazon to block the ASIN from receiving any additional unverified reviews. Similarly, after receiving 6-8 verified reviews in a day, Amazon would block the ASIN from receiving any additional verified reviews. And customers would see some form of this message:
Pro Tip: Unverified reviews are reviews in which the reviewer has not purchased the product on Amazon, or has purchased with a heavy discounted code
From what we could tell, the first time a review block was initiated, the product was put on a 7-day review block (for both verified and unverified reviews). We believe that during this time Amazon conducted an audit, combing through the account to try to develop an understanding of the authenticity of the reviews. We believe Amazon would look at things such as the content of the seller’s email follow-up sequence, the referral path of the reviewer, etc. If everything checked out, the product’s review block would be lifted (meaning it was again eligible to receive reviews), and if things did not check out, the account was suspended.
But passing the initial suspension did not mean you were completely in the clear! The majority of products that continued to exceed the Amazon imposed speed limit would have the review block reinstated. If a product reached the limit of 3 unverified reviews in a day, it would be blocked from receiving any additional unverified reviews. But it could still receive verified reviews. Similarly, if 6-8 verified reviews were left in a day, then the ASIN would be blocked from receiving additional verified reviews. These blocks seemed to last for 1-3 days at a time. We saw several listings cycle through this process multiple times.
For our clients with products selling 1,000+ units per day, receiving 6-10 reviews in a day is not uncommon. It was quite surprising that Amazon would choose to use a static number versus a percentage of sales. And it creates a very frustrating experience for sellers.
This review blockage was not exclusive to ASINs offered by third-parties. Many of Amazon’s own high-volume products had review blocks in place at one point or another.
At the time of this posting, here is a high volume product (#3 best seller in Beauty & Personal Care category) that currently has a block in place for unverified reviews. When you click on the star rating, you will see a pop-up stating that your review cannot be accepted at this time.
While review blocks are still happening, from what we’ve observed, as of mid-June, there have been far fewer than in May. We don’t yet have the answers as to why. Perhaps Amazon has adjusted the speed limit to a percentage of sales rather than a static number. Maybe they decided that one account review analysis was enough. Only time will tell.
We are grateful to Amazon for stepping up their game in terms of monitoring and enforcing the rules when it comes to reviews. A lot of the existing rules harm the ToS-abiding sellers while bad actors gain a competitive advantage by cheating the system. We really hope that, at the least, this is an indication of Amazon beefing up security and enforcement of their inauthentic review policies and programmatic monitoring.
Why this change was so scary.
This Amazon review policy change challenged some of the fundamental dynamics of being a third-party seller on Amazon. The very reason our customers have been able to build successful million-dollar businesses (and sometimes tens of millions of dollars) is they are able to develop competitive advantages in their market. Those advantages range from the ability to drive keyword ranking in organic search results, drive better conversion rates through product photography and better listing copy, having a better product and so many others.
A seller’s ability to drive quality reviews faster than their competitors is an important advantage in a savvy seller’s tool belt. Being able to drive reviews faster than your competition allows you to enter an existing market with 0 reviews, strategically drive reviews to catch up with competitors that may have hundreds or thousands, and achieve maximum sales potential. If you and your competitors were restricted to driving no more than 10 reviews per day, it would be impossible to catch up to existing players and obtain a relatively competitive number of reviews in a market, which could severely inhibit your ability to ever drive meaningful sales volume in that market. If this change came to fruition, it would not be as impactful if Amazon made the systemic review change we anticipate them to make, which we talk about in a podcast episode here.
The opportunity for success as an FBA seller exists in the competitive advantages. Any erosion of opportunity for competitive advantage will make it more difficult for new products and new sellers in existing markets. We firmly believe that the high degree of competition on Amazon has been key to Amazon’s success as well as our clients’ success, so any potential infringement on that is intimidating.
Change 3 of 6: Reviews Split By Variation
The way that reviews are being attributed to variations seems to also be apart of the Amazon review policy change. It used to be that if you had a listing with variations a, b, and c, the reviews for each of those variations would be aggregated for a total review count and star rating that applied to each variation on the listing. So the total number of reviews for the each ASIN would be the same. That means if someone bought your purple flashlight and left a review, that review would apply to the red variation, the black variation, and the green variation.
Not so in the new world of the Amazon review policy! Amazon is slowly making a switch in the way that it handles reviews on listings with variations. Instead of applying a review for a variation to the whole listing, if someone leaves a review for, say, that purple flashlight, it will only show up on the purple flashlight variation.
So as a shopper is clicking through the product variations, the star rating and the review count under the title will change. That means if no one has leaves a review for your green flashlight yet, it will show up as having no stars and zero reviews!
Here are a few examples:
This Amazon review policy change in the way that Amazon is handling reviews for variations also means that if someone does finally leave a review for your green flashlight and it’s a 1-star, your rating for that variation is going to be really low. Even if you have a 5-star rating for the other variations. This is potentially bad news for sellers. It means that every variation you have is going to have to be top quality. And your strategy for providing stellar customer support and preventing bad reviews has to be good.
Change 4 of 6: Seller Feedback Lost to a New Rolling 12 Month System
Have you ever jumped in your car to go somewhere and realized that you don’t have your cell phone with you? You run back inside and spend the next 15 minutes tearing your house apart trying to find it? Yeah. This is nothing like that.
This is like realizing you don’t have your cell phone, and running back inside to look for it except now your house is gone. And so is your cell phone, which you left on the kitchen counter by the way.
Seller feedback is being moved into a rolling 12 month system. This new system is especially significant for Retail Arbitrage, Online Arbitrage, and Wholesale people. Seller rating is a big part of what allows RA, OA, and Wholesale sellers to win over customers.
You know how when a shopper clicks on New (10) from $5.90 they are taken to a page with all the offerings on that listing. And each of those offerings is being sold by a seller with a rating. Well now instead of seeing that seller’s overall rating, Amazon is showing customers a rating from “over the past 12 months.”
So if you got four 1-star reviews in the last 12 months, those hold a lot more weight than they used to. Let’s say you used to have 2,000 seller reviews and 5 of them were 1-star reviews. That probably wasn’t pulling down your overall rating too much.
But now that Amazon is only looking at that last 12 months, it’s only looking at the last 500 reviews that you got. And four of those 1-star reviews happened during that time. Now you’re feeling those 1-star reviews in your rating.
But this change is not happening across the board. We’ve only seen certain seller accounts where the change has been implemented. So if you haven’t been affected yet, consider yourself lucky. This inconsistent introduction of a new system could end up being totally unfair to those sellers who have been pushed in already and whose star ratings may look less favorable than their competitors who are still part of the lifetime review system.
Under the rule of the new 12-month rolling reviews, consistency is the name of the game. You have to deliver an awesome customer experience with every single product on every single order if you want to be among the top-rated sellers on the platform.
Change 5 of 6: “The Great Amazon Reviewer Purge”
Amazon is getting rid of thousands of top reviewer accounts. After Amazon banned incentivized reviews in October of 2016, top reviewers started being kicked off the platform if Amazon suspected that they were not complying with the changes.
Amazon has been purging their Top 10,000 reviewers, banning them from the platform. In the subreddit The Great Amazon Purge, you can track how many reviewers have been removed so far. The subreddit has actually been tracking the number of removed Top 10,000 reviewers since February of 2016.
At the time of this posting, 4892 of the Top 10,000 reviewers have been removed.
Change 6 of 6: Suspension Changes And Account Bans
While the majority of Amazon sellers cite their number one fear as having their privileges revoked from selling on Amazon, we have very rarely seen this actually happen in the private label world (and for context, it was one seller of the 10,000+ we’ve worked with and was due to continued review manipulation). An Amazon account suspension is incredibly scary and can be devastating to monthly financials too. From our limited experience, intuitively, we’d guess the average account suspension lasts no more than a week (would love some data on this if anyone wants to share!). From our sources, there is a new rule for suspensions due to review manipulation, and it comes in at whopping 21 day minimum suspension! That is tough!
From what we know, if there is an additional review manipulation infraction, the seller account will be BANNED. The kiss of death. Unfortunately, we’ve seen this happen to two major accounts and I imagine many others (we did not dig into the data around account suspensions/bans, we’ve merely heard of these instances through the grapevine). Unfortunately, we do not have any specific information around the why behind these accounts being shut-down by Amazon.
How To Move Forward With the Amazon Review Policy Changes
The problem with any definitive suggestions in moving forward on Amazon, especially when it comes to reviews, is that we are not 100% confident in where the line of appropriate behavior versus non-ToS compliant behavior falls. If all of our questions could be answered, or the ToS was completely comprehensive, then as a company, we’d feel much more confident in providing a “best review strategy”.
The short answer is that right now, we suggest being conservative in your review acquisition strategy. This is especially our advice if your account has been suspended due to review manipulation. Over the next few months, we’ll start to develop a better conceptual understanding of what the Amazon review policy and what’s acceptable vs unacceptable.
Some account managers for our largest clients have even made the suggestions to stop sending all email follow-ups that mention the words review, feedback, etc. We are not saying that well-worded email follow-up sequences will lead to suspension, we just want to provide you with as much information as possible so you can make the decision about what activities to leverage in your review acquisition strategy.
It should go without saying that as a company, our suggestion is to abide by Amazon’s TOS when it comes to obtaining reviews. Knowing exactly what is and is not within TOS is another question. We still see many sellers participating in review groups (predominantly those in Facebook groups), and we cannot encourage you enough to stop this. We have strong evidence that a number of these groups have had the brands participating in these groups suspended. For those that are unfamiliar, there are secret Facebook groups in which a buyer offers to purchase a product at full price and leave a review, and the seller reimburses the buyer through a payment provider such as PayPal.
As our understanding of what is/is not acceptable in terms of review acquisition strategies, we will be sure to post updates. Reviews are a critical component of Amazon success and can be a major competitive advantage. We want to make sure that you are best equipped to avoid unintended suspensions and gain a competitive advantage with the most powerful, yet TOS compliant, review strategies available.
For more Viral Launch, data-driven insights. Subscribe to the blog below and we’ll notify you whenever we have a piece of big, impactful information, like this, so you can stay on the cutting edge of Amazon seller success.
