If you own a computer and have a disposable income, chances are good that you have bought something via Amazon.com, a well-known site dedicated to being the Internet's largest store.  The site boasts many features, including discount prices, lists of recommendations for frequent users, and intuitive search features.  A past blog post on The Bivings Report highlights one of Amazon's recent user-friendly upgrades.

An associate of mine recently praised Amazon because of the comments feature on the individual items' pages.  At the bottom of a specific product's page, after the official description and details are listed, comments from users/buyers are shown.  The comments deemed "Most Helpful" are the first ones that you will see, while on the right side of the page are the most recent ones.  Directly above the Most Helpful comments is a small graph of metrics, showing the number of votes, what were those votes, and the overall average of the votes.  Users can assign a numeric rating to the product-a ‘star' system-on a scale from one to five.

Being the skeptic that I am, I questioned my friend's praise of the comment system.  Typically, sites with comments vary wildly in usefulness.  Comments on websites can come in many forms: helpful, useless, curse word heavy, angry, naive, etc.

However, I was pleasantly surprised as I delved into the comment sections on Amazon.  Most of the posts were valuable, or at the very least, civil.  There were few, if no, drunken rants to be found.  I did notice that the overall rankings for the products seemed a little high.  I figured that it may be just a coincidence, but I decided to do a quick analysis to comfort me.

After selecting a product category that had many similar, yet different products (TV show box sets on DVD), I went through 100 product pages.  On each, I noted the number of 1 star, 2 star, 3 star, 4 star, and 5 star reviews, as well as the overall average of the reviews.  I only rated one randomly-selected season per TV show.  The complete data set can be found in a spreadsheet below.

The first detail of interest is the fact that a vast majority of votes were for 5 stars.  In fact, the number of 5-star reviews is nearly triple the amount of all the other scores put together.  The results are summarized below in a pie chart, made using ImpactWatch:

Why so many 5-star reviews?  I came up with many conclusions as to why users dominate the clickable 5-star vote button:

• Users typically rate products that they would buy or enjoy already, instead of ones that they hate and to which they'd give lower scores
• Users are more generous online than they would be offline
• TV show box sets are rated by quality of the show, and not the quality of the actual DVD, leading to higher scores
• Users think little about the rating; a 5-star rating essentially being "I liked it!"

Of course, this tendency to give out high ratings led to the majority of products being rated an average of 4.5 or 5.  Results are summarized below:

The overall average of the averages is 4.545.  I suppose that I should start saving my money, because the Amazon community suggests that I purchase every DVD box set available. 

Perhaps a different product selection would have yielded different results, but I doubt it.  While Amazon is a great online resource, its comments feature seems to have fallen prey to fanboy-ism, resulting in ratings skewed higher than they should be.

While it would not be simple to overhaul the entire site, my suggestion to combat this problem would be to expand the ratings feature, just by increasing the available ranking choices from 5 to 10.  I have found on other websites that ten-point scales lend themselves to a greater variety of ratings.  Amazon probably does not want to change a thing though, because higher ratings may lead to more sales.  Was this rating system a clever marketing idea from the start, or did the company luck out?  I guess that's for only the CEO to know.

[This blog post has been rated 6 stars.]

Amazon Comments Data Spreadsheet 

PDF of Amazon Comments Graphs