topnav

Series Info...Trials, Triumphs & Trivialities #212:

Collective Choice: Ratings Ease of Use, Part One

by Shannon Appelcline
2009-09-18

Over the course of this column, I've talked about ratings on and off. The majority of the time, I've talked about the underlying skeleton of a rating system: how you build scales to encourage ratings, how you use math to smooth ratings, and other such topics.

However, I also think the user interface of a rating system is quite important. More specifically, what you make easy to do and what you make hard to do will have important effects both on the usefulness of your rating system and how it's used.

In this, the first of two articles, I'm going to talk about how ease of use impacts the ultimate consumer of a rating system, and how making a rating system easier to use for a consumer is almost always a win.

A lot of my recent thoughts come from many uses of the Amazon Marketplace and eBay to buy books over the last several months. More than once I've been annoyed by things that I (the rating consumer) couldn't do.

Ease of Comparison

Amazon has increasingly become a great place to buy used books. That's because through their Marketplace (formerly zShops) they've aggregated tens of thousands of used book sellers who bid against each other (using implicit Dutch auctions) for the opportunity to sell you a book.

For me as a consumer there are three important elements that I use to decide which seller to buy from: book condition, book price, and seller trust. Amazon makes it relatively easy to see all of this data on a single comparison page. If you request the page for a book of interest, the price from each seller is very prominent (and used to sort the page). The condition is equally prominent. The percentage positive rating is a bit smaller and mixed in with other text, but if you take the time, you can easily make it out on this comparison page. About the only thing I'd like to do which I can't is to sort by each of those three fields.

I sometimes feel like I beat up on eBay unfairly in these articles, but the truth is that they developed a system back at the dawn of the internet age and they haven't seen fit to update it that often or that notably. At eBay when I search for a title I see a listing of pictures, titles, seller names, and prices. In order to find out a rating for someone, I have to click through into the actual item. In other words, they offer no ease of use for comparing ratings.

This doesn't per se make me less likely to shop at eBay. But I think it does make me more likely to get involved in a bad transaction at eBay and that has the possibility of making me use eBay less. Even more generally, because of the ease of use issues, I pretty much ignore the ratings at eBay (except in extreme cases), while at Amazon the ratings drive my every purchase.

Ease of Filtering

I've talked already about my desire to sort searches by rating, something which neither eBay nor Amazon allows. However, I'd go a step further and say that what I'd really like to do is filter my searches by ratings.

In that lookup example that I offered for Amazon, I noticed that one of the sellers has a very bad rating of 89%. That's pretty horrific given the low incidence of negative reporting that happens in any rating system. At Amazon I'll often choose a more expensive copy of a book if their rating is just a few points higher, let alone 10% higher.

Any rating system would thus be more useful to me if it was easy to automatically exclude items from sellers with ratings beneath a certain level. Sure, I can do it by hand (meaning, that I just ignore ratings below a certain threshold), but not only does that introduce human fallibility, but it also just isn't as convenient.

There are several other ways that a rating-driven site could do to improve its ease of filtering. I'd also like to be able to filter out "untested" stores with too few ratings, and I'd like at least the option to filter out any seller that I personally gave a bad rating.

It's important to note that these changes (and any ease of use changes) offer two benefits to a rating system. First, they improve the odds that consumers can use the system to its full potential. Second, they make ratings more important and thus something that a ratee has to pay more attention to. If an Amazon seller knew that he could be filtered from searches if his rating dropped, he'd work that much harder to keep it up. Similarly, if he knew that a storefront with too few ratings would be filtered out from some searches, he wouldn't see starting over as a very viable option.

Ease of Access

I want to close out my look at consumer accessability with one "ease of use" issue for ratings that should be obvious, but which both Amazon and eBay fail at: the ability to easily browse ratings that you want to.

Sometimes you'll find a seller that's borderline. You don't like the amount of bad ratings it's gotten, but because of good price, access to a rare item, or for some other reason, you at least want to consider them. In this case the obvious thing to do is to look at their detailed rating responses and read over the bad comments.

To my knowledge, neither Amazon nor eBay offers any way to read just the neutral and/or bad feedback for a seller. As a result, at boths sites I have to search through piles of good ratings to find the bad ones. Since a "bad" seller is usually just a few percentage points different from a "good" seller, and since some of these sellers can manage thousands of transactions a month, this is a big deal.

There's no doubt that this missing ease of use is a disservice to me, the consumer, because it makes the ratings less useful. However, I also think that it can be a disservice to the ratee. Amazon in particular seems to attract ill-informed ratings at times. One storefront that I looked at recently had a low ratings percentage of 95% or so, and it turned out to be entirely due to people complaining about a toy that they bought breaking very quickly. It had nothing to do with the store, its shipping, or its customer service; it was about the manufacture of something that this store front had the ill fortune to carry. But anyone who just glanced at their ratings percentage would probably simply avoid them--and Amazon makes it hard to do more than this.

So poor ease of use can do damage all around.

Conclusion

Ease of use is very important to the usefulness of your ratings. If you've built a good ratings system, perhaps using some advice from my previous articles, you'll want to do everything you can to really open it up and let consumers really use it: for filtering, for comparison, and to just read through ratings of interest. This will all improve the rating system itself.

However there's a flip side to all this. Where I've talked here about ease of use for a consumer who comes in and makes use of the rating system, there's also the question of ease of use for a rater, who is adding to the rating system. Here I think there are ease of use issues that can both benefit and hurt a rating system, a topic which I'll cover in my next article, in ten days.

[ <— #211: Testing Characters, with a Look at Battlestar Galactica | #213: Collective Choice: Ratings Ease of Use, Part Two —> ]

Miniseries ...