Our rating system

We realize that we’re not professional literature critics — we’re just a group of readers who love to read and write about speculative fiction — but we started this site to help SFF readers decide what to read and most consumers, whether of fine dining, vacuum cleaners, or fantasy novels, seem to like some sort of quantification. Thus, we assign “ratings,” even though a few of us find the notion disagreeable. We challenged ourselves to come up with a standardized rating system. Though Marion and Ryan a couple of us are notoriously stingy with our stars, we still managed to come up with a scheme that we could agree on:

I loved pretty much all aspects of this book: story, style, and characters. It will be one of the best books I read this year. Perhaps it blew my mind or changed how I view the world. A 5-star book is distinguished from just a really good book by the level of depth, emotional impact, and style. This book is awesome, so go out and get it today even if it means wading through a horde of orcs.

This is a really good book which I completely enjoyed. It was definitely entertaining, but didn’t blow me away. Maybe it doesn’t quite have the intellectual or emotional wallop of a 5-star book, or maybe I was hoping for something transcendent and it just didn’t quite deliver that. But still, I strongly recommended that you let the orcs pass by, then get the book, even if you have to deal with a straggler or two.

This book was enjoyable and there were qualities that I admired about the work, but it was marred by one or more noticeable flaws that kept it from being “really good.” Still, the good outweighs the bad, so pick this up if you have a long wait while the orc horde goes by.

I didn’t like it and I don’t recommend it. I had to force myself to read it (and it’s possible I skimmed parts). Perhaps the author’s goals are interesting or ambitious but the result is disappointing. Or it might have been boring or had major plot holes or mechanical problems. Or maybe I recognize that I am not the right audience for this book, so it gave me no enjoyment. I think you’re better off counting the orcs in the passing horde than reading this book.

This novel was dreadful in all respects, or was so impossibly poor in one area that no more positive qualities it might have could pull it out of the mire, or it was on a wavelength I just couldn’t tune into. It might have been an act of self-flagellation, but I finished it (possibly skimming) maybe just to see how bad it could get, and maybe just to generate a snarky review if it was written by a popular author. I’d recommend that you throw yourself into the midst of the orc horde instead of reading this book.

I couldn’t finish this book, maybe because it was horrible or boring, or maybe because it was just totally not my thing and I wanted to move on. Give this book to the orc horde. Life’s too short to read bad books!

Of course, we didn’t come to this without a lot of discussion. Here’s what Ryan and Tim had to say:

Ryan: When assigning ratings, I can’t help wondering about the dedication it would take to devote months, perhaps years, on a story or a character or a journey. It must take a lot of time to create a character like Neal Stephenson’s Hiro Protagonist. How odd it must be to devote one’s creative spirit to a story, only to have random strangers represent the entirety of their response to that story in just a few stars. (Preferably 4 or 5.) I imagine it must be disappointing for the author, if only because as someone who writes reviews, I sometimes find it irritating that I might spend an hour or so writing a review only to have readers glance at the number of stars I use to summarize the impact a work made on me. It may not make a lot of sense, but the number of stars associated with a novel or a number followed by a % is more likely to convince readers to move to a city like China Miéville’s New Crobuzon than anything I write about it. Which is why I give out 5 stars sparingly.

Tim: There are a lot of issues with assigning stars to novels. Of course we need a bit of a concrete rating system, but how are we to manage all the factors? Is it ‘good’ according to me and only me, the reviewer, that fiend who secretly (gasp) doesn’t much care for Terry Pratchett novels yet has the temerity to rather like confusing, oddball curiosities like Lud-in-the-Mist? Or is it ‘good’ in relation to other books in the same series, or in relation to the genre as a whole? I personally tend to rate books in relation to what I believe their position is in the subgenre from which they hail. I’m far less a fan of romantic fantasy than I am of epic fantasy, but if a good romantic fantasy dances hopefully by in its best dress, I’m not going to scoff, turn up my nose, and send it over there with the other riff-raff. Subjective I may be, but there are all sorts of fantasy readers, and whether or not I am the target of a novel is something I try to keep separate from my review. If I think a novel is a good example of urban fantasy, it gets a high ranking. Thus when I’m reviewing an IRON DRUID novel, for instance, I’m reviewing it in relation to other novels in the subgenre of urban fantasy. There’s something to be said for the purely subjective experience, I suppose, but I do tend to find this way of doing things more fair to viewers. At least if I’m forcing myself to contextualize the novel I’m not rating it a star lower simply because I happened to be suffering from indigestion most of the way through chapters 32-56.

Ryan: If I assign a novel 3 stars, there were likely things that I admired about the work. For example, although I am a big fan of William Gibson, I was not very impressed with Spook Country. Still, Gibson is an outstanding writer and the mood of the novel stood out to me. I do not regret having read Spook Country, and would consider reading it again to see if I might enjoy it more now that I’m older. Although some might refer to the 2 star rating as “mediocre,” I tend to have more in mind than “mediocrity” when assigning 2 stars. In my opinion, there is an audience for the 2 star novel. However, the novel did not speak to me. When reading SFF, I find that I often assign 2 stars to works within a series. There is certainly an audience for Robert Jordan’s WHEEL OF TIME series, but some entries are far weaker than others. (Oddly, I’m inclined to rate the series 5 stars, though I’d likely not rate any work within that series a 5). At other times, I might rate a novel 2 stars if I find the author’s goals to be interesting or ambitious but the result disappointing.

Tim: For me, 2 stars is a failing grade, the point at which the novel’s issues have swallowed up the reading experience to some extent, but it has some good points remaining. 1 star is something I’d use very sparingly, indicating that the novel was simply dreadful in all respects, or was so impossibly poor in one area that no more positive qualities it might have could pull it out of the mire.

Ryan: Assigning stars to novel is a ridiculous pastime. The only thing I can say in its defense is that it sometimes leads to an interesting discussion. “Really? Just three stars? Justify your rating!” In other words, the imperfection of star ratings is often what makes them so fun. It’s just unfortunate when counting takes priority over thinking. So readers are right to be wary of star ratings. Fortunately, they can read the entire review. Or better yet, they can read the book and consider their own response to the story. Like everyone else, I’ll be interested to see how many stars they assign it.

Tim: I like to think this is a fairly just way of dealing with the necessity of assigning rankings to these texts. I admit freely, however, that I often go back and forth on rankings quite a bit, and I often try to give positives and negatives for various types of readers. I encourage anyone who wants clarification on my opinions to ask me via a comment.

So, readers, how do you feel about ratings for books? Is the rating more or less important than the review? Do you prefer to have a quantification scheme or would you rather read a review to help you decide whether to read a book? How do you use (or not use) ratings? If you rate books on your blog, at Amazon, or at Goodreads, what is your rating scheme like?