There are a lot of rating systems, so it can be hard to pick one. After some considering, I’ve been using this system:

⭐ Regret
⭐⭐ Meh
⭐⭐⭐ Satisfied
⭐⭐⭐⭐ Impressed
⭐⭐⭐⭐⭐ Transcendent

What makes for a good rating system? Let’s look at a few examples to draw some conclusions.

Thumbs up or down

The simple binary system is perhaps the perfect distillation of the critic’s task. Is the thing good or bad? No ambiguity.

Five stars then? Nope. I rate it ⭐⭐. There’s just too little nuance. It works for Siskel and Ebert because their rating is actually 4 distinct states:

Review Siskel Ebert
Two thumbs up 👍 👍
Siskel likes 👍 👎
Ebert likes 👎 👍
Two thumbs down 👎 👎

Even when they agree, there is nuance in their review that isn’t reflected in the rating. For instance, neither liked Brazil, but Gene liked it better on the second viewing and thought it could be saved with further editing. Roger just hated it.

Michelin Guide stars

The official description:

  • ⭐ “High-quality cooking, worth a stop” (Cuisine de qualité, mérite une halte)
  • ⭐⭐ “Excellent cooking, worth a detour” (Cuisine excellente, mérite un détour)
  • ⭐⭐⭐ “Exceptional cuisine, worth a special journey” (Une des meilleures cuisines, vaut le voyage).

Note that a Michelin star is a huge deal for a restaurant and it’s not a great scale for the whole range of things. It’s secretly a 4-point scale with almost every restaurant getting zero stars.

I rate this scale ⭐⭐ for anything besides high-end experiences. (For those, it gets ⭐⭐⭐, but you have to decide which system I used.)

Hundred point scale

So this is apparently how wines are rated. Metacritic uses 100 points too. It’s also effectively the Board Game Arena rating system because many people use decimals (6.3, 7.6, 8.1, etc.). This scale has plenty of nuance because there’s space in the rating to put slightly better things over slightly worse things.

The downside is a 100 point scale lacks even the semblance of conviction. An 82 point thing is clearly better than a 79 point thing. But where do you draw the line between worth the investment and not? Because of this, scores tend toward inflation. If 80 is the culturally accepted cutoff, a 79 feels harsher than it otherwise would be.1

Unless you review hundreds of things, this system is ridiculous. ⭐

Ten point scale

This includes IGN, IMDb and Famitsu2. This scale shares many of the same problems as the 100 point scale. People tend to use 1-6 entirely unused and the real system works out to be:

  • 6 Bad
  • 7 Ok
  • 8 Good
  • 9 Great

In other words, a 4 point rating. I give it ⭐.

5 ⭐s

The 5 star system thrives in product and service reviews. If you ever look at Amazon ratings, you’ll notice that people tend to use 1 ⭐ and complain that there isn’t a way to give zero. 5 ⭐ ratings seem way more common than they have an right to be. J-shaped distributions occur because people tend to either like a product (it affirms their decision to buy it) or hate a product (because it disappointed somehow). My favorite reviews are 1 ⭐ reviews in the form “I ordered this for [some important event] and it was delivered late!” That must suck for you, but it tells me nothing about the product itself.

Eagle-eyed observers will notice this is essentially my system. If it’s to work, I need to be vigilant about giving more stars in the middle of the range than towards the end. So ⭐⭐⭐ out of 5.

Solomon scale

Named after Paul Salomon who explained the scale in Episode 184 of the Decision Space podcast. (Go to around 15:00.) The scale is simple:

  1. Needs more work
  2. Fine
  3. Good

He pitched it for game designers looking for ratings from play testers. It has the sharp decision of the thumb scale with a tiny bit of room for greatness at the top. It’s designed for regular things in exactly the way Michelin stars are not. For that purpose, it’s a ⭐⭐⭐⭐⭐ on my scale.

So why use the 5 ⭐ system?

Ultimately I want to sharply divide the stuff I really like from the stuff I don’t with a tiny bit of room for nuance. For instance I don’t recommend Cibele or Wasted Pizza. But one (Cibele) is substantially better than the other. A binary recommendation works better when aggregating many opinions. Something with a tiny bit more give suits me better.

  1. Additionally, the US grading system probably influences people to assume 60 as a failing grade.↩︎

  2. Ok, technically Famitsu gives games scores between 0 and 40 because their scale starts at zero and there are 4 critics.↩︎