Community Focus: BGG Ratings Part 1: Know Thyself

John Parker
Feb 27, 2017

Setup

The debate about BGG rankings comes up frequently in the BGG forums, on tabletop gaming podcasts, and around the gaming table. In this 6-part series of articles we will look at the BGG ranking system from the perspective of the BGG user ratings: the BGG recommended rating criteria, user rating methods, and some user practices. Given its high visibility in the BGG community, we will pay particular attention to the pandemonium that occurred a year ago (January 2016) as Pandemic Legacy: Season 1 (or simply Pandemic Legacy) raced to the top of the charts and made a hot zone of the forums. (I actually started this series then, but am only now getting back to it). Here are the six parts:

This being the first part is probably a case of burying the lead – the most interesting aspect of this series is the analysis of the rating activities for Pandemic Legacy – but I want to preemptively answer some questions about BGG ratings and what I do. Since I will be analyzing data submitted by other users; their ratings, collections, and play data, I will start by presenting my own data and methods for context and in the interest of full disclosure.

Important Note: Although I will use personal tracking and rating information in this analysis and it is public information, I will absolutely not present any individual personal data with the person identified. This series will provide statistical information calculated from the individual source data all collected from Boardgame Geek. I used primarily the Ratings and Comments pages for the games in question and the user profile and collection pages to compile the data. Some additional specific pages will be mentioned throughout.

Round 1: Tracking Personal Data – What

Although a BGG user does not need to track any personal collection or play information to be able to use the rating system, though some contend that this should be a requirement, the data provided in their personal tracking is informative when trying to discern how BGGers use the rating system. Since I have used this data for my analysis, I will present my personal level of tracking and method.

I have been tracking the following consistently since Jan 2014: (prior to then, when I was sane, I just used the personal collection as a wish list)

  • Collection Information:
    • Want to Play
    • Wish List
    • Owned/Preordered/Previously Owned
    • Private Information: when and where acquired
  • Play Information:
    • Base Games
      • I count the first play of an expansion so I know when/if I added it to the game.
      • These are mostly tabletop, face-to-face plays.
    • Online/Android Plays
      • I track online or Android plays if they are a replacement for a tabletop experience – that is, synchronous gaming with someone I would play with at my kitchen table if I could.
      • I will count 1 play (so I know that I have played the game) for those online or on Android.
    • Number of Plays
      • If multiple plays in a session – even for fillers – I include the number of individual plays.
    • Players (that I know – otherwise use “Unknown” to get an accurate player count)
    • Winner (except “party games“)
    • Scores (usually for each game unless a # of wins for a series of plays is more appropriate – e.g., Backgammon, Cribbage, etc.)
    • Location (though inexact for generic locations; “Bar”, “Hotel”, etc.)
    • Comments (primarily to document game state specifics – have I played all the factions, etc.)

Round 2: Tracking Personal Data – Why

Something informative in this self-review process is the degree that any tracking rationale or method is very personal and unique to the individual. The reasons for tracking are personal and the method used is presumably related to those personal reasons.

There are several good reasons for tracking collection and play data on BGG and at least twice as many for not bothering. Here are the main attractions of the practice for me:

  • Game Night (especially since I am usually the curator and tour guide for these)
    • Browsing and querying my Collection for a game to play – usually when I am not in proximity to my game shelves (like the couch in the next room).
    • Easily generating a list of games with BGG links – “Hey guys, here’s a list of 6-player games we can play tomorrow night…”
    • Easily identifying games that my fellow gamers have played – they may still need a rules refresh, but probably have an idea about what to do – or not played – we will need some time to teach the game.
    • Easily identify games that I own, but haven’t played – the shelf of shame.
  • Tracking
    • Tracking my game playing behavior and trends.
    • Tracking my “investment” – I’ve spent a substantial sum on games. Am I getting a reciprocal enjoyment from them?
  • Game Design and Study
    • Identifying games “of a type” (theme, mechanic, weight, etc.) for comparison or study.
    • Fodder for blogging.
  • Game Purchasing
    • Curb my enthusiasm to buy a new game when I own plenty that have not been played recently or at all.
    • Quickly identify games to look for at a store or in a sale.
    • Publishers and designers who affect my game experience one way or another.
  • Personal
    • Being able to respond to false absolutes, “I/You never/always win.”
    • Satisfy my OCD.

Everyone has their own method, so generalities cannot be easily made from a small amount of data. Having worked in software development and data analysis for almost 30 years, this is no surprise, but the degree is high compared to most data collection because there is no common goal of the data for the community that records it.

Round 3: Tracking Personal Data – How

Tracking this data is easy enough on BGG, but it can still be quite tedious. Thanks to the BGG 4 Android app, though, tracking plays and updating my collection is a snap. I highly recommend this app for any BGG user, even if you don’t track personal data. I assume that there is an iOS equivalent since Android apps are usually a subset of what is available on iOS.

Round 4: The BGG Rating Scale

In case you haven’t read it or it changes in the future, here is the BGG recommended rating scale:

  1. Awful – defies game description.
  2. Very bad – won’t play ever again.
  3. Bad – likely won’t play this again.
  4. Not so good – but could play again.
  5. Mediocre – take it or leave it.
  6. OK – will play it if in the mood.
  7. Good – usually willing to play.
  8. Very good – enjoy playing and would suggest it.
  9. Excellent – very much enjoy playing.
  10. Outstanding – will always enjoy playing.

I am not sure if the text related to each BGG rating should be considered as directions, advice, recommendation, example, etc., but the commonality and exceptions in the scale provide insight into the intent:

  • The rating is related to:
    • The act of playing the game. Except 1 and 5, the word “play” is used in the description.
    • Personal experience.
    • Emotion/Feeling.
  • Exception:
    • The “1” rating is a statement about the game itself.
    • Does this description “allow” for a rating of “1” on a game that you would never play because you don’t consider it a game?

Round 5: My Rating Method

This is my rating method, such as it is. Though I am sure that I deviate from it periodically, it is what I typically do.

  1. I generally follow the BGG recommended rating scale – based on experience with the game and desire to play it. (More on that in Round 5).
  2. I use only whole numbers. (A 10 point scale precise enough for me).
  3. I try to rate every game that I play (enough to have an opinion).
  4. I usually rate after 1 play unless I know a second is coming soon.
    1. I indicate in the rating comment “First Impression” for anything where I haven’t had enough plays to be more certain the rating will be close.
  5. I will update ratings periodically, but don’t go through my entire list of ratings and recalibrate.
  6. I rate based on my experience at the time I was actively playing the game.
  7. I occasionally rate games that I played a long time ago and they come to mind for some reason (particularly family or children’s games). I rate these based on my memory of how much I enjoyed them and provide a comment to that effect.

Note: I am not a reviewer. If I was, my method would be different (e.g., I would wait until I had a handful of plays to rate, I would apply different criteria than those in the BGG scale, I may not use the BGG rating system at all, etc.).

Round 6: The Numbers

Although, it may not be that interesting at this point, for the record when I report on these stats in the next article these personal stats may be of interest. At the time that I researched the data for this article my stats related to this article were:

  • Collection
    • 310 Owned
    • 30 Previously Owned
    • Pandemic (Base): Owned
    • Pandemic Legacy: Season 1: Owned
    • Twilight Struggle: Not Owned, Want to Play
  • Plays
    • Total: 1931 (since Jan 2014)
    • Most: 102
    • Though this is an undercounting of combined Android and tabletop plays of Star Realms indicating that it was a “thing” for a while for a couple friends and me. Otherwise I track actual plays.
    • Games Played 18 or more times: 25
    • H Index: 21
    • Pandemic (Base): 24 plays
    • Pandemic Legacy: Season 1: 17 plays
    • Twilight Struggle: 0 Plays
  • Ratings
    • Total: 412
    • Ratings of 10: 5
      • The Castles of Burgundy (Ravensburger English/French Edition) (2011)
      • Caverna: The Cave Farmers (English Second edition) (2014)
      • Mice and Mystics (English first edition) (2012)
      • Pandemic Legacy: Season 1 (English first edition (blue)) (2015)
      • Puerto Rico (English second edition 2003) (2003)
    • Ratings of 1: 2
      • LCR (1983)
      • Tic-Tac-Toe (-1300)
    • Average rating: 6.4 (I have a fairly regular Gaussian distribution)
    • Pandemic (Base): 8 – though it may be overrated today since I don’t really want to play it without some expansion(s) included
    • Pandemic Legacy: Season 1: 10 (Rated after I played through the whole game)
    • Twilight Struggle: Not Rated

Your Turn

Do you rate games on BGG? Do you track other statistics (e.g., your collection, your plays, etc.)? What is your method for rating them? Did you watch or participate in the rating discussions related to the rise of Pandemic Legacy: Season 1? Read Part 2 for more on that.

If you find this article interesting, you may want to check out all the articles in the Analysis Paralysis category where we analyze and discuss issues in the tabletop industry and community.

Continue with this series:

Subject
Analysis Paralysis
Community
Game Industry