Playtesting

A few words about the playtesting approach are in order so they are not necessary in every posting. While the specific variations in this makeover are relatively simple, a valuable aspect of this makeover is to review the testing approach to a highly variable game.

Approach

For playtesting an individual mechanic, I am introducing what I call a “Gamelet.” A gamelet (like an applet compared to an application) is a mini-game in the sense that it does the functions of a game, but in a very narrow sense. A good gamelet will exercise one mechanic in a very limited sense. In this case there are also Attributes that impact the operation of the mechanic. So there is a grid of Attributes and Gamelets to test if each attribute is tested separately.

Dice Permutation 1

 

Attribute 1

Attribute 2

Attribute 3

Gamelet A

 

 

 

Gamelet B

 

 

 

Gamelet C

 

 

 

 

However, I need to test how the different attributes affect each other also. Since I don’t have to test the same attribute applied twice in one scenario, those scenarios are not applicable.

Gamelet A

Attribute 1

Attribute 2

Attribute 3

Attribute 1

NA

 

 

Attribute 2

 

NA

 

Attribute 3

 

 

NA

 

Gamelet B

Attribute 1

Attribute 2

Attribute 3

Attribute 1

NA

 

 

Attribute 2

 

NA

 

Attribute 3

 

 

NA

 

Ultimately, then I end up with a grid for each gamelet or, more precisely a 3D matrix with two dimensions of Attributes and one dimension of Gamelets. The testing scenarios, then are [#Gamelets] x [#Attributes] x [#Attributes – 1]. So if I have 5 Gamelets and 3 Attributes, the total number of test scenarios is 30. I expect to have closer to 20 Gamelets and 5 Attributes, which creates a total of 400 test scenarios.

But we aren’t done yet. Each of those Attributes has 2 – 6 possible Values. Let’s say they each have 4 values to simplify. The number of permutations for 5 Attributes each with 4 Values is 1024. That puts our total possible game states at 400 x 1024 = 409600! This is just considering the mathematical variability of the game. Add in the testing iterations required when problems are discovered and then add in differences in the player tactics and strategies and the testing requirements have not only gone through the roof, but have escaped earth’s atmosphere.

Wow, all the more reason to make the testing fun! If the gamelets can be compiled into a game, it certainly will have a high degree of variability.

Testing 409600 different game states is a practical impossibility, so I will have to be smart about testing to get a sense as to how each variable impacts the game play. There are a couple approaches to testing that can help reduce the actual tests required:

Combine Like Scenarios: In this case, I have selected gamelets that are intentionally different, but in a game, there may be various actions or situations that are similar and testing can be done primarily on one and the impact inferred on the other.

Perform Thought Experiments: While the Thought Experiment is a tool that must be in every game designer’s bag of tricks, it is even more important in testing high variability game states.

Test Individual Attributes: This is probably the natural approach to testing for most designers. In fact, the designer most likely (and probably should) be introducing new Attributes to the game over time and so these will be tested as they enter the game. In this makeover, I am intentionally starting with a list of Attributes. So I need to test how each Attribute impacts each gamelet before testing combinations of Attributes. In this case, I will have to be extra diligent to understand the impact because I will have to infer impacts from the combinations later.

Test the Extreme States: In testing individual Attributes, I will want to pay particular attention to the extreme states and to note whether the impact is relatively linear across all values between the extremes. Example: if Attribute A has possible Values of 1, 2, 3, and 4, does the change in value from 1 to 4 have a linear impact on the game or does it act very differently, say at a value of 2?

Test Break Points: If the impact between extremes is not linear; there is a break point at which there is a big change to impact, then pay particular attention to these as well.

Test Combinations of Expected Dependent Attributes: Presumably some Attributes are more co-dependent than others. Special attention will need to be paid to these to determine where they may have break points as a combination.

Test a Random Sample: Finally, test a random sample of combinations of all Attributes. This may reveal other co-dependencies that were not anticipated and other combinations that just don’t work.

Test Outliers and Break Points: If any problems were discovered in the sample testing, then dig into them.

Breaking it down this way makes the number of variations/arrangements seem astronomical, but keep in mind that the number of variations/arrangements possible for a deck of 52 cards is 8×1067. When designing a game with a large number of dice or cards, these numbers are typical. For more information on how probability affects game design, read the great article “Probability for Game Designers” by James Earnst (of Cheapass Games) on the League of Game Makers.

What other testing approaches do you use to ease the burden of testing a high degree of variation?