In 1961, physicist Edward Lorenz carried out simulations of climate models. Processing data on temperature, pressure, winds and everything else he might need, he managed to organize a model that related the progression of weather conditions day by day in his computer-simulated environment.
When he needed to save his experiment, due to technological limitations, and continue the process later, Lorenz chose an intermediate step, replicated the data stored on the punch card he was using (1961, remember?) and ran the simulation again. Before long, the subsequent results of Simulation 2 were totally different from the results obtained previously in Simulation 1.
This variation was due to a minor difference in the computer’s data storage. While the punch card delivered three decimal places of accuracy, the internal memory was storing six.
This chance made Lorenz see how minor variations in the initial conditions of a system could generate totally different later conditions. Hence the phrase “A butterfly flapping its wings in Brazil could cause a hurricane in Texas”.
Butterfly effect?
Perhaps you know this as the Butterfly Effect, a fundamental element of Chaos Theory, which has Edward Lorenz’s studies as one of its bases and forms one of the most interesting areas of study in Probability Theory.
This type of occurrence also gained its own name, which defines processes, systems and other activities that are not random, have known rules and initial conditions, but with unpredictable results: Stochastic.
On the other hand, there is a term in Probability Theory that defines the opposite result. A system that has known rules and initial conditions, whose progression and results can be predicted through a model that will always generate the same results: Deterministic.
After the brief explanation, can we finally talk about what all this has to do with making games? Let’s start with Benchmarking.
Read all our Games Design Posts
When make benchmarking?
Benchmarking is a process that must be done at the beginning of the design of a game project in order to understand the possible performance of a product and mitigate its risks by analyzing the industry conditions and the main competing projects that are similar by characteristics of gameplay, genre, theme, among others that can be considered.
This activity is something that involves a lot of the Business Intelligence area, with several researches involving chart analysis, understanding of market trends, sales progressions and user behavior predictions. All this to carry out the risk mitigation of an investment that can last for years and involve considerable sums of resources.
On the other hand, all this is oriented to the concept established for the project and the vision of the development team that filters the way it imagines the project will be seen and understands the features designed in parallel with others already applied in the industry or similar.
This brings us to the starting point of this text: benchmarking processes are often seen as processes with deterministic outcomes. On the other hand, they are much more similar to processes with stochastic results.
Thinking of benchmarking as something deterministic implies that the closer you get to the parameters of a benchmark, such as using features, aesthetics, being present on similar shelves and in similar market conditions will necessarily bring a similar result.
Nothing could be further from reality.
When we stop to think about it, we begin to realize how absurd it is to imagine that relationships could exist in this way.
“Game X uses a retro aesthetic, presents multiple choices for the Player’s journey and has a customization system that is very reminiscent of that other famous game. It sold over ten million copies across all platforms in 2018 with this Marketing strategy, so if I go in a similar direction, that means I’ll get at least a fraction of that result.”
Have you ever heard this somewhere? It’s the Pseudo Science of Benchmarking for Games. DNA Reprogramming Applied to Games. Quantum Coach of Interactive Entertainment. And just for mathematical information, Ten Units is a fraction of Ten Million, which does not guarantee that you have had a relevant or desired fraction of success.
What I mean by this is that superficial parameter analysis is not enough for good benchmarking. Understanding how many units a game has sold or what its features are individually will not be information that alone will give the correct analysis picture.
This is because of the three decimal places that you are not counting in your analysis, that is, the factors that go unnoticed if you only perform a quantitative validation of the design without taking deeper elements into account.
Let’s open a parenthesis
Opening a parenthesis for Game Studies, there are theorists who argue that Games are only Games in fact when they are being played. Otherwise, the physical (or digital) object by itself is just an object, which is not endowed with playful features outside the Magic Circle. This, despite sounding a bit esoteric and philosophical, is important.
This means, in practice, that Games have a Player component that completes the interaction and makes it gain meaning. Games are not interesting as works to be exhibited or just watched (although some Board Games are true miniature works of art and some Digital Games are movies with Quick Time Events). Thus, the Games depend on the player to materialize and this dependence occurs at various levels.
About these elements, the characteristics of the audience and how a clear understanding of the Design of your project can help you to achieve a coherent Benchmarking, we will deal in the next chapter of this text before we start writing a monograph.
But the record is to be remembered in the future, the small decisions you make in the concept of your project impact much more in the long term than the big Tags, Bullet Points and Advertising Videos. These are the three decimal places that differentiate your game from other companies’ games and they contemplate the features that make your game what it is.
