TDD in gamedev – how? when? why?

Is TDD used in gamedev? If you don't know the answer or the answer seems obvious to you, this article may surprise you.

What is TDD?

First, for those just exploring the topic, a brief (simplified) explanation of concepts.

– (Automated) test – a piece of code whose task is to test some functionality of regular code.
– Unit test – a test that checks the operation of a piece of code in isolation from the rest of the code.
– Integration test – a test that checks how connected pieces of code work together.

TDD (Test-Driven Development) – a program design strategy where programming begins with a unit test and follows the pattern:
1. Red – failure. We create a test that fails.
2. Green – works. Acting along the path of least resistance, we make the test pass.
3. Refactor – improvement. We improve the code while preserving the obtained functionality.

In theory, each of these points should take only a moment, hence:

"TDD actually is the constant fast switching between extending the tests and making them pass continuously. So as you write tests, you write code to satisfy them basically at the same time. This allows you to instantly test what you write, and mainly use tests as specification of what the code should actually do, which guides the thought process to make you think about where you are headed to, and to write code that is more structured and testable from the very beginning." Kovarex, creator of "Factorio"

Note: It's worth distinguishing TDD (where tests drive design) from later adding tests to existing code. The latter approach is more common in gamedev – tests appear only once the mechanics have stabilized.

TDD Enthusiasts

When dealing with programming, at some point we come across the topic of "clean code". We might also reach for Robert Cecil Martin's book "Clean Code: A Handbook of Agile Software Craftsmanship", which, incidentally, I found at the top of the list: "gift for a programmer": "You have no idea what language your loved one programs in, no matter! Give them this 'Programming Bible' by Martin – also known as 'Uncle Bob' – it will surely come in handy."

In this book, in the chapter: "What Is Clean Code?" Martin quotes Dave Thomas:

"Clean code can be read, and enhanced by a developer other than its original author. It has unit and acceptance tests...."

and adds himself:

"Dave is right. Code, without tests, is not clean. No matter how elegant it is, no matter how readable and accessible, if it hath not tests, it be unclean."

In the epilogue, he confesses: "Since learning TDD from Kent Beck in 1999, I have indeed become obsessed with test-driven development." and adds that taking off a wristband with the inscription "Test Obsessed" would be like a betrayal of professional ethics for him, which he cannot allow.

In reality, those teaching "clean code" often steer towards TDD, and the "instructors" themselves gush with enthusiasm over this technique. It's supposed to be faster, better, and more reliable. Test-driven development works not only in code creation but also in industry and other fields.

It's hard to find information about the disadvantages of this approach in the materials. Watching a video where "Pong" was made in Godot and reading online comments saying "I always start with a test," I thought it was a must in solid programming, including gamedev. After all, no one mentioned it wasn't used in gamedev, and "Uncle Bob" couldn't be wrong, right?

My Experience with TDD

For over two years, I've been creating my own anime-style fighting game called "Lightning Punch." For most of this time, I worked as a TDD enthusiast. I wrote hundreds of tests and experienced both the advantages and disadvantages of this approach, which I will briefly present here.

It's worth noting that Godot has a special testing addon called GUT. Testing in Godot is thus much easier. While in other languages, a large number of tests can prolong compilation time, in GDScript, which requires no compilation, testing is fast and less problematic.

Is TDD used in gamedev?

It must be said clearly: Although TDD has its share in creating tools, systems, and game engines; the TDD technique is extremely rarely used as the main approach in creating the games themselves. Even automated tests are not very popular in gamedev, let alone test-driven development, which remains on the margins as an expression of peculiarity and eccentricity. There is a widespread belief that TDD is unsuitable for game development.

On Reddit, one user even said about using TDD in gamedev:

"There’s no way any sane individual would do this."

As a counterpoint to this, Alistair Doulin ("Battle Group") as early as 2011 spoke about the benefits of using TDD in gamedev. Ariel Coppes, creator of the upcoming "Ship Miner," also praises TDD. Among the really well-known, on forums you can find that studio Rare develops with TDD:

speps on July 19, 2017: „If you want to experience that in the AAA game industry, I work at Rare Ltd (UK, part of Microsoft Studios). We've been doing TDD and Continuous Integration for about 5-6 years now starting with Kinect Sports Rivals. For Sea of Thieves[0] (our current project), we've extended that to Continuous Delivery and it's been just a complete game changer in regards to overtime and crunch. For example, the onboarding exercise is basically a TDD kata that has to use our coding guidelines and build farm so you get to know how we work without having to dive right away into Unreal Engine 4. We use TeamCity for our CI solution.”

Also Kovarex, creator of "Factorio," uses TDD:

"So after the "AHA" moment of realizing what TDD really is, I started to be instant fan. I'm now putting a lot of effort to try to follow the TDD methodology as much as possible, and to force it on others in the team as well. It feels slower, to write tests even for simple pieces of logic that just bound to be right, but the test proved me wrong several times already, and prevented annoying low-level debugging sessions in the near future.” Kovarex Friday Facts #366 - The only way to go fast, is to go well!

Why was choosing TDD a mistake for me?

I strictly followed the TDD methodology from the very beginning. Although it gave me much joy, I eventually got lost in it all and had to start over, the normal way.

Test-Driven Development is unsuitable for prototyping. In general, excessive focus on code cleanliness and implementation during prototyping doesn't make much sense.

There are specific reasons why Test-Driven Development doesn't work particularly well in games:

1. TDD forces fragmentation and independence of modules, which can be good, but in games, it's not always practical. Additionally, integration tests have much greater value in games than unit tests. A very high percentage of unit tests can lead to them failing every time you intentionally change the implementation, i.e., they hinder your progress.
2. In games, everything is subject to change in the search for better gameplay; we don't have such a clearly defined goal to strive for. With TDD, you can precisely implement the programming assumptions, but what good is that when halfway through it turns out the idea was bad and the game isn't fun? TDD works better when we have a strictly defined goal, which in gamedev is more fluid.
3. In games, there are many things that are exceptionally hard (or impossible) to test automatically. How do you automatically test fun? How about shaders? Smooth animation appearance or good rendering?
4. When creating a game, we often learn what we really want while programming and manual testing. TDD, requiring us to decide upfront what we want by writing a test, makes it harder to discover such new solutions or game mechanics.
5. When working with TDD, it's easy to lose sight of your main goal, which is to make a good game, not reliable and good code.

Does TDD make changes easier or harder?

The accusation against TDD is that it hinders changes and cements implementations. Paradoxically, proponents of this methodology present the ease of making changes later as TDD's main advantage. Who is right?

Both sides are right, as it depends on the type of project and the changes whether TDD helps or harms. If we had, for example, a complex economic strategy and now needed to add some new mechanic or change an existing one, the broad test coverage provided by TDD would be a godsend and very helpful.

If, however, we wanted to completely overhaul the controls in a platformer to make it more enjoyable for the player, the presence of tests would only get in the way, and experimenting and creating new mechanics would be significantly hampered by TDD.

Faster or slower with TDD?

The accusation leveled against TDD in gamedev is that it significantly slows down development. Proponents of TDD, on the contrary – recommend it to create games faster. Who is right?

Programming with TDD is generally slower but smoother. It saves time because the created code is more solid and easier to debug. In gamedev, however, when you have to delete dozens of tests or stop for a long time to figure out some clever way to test something untestable, it can significantly slow down the project.

What TDD taught me

Before learning the TDD methodology, I had a tendency to create large chunks of code at once and then debug them. Thanks to TDD, I learned to work in small steps, which, though not always, is often very helpful.

I now treat TDD as a useful workshop skill. I reach for tests when it's convenient for me, not all the time.

Test-Driven Development – Without Tests?

Einstein conducted thought experiments, and we can conduct mental tests. One advantage (which is sometimes a disadvantage) of TDD is that when creating the test first, we first have to think carefully about what we want to achieve.

Even when I don't write tests, I often use the TDD approach in the sense that I first create a kind of test in my head, i.e., I decide what exactly I need to do and how it should look, then I code it quickly, and finally I refactor. Of course, this way I don't have the advantage of TDD, which is automated test coverage, but I do have the advantage that coding this way makes it harder to get stuck on what I'm doing.

Similarly with fixing bugs. Seeing a bug in the code, instead of fixing it immediately, I first test manually to reproduce the bug ("Red") and only then fix it ("Green"). This is useful because it sometimes turns out that the "bug" didn't need fixing at all or was more complex than I thought.

And yet some use TDD in gamedev? Factorio?

The TDD approach comes with a range of advantages and can be particularly beneficial when creating games with complex systems. In games like Factorio, where precise systems are key (e.g., production chains), TDD makes sense because the requirements are deterministic. In games based on the 'feel' of gameplay (like fighters or FPS), TDD is less useful because it's hard to encode the fun of gameplay in tests.

Reviewing what developers using TDD in gamedev write, it's clear that they don't adhere to this approach too rigidly.

The worst thing you can do is follow the theory too slavishly and literally. Gamedev differs in specifics from other IT sectors, and because of this, it seems that TDD requires special adaptation for games. For example, tests can be manual or integration, not always unit. Not everything must be created using TDD, but rather utilized where needed. TDD used as a tool can sometimes help significantly.

Regarding adapting TDD to games, it's worth quoting Kovarex's words again:

"If tests should be truly independent, test C, should have some mocks of A and B, so the test of C is independent of the system A + B working correctly. The consensus seems to be, that this leads to more independent design etc.

This might be applicable in a lot of cases, but I believe that trying to have this approach everywhere is close to impossible, and it would lead to a lot of clutter and I'm not the only one having problem with this.

(…)

For this, I implemented a simple test dependency system. The tests are executed and listed in a way, that when you get to debug and check a test, you know that all of its dependencies are already working correctly. I tried to search if others use the dependencies as well, and how they do it, and I surprisingly didn't find anything.”

Therefore, some adaptation and development of implementation methods for TDD in gamedev are still needed.

Summary:

1. The TDD technique, though recommended in other IT sectors, is not commonly used in gamedev.
2. Both the enthusiasm of supporters and the hostility of opponents of TDD are exaggerated.
3. It's worth trying TDD and drawing the best from it, but one should not follow this method literally and rigidly. Rather, treat it as an additional workshop skill.

TDD is a tool, not an obligation – use it where it brings benefits. And only when it actually does bring benefits.

Sources:

https://www.factorio.com/blog/post/fff-366
https://news.ycombinator.com/item?id=14802333
https://www.gamedeveloper.com/programming/test-driven-game-development-experiences
https://arielcoppes.dev/2023/10/29/tdd-to-make-games.html
https://monoflauta.com/2022/05/12/doing-tdd-in-indie-game-projects/#comments