In a previous post, I explained why writing clean code is more profitable both for the developers and their managers: it will help them build faster, with fewer bugs and save both time and money in the long run. This time, I’d like to show with some basic math how a test first approach can help speed up your development, among other benefits.
Back when we first decided to move to Kotlin, our managers didn’t seem very keen to accept the change. Since the language was quite new, wouldn’t it be more expensive to train everybody to adopt it? Wouldn’t it be more complicated to find new developers that would know the language? Wouldn’t it be more complicated for new additions to the team to understand our apps and fix bugs if they’re not familiar enough with the language?
As an attempt to convince them of the extreme benefits we would have if we ditched Java, I tried to explain how Kotlin would help us produce safer and cheaper code using words like improved null safety, greater readability, better maintainability. But it turns out that unless I could back it up with numbers, managers didn’t really believe my claims. They’d heard similar ones in the past, and they hadn’t necessarily seen the effects their authors advertised.
Eventually, we managed to convince them, but I sometimes have the same type of debate when it comes to software quality, and especially whether it’s worth writing automated tests for my code or not. After all, writing tests takes time, so logic tells you that if you skip the tests, you will be able to produce value quicker, won’t you? Well, instead of shoving my copy of “TDD by example” in their face, hoping Kent Beck will have more success convincing them, I tried to do some math.
A bit of math
It’s hard to pick exact numbers to illustrate my point, so let’s choose some arbitrary values first. As we’ll see later, the actual values don’t matter much, but they will help us perceive the differences more easily. Let’s assume we want to evaluate the time it takes to build a program that meets a certain number of requirements. We will implement them step by step, checking regularly that our code does what it’s supposed to do. We’ll keep each step relatively small so we can easily find bugs (debugging a huge amount of code is always much more complicated that a small number of changes).
Given that our steps are small, let’s assume we will spend 10 minutes implementing the requirement and 5 minutes coding the corresponding test. Running all our automated tests would only take a few seconds, so that time will be negligible in comparison. Testing that feature manually would be quicker than writing the test for it, but longer than running an automated test: I need to start my app, navigate until I reach the proper state, then effectively check the feature. Let’s say I spend on average 1 minute doing that.
We can now compare the amount of work I have to do when I use TDD and when I use manual testing:
- With TDD, I’ll start by writing a new test (5 min), implement the feature (10 min) and run my complete test suite (0 min). So a total of 15 minutes. The next step will be the same, so each new feature will add another 15 minutes.
- With manual testing, implementing the feature will take 10 minutes too, but testing it will only be 1 min, so 11 minutes in total. When I implement my second feature, I’ll spend 10 minutes coding it and 1 testing it, but will spend another minute to verify I didn’t break the first feature (so 23 minutes for 2 features). With the third feature, I’ll have to spend 2 extra minutes (one minute by feature, so a total of 36 minutes).
What we can see is that based on those numbers, it seems quicker not to write tests at first, which seems to confirm the common intuition that writing tests add time. But the problem is that with each new feature, the time to check none of the previous features was broken is going to add up. After the ninth feature, the TDD curve drop below the manual test one.
If you keep that trend for a while, the difference will become even more obvious.
What starts to appear in those graphs is that with TDD, the time it takes to add a new feature is pretty much constant: testing all the previous features will only take a few seconds and can be completely ignored. So the growth is perfectly linear (t = 15 * n). With manual testing on the other hand, that times increase with each new feature because manually retesting everything takes longer and longer. With that simple model we have a quadratic growth of the curve (t = 10 * n + n * (n + 1) / 2).
This is is why the values we chose, even if totally arbitrary, did not matter much. Whatever those numbers actually are, you will be comparing a linear series with a quadratic one. Even if you tweak the numbers to maximize the gap between the two curves at first, you will eventually reach a point where the red line jumps above the blue one.
You could suggest slowing the red curve by not retesting everything all the time. This is indeed a possibility (and actually what we tend to do in practice). But please note that until now, we have been assuming a best case scenario: everything we coded worked on the first try. The problem with that attitude is in case you break something, it will take you longer to notice it. By the time you discover something no longer works as expected, it might take you a long time to track down the change that created that bug. Even worse, when you find it, you might realize that you’ve done a few more changes that rely on this faulty one. In those cases, the best option is often to go back to the last state when the code worked as planned. Which in TDD is basically the last time you ran your test suite. With manual testing, if you don’t test ALL your code EVERY time you alter it, it is going to be painful to track the last time you were sure your code worked (remember that checking your code at a certain state takes time because it implies running all the tests, which is not reasonable if you have lots of commits to check).
Lower your mental load
Now those numbers might be a good tool to convince a manager. But there’s much more to it than that. As a developer, there are a lot of other benefits to writing automated tests first. And most of those rely on a simple principle: TDD helps you break down your tasks into very small steps.
With manual tests, checking your code is costly, which will deter you from testing often. But if your tests are automated and run in a few seconds, you can check the correctness of your code as often as you want, and take steps as small as you want.
This leads to a few benefits:
- Running your code and realizing you solved a certain problem is always rewarding. But having to wait a long time before you test it will delay that satisfaction. Validating smaller steps often on the other hand means you will experience that gratifying feeling more often, which is excellent to keep your motivation high. Nothing is more frustrating than the thought your goal is still ages away. Every experienced hiker knows that in case you’re facing a long journey, it is tremendously helpful to mentally split your trip into several legs and focus on one at a time.
- Focusing on smaller tasks also limits the number of concepts you’re juggling within your mind at a given time. Building a mental model of the problem to solve and designing the proper algorithm to tackle it takes some effort and concentration. If you get interrupted in the middle of your task, those efforts will be wasted. The smaller the task, the easier it will be to have a complete understanding of the job at hand and play with it in your head until it’s completed.
- By adding a new failing test each time you get started on something, you’re also adding a clear milestone to track your progress. If you need to stop coding at some point, it will be easier for you to pick up where you left off: whatever test does not pass is your next task.
- Furthermore, the list of test cases gives you a clear list of the scenarios you’ve handled, so you don’t have to rely on memories of things you tested manually the previous days. Adding tests is like keeping a written list of all the things you’ve done.
As a mobile developer, I have to admit that the temptation to run my app and check how it behaves is generally there whenever I add more code. But as we’ve seen, this is just going to slow me down. Moreover, the desire to be able to test my assumptions rapidly might influence my design and push me to keep my business logic closer to the GUI layer. On the other hand, writing automated tests might seem counter-intuitive at first, but it will definitely speed up my development and reduce the amount of concentration I need to complete each step.
Unfortunately, not every test can be automated easily. Some architectural layers are naturally easier to interact with than others. So if you want to produce value quickly, the whole goal is going to be to maximize the testability of your code. And the best way to do that is to start by writing the tests!
Click 👏 to say “thanks!” and help others find this article.
If you need a Kotlin workshop, check how we can help you: kt.academy.