More important than testing is knowing what to test and how often to test. Here's how you can optimize the time and money you spend on automated software tests during development.
Overview
Automated testing is the heart of any software development. Particularly in today's development environment where microservices architecture is widespread and a DevOps culture is growing, automated testing becomes a must in any project because it provides quick feedback if something goes wrong.
But remember: more important than testing is knowing exactly what to test. If you know what to test, you will put your time and effort where they need to be.
The Test Pyramid
Image courtesy of Medium
The test pyramid was first mentioned in Mike Cohn's book Succeeding with Agile. The test pyramid is a way to illustrate that you should focus more on tests that are at the base of the pyramid (unit tests) and less on tests that are at the top of the pyramid (end-to-end tests). To explain why the test pyramid is structured this way, let's review the purpose of each testing layer.
Unit Tests
Unit tests are the base of the pyramid. Their purpose is to test the expected behavior of a unit of code in isolated conditions. For example, suppose you want to test a method that retrieves data from your database, does some processing, and returns information to the user. In this example, though the retrieval of data happens in the same method, it's done by another layer. In this case, you can just assume that the data would be returned correctly (by mocking it) and test what really matters: the processing of data and the returned information to the user. You isolate the method being tested from other necessary calls (database, another REST endpoint, etc).
Since this is the kind of test you'll be writing more, your tests need to be fast and provide quick feedback to the developers making changes to the code.
Example frameworks: JUnit and Mockito.
Integration Tests
As the name suggests, an integration test validates system integration. This can be a database call, an external API call, etc. If you only write unit tests, you can't be sure that the query you are running in your database is actually working and retrieving the correct information.
To test the integration, you don't actually have to call a production database or an external API that's deployed to the production environment. Instead, you can set up an environment that simulates those environments and call them in your tests.
Example frameworks: MockServer and Testcontainer.
End-to-End Tests
Think of E2E tests as a user who is actually interacting with your system. For example, if you have an e-commerce website, you want to make sure that the user can log in, search for products, add them to the cart, and check out. As you can imagine, these tests are very expensive to write, they run slowly, and they are difficult to maintain. These tests also need to interact with external systems (for example, a payment system).
Example framework: Selenium.
Pyramid Overview
The higher the level of the test (end-to-end tests), the fewer tests you should run. The lower the level of the test (unit tests), the more tests you should run.
As Martin Fowler states:
"Tests that run end-to-end through UI are: brittle, expensive to write, and time consuming to run. So the pyramid argues that you should do much more automated testing through unit tests than you should through traditional GUI based testing."
Does this mean that you should eliminate all high-level tests? The answer is: it depends. For example, if you are building a mission critical system where errors can cost a lot of money or other damage, it doesn't make much sense to just have unit tests. You need to make sure the system is working end-to-end with all of the integrations, validations, etc.
If you are building a system that is not critical, perhaps you would just spend the effort of high level integration tests on a subset that is very important for your business. For example, the ones where user access is frequent. You want to make sure that this part is not affected by new changes that are deployed.
Legacy Systems Built Without Tests
Sometimes you'll need to maintain a legacy system in which very little to no testing was done. Which path should you follow if you want to introduce tests? Would you spend more time doing unit tests or E2E tests?
In her article titled "The Test Pyramid," software developer Jessie Leung argues that if you face a situation like this, you may want to start with E2E tests first and then gradually move to unit tests. The reason is that normally a system that wasn't designed with testing in mind would have to be refactored to support unit tests, adding potential risks for breaks. Once you have a couple of E2E tests that give developers confidence to refactor the system, then you can slowly start introducing unit tests.
I hope you enjoyed this article. Do you agree with the test pyramid basics? Can you share the experiences you have had? Feel free to leave a comment.
