The concept of testing pyramid is really all about balancing between different types of tests and different levels of testing. The ultimate goal here is to make sure the tests are executed quickly, locate the failure as soon as possible, debug, and move on.
If we manage to achieve this, we are going to gain more confidence in the software that we are building, and achieve overall stability which will then help us create the opportunities to have frequent production releases. Every team wants to have frequent deployments to production with small chunks of changes. And if we combine all levels of testing with things such as CI, which I’m going to talk about soon, we will be able to have production releases every day.
How to successfully build a testing pyramid
Led by the experience, I strongly believe that people who are working on setting up the CI, the pipelines, and the people who are in charge of setting up the testing pyramid should work closely together. And, it’s not enough to have all levels of testing. You need to have an infrastructure to help you know when you’re going to execute these tests. CI/CD tools play a major role here. They provide us with the infrastructure so that we would know which test is going to be executed at any given moment and what value we are going to get from it. The ultimate goal here is to raise the quality of software and one of the most important things is something we like to call code coverage.
We try to get a certain percentage of coverage with a lot of tests, whether these are unit tests or some other levels of testing. But, when it comes to testing pyramids, I think a smart code coverage is more important than a certain number. Meaning that it’s quite important for the team to know that we are not testing the same thing twice with unit integration or even three times. We really want to split the responsibilities between developers and QA engineers and always have optimal level of testing and optimal number of tests. Finally, I think all of these things couldn’t be done without the infrastructure for testing, and those are test environments.
From my experience, when we talk about traditional regular web applications, having two test environments is really important to get the biggest value from the testing pyramid. This way you enable your team members to optimize the scope of testing to have the customizable scope and to locate specific types of tests to the specific environment. This allows you to slowly grow to the production environment and gain confidence along the way to move on.
In the image below, we can see a simplified testing pyramid. I think every team should make it simple. And we can see the broader scope is at the top and the narrow one is at the bottom. With narrow scope we are actually talking about the unit test. With unit tests we are testing classes, we are testing modules, even objects independently. And then, we can move up to the pyramid and follow its shape. It’s called a pyramid for a reason.
We can see that the biggest amount of tests are at the bottom. This is because unit tests are run quickly and they’re easy to maintain. It’s cheapest to write the unit tests and compare them to the higher level of testing, such as acceptance tests, where we have even UI tests. These are not easy to maintain, they’re slowly executed, and they are more prone to failure than unit tests.
Testing Pyramid
Broader scope at the top and narrow at the bottom
Follow the shape of the pyramid
QA responsibilities
API testing:
– bypass the presentation layer
– less expensive to write and maintain
– generate data for UI test
UI testing:
– cover just the best-case scenarios
– use for critical paths
– time factor
Enterprise solutions and startup culture
What are the QA engineer’s responsibilities in terms of the testing pyramid?
As I said, there are integration tests which are testing components and interaction between the components. But, usually, developers are responsible for both unit and integration tests.
Of course, QA engineers can be responsible for integration tests as well, depending on the software that is being built and the business model. This time my focus will be on acceptance tests, that is, API testing and UI. API testing is my favorite.
Why acceptance tests?
API tests are executed really quickly. They are less expensive to write and maintain when compared to the UI tests. You can use them to check just the status code. You can dive into it more deeply and you can go and inspect, for example, json responses and validate and assert some of the attributes you have. It’s up to you.
Another benefit of API tests is that you can generate data for your UI tests through the API testing. So you can use your API, rest API requests to prepare, and then clean up data. When it comes to UI, my advice is to go with the best case scenario which is to use it for critical end-to-end paths. This is a kind of a smoke test. We usually tend to perform smoke tests after deployment to production and it doesn’t have to be longer than 15, 20 minutes. So my advice would be to keep it small, to have just a few UI tests. Time factor is important. And, executing this test can be time-consuming and maintenance of failures can be quite exhausting for the whole — it can stop the whole process and take too much time for no reason whatsoever.
Types of environments for testing pyramids: enterprise solutions and startup culture
Another thing I’d like to point out is the importance of two types of environments where you should set up a testing pyramid. We have teams that are developing enterprise solutions, and we also have teams that are working in a startup culture. Teams that are working on enterprise solutions have a better opportunity to build a successful testing pyramid because they usually have more time and they’re able to dedicate their time to the quality first, and not delivery which is the case in startup culture.
So building a testing pyramid in an enterprise environment can be more complex. On the other hand, the startup cultures are usually focused more on delivery, meaning you should be smart when it comes to implementing a testing pyramid in this kind of environment. Try to focus only on unit and acceptance testing, and try to make the best of it, to get the best value. Finally, exploratory testing, which, in its core, is manual testing is at the top of the pyramid.
We cannot automate everything and usually the QA team is responsible for this. But I would definitely encourage the whole team to do exploratory testing.
Teamwork makes the dream work
Also, keep in mind that it is crucial that you should implement the concept of testing pyramids in your team and help them understand it and use it in future. This should not only be the QA responsibility. Split the responsibilities, and motivate your team to contribute to a healthy testing pyramid. Everyone should be engaged.
