Avoid The Inverted Test Pyramid

Yesterday we explored the Test Pyramid. We saw how this concept should shape our testing strategy.

Unfortunately, the Test Pyramid is not the norm. In reality, the Inverted Test Pyramid is more common:

In organisations applying the Inverted Test Pyramid, Manual Testing dominates the testing strategy. Sometimes there may be a suite of automated system and integration tests, and if lucky, a handful of unit tests. 

Developers are often frustrated with the automated system and integration tests since they frequently fail. Why are these tests so fragile in the Inverted Test Pyramid? Because without a suite of unit tests verifying local application correctness, integration and system tests must do double duty. Tests intended for asserting high-level integration and system operation are responsible for verifying all code they traverse! No wonder they break frequently. It doesn’t take long for repeatedly failing tests to be ignored or commented out. And as soon as that happens, the reliance on Manual Testing increases further.

By why are broken integration and system tests hard to repair? Because they can cover hundreds of lines of code, they are difficult to debug. Did this test fail because we have a bug in our business logic? Or there is a problem with the database? Or is the issue in our web layer?

Heavy reliance on Manual Testing makes sense in the short term. Testers are only needed when code is modified. It’s a reactive approach to testing—we test the current, momentary changes. Therefore, every time a few code lines are changed, the application should be fully re-tested. That is cost prohibitive—a tester can only test so much. So uncertainty and bugs start slipping into the code to be discovered by unhappy end-users.

On the other hand, with the Test Pyramid, unit testing is a forward-looking strategy. Here we write unit tests to prevent future code changes from breaking our application’s behaviour. For a software system, the future is its useful lifetime. Yes, it initially costs more time to create unit tests. However, once they are in place, they will keep verifying the program’s correct operation forever at almost zero cost.

Please avoid the Inverted Test Pyramid and switch gradually over to the Test Pyramid. If you are working for an organisation using the Inverted Test Pyramid, please forward this article to your manager!