Just enough testing
Adopting a test strategy of “test everything all the time” simply does not scale. As your application’s codebase grows and becomes increasingly fragmented across microservices and infrastructure stacks, this all-encompassing test strategy will require you to continually add more tests, spreading those tests across more service boundaries. No matter how much you optimize the performance and parallelization of these tests, they will inevitably take longer and longer to run and become more complex to orchestrate.
Test coverage can be limited to the minimum level by adapting it according to production stability and risk of bugs: reduce tests for highly stable user experiences and temporarily increase coverage for unstable ones until they stabilize. Remove tests that rarely or never fail, as they are probably not testing anything that is likely to break in production. Also remove flaky tests that often fail initially and pass when retried. Flaky tests will promote a culture of easily ignoring test results and quickly erode confidence in your test strategy, similar to an alert that is always ignored.
While removing tests that provide little or no value is a sensible strategy for achieving an efficient balance between delivery and stability, this approach should always be used with caution. You must fully understand the purpose of a test and be sure of its lack of value before removing it. You can also consider running the test less frequently before removing it altogether.
Mutation testing is a technique that involves deliberately introduc‐ ing bugs and seeing which tests catch them. As your application matures and grows, you can use mutation testing as a method for identifying useless tests.
Your test suites can also be limited by the types of tests that you write and execute. A sound strategy to apply to serverless applications is to prefer static testing as far as possible. Static testing does not require deployment of microservices to the cloud and can be implemented via unit tests and static analysis. These static tests can be further limited by primarily focusing them on your critical paths.