March 30, 2020

The world is turned upside down as the COVID-19 ravages through the modern world. Who would have thought that on the cusps of the fourth industrial revolution, the very technologies that made our world flatter and connected allowed an epidemic to become a global crisis? While the nations are making community efforts to stop the pandemic, there are a lot of analogies that we can draw from the COVID-19 responses and software testing. These are the lessons that I learned as a software developer and tester and I wish people would have heeded.

1. Test early, test often

One of the mantra in software testing is to test early and test often in the software development life cycle. As one moves from requirements gathering, writing a spec, coding, testing and releasing, the cost of fixing a bug increases exponentially. When a bug is released to the users, debugging, root-causing, and fixing it becomes dramatically difficult and expensive. More importantly, the trust of the user is lost. Trust is something that is hard to get it back once it is lost.

If you look at how different countries are faring about the pandemic, the countries who acted and tested early are relatively containing the virus. Taiwan, for instance, began monitoring passengers arriving from Wuhan as early as December 31st, 2019, way before the world started to notice the new virus from China. As of March 27, 2020, Taiwan has 252 confirmed cases and 2 deaths.

South Korea had an early outbreak in February, but soon implemented an extensive testing system in place and was able to stem the outbreak to slightly over 9,000 as of now. This corresponds to the number of tests performed by South Korea. So far, the country has performed 316,664 tests, far outnumbering those of any countries in the world.

2. Containment

Quarantines, shelter in place, shutdowns, lockdowns, and banned or restricted entries are all measures that affected countries are using to contain the virus. In software development, this corresponds to testing small contained batches, starting from unit tests, moving to integrate other component and then finally doing end-to-end tests before the release. By starting from unit tests, you can find issues early and easier to fix them in place. If you wait till end-to-end tests to find issues, you will have a hard time determining where the bugs are.

To write good unit tests, you need to contain the environment in such a way that you have a total control. To test functions that rely on other components and dependencies (like the database or internet connection), there many mocking frameworks that you can use. Mockito, for instance, is a popular mocking framework for Java. Other languages have similar mocking solutions as well. By using mocks instead of real dependencies, the tests run fast and reliably, allowing the developers to track down any issues in a more agile manner.

As you move later in the development cycle, as more components are integrated, testing becomes fragile, slow and unreliable. For this reason, the best testing strategy is to create far more unit test than any other types of testing. This results in a testing pyramid:

A non-pattern is an upside-down pyramid, like an ice cream cone, where there are more end-to-end tests and very little or no unit tests.

Another non-pattern is an hourglass where you have significant number of end-to-end and unit tests, but no integration tests in the middle.

3. Traceability

One of things that South Korea did to flatten the curve is to make the movements of all infected patients publicly available. This allows the disease control center to trace and test those who might be infected. For instance, when a new case is found, all mobile users in the affected area get automatic notification. This is probably hard to replicate in other countries because it raises privacy concerns.

In software world, traceability is crucial for software development and services. By analyzing user’s clickstream data, for instance, the company can make a decision on how to develop or market their products. Having a good logging mechanism in place also makes it possible to monitor and setup automatic alerts and investigate hard to replicate bugs later.

4. Mechanism and Policy

While the above points are best practices and common knowledge among developers and testers, not a lot of companies (big or small) follow the guidelines. The reason is very simple. Time and pressure. Developers are often pressured to deliver on a short time schedule. Often, they find it difficult even to finish coding, so they sacrifice what they deem as “good-to-have” features. Unfortunately, testing is considered something that you can cheat. Such short-sighted visions result in wee hours of massive bug hunting later.

The sad truth is that no developer wants to write buggy code. No executive or manager intend to produce bad quality products. The problem is not intention. Mechanism and policy are the crux of the matter. Unless the company or the department sets a certain policy regarding software development process and testing, manager and developers will follow their instinct to make things work from their perspective. This is best expressed by Jeff Bezos’ famous quote,

Good intentions never work, you need good mechanisms to make anything happen. – Jeff Bezos

The state of Washington governor Jay Inslee was slow to order a state-wide shut in place order, not following the leads of other states like New York and California. Instead, he resorted to pleading the citizens to limit unnecessary outings. As we know, his good intention did not work and he was forced to announce state-wide shutdowns later.

Good code check in policies should include getting code reviewed, having unit tests written and having good code coverage. Of course, such policies work best when they are automated and good monitoring and enforcing mechanisms are in place. For this reason, it is important to have CI/CD (Continuous Integration/Continuous Deployment) in place early in the development life cycle.

Conclusion

We cannot yet fathom how the COVID-19 will change our lives. The virus will keep on mutating and software technologies will keep on evolving. No matter how good the mechanisms are, it is up to the individual team or the developer to practice them on daily basis. Any test code written will be outdated as the code under test changes. One of the common objections I hear people say about writing test code is that they require constant maintenance. Of course, they do. It is like washing your face or brushing your teeth every morning. It would be preposterous not to do it just because you must keep on doing it every day. Ultimately, hygiene and good common sense will help us enjoy healthy lives. Likewise, good software testing practices allow us better quality products and more time for the developers. Happy coding and testing!

Written by CHANGSIN LEE at changsin@testworks.co.kr

Testworks Inc., Seoul, Korea, CTO, Research & Development
Amazon Corporation (2014-2019), Seattle, WA, Sr. Software Development Engineer
Microsoft Corporation (1999-2014), Redmond, WA, Software Engineer