My very first job out of college was the management of a system test environment for a client server application my employers, Andersen Consulting, were building for the local Baby Bell back in 1995.  In that role, I learned very quickly that how you test, and, even more importantly, when you test, can make or break any software-related process or product.

And yet, in the 20+ years since that baptism by fire in the world of large scale systems integrators, I continue to be amazed by how many people leading and shepherding enterprise-scale applications live in denial of two fundamental truths about software QA.


Truth #1: There are only two kinds of testing.  The cheap kind and the expensive kind.

The cheap kind of testing is where your organization hires a QA team, builds a production-replica environment, creates a test plan based on your business and functional requirements, constructs production-scale test data, and executes in a way that exercises every possible code path and permutation.

Now, to be clear, all of the things I’ve described above do cost a significant amount of money, and you can choose to skimp on that expense as much as you like.

But, remember that there is another kind of testing.  The expensive kind.

The expensive kind of testing happens when your end users have become your testers.  Not because they’ve volunteered for this honor, but because you didn’t make the investment to ensure that your product has been cheap tested before it reaches them.

Expensive testing uses real users and their real data.  If you’re a bank, that means you’re using someone’s real bank account and their real money as your test data.  If you’re a healthcare provider, you’re using their actual PHI as your test data. When errors occur, they happen in the real world.

And, rest assured, these errors will occur because of the other fundamental truth about software QA.


Truth #2: The testing ALWAYS gets done.

One way or another, someone will always find your critical bugs: either your testers, or your users.  It’s inevitable.

If you find a bug the cheap way, you can make a hot fix, or delay a release, or any number of things to remedy it before it goes live.

If you find a bug the expensive way, you could find your company on the 6 o’clock news.

Yes, QA is always an afterthought and always seems like just a rubber stamp that can easily be stricken from the bottom line to boost your margins and cut overhead.

But making the investment to do true, honest testing can mean the difference between fame and infamy for your firm.