Unit Tests <> QA
March 15th, 2007[ Software Development ]
Based on this post, I’m guessing that secretGeek isn’t a hardcore fan of test driven development(TDD). I’m actually unsure whether I’m a fan of pure TDD as I’ve never adhered to it in the “pure” sense of always writing a test before you write code.
What I can say is that I’m a huge fan of test driven bug fixing(TDBF), and yes I did just make that up, catch phrases really are that simple. What’s TDBF? When you find a bug, in development or QA or whenever, you start by writing a unit test to reproduce it.
Here’s why I’m a fan of this. What’s the generic process for fixing a bug?
- Reproduce: Hack away in a gui, web app or debugger until you can continually reproduce the bug in a controlled environment.
- Research: Once you can reproduce it then it’s recon time to understand the bug.
- Solve: Design an appropriate solution and implement it.
- Validate: This consists of doing exactly what you started with, trying to reproduce the bug by hacking away in a gui, web app or debugger, etc.
The first and the last step are almost identical. You will likely add to the validate stage, possibly testing in varied environments etc. No matter how you do the first step, you will have to reproduce that step again at the end. So why not start by capturing it in code? Do that and you’ve not only accelerated the last step but you now have a repeatable means of ensuring this bug doesn’t exist tomorrow or a year from now.
My take on unit tests is that you are capturing debugger statements in a usable form NOT performing any form of quality assurance. Unit tests should never ever ever ever ever be considered a replacement, in any form, for QA.
Think of this in terms of development cost. Everytime you pay a developer to fix a bug without using unit tests, you are paying that developer to write a lot of code that escapes into the void. Sure you may not think of the act of reproducing a bug as writing code but it’s a small shift for it to be just that. Without that, this ‘code’ is never checked in anywhere, it’s likely not even saved in any fashion. In a sense, you are throwing away valuable code.
So I agree with secretGeek on that point, however, the flaw is in allowing unit tests to be viewed as replacing QA. It’s not a flaw in TDD itself.
Will you have to refactor your unit tests? Definitely and if you’re going to have unit tests then I suggest you stop thinking of them as some appendage. If you’re building a software product and you’re using unit tests then you must think of them as being part of your product.