Does anybody have a really great QA testing program for their software? I’d love to learn about it, because I think I have yet to see a top-notch one in 14 years in the industry.
In this period of severe contraction in the software industry, which started in early 2000 when all the Y2K QA testers saw their jobs evaporate, it’s not uncommon to see QA positions hit harder than development. It’s pretty hard to justify eliminating a programmer instead of a QA tester.
Yet testing is not something anyone responsible for the product can afford to slight.
When it comes to QA testing resources, companies are faced with three different profiles of employees that gravitate to different functions at the company. Each profile has its advantages and disadvantages when it comes to performing testing.
Read on for a discussion of these profiles and which best meets your QA testing goals.
[private]
Three Profiles of QA Testers
Here’s some more detail about the three profiles of testers.
The first profile is that of your software developers. When it comes to accepting feedback on their software, programmers respect programmers. If all your QA testers were programmers, you’d find developers amazingly receptive to their input. But people who can program usually don’t want to be QA testers.
In addition, what programmers like to test is often not what needs to be tested from an end user point of view.
Despite all this, TQM and Six Sigma programs often call for testing by engineers as soon as new code is checked in. One reason for that is that if quality control is seen as a separate function, developers don’t consider quality their responsibility at all.
The second profile is QA specialists. While these individuals generally do not have the depth of technical knowledge of a programmer, QA specialists understand quality and process issues, and know where to dig for problems. Yet because they often focus on increasing their technical knowledge of the product — to gain credibility with developers — they don’t always manage to fully understand how customers use the software.
The third profile is consultants, trainers, and other non-technical customer-facing employees. These folks know exactly what customers want, and they cite some dramatic examples of customer needs and how the software is ineffective in meeting them. But they usually don’t understand testing principles. They probably won’t be methodical enough, especially since they have other tasks connected to external customers which take priority.
So Which Profile Do You Go With?
The answer is like so many others: you’re better off building a QA testing effort out of a combination of all three types of testers. Each profile is best used with different components of the overall testing effort needed.
Start Upstream: Input During Design
The best way to reduce your testing requirements is to get customer input early on, while you’re designing the software to be developed. However, it’s unlikely that you’ll get much good input from customers directly — it’s as hard as getting beta customers to test the software. So this is the time to bring in your trainers and consultants to be the voice of end users.
Trainers, consultants, and others of this profile will do well with verbal interactions where they can provide immediate, non-technical feedback. Have someone on the development team take notes on their input, or nothing will be written down.
Unit Testing: the Quick Feedback Loop
The concept of testing code and immediately communicating results is promoted by quality programs because it nips many problems in the bud. It constitutes an early warning system to alert programmers to patterns in the bugs they have created. The resulting insight leads to fewer mistakes going forward.
Unit testing can be done as soon as a unit of code is made available. Here’s where the most technical profile of testers — programmers — can be very useful in devising solid tests of limited scope that provide quick feedback to the coder.
Unit testing, because of the well defined scope of each test, is particularly appropriate for automated testing. And programmers are usually motivated by the challenge of developing automated tests.
It is important to have a section of code tested by programmers who did not do any coding on it.
System Testing: an Overall QA Mentality
System testing, with its many interdependencies and chains of functional units tested previously as standalone code, is best left to the QA specialists. This is the profile that understands how to cover the most territory in the limited time available.
This area of testing is often what constitutes QA testing in the eyes of most companies. Yet the scope of testing needs to extend well beyond this.
Alpha Testing
With alpha testing, you can again bring in the customer-facing, non-technical profile to work with the whole product the way they see customers use it. No matter how thorough the system testing was, these non-technical users will find things that don’t work.
It’s a challenge to automate the testing efforts of this third profile, but if you can manage to do this, you can accumulate a significant amount of institutional knowledge on how the product is used, to the great benefit of your testing efforts. But automation won’t happen at this point without footwork by the QA specialists. The QA specialists will probably have to take care of automating a test that has been demonstrated to them by a non-technical user.
If nothing else, business case tests can be recorded as written documents for repeated use by those less familiar with end users.
Beta Testing: the Final Pass
Finally, beta testing by customers — usually with significant assistance from implementors and trainers — rounds out the testing required to get your product ready for commercial launch.
QA Testing: a Meta-Job
When you look at how different aspects of QA testing calls on different employee profiles and job functions, as well as beta customers, it becomes obvious that that QA Testing function at your company is a meta-function. It goes beyond the scope of any single job tile or department. It spans departments that will never be headed by the same member of the management team.
So it becomes critical that a quality specialist, with a profile somewhere between business and technical users, be designated to help the QA Testing meta-job coalesce across job functions and departments.
Only then will the testing effort be complete.
— Jacques Murphy, Product Management Challenges
ProductManagementChallenges.com
[/private]