Test levels? Test types? Test Varieties!


23_q74tnWhen organizing testing, the test manager adhering to the traditional view on testing had to structure the testing activities in a hierarchical way, based on quality characteristics. But a test manager often distinguished various stages too. Defined terms such as Test Level, Test Type, Test Phase and Test Stage were often used.

In today’s view on testing, the people involved in testing are hesitant to use the word ‘Test Level’ since it seems to imply that various groups, based on various hierarchical responsibilities, will perform various testing tasks without any interaction between these test levels. Moreover, many testers have often struggled to distinguish between Test Levels and Test Types. And a Test Stage – is that identical to a Test Level or not? What should be our focus when organizing testing?

All testing activities must collectively cover all important areas and aspects of the system under test – that is the main objective. To cope with the confusion around how to distinguish testing tasks, we introduce the term Test Variety.

The term Test Variety aims at making all stakeholders aware that there will always be different needs for testing, and therefore different test varieties will have to be organized. Whether these are organized separately or combined depends on the situation. There may be many reasons for having different test varieties. For example, there are different stakeholders who ought to be involved; programmers have a different focus in their testing than business representatives do. This is often related to responsibility and accountability for testing activities. The quality characteristics that have to be addressed form another reason for distinguishing test varieties. Maintainability for example, demands totally different testing activities than usability does.

Traditionally, different aspects were separately approached as a group of testing activities that had been brought together in a test level. Many people know the ‘functional acceptance test’,

a name that already indicates that testing was not complete because it obviously didn’t focus on non-functional aspects. In the new view, functional and non-functional testing can be seen as test varieties. Depending on the circumstances, such as the application lifecycle model that is used, these test varieties are organized either together or separately. The main concern is that all relevant test varieties are carried out one way or another.

So let’s use the term Test Variety, to make everybody involved aware of the fact that there are different points of view towards testing activities, and we can make sure that the interests of all stakeholders will be covered by addressing these in a well-considered way.

Rik Marselis


Rik Marselis is principal quality consultant at Sogeti in the Netherlands. He has assisted many organizations in improving their IT-processes, in establishing their quality & testing approach, setting up their quality & test organization, and he acted as quality coach, qa-consultant, test manager and quality supervisor. Rik uses his more than 40 years of experience in systems development and quality and testing to bring fit for purpose solutions to our clients. He focuses at three major tasks: * Consultancy on Quality engineering & Testing in the broadest sense (quality & test policy, project startup, process improvement, coaching, second-opinions, etc…) * Develop and give training courses for both novice and experienced testers (Rik is an accredited trainer for TMAP, TPI and ISTQB certification training courses) * Research and development of the quality engineering & testing profession. Rik has contributed to over 20 books on quality and testing, of which 5 as an main author and 5 as project leader. His most recent book in the TMAP body of knowledge is “Quality for DevOps teams”. Rik is a much-appreciated keynote-speaker and workshop-host at conferences (he has presented at conferences in over 15 countries).

More on Rik Marselis.

Related Posts

Your email address will not be published.