English | EN 
The syllabus of the ISTQB® foundation level contains a definition of the purpose of configuration management within the context of software testing. Three preconditions are defined by the syllabus to achieve the introduced purpose. This preconditions can be summarized into traceability in between test items, testware items and referred documentation in a given configuration. To fulfill them, configuration management has to be a determined subject within the test planning phase and proper tooling or infrastructure have to be identified and considered.
Test Automation Frameworks implicitly consider configuration management
Nevertheless, configuration management for software testing often stays unmentioned or insufficiently covered. When software testing is automated or semi-automated, configuration management is implicitly covered in standard framework architecture – but often to an unknown or unaware degree level. This is caused by the given requirement to configure the automation software and the software under test and knowledge about software configurations result in configurations for various test items or testware items.
The traceability regarding certain configuration naturally decreases in manual testing environments when configuration management is not considered in test planning. In manual testing, software under test configuration might be documented but not managed well. Nevertheless, if automated or not: configuration management has to be considered more consequently in software testing projects from the start.
The scope of configuration management in software testing
As mentioned, software under test configuration is certainly to be managed. In most testing projects, not only one software version has to be covered. This implies the requirement to manage various test environments and to keep track of corresponding test data, test scripts, test automation frameworks, reporting tools, defect tracking and more. If configurations are not managed well, a necessary update of a test script or test data for a newer software version could create an incapability to cover previous software versions in software testing – e.g. after a defect occurred in the live software version, retesting or retracing the same could become impossible or expensive in the available test environment.
Therefore it is important to consider the software deployment cycle in the test plan not only regarding software test cycles but also regarding software test configurations. Test items and testware have to be preserved as long as a certain software version is available. Usually, one ore more reference and release test environments are being provided to maintain a basic capability. Nevertheless, configurations of the same are not being guaranteed to provide the correct comprehensive configuration to maintain a required level of representation and reliability. Test planning should consider the mismatches in configurations that might apply in the different test environments and describe actions to compensate them, if they are inevitable in terms of adjusting configurations itself.
One of the three preconditions for configuration management is commonly left behind
When mentioning configuration management, typically there is thought of settings, run parameters or software versions. In experience, test data could sometimes be left out in consideration. Most common is the insufficient coverage of the third precondition, mentioned above as traceability of test items and testware items towards referred documentation. Especially in agile environments, where one of the statements of the manifest is “Working software over comprehensive documentation”, fulfilling the corresponding precondition might sound difficult to achieve.
Therefore, there is an important requirement to focus on the same. An easy way of referring documentation (which does not have to be a comprehensive specification document) are for example tags, comments and links. It is important to not maintain too many configurations at the same time, so a test configuration plan will be needed which clusters test items and testware items and documentation.
Consequently, configuration management is a basic requirement for maintenance of testware. Maintenance is not only a quality attribute for a software under test but also a quality attribute of software testing itself. Configuration management is, therefore, a fundamental requirement to achieve qualitative software testing.
[References]
ISTQB® Foundation Level Syllabus, Version 2018, invoked on 02/09/2019
ISTQB® Advanced Test Automation Engineer Syllabus, Version 2016, invoked on 30/03/2019 on https://www.istqb.org/downloads/category/48-advanced-level-test-automation-engineer-documents.html
Manifesto for Agile Software Development, invoked on 30/03/2019 on https://agilemanifesto.org/iso/en/manifesto.html
