English | EN Is it a good idea solving test environment problems “later”?
Apr 18, 2014
In recent years, many companies focus on improving their test process. The aim is that the improvements will lead to measurable improvements in quality, faster time-to-market, cost reduction and more efficient IT operational processes. Still, they think it is best to first establish a good test process and then consider if they need to improve their test environment.
World Quality Report 2013-2014 found that many companies have challenges with the test environment and test data. It is a new area of concern.
The problems are often put forward as “statements” from the test managers and testers. They struggle with downtime and interference with each other during the tests. Too much is often going on in the same test environments simultaneously since projects often have the same delivery deadlines. At the same time, they do not get much sympathy from management. The project teams are seldom responsible for delivering the test environments – it is delivered by an infrastructure partner. Reports “document” that the test environment is ‘up’ and available according to SLA requirements or other agreements with infrastructure partner. However, I have been test manager in several projects where the test environment was not available half the of the test period.
Examples of problems:
- “Someone” has added new code or changes in the test environment leading to serious errors.
- “Someone” has given the test team the wrong version of the code because of lack of configuration management.
- “Someone” is running a job that leads to poor performance.
- The data is too old and outdated.
- There are not enough devices available for testing apps. This leads to manual tests with the most common devices and poor test coverage.
Assessments we have done for clients show us that these challenges are seldom documented by the test team, and the management does not take decisions based on emotions. They need facts. Problems during the test periods must be documented so that the management can find the right balance between risk, design, management and support, and identifying the optimal level of investment in test environments. 40% of the total test costs are related to the testing hardware and infrastructure. Professional Test Environment Management (TEM) services enable the organizations to speed up their software release schedules by up to 25%, cut infrastructure costs by 5-10% and increase item team productivity by up to 30%.
In other words, it will be profitable to improve the test environments. Test environments are complex and the challenges will increase with the need of apps testing on devices in addition to traditional tests. It will require professional test environment management and test data management, with SLA agreements that secure stable and available test environments.
Test Environment Management services provides a holistic approach to the area of technical testing services, from Service Management, Data, Build, Architecture, Infrastructure, Applications and Virtualization.
- Test Environment Management (TEM) Services.
- Test Data Management (TDM) Services.
- Development Operations (DevOps) Services.
- Infrastructure, Virtualization & Cloud (IS) Services.
- Wide-angle Application Rationalization Program (WARP) Services.
- Service Virtualization (SV) Services.
To secure expected effect of the improvement of the test process it is necessary to also improve test environment and test data management. It must be seen as key areas in the improvement program. A good advice to all testers struggling: document and report the challenges you have with the test environment.
References:
- World Quality Report 2013-14
About the author
