As an industry, it took us a while to move from Quality Assurance to Quality engineering. However, the further enhancements in Quality engineering are happening at a very fast pace.
Even today we know, Testing is ~30% of overall project cost. Despite bringing in the automation aspects and the new age tools into play, we still struggle to deliver quality product at an optimized cost.
Here the word “OPTIMIZE” is of importance. As a practitioner, I believe there are multiple usage of AI /ML form bringing OPTIMIZATION aspects in testing.
Let me take a few examples:
Not even getting to automation and complexities of NLP (Natural language processors etc), the most simplistic area to optimize is the existing set of test repositories. We often see in second or third generation outsourcing deals, clients carrying a huge test suite; manual most of the times.
We can bring the use case of similarity analytics to reduce the redundancy in the test suite and optimize it to have more unique test cases that test a specific path or functionality. When used in the right way, it can trim down the test suite size and thereby the efforts required to maintain and execute. In my experience when used with right algorithms, it can easily optimize the suites ~10-15%.
The similar concept then can be extended to Defects de-duplication and reduce the time and effort spent on overall defect management (triaging, RCA, resolution and retesting). Extending this example – it can also be leveraged to find de-duplication on automaton suites.
For many years we have been speaking and implementing shift left techniques to build the quality of the systems and in my opinion its high time we start focusing on shift right. A very simple example would be to perform “usage based testing” or “targeted testing”. We often either undertest or over test the systems which results into defect leakage and high cost.
This is another high potential area to get the AI/ML aspects. How can we analyse and learn the production patterns and compare it with the tests that are being performed in the test environment? This kind of analysis will give a clear picture on the e2e scenarios that are missed in the testing or the % coverage of certain areas. Usage based testing is not only effective but also high on user experience and especially critical when you must do best in limited time.
However, as a technology group, we always focus on functional automation to speed up the things. With advancements in tooling, we have achieved higher level or test automation but in reality we still find ~30-40% of test automation needs babysitting. Auto healing, self-adapting automation is the need and definitely there are huge list of tools that helps with AI in automation – Autonom IQ, Functionize, Mabl, Parasoft SOAtest etc. But most of them still focusses on the automation of functional test cases.
While Continuous testing is important, a clear focus on other areas to implement and leverage AI/ML can improve the overall benefits.
Simple usage of AI/ML to find the testing focus areas based on high defect prone areas, or prioritizing high defect yielding test cases upfront in the test cycles could add more value from an end to end solution than focusing on functional test automation alone.