English | EN 
What are the QA Challenges?
There is a constant ask from the customer on how to optimize the overall QA (Quality assurance) activities in terms of reducing cycle time, improving quality by reducing production defects, focused testing to get maximum defects in early development phases. Apart from this, most of the customers are adopting digital platforms such as PaaS (Platform as Services) & SaaS (Software as Services) solutions for faster delivery, so how can the QA Team keep pace with development and subsequent validation activities, by automating test case generation. Can we get insights into what areas to automate? Will there be any prediction on what will be the number of defects found, test cases need to be written based on the release magnitude. To get these answers, let’s explore the solutions available which we can leverage.
Technologies such as Analytics, Artificial Intelligence (AI) & Machine learning (ML) can be leveraged to bring efficiency and effectiveness for end-to-end QA activities. If we look at AI & ML in Quality assurance, broadly there are two categories
- Testing with AI
- Leverage AI, Cognitive solution for testing and quality assurance
- AI Testing
- Validate AI, it’s outcome and algorithm, training the AI model and test the outcome of AI model
In below blog, we will primarily focus on Testing with AI
What could be possible solutions?
There are two broader categories where AI solution can be leveraged in quality assurance
- AI for QA – Prescription & Prediction
- This technique collects historical data from your SDLC Ecosystem and then applies analytics, ML algorithms, and Cognitive techniques to suggest the possible options for decision making. It also used to predict the future quality for upcoming releases based on historical data and current release information as a parameter
- AI for Test case generation
- This technique focuses on automating the test case generation activities based on ML & NLP (Natural language processing) algorithm
How do these two solutions work?
- AI for QA Prescription & Prediction:
- It gathers your Release, Sprint, Requirement, Test design, Test Execution, Defect management, their linkage information, code commit log, production defect and logs, user feedback, and social media data from different sources. Pull all the raw data into a DataMart and then apply the MI algorithm and analytic on the data and then decode this complied information using the BI tool. This process looks complicated but most of the SDLC tools expose their data and CRUD (Create, Read, Update & Delete) operations via rest API’s and you can develop the connector to pull this information. Once built, it can be reused across projects and engagements within the organization.
- AI for Test case generation
- AI and machine learning technologies are being used to continually scan the user journey, for e.g. how a user interacts with an application and navigation and captures the relevant information. Techniques such as AI & NLP can be used to generate the application print. NLP & AI can then generate test cases that can be used for functional and non-functional testing. This operation is normally performed using market/ vendor developed tools. Some tools provide features to auto-heal the test cases if there is a change in the application flow or screens
Possible use cases:
With the above possible solutions, there are different use cases that can be implemented to expedite QA activities. Below are few sample use cases or usages of this technology
Tool & Technology available in the Market:
To leverage these AI-driven techniques to optimize and expedite your QA activities, there are different technology platforms and vendors tools available. Below is a snapshot of those tools and technologies.
AI for QA Prescription & Prediction:
This solution can be built by understanding client application and tool landscape so that there will be limited operation and maintenance cost. There are three major components required to build this solution
- Connector to pull SDLC & ecosystem data: This can be built using Java or .net platform leveraging tool Rest API and importing the data at regular intervals.
- ML & NLP – R & Python machine learning, text mining libraries to process the structured and non-structured data. Use linear regression, classification algorithms for prediction and prescription use cases. Leverage Cognitive services and NLP technique to build additional business logic and generate the prescriptive and predictive data
- BI Tools – the BI or analytics tools can be used to visualize the processed prescriptive and predictive results/ KPI’s
AI for Test case generation:
This solution developed by different vendors using Machine Learning techniques and models. Most of the customers use these tools to add velocity and improve the quality of their end-to-end QA activities, below are few market-leading tools which have started penetrating in this area.
- Functionize
- Functionize uses artificial intelligence to create tests that become smarter with use. Every time a test is run, Adaptive Event Analysis Engine learns your site, and creates tests that become even more stable over time. These powerful functional tests can then be run in several ways and in conjunction with a number of integrations.
- Mabl
- Once you point Mabl to your application, it will crawl your app’s screens and begin to run default tests that are common for most applications. It uses machine learning algorithms to improve test execution and defect detection. Mabl uses proprietary machine learning models to automatically identify application bugs, including visual regressions, JavaScript errors, broken links, and increased page latency
- Applitools
- It uses cognitive vision technology for visual scanning of the application. Using visual technology, it runs an eye visual test, drives the application through a series of application states, and for each state executes a checkpoint which captures an image of the application in that state. Eyes compare the sequence of captured images to the sequence of reference images stored in the baseline of that test and reports any significant differences. Once captured it automatically runs functional and Visual AI-powered tests at scale across every app, browser, OS, and screen size. Applitools Eyes can be integrated into other test automation and ALM frameworks such as Selenium, Appium, MS Coded UI, Atlassian, Rally and others
Above are a few reference tools, which represent how ML techniques can leverage for Test automation and Test case generation
Benefits of leveraging AI-assisted technology for QA
There are tangible and intangible benefits of leveraging both the solutions, based on client product criticality, risk appetite and vision to align software delivery with DevOps and AI platform, any one of the below solutions can be leveraged based on the organization requirements
The first one, AI for QA Prescription & Prediction can be built as a platform and can give below benefits:
- Improved requirement coverage and build linkages across SDLC components from requirement to production feedback
- Faster time to market – Reduces test planning effort up to 50% and Test creation effort by 30%
- Prediction for release and capacity planning
- Test governance and identifying the process gaps
- Track the business and operation KPIs
For 2nd technology, AI for Test case generation, below benefits can be leveraged
- Self-healing tests that update autonomously in real-time, and can be integrated into CI/ CD, DevOps release train
- Broken tests are identified and fixed due to application changes, no more test maintenance
- AI to reduce maintenance efforts and improve productivity
- As these vendor tools come with a license cost, the ROI can be calculated based on application usages, reduction on test creation, test execution and maintenance efforts and may vary from application to application.
Why should customers and projects invest in this solution?
Looking at these AI-assisted testing solutions, customers, and projects can reduce the cycle time, improve productivity, build transparency, and linkages across the SDLC Ecosystem. Although there will be cost for initial investment, ROI can be achieved by looking at the above tangible and intangible benefits.
Reference:
https://www.functionize.com/architect/
Https://www.functionize.com/blog/automating-your-testing-with-functionize/
https://applitools.com/blog/what-is-applitools/
