English | EN 
Considering the rapid progress in Artificial Intelligence (AI), the significance of traceability in both testing and requirement engineering cannot be overstated. As AI technologies become increasingly pervasive across various industries, from healthcare and finance to autonomous vehicles and smart cities, the need for reliable, transparent, and accountable AI systems has never been greater. Native traceability should be established as a foundational goal. To achieve this, establishing clear and documented links between requirements, test cases, and AI results must be applied. This link should be able to provide a chain of causality for any stakeholder working with those AI generated results.
In this article, I will delve into the pivotal role that traceability plays within AI and offer insights into the various aspects of traceability that warrant thoughtful consideration. The widespread application of generative AI highlights the necessity for results that can be traced back. This is particularly important in maintaining the quality, reliability, and credibility of the outcomes.
The unique challenge for traceability posed by AI systems.
Traceability as the capacity to track and monitor any artifacts throughout the software development life-cycle (SDLC). This involves establishing and maintaining relationships between requirements, test design, test results, ticket systems, code, design documents etc.
Generative AI systems are complex and multifaceted, utilizing large datasets and a vast number of interconnected components. The results produced can pose difficulties for human interpretation. Seemingly, those results can show unexpected behaviors and unpredictable outcomes if not enough effort and AI expertise are applied to interpret them. However, it’s challenging and demanding to test and validate systems with the help of AI, particularly when the process lacks traceability.
The Multifaceted importance of traceability in Test Engineering
Accountability and transparency:
Stakeholders, including developers, test engineers, requirement engineers and end users, need to have a clear understanding how the AI system is making decisions. Furthermore, they must understand why certain outcomes are produced and how potential risks can be mitigated. Traceability provides a transparent view in all involved processes, especially within the testing process. In addition, a clear audit of the current state of development or the current software quality can only be achieved with well-established traceability. It also offers visibility and insights on responsibilities and accountability.
Risk management and mitigation:
In safety-critical but also business-critical applications, failures or unanticipated behaviors are a significant risk. Traceability is a necessary instrument to help in identifying and mitigating the same by ensuring that requirements are adequately tested, test coverage is met, and defects or anomalies are identified, documented, and addressed. This way, potential gaps, inconsistencies, and areas of concern can be identified. AI usage should address this by creating results that immediately reflect a traceable connection from requirements to test results. The results should include a sequence of cause and effect, as well as arguments that can be analyzed.
Compliance, standards, and regulations:
In almost every field of business, compliance with certain standards and regulations is mandatory. It’s crucial to have a verification process in place to confirm compliance adherence. A clear audit trail must be established, so all activities, decisions and changes through the development and test process can be traced. For AI systems, it should be evident that all requirements related to industry and organizational standards, as well as corresponding regulations, have been properly implemented. That includes policies and strategy definitions – e.g., an AI system would have to follow a test policy and organization-wide coding standards. Traceability must ensure that all requirements are correctly implemented, validated, and verified. AI outcomes must reflect these.
Final thoughts
To conclude, traceability is an integral and indispensable aspect in software engineering. This includes software and test engineering that incorporate AI-generated results. Moreover, it provides a basis for a shared understanding of software and documentation artifacts. It is fundamental for facilitating risk management, compliance, validation, optimization, scalability, change management, maintenance, and other aspects of the entire software life cycle.
The further development of AI technologies will continue to evolve the capabilities to achieve a goal of AI native traceability. It must be a priority to ensure that trust and efficient usage of AI is further established. Nevertheless, organizations should focus on developing and implementing robust, reliable, and scalable traceability frameworks for AI-generated results. In this manner, organizations can fully leverage the potential of AI technologies within software and testing engineering.
