Testing of Artificial Intelligence


Artificial Intelligence (AI) : It is something that many people only know about from Hollywood films, creating the impression that it will not impact their lives in the near future. In reality, AI is already changing our daily lives in ways that improve human health, safety, and productivity. Unlike in the movies, there is no race of superhuman robots on the horizon. And while the potential to abuse AI technologies must be acknowledged and addressed, their greater potential is, among other things, to make driving safer, help children learn, and extend and enhance people’s lives. In fact, beneficial AI applications in schools, homes, and hospitals are already growing at an accelerated pace.

Anyone responsible for producing software knows that the traditional ways of developing, testing and delivering software aren’t adequate for meeting this new demand. Not long ago, most companies were releasing software annually, bi-annually, or quarterly. Now, iterations commonly last 2 weeks or less. We adopted Agile and DevOps in order to move beyond that acceleration plateau. Today, many organizations are talking about Continuous Testing and trying to implement it. Nevertheless, when we look into the future, it’s clear that even Continuous Testing will not be sufficient. That’s where AI and Machine Learning enter game. They can, and will, take over the complex aspects of software development and testing. AI is perfectly able to advance software testing by automating tasks that involve self-learning, and which traditionally required human cognition.

On this topic we have written a paper that considers why we need to test AI and whether we should test it using well-known software testing skills, or do we need new and additional skills. We present general ideas, definitions and guidelines for the testing both ‘of’ Artificial Intelligence and ‘with’ the assistance of AI.

Among the topics covered in this paper are:

  • Terminology of AI and Robotics
  • The six angles of quality for AI
  • Traditional testing skills that remain relevant
  • New quality engineering skills that are needed for the testing ‘of’ AI and/or testing ‘with’ AI
  • Related fields of expertise that become relevant, such as sociology and psychology
  • The importance of controlling input of learning machines because the output cannot easily be predicted. Download the paper here.
Rik Marselis


Rik Marselis is principal quality consultant at Sogeti in the Netherlands. He has assisted many organizations in improving their IT-processes, in establishing their quality & testing approach, setting up their quality & test organization, and he acted as quality coach, qa-consultant, test manager and quality supervisor. Rik uses his more than 40 years of experience in systems development and quality and testing to bring fit for purpose solutions to our clients. He focuses at three major tasks: * Consultancy on Quality engineering & Testing in the broadest sense (quality & test policy, project startup, process improvement, coaching, second-opinions, etc…) * Develop and give training courses for both novice and experienced testers (Rik is an accredited trainer for TMAP, TPI and ISTQB certification training courses) * Research and development of the quality engineering & testing profession. Rik has contributed to over 20 books on quality and testing, of which 5 as an main author and 5 as project leader. His most recent book in the TMAP body of knowledge is “Quality for DevOps teams”. Rik is a much-appreciated keynote-speaker and workshop-host at conferences (he has presented at conferences in over 15 countries).

More on Rik Marselis.

Related Posts

Your email address will not be published. Required fields are marked *