proven expertise

Testing of Artificial Intelligence

In search of new AI quality engineering skills

Smart software. It’s everywhere. It’s changing the way we work, manufacture, communicate, entertain ourselves, manage our health, money, fitness, and more. Smart software is embedded in so much of what we do and use in today’s digital world. But I believe that this presents us with a real problem in terms of quality. Why? Because the actions and responses of intelligent machines will differ over time and thus are less predictable than traditional IT-systems. And also, because the manually intensive way we’ve developed, tested and delivered software up until now, is no longer enough to meet the escalating demand for new and innovative applications, in every walk of life.

Learning curve

Yet – as is always the case when new capabilities and technologies enter the fray, while we will still need the same solid base of testing knowledge as always, – there is a steep learning curve needed for everyone involved. Since currently there aren’t enough people skilled in the new technologies, including AI, data analytics, robotics and machine learning.

This is something that Humayun Shaukat, Toni Gansel and Rik Marselis, discuss in our new opinion paper ‘Testing of artificial intelligence – AI Quality Engineering Skills (An Introduction)’.

Embracing new skills

This testing of AI will require a new skillset related to interpreting a system’s boundaries or tolerances. Indeed, as our paper points out, the complex functioning of an AI system means that the focus of testing shifts from output to input to verify a robust solution.

That’s not to say people that test must throw out everything they’ve already learned. In fact, we will still see many of the established test approaches, test design techniques and testing practices being applied in the AI-led landscape. It’s more that their skillset will need to expand as AI plays an increasingly major role. For example, as well as skills from the likes of TMap and ISTQB, AI quality engineers should have expertise in A/B testing, metamorphic testing and multiple types of non-functional testing, amongst other techniques that have gained new importance.

Cross-functional teams must hone their skills in what our paper describes as “the six angles of quality” used for testing AI, robotics and other modern technologies. These six angles are: mechanical, electrical, information processing, machine intelligence, business impact, and social impact. The first three are well-known and enough knowledge and experience is available. The latter three need research and development of new approaches and skills of which our paper sketches the first outlines.

Tomorrow’s world

What does all of this mean for today’s – and tomorrow’s – QA and testing profession? We believe AI and other smart technologies broaden our horizons and opportunities. Even though the technology can take over many of the mundane, repetitive tasks, the need for skills and knowledge in areas such as software engineering, informatics, applied statistics, analytics, new programming languages, and so much more, make this an exciting new world in which to work.

Download the paper “Testing of artificial intelligence”.

To find out more, please contact Rik Marselis at or Humayun Shaukat at


More tips can be found in the NEW book 'Testing in the Digital Age' (available 1st June 2018)

Rik Marselis
Rik Marselis
Portfolio Leader
+31 886 606 600

Neueste Artikel

  • Paul Saunders
    Paul Saunders
    Global Marketing & Communications Director