English | EN 
In a couple of blogs, I look to the future of IoT and testing. My last blog (From SoS to MoM) describes how the test cases do the thinking. In this blog, we build further on the System of Systems(SoS) concept to Model of Models(MoM) and consider the world where multi model predictions become a new form of testing.
The rapid growth of products associated with IoT is having consequences in how we do system design. Systems we now develop are (parts of) an IoT solution or are strongly related to the IoT. IoT brings a new experience to new and existing products. A new set of properties are available in IoT devices and we can connect them all. Combining new properties together with all the connectivity options gives us an infinite number of situations one IoT solution can be in.
IoT can be defined as a System of Systems. SoS have the challenge of what will be computed where (look at “System Computing Challenges in the Internet of Things”). In the (near) future major challenges such systems face is in terms of usability, performance, security, and reliability.
Multi-Model predictions
IoT solutions or SoS operate in their own and/or each other’s environment. A new IoT solution in the neighbouring environments can cause new and unwanted effects for an existing IoT solution.
Take for example a car with an adaptive cruise control. This cruise control has radar information from the front of the car to see whether some other object is in the front of the car, how far it is and how fast it is travelling. This is all tested and approved to be road legal and you can drive a car that keeps its distance to the car in front of you automatically.
Now a new brand of car enters the market with a new type of driver aid. This can be another type of radar, or maybe a whole new concept on automatic driving aids. However, in some way there is the possibility it can interfere, disrupt the radar of your car or otherwise influence the results of your adaptive cruise control. A new environmental element is now added to the number of possible situations your SoS or IoT solution can be combined with.
Figure 1. The so called “plume” forecast, where we see uncertainty rise in time for multi model results.
Models are often used when looking at the behaviour of a system (or SoS) in its environment. It is good practice to look at multiple concepts at an early stage. Concepts can be tried out and faults can be detected at this early stage. In the example of the adaptive cruise control, we want a whole lot of models of different environments to be combined with the model of the adaptive cruise control. This helps testing at an early stage a lot. Not a lot of real-life crash tests are needed to prove that the adaptive cruise control is safe to use.
New driver aids are coming to the market, but also new traffic situations the adaptive cruise control must deal with. The set of environmental models grows as time goes by. Even when not updating the adaptive cruise control a weekly test with new models should be done to keep it a safe function.
Singularity meteorologists
This concept can be put to very good use in IoT environments. Why not come up with new environmental models constantly and use an evolutionary algorithm to come up with extreme test cases that prove an IoT solution is still safe to use (no critical limits/boundaries of the system are crossed). We can even go so far as to create a set of predictive modelling situations where we try to induce these extremes. The same is done in weather forecasting. A set of models is run with a given set of parameters to see where temperature will end up in a couple of months (see Figure 1 and the “Uncertainty in weather and climate prediction” article in the discussion Meeting Issue ‘Handling uncertainty in science’)
Figure 2. Multi model predictions give specific insight in future results with corresponding uncertainty.
Let’s use this concept in IoT testing. Test engineers set up the right set of modelled environments to interact with the model of a SoS. They run all these combinations, to see what in time will be possible end-situations for quality attributes like robustness (can we find a situation where a SoS can crash?) or timing (is timed information still in time). The test engineer becomes a technology meteorologist. In the future, test engineers can keep an eye out on running systems and how good they keep on working. They will work with and create pictures like Figure 2. With more and more intelligent systems it may even go so far that the field of testing moves towards monitoring the degree of intelligence of SoS.
A test engineer becomes the singularity meteorologist of the future! Read more on singularity in this great essay Vernor Vinge wrote already in 1993.
IoT needs smart test cases
Internet of Things with its Systems of Systems gives us infinite test combinations and growing test environments. Running tests in IoT environments needs another approach to testing:
Technologies like artificial intelligence can help us in testing IoT solutions. Specifically, an evolutionary algorithm can be used to generate test cases. Together with the use of multi model predictions for test environments, testing becomes smart testing.
The future sees test engineers as technology meteorologists that use smart sets of test cases which evolve to the best fit of addressing problem areas for an IoT solution in the right variety of situations.
