November 21, 2016

From Systems of Systems (SoS) to Models of Models (MoM)

BY :     November 21, 2016

Internet of Things has been growing exponentially; testing must keep up. You might have read this in an article I published last year [TSE]. In this blog (part of a series) you will read about the technology that we need, in order to restrain this exponential growth of test cases. Let’s take a look into the future of IoT testing where eventually the test cases do the thinking.

Systems-1 Figure 1. Forecast of the growth of IoT Things up to 2020 (source: Cisco).

We are experiencing devices we have seen in science fiction movies decades ago becoming a reality. Our clothes can sense our well-being, parts of body scans can be done by smart phones, predictive maintenance is done on an oil rig in the open sea and cars are driving themselves. These are but examples of the high-tech devices or systems we are working with today. These high-tech systems are measuring large volumes of data and exchange it with fellow systems and beyond. Systems collecting and exchanging data are of all ages. We see the number of devices and the amount of data exchanged growing to epic proportions (Cisco was already predicting this years ago, see figure 1). Let’s put all of this under the term Internet of Things (IoT )in short).

From the science fiction examples, two types of IoT solutions can be distinguished. A set more associated with consumer electronics and a set more associated with industrial solutions. The latter is often referred to as the Industrial Internet of Things (IIoT). Where IIoT focuses more on high-volume “things” and big data the elements that make an IoT solution are the same. In this article, we only refer to the term IoT in the context of testing.

The rapid growth of products associated with IoT is having consequences in how we do system design. Systems we now develop are (parts of) an IoT solution or are strongly related to the IoT. IoT brings a new experience to new and existing products. A new set of properties are available in IoT devices and we can connect them all. Combining new properties together with all the connectivity options gives us an infinite amount of situations one IoT solution can be in.

IoT as Systems of Systems
Internet of Things (IoT) is a Systems of Systems. Each system can be extracted into its bare elements. Eventually, we come to the smallest possible (still functional and distributed) parts of a so-called IoT-eco-system. Let’s call them microservices. The monolithic IT architectures do not align with an environment where every device has a computer and wireless connection as is the case with IoT solutions [UMS]. Microservices are what we need in order to let the individual systems within the Systems of Systems interact and share data.

A good example of Systems of Systems (SoS) is fitness tracking. The wristband collecting fitness data of a person is a system. The system storing the data that is being collected is obviously a system and the website that lets people compete against each other in for instance running by comparing collected data is another system. Together with your
smart phone that adds more functionality to the collected data (for instance adds location information, but also shows what music you were listening while running), we have systems of systems that give us fitness tracking.
The microservices part in this example can lie in a very small service recognizing you are running, another service that reacts to this and starts to send data to my running competition website and maybe yet another service publishing on my Facebook timeline that I am running and listening to a specific song.

Systems-2 Figure 2. Multiple fitness tracking solutions.

SoS have the challenge of what will be computed where (see [SCC]). In the near (near) future major challenges such systems face are in terms of usability, performance, security, and reliability.

Testing a large spread connected IoT-eco-system is a challenge. Functionally testing the separate elements (microservices) can be done but looking at the whole solution with all non-functional attributes a number of possibilities to test explodes. We need to look for new technologies to help us test all of this.

Models of Models
A side effect of the SoS is that testing the full solution only becomes possible after integration of all separate systems. The challenge here lies in the fact of testing at an early stage. Creating models for all separate systems (or microservices for that matter) can help test and develop before integration takes place. This implies the use of Models of Models (MoM). Models are needed up front to see if an IOT ecosystem can work, or maybe just to look at the implementation and try it out with limited functionality and do crowd testing. A set of models can be calculated to see what possible outcomes a situation may have. This is often used in meteorology: multi-model predictions that give the degree of uncertainty towards the future behavior of systems of systems! A great example of this is shown in figure 3.

Systems-3 Figure 3. Multi-model predictions within meteorology for detecting trends of the weather using todays measurements as a starting point. Good example to see that uncertainty rises with the time passed within the models.

Functional aspects are ideally tested at the microservices level, whereas the non-functional aspects of the full IoT solution can now early be tested in a model environment. In my next blog, I will go into more detail how models help us to give the insight to see when systems of systems will crash and what role of a test engineer will become the Model of Models spectrum.


[TSE] Tom van de Ven and Alexander van Ewijk: Der Testaufwand steigt exponentiell durch das Internet der Dinge (IoT), In:, 2015, Titelthema “Internet der Dinge”

[SCC] Alur R., Berger E., Drobnis A. W., Fix L., Fu K., Hager G. D., . . . Zorn B. (2015). System Computing Challenges in the Internet of Things:
A white paper prepared for the Computing Community Consortium
committee of the Computing Research Association.

Tom Van de Ven


Tom van de Ven is active in the field of High Tech testing for 12 years. As a High Tech test expert he is a frequently asked sparring partner for Sogeti High Tech customers with regard to starting/professionalizing test projects. Besides a multitude of test assignments (eg. in the field of healthcare, semiconductors, agriculture and automotive) he is an active member of the Sogeti High Tech Test Competence Centre and a speaker for High Tech seminars. Tom uses his experience in a role as a coach for (starting) High Tech test engineers and is constantly looking for improvements in High Tech test methodologies (now working on the book that combines Internet of Things and TMap: IoTMap!!! Release date: 2016). He also teaches and develops several testing courses in the embedded and high tech domain. If not teaching, testing a tunnel or promoting “Quick Tech Testing” you can find him setting up a high tech test automation framework for the odd customer.

More on Tom Van de Ven.

Related Posts

Your email address will not be published. Required fields are marked *

  1. ohhani baek · January 5, 2017 Reply

    Thanks for Providing Nice information about the IOTconcept.It Helps to gain the Knowledge on the new techniques.IoT technology is Good for students.

*Opinions expressed on this blog reflect the writer’s views and not the position of the Sogeti Group