From Systems of Systems (SoS) to Models of Models (MoM)

Internet of Things has been growing exponentially; testing must keep up. You might have read this in an article I published last year [TSE]. In this blog (part of a series) you will read about the technology that we need, in order to restrain this exponential growth of test cases. Let’s take a look into the future of IoT testing where eventually the test cases do the thinking.

Figure 1. Forecast of the growth of IoT Things up to 2020 (source: Cisco).

We are experiencing devices we have seen in science fiction movies decades ago becoming a reality. Our clothes can sense our well-being, parts of body scans can be done by smart phones, predictive maintenance is done on an oil rig in the open sea and cars are driving themselves. These are but examples of the high-tech devices or systems we are working with today. These high-tech systems are measuring large volumes of data and exchange it with fellow systems and beyond. Systems collecting and exchanging data are of all ages. We see the number of devices and the amount of data exchanged growing to epic proportions (Cisco was already predicting this years ago, see figure 1). Let’s put all of this under the term Internet of Things (IoT )in short).

From the science fiction examples, two types of IoT solutions can be distinguished. A set more associated with consumer electronics and a set more associated with industrial solutions. The latter is often referred to as the Industrial Internet of Things (IIoT). Where IIoT focuses more on high-volume “things” and big data the elements that make an IoT solution are the same. In this article, we only refer to the term IoT in the context of testing.

The rapid growth of products associated with IoT is having consequences in how we do system design. Systems we now develop are (parts of) an IoT solution or are strongly related to the IoT. IoT brings a new experience to new and existing products. A new set of properties are available in IoT devices and we can connect them all. Combining new properties together with all the connectivity options gives us an infinite amount of situations one IoT solution can be in.

IoT as Systems of Systems
Internet of Things (IoT) is a Systems of Systems. Each system can be extracted into its bare elements. Eventually, we come to the smallest possible (still functional and distributed) parts of a so-called IoT-eco-system. Let’s call them microservices. The monolithic IT architectures do not align with an environment where every device has a computer and wireless connection as is the case with IoT solutions [UMS]. Microservices are what we need in order to let the individual systems within the Systems of Systems interact and share data.

A good example of Systems of Systems (SoS) is fitness tracking. The wristband collecting fitness data of a person is a system. The system storing the data that is being collected is obviously a system and the website that lets people compete against each other in for instance running by comparing collected data is another system. Together with yoursmart phone that adds more functionality to the collected data (for instance adds location information, but also shows what music you were listening while running), we have systems of systems that give us fitness tracking.
The microservices part in this example can lie in a very small service recognizing you are running, another service that reacts to this and starts to send data to my running competition website and maybe yet another service publishing on my Facebook timeline that I am running and listening to a specific song.

Figure 2. Multiple fitness tracking solutions.

SoS have the challenge of what will be computed where (see [SCC]). In the near (near) future major challenges such systems face are in terms of usability, performance, security, and reliability.

Testing a large spread connected IoT-eco-system is a challenge. Functionally testing the separate elements (microservices) can be done but looking at the whole solution with all non-functional attributes a number of possibilities to test explodes. We need to look for new technologies to help us test all of this.

Models of Models
A side effect of the SoS is that testing the full solution only becomes possible after integration of all separate systems. The challenge here lies in the fact of testing at an early stage. Creating models for all separate systems (or microservices for that matter) can help test and develop before integration takes place. This implies the use of Models of Models (MoM). Models are needed up front to see if an IOT ecosystem can work, or maybe just to look at the implementation and try it out with limited functionality and do crowd testing. A set of models can be calculated to see what possible outcomes a situation may have. This is often used in meteorology: multi-model predictions that give the degree of uncertainty towards the future behavior of systems of systems! A great example of this is shown in figure 3.

Figure 3. Multi-model predictions within meteorology for detecting trends of the weather using todays measurements as a starting point. Good example to see that uncertainty rises with the time passed within the models.

Functional aspects are ideally tested at the microservices level, whereas the non-functional aspects of the full IoT solution can now early be tested in a model environment. In my next blog, I will go into more detail how models help us to give the insight to see when systems of systems will crash and what role of a test engineer will become the Model of Models spectrum.

References:
[TSE] Tom van de Ven and Alexander van Ewijk: Der Testaufwand steigt exponentiell durch das Internet der Dinge (IoT), In: ObjektSpektrum.de, 2015, Titelthema “Internet der Dinge”

[SCC] Alur R., Berger E., Drobnis A. W., Fix L., Fu K., Hager G. D., . . . Zorn B. (2015). System Computing Challenges in the Internet of Things:
A white paper prepared for the Computing Community Consortium
committee of the Computing Research Association.