I am a big fan of playing (strategic) tabletop wargames. I play all kinds of settings: fantasy, science fiction or historical (for example I play: Warhammer, Frostgrave, Star Wars X-wing and Flames of War) and aside from playing games I like to collect and paint miniatures to use in those games. Looking at tabletop wargames (see for example this list on wikipedia), they can be decomposed into a couple of components:
- a rulebook,
- the miniatures,
- a table with scenery and
- the players.
Where is the link to the Internet of Things (IoT) you might ask? As I am playing these games for many years I noticed a trend when a new game (system) is released. Here starts the big analogy with IoT solutions. The following steps come by when a new game is introduced.
- New rules defined. A new game is defined by a new rule set (or rulebook). This can be bought and players can collect their own miniatures and scenery to play with. This is part of the hobby and fun to do.
- Building up the environment. A lot of miniatures and scenery is collected and painted before the first game is played but even more is added when the first games are played. New tactics call for new miniatures all the time!
- Game experiences exchanged. After some time more and more players have played with the new rule set and online forums are filling with questions, personal reviews, miniature guides, rants and thoughts on interpretation of the rules. Often so called house-rules become largely accepted and the initial rule set changes in the tabletop wargames community.
- New versions/expansions released. Based on all the feedback from the field a new version is released. Specific magical weapon combinations may be too powerful and need to be more expensive or the game is so successful an expansion with new space ship pilots is introduced.
This is a cycle that repeats itself on and on. Actually lots of popular tabletop wargames continue for 10 years or more in this way.
Let’s take a look at the introduction of an IoT solution. This often starts at the point where I can buy a “Thing”. This new product has new rules, and I will deploy it in a new and unknown environment. Data is collected during operation (giving the producer insight in specific use and problems in the field). New versions of this “Thing” come to the market and maybe new add-ons will be released. We see the same four steps come by in an IoT product development cycle.
If this analogy holds it is interesting to look at how testing is done for tabletop wargames. New games have been released for decades with success. Let’s see what we can learn from their experiences with respect to the relatively new world that is now called IoT.
A tabletop wargame is tested before being introduced. It is often done in-house with a small set of very experienced people). It is impossible to test all in-game combinations. With tabletop wargames it is accepted that a game needs a couple of versions to mature and the entire wargames community is happy to play and test and play and test. Most important thing is that a good dialogue exists between the wargames community and the producers. You have to take your users seriously and communicate and give regular updates. The Wargames producers make very good use of the crowd that buys and plays the game. The crowd consists of wargame veterans, rookies and all in between. They will find weak spots in the rule set in no time. In this way over-powerful combinations get discovered more quickly opposed to creating a big list of all combinations (a more formal test approach). The so-called “Meta rule set” of a wargame is what the wargames producers are looking for.
This experience can be applied to testing IoT solutions. Before releasing a product an in-house quality check needs to be done. The crowd buying a “Thing” as part of the full IoT solution needs to be informed about the current status. The IoT solution must look good and perform well in order to get people to be enthusiastic enough in using it without it being fully matured in functionality. In this way new versions can be created based on the feedback and data collection of the users. Soon new and better functionality is made available and more users buy the product.
The way of working for releasing tabletop wargames can be used in IoT testing. A combination of the following elements helps shorten time-to-market and give confidence in a good experience of the IoT solution:
- in-house quality check
- crowd testing and informing the crowd
- a good looking product
- new versions released quickly
With this final checklist I would ask you to set up your version of a crowd test (big or small in participants). I would definitely like to hear about the “Meta rule set” of your “Thing” under test!