Anti-cheat software is part of an endless cat-and-mouse game: developers find a way to detect and punish cheaters, who in turn try to evade detection by building better cheats. This cycle can only be halted momentarily, when either developers or cheaters manage to out-smart the other. A few years ago, a game editor managed to do exactly that, in a brilliant way.

Summer 2016, everyone is stuck to their phones playing Pokémon GO, the newest game by studio Niantic where you roam the streets to catch virtual monsters in augmented reality. Among the nearly 1 billion players who downloaded the app, inevitably came people with the intent to break the game or give themselves an unfair advantage.

How to catch cheaters?

As a multiplayer game development company, the last thing you want is to have players frustrated by a large number of cheaters who take the fun out of the game. Games that are free to download and play (like Pokémon GO) have it even worse, because hackers can easily create another account with no cost and very little effort when they get banned.

The most simple way to find the most obvious cheaters is to use a Report System. When a player sees potentially fraudulent behavior from another player, they can submit a report which will be reviewed manually by an expert from the company who can make an informed decision. This kind of human-based system is great for UX: it empowers the player to actively participate in cheat prevention instead of being frustrated by a weak anti-cheat, but it also has its limits. Indeed, it’s expensive to hire manual reviewers and the process is likely to take several hours or days, especially given that cheaters don’t wait for office hours to use their malicious software. Also, in the case of Pokémon GO where user interaction is minimal, it’s hard for a player to determine whether their opponent is a legitimately strong player or someone who used automated tools to reach a high level.

This brings us to another type of common cheat detection technique, which is rule-based detection. Here, the developers study through reverse-engineering or in-game behavior how various hacks work, and try to come up with rules which can accurately detect them without false positives. Such rules include verification of the game’s integrity (many cheats modify game assets or inject custom libraries), or measuring the number of actions made each second to expose super-human performance. Given sufficient performance and accuracy, automated detection can go as far as processing every game in real-time and banning cheaters on the fly.

However, rule-based algorithms now belong to the past thanks to the rise of machine learning which gave us ways to design extremely precise and versatile detection systems which do not rely on humans to create rules for them. One such example is the use of deep neural networks by the famous Valve Anti Cheat on multiplayer games of Counter Strike: Global Offensive, as presented by John McDonald at GDC 2018.

Still, machine learning models have one major downside: they have to rely on huge annotated datasets to be trained correctly.

The case of Pokémon GO

The limitation of needing large datasets of player traces already labeled as legitimate/cheat is usually solved by making those annotations manually or using some elementary rule-based classifier. However, both of those methods are able to reliably catch only the most obvious and simple cheats, and the more subtle and advanced ones are likely to be included among the legitimate players in the AI’s learning samples. That’s where Niantic implemented a genius design which may actually be the first of its kind in anti-cheat software.

At the game’s release, several protections were put in place to make sure Pokémon GO’s APIs would be abused as little as possible : the app’s executable was protected by SafetyNet which ensures it runs only in a trusted, non-rooted device. Also, some parts of the code were heavily obfuscated and communication with the official game servers was done using a serialized format called Protobuf, which meant that it wasn’t easy to bypass SafetyNet by reimplementing your own fake game client. After a few days, some teams of elite reverse-engineers managed to break through all of those protections, and made a custom library which was able to interact with the Pokémon GO servers. This allowed modders to create automated maps which showed the location of rare Pokémon as they were appearing, but some others with worse intentions started to grind experience points 24/7 using bots and get unfair advantages in battles. Everyone thought that was the end of it, but they were missing a crucial piece of the puzzle…

During reverse-engineering of the app, one of the fields of the main API call was some sort of complex hash which was thought to be some sort of integrity check, but people working on it quickly realized that the servers did not care whether this field was valid or not when receiving a client packet. This field was quickly dubbed Unknown6 (it was in 6th position of the Protobuf query) and illegitimate clients were simply using an empty value for it. However, a few weeks after release, Niantic servers started only accepting packets with a seemingly valid Unknown6 value. One immediate effect was that all bot traffic was immediately brought to a stop until someone could figure how this value was computed by the game :

Source : Niantic Blog Post after activating Unknown6 verification

How Unknown6 changed the world of anti-cheat

After more than three days, a team of 17 experts managed to understand and implement the code behind Unknown6. Once again, modders and cheaters thought they had won because their activities could be resumed, but the real winner of this battle was actually Niantic.

Indeed, during the first weeks after release, the game servers were experiencing traffic both from legitimate players and from cheaters. Remember what I said about machine-learning based anti-cheat ? Not only do they need many samples of player actions, they also rely on accurate labels of those traces. And Unknown6 was exactly designed for that: before this field was fully reverse-engineered, every packet with a valid signature necessarily came from a legitimate app, and the rest of the samples could be classified as cheaters. By initially ignoring Unknown6, hackers gave Niantic exactly what they wanted : a reliable label to train their anti-cheat AI. Even after cheaters started to use correct values for Unknown6, it was already too late: they were now fighting against a machine-learning model trained on billions of interactions, where any suspicious activity could get their account automatically banned in a second.

In conclusion, even though I was closely following and supporting the effort to reverse-engineer the game at that time, I still admire the great design put together by Niantic. I still enjoy playing Pokémon GO from time to time, and while it might be hard nowadays to find a live open-source map of the best Pokémon spots in your area, that’s a small price to pay for a game without a million hackers bothering you!