March 11, 2015 at 06:47Protection against cheating in online games
This article is about how we made the system of protecting the HTML5 browser game from hacking and falsification of the results, what difficulties we encountered in this, how we solved them and what we got in the end. The main and familiar problem of such games is the ability to write a bot that will automatically pass this game. The development of such a bot is facilitated by the fact that the game code is in the public domain. The situation was complicated by the fact that real prizes were announced, including an iPad, concert tickets, USB flash drives, etc.
The article will be useful mainly to those who make HTML5 / Flash games and care about their safety; those who pay for the development of these games; and a little to those who are called to fight with bots. And, of course, to those who wrote this article. Because we hope that it will be the beginning of a productive discussion about how browser game developers can resist cyber crooks.
A little about the game itself.
Unfortunately, we cannot show the game itself at the request of the customer. Despite all the techniques described below, unfortunately, “ignorance” is a key element of protection. That is, the disclosure of methods in relation to a specific game will significantly reduce their effectiveness.
We are web-developers, but have not previously been involved in game development. At the very beginning we gave birth to many funny decisions. For example, we made an important architectural mistake - we decided to generate a level on the client in the browser; and as protection they relied on obfuscation of JS code. True, comrades from neighboring departments said that for quality protection they would have to “play” the game on the server and compare with the client. But then we laughed that it was very difficult.
I repeat that the main requirement was to provide protection against markups. But our team decided first to make the game work in principle. And the development of protection was postponed until later. It seemed logical: first we do something, and only then we protect this “something”.
As soon as we began to engage in protection, we immediately faced the basic problem of all browser games arising from the client-server architecture. Namely: the player sees the entire client code of the game and one cannot blindly trust the data that has come to the server. If we can deal with the second part of the problem, then the possibility of analyzing and changing the source code by any user really puzzled us.
We started with a simple solution - obfuscating JS code and encrypting data sent by the client to the server. But the customer had a killer security team, which in 15 minutes was able to record an unrealistic game score, having a completely obfuscated code. Data encryption during transmission is also not an option - the keys are at the client’s, and they can easily be substituted in the right place. So in fact, both of these solutions turned out to be effective only for very lazy hackers.
Moreover, Flash games are also subject to similar reverse engineering, and those developers are also in search of a security solution.
The situation was complicated by the fact that our game has real prizes. In this case, its hacking moved from the category “hack for fun” to the category “hack for money”. This game is more than attractive to specialists of a higher class than the “novice hacker”.
Despite the fact that a lot of online games have already been developed, very little information has been found on methods to combat their hacking. Here are some of the recommendations we’ve drawn from the Internet:
We brainstormed and also came up with a lot of ideas. They were successful, they were not very. To the category of not the most successful can be attributed, for example, the withdrawal of captcha “before”, “during” and “after” the game. Or, say, taking screenshots during the game and sending them to the server for verification. The most interesting of the implemented methods are described below.
After much thought and resistance to the difficult decision, we came to the conclusion that playing (emulating) the game on the server is clearly a necessary thing. If not to say - inevitable.
To do this, we began to generate the game (the location of bonuses and obstacles) on the server and
send to the server not the end result of the game, but only the events that took place in it: collisions with obstacles, bonuses and keystrokes. For each event, we knew the time, coordinates, keystrokes or bonuses / obstacles that the player encountered.
There was an option to send only keystrokes, and on the server to calculate the result at the end of the game. But we are faced with the problem of different game speeds on different devices and in different browsers. There is a high probability of a strong discrepancy between the game process on the client and on the server, since this method is highly sensitive to inconsistencies. For example, if suddenly the client does not send just one collision with an obstacle, then the emulated game from this moment will go completely different. After all, in collisions, energy and speed are lost.
Another option is to process events on the server and immediately send the event results to the client. But at the same time, very large traffic occurs between the server and the client, the response time of the game increases, and the load on the server increases sharply. So work, for example, MMO games. But for a small Collect & Run game, we found this impractical.
I would especially like to note the fact that the devices on which the game was launched had different performance. On fast devices, the game went smoothly, on slow - jerky. Due to this difference, we had to abandon the time reference during emulation. Instead, we viewed the events of the game as something happening in some coordinates. For example, at first we considered the jump for the bonus as “pressing the button up at the 10th second, collision with the bonus at the 12th second”. But the following suited us better: “pressing the button up when the player’s X-coordinate is 1000, collision with the bonus when the player’s X-coordinate is 1100 and the player’s Y-coordinate is 300”. In this case, both fast and slow devices were emulated more or less the same.
As a result, we decided after the game to emulate on the server every event sent by the client, and additionally calculate the probability of its occurrence. One of the tasks that we solved at the same time was the creation of the same collision calculation algorithms on the server and on the client. It was necessary to achieve the same accuracy, including rounding.
As a result of the emulator, we get on the server a sequence of game events in the same form as in a real game. After emulating the events, we move on to analyzing what we got.
This is the most interesting block. Here are the most successful cheat detection methods. They are all different, run sequentially, you can add more if necessary.
Each method has its own weight in the calculation, the value of which is adjustable. By the way, it makes sense to make the maximum possible number of parameters in the settings. For it is not clear in advance how sensitive or rude this or that algorithm will turn out in reality.
During operation, the analyzer sets values for each method from 0 to 1 (0 - the game is beyond suspicion, 1 - the test is completely failed) and at the end it calculates the “Game Suspicion Index” (also from 0 to 1).
This index is used to automatically tag the game. If it is more than a threshold, then the game is considered hacked. A specific threshold value is selected empirically, we got 0.4.
But that is not all. Something needs to be done with all these calculations. It is one thing to program the algorithms described above, another is to give the game administrators the results of work in a readable form. A player who is blocked for hacking can apply for support with complaints, and they must have evidence of the decision made. Therefore, it is recommended to give administrators the maximum amount of information so that they understand and justify why a particular game was identified as hacked.
To do this, in the admin panel, we made 4 pages.
By the time the game started, we did not have time to automatically block suspicious users. This would greatly simplify the work of the game administrators, as a person is not able to view thousands of suspicious games per day.
It was very interesting to analyze the top players a few days after the launch of the game. One cracker followed a simple path and collected all the elements, but never jumped. He also collected all the “false” elements. Another “champion” has evolved sequentially. At first he had a dozen fair games, then a lot of “empty” ones. And in the most recent games, the suspiciousness index went wild. Some “winners” completed the game in less than 10 seconds. Some people played the game only once and immediately hit the top. This, by the way, can be used as an analyzer method - few people are able to make the maximum the first time. There were “champions” in one second gaining an unrealistic amount of points. Etc. All of them were blocked and were not allowed to real prizes. No complaints were received from them.
After reviewing several dozens of real games, we updated some parameters of the threshold values in the direction of increase. For example, the maximum value of a game’s suspiciousness index when a game is considered hacked can be increased. In other words, the initial settings were very suspicious.
In reality, we got a positive result of this system. The display of results in the admin panel helped a lot. In general, we can say that the system has proved to be quite viable, the customer was satisfied.
When protecting against account markups, we used the data of only one game and tried to catch it on internal inconsistencies. We proceeded from the fact that the game is completely vulnerable: all sources are available to the player, and the incoming data is unreliable. But the game has a part inaccessible to hackers - this is the data of all games. There is an assumption that using the database played by thousands of players correctly can improve the accuracy of hacking detection. Now we believe that the database of game results is the main advantage of developers over hackers.
And what do you think, Khabrovites? Share your thoughts and approaches to protecting online games.
The article will be useful mainly to those who make HTML5 / Flash games and care about their safety; those who pay for the development of these games; and a little to those who are called to fight with bots. And, of course, to those who wrote this article. Because we hope that it will be the beginning of a productive discussion about how browser game developers can resist cyber crooks.
What game is this?
A little about the game itself.
Unfortunately, we cannot show the game itself at the request of the customer. Despite all the techniques described below, unfortunately, “ignorance” is a key element of protection. That is, the disclosure of methods in relation to a specific game will significantly reduce their effectiveness.
- Type of game: Collect & Run.
- Examples: the familiar Mario Bros. and his clones
- Duration of the game: approximately 1-2 minutes.
- Purpose of the game: score the most points.
- The main elements of the game:
- The character.
The main character is always moving. Can bounce and crouch in order to collect bonuses and avoid obstacles. - Bonuses.
May increase points or character movement speed. In order to collect them, the character must bounce. - Obstacles.
Barriers or objects moving towards the character. In order to avoid them, the character must crouch or bounce. When faced with an obstacle, the character loses speed.
- The character.
- Game characteristics used in calculating the result of the game:
- Points earned. Increase when collecting bonuses.
- Character speed. Decreases over time. Decreases when colliding with an obstacle. Increases when collecting bonuses.
- The time spent on the game.
- Technologies used:
- Client: HTML5. Reason: it was required that the game run on all devices. JS-framework: Pixi.js ( http://www.pixijs.com )
- Server: php, framework: zend
- Customer goal: advertising your brand through the game. The gameplay was associated with the services of the company; logos, company colors and all that jazz.
- The game was launched for a certain period, after which players with the best results were awarded valuable prizes.
- Based on the foregoing, the customer immediately emphasized that the game should be protected from falsification of results. The customer values reputation. He did not want information to appear after the game was released in the media that their branded game was hacked.
What kind of team?
We are web-developers, but have not previously been involved in game development. At the very beginning we gave birth to many funny decisions. For example, we made an important architectural mistake - we decided to generate a level on the client in the browser; and as protection they relied on obfuscation of JS code. True, comrades from neighboring departments said that for quality protection they would have to “play” the game on the server and compare with the client. But then we laughed that it was very difficult.
I repeat that the main requirement was to provide protection against markups. But our team decided first to make the game work in principle. And the development of protection was postponed until later. It seemed logical: first we do something, and only then we protect this “something”.
The problem
As soon as we began to engage in protection, we immediately faced the basic problem of all browser games arising from the client-server architecture. Namely: the player sees the entire client code of the game and one cannot blindly trust the data that has come to the server. If we can deal with the second part of the problem, then the possibility of analyzing and changing the source code by any user really puzzled us.
We started with a simple solution - obfuscating JS code and encrypting data sent by the client to the server. But the customer had a killer security team, which in 15 minutes was able to record an unrealistic game score, having a completely obfuscated code. Data encryption during transmission is also not an option - the keys are at the client’s, and they can easily be substituted in the right place. So in fact, both of these solutions turned out to be effective only for very lazy hackers.
Moreover, Flash games are also subject to similar reverse engineering, and those developers are also in search of a security solution.
The situation was complicated by the fact that our game has real prizes. In this case, its hacking moved from the category “hack for fun” to the category “hack for money”. This game is more than attractive to specialists of a higher class than the “novice hacker”.
Tips from the Internet
Despite the fact that a lot of online games have already been developed, very little information has been found on methods to combat their hacking. Here are some of the recommendations we’ve drawn from the Internet:
- Hide, protect and encrypt game data:
- The variables responsible for the game score should not be made in the global scope. They are recommended to be declared private and placed in closures.
- Do not send account values in clear form, but be sure to encrypt.
- Encrypt the addresses (uri) to which the account is transmitted.
- Obfuscate the part of the code that is responsible for calculating and sending the invoice.
- Send along with the results of the game license keys and check them on the server.
- Pros: relative ease of implementation
- Cons: as already mentioned above - breaks in 15 minutes
- Fool hackers:
- If a cheat is detected, the account will still show the hacker in the list of records his result. Let him think the job is done. But other users should not see this.
- When a game is defined as hacked, then postpone the punishment for a random period (for example, from 1 to 3 days). The hacker will use several methods and periodically he will see what he is doing. But what exactly turns out, it will be difficult for him to understand, and on what exactly he was caught in the end - too.
- Check the realism of the game data:
- Check on the server the theoretical possibility of the values sent. For example, energy cannot be negative, the score cannot decrease, the player’s position always increases, etc.
- Send game data to the server and compare data with the same data of other users. For example, if a player fired 200 shots and reached the score of 200,000, and the rest of the players after making 200 shots reached the score of 5000, then this is an occasion to doubt the honesty of the game.
- Delete games that lasted less than (more) a certain time.
- Pros: relative ease of implementation
- Cons: does not give full protection, adds an extra 5-10 minutes to organize a hack
- Organizational impact on accounts:
- Add importance to the account so that it is a pity to lose it. For example, to play this game with real prizes, you must first achieve something in other games without prizes.
- Make account registration paid.
- Ban IP hackers at the very first attempt to hack. In general, do not give them the opportunity to understand.
- Emulate the game on the server:
- Make 'instant replay' on the server. After passing the level, send data, according to which the events in the game are restored and its result is calculated again. Then all this is compared with what the player sent, and if the difference is big, then this is a hack.
- Pros: gives a greater degree of protection
- Cons: Events dispatched at the end of the game can also be tampered with.
We brainstormed and also came up with a lot of ideas. They were successful, they were not very. To the category of not the most successful can be attributed, for example, the withdrawal of captcha “before”, “during” and “after” the game. Or, say, taking screenshots during the game and sending them to the server for verification. The most interesting of the implemented methods are described below.
Game emulator
After much thought and resistance to the difficult decision, we came to the conclusion that playing (emulating) the game on the server is clearly a necessary thing. If not to say - inevitable.
To do this, we began to generate the game (the location of bonuses and obstacles) on the server and
send to the server not the end result of the game, but only the events that took place in it: collisions with obstacles, bonuses and keystrokes. For each event, we knew the time, coordinates, keystrokes or bonuses / obstacles that the player encountered.
There was an option to send only keystrokes, and on the server to calculate the result at the end of the game. But we are faced with the problem of different game speeds on different devices and in different browsers. There is a high probability of a strong discrepancy between the game process on the client and on the server, since this method is highly sensitive to inconsistencies. For example, if suddenly the client does not send just one collision with an obstacle, then the emulated game from this moment will go completely different. After all, in collisions, energy and speed are lost.
Another option is to process events on the server and immediately send the event results to the client. But at the same time, very large traffic occurs between the server and the client, the response time of the game increases, and the load on the server increases sharply. So work, for example, MMO games. But for a small Collect & Run game, we found this impractical.
I would especially like to note the fact that the devices on which the game was launched had different performance. On fast devices, the game went smoothly, on slow - jerky. Due to this difference, we had to abandon the time reference during emulation. Instead, we viewed the events of the game as something happening in some coordinates. For example, at first we considered the jump for the bonus as “pressing the button up at the 10th second, collision with the bonus at the 12th second”. But the following suited us better: “pressing the button up when the player’s X-coordinate is 1000, collision with the bonus when the player’s X-coordinate is 1100 and the player’s Y-coordinate is 300”. In this case, both fast and slow devices were emulated more or less the same.
As a result, we decided after the game to emulate on the server every event sent by the client, and additionally calculate the probability of its occurrence. One of the tasks that we solved at the same time was the creation of the same collision calculation algorithms on the server and on the client. It was necessary to achieve the same accuracy, including rounding.
As a result of the emulator, we get on the server a sequence of game events in the same form as in a real game. After emulating the events, we move on to analyzing what we got.
Game analyzer
This is the most interesting block. Here are the most successful cheat detection methods. They are all different, run sequentially, you can add more if necessary.
- Collision coordinates.
Collision with an obstacle or bonus occurred in the browser on the client at the point where this item was generated on the server. - Increase character position.
The character’s position in the game should increase with time. It cannot decrease or remain the same. - The character has reached the end of the level.
The maximum character position does not exceed the length of the level generated on the server. - The duration of the game.
It is physically impossible to complete in less than one minute. Therefore, games that have passed in 30 seconds or 2 minutes are considered hacked. - Monitoring of speed and energy indicators.
The values of indicators in the browser on the client should be correlated with game events (collecting bonuses, collisions with obstacles). When taking some bonuses, the character’s speed increases, when colliding with obstacles it decreases. Also, collecting some bonuses increases the player's score. Knowing what items the player collected, you can calculate on the server what values he should have: speed and amount of points. - Jump before taking the bonus.
Bonuses hang high to collect them. The character must bounce. Here it is checked whether there was a jump before taking. - Jump in front of an obstacle.
In order to avoid a collision with an obstacle lying on the ground, the character must bounce. For each obstacle, we check whether the player jumped to the obstacle (pressing the jump button) and if the jump path blocked it. - The right keys.
Checking the pressing of strictly defined keys: “up” and “down”. If you pressed something else, then this is suspicious. - Bonuses and obstacles are collected once.
The user can collect one bonus (or obstacle) no more than once. - Collisions on the client.
Comparison of collisions that occurred in the browser on the client and received on the emulator. - Collisions on the emulator.
Comparison of collisions that occurred on the emulator, but did not occur in the game. This method was introduced in order to have different weights of this and the previous methods. We thought that if there is a collision on the emulator, but not in the real game, then this is less suspicious than when there is a collision in the real game, but not on the emulator. - The taking of false elements.
When generating the game, we added several bonuses that are impossible to take in the game. For example, they are slightly higher than the character can jump; or there are two bonuses that are nearby, but physically you can collect only one of them. And we check whether the hacker got on the bait and whether he took these bonuses. - Fingerprint analysis.
Before the game, when registering and the player’s login, we collect information about the client (browser plug-ins, http headers, ip, installed fonts, screen size and color). This does not help us understand if the game has been hacked. But in the case when administrators mark a player as a hacker, the analyzer scans all the players in the system and finds similar “fingerprints”. Thus, it is possible with certain probability to detect hackers who are trying to break the game under one account, and play “cleanly” under another.
Each method has its own weight in the calculation, the value of which is adjustable. By the way, it makes sense to make the maximum possible number of parameters in the settings. For it is not clear in advance how sensitive or rude this or that algorithm will turn out in reality.
During operation, the analyzer sets values for each method from 0 to 1 (0 - the game is beyond suspicion, 1 - the test is completely failed) and at the end it calculates the “Game Suspicion Index” (also from 0 to 1).
This index is used to automatically tag the game. If it is more than a threshold, then the game is considered hacked. A specific threshold value is selected empirically, we got 0.4.
Evidence administration
But that is not all. Something needs to be done with all these calculations. It is one thing to program the algorithms described above, another is to give the game administrators the results of work in a readable form. A player who is blocked for hacking can apply for support with complaints, and they must have evidence of the decision made. Therefore, it is recommended to give administrators the maximum amount of information so that they understand and justify why a particular game was identified as hacked.
To do this, in the admin panel, we made 4 pages.
- Information about the game - Emulator
Table with real and emulated events of the selected game. The columns are as follows:- item type (bonus, barrier)
- X- and Y-coordinates of the element
- time on client
- X- and Y- coordinates of the character (on the client)
- fact of collision of character and element (yes, no)
- X- and Y-coordinates of the character (emulated)
- the probability of a collision at a specified time on the client
- Information about the game - Analyzer The
general index of suspiciousness of the game and a table with the results of the analyzer for each method. - The analyzer
displays a table of generally all suspicious games, based on the threshold of suspiciousness (0 - no suspicion, 1 - definitely hacked). From experience, we got a value of 0.4. - Player Page
Added to the player profile “Player Suspicion Index”. He is calculated based on the suspiciousness indices of all his games.
By the time the game started, we did not have time to automatically block suspicious users. This would greatly simplify the work of the game administrators, as a person is not able to view thousands of suspicious games per day.
How did everything go
It was very interesting to analyze the top players a few days after the launch of the game. One cracker followed a simple path and collected all the elements, but never jumped. He also collected all the “false” elements. Another “champion” has evolved sequentially. At first he had a dozen fair games, then a lot of “empty” ones. And in the most recent games, the suspiciousness index went wild. Some “winners” completed the game in less than 10 seconds. Some people played the game only once and immediately hit the top. This, by the way, can be used as an analyzer method - few people are able to make the maximum the first time. There were “champions” in one second gaining an unrealistic amount of points. Etc. All of them were blocked and were not allowed to real prizes. No complaints were received from them.
After reviewing several dozens of real games, we updated some parameters of the threshold values in the direction of increase. For example, the maximum value of a game’s suspiciousness index when a game is considered hacked can be increased. In other words, the initial settings were very suspicious.
In reality, we got a positive result of this system. The display of results in the admin panel helped a lot. In general, we can say that the system has proved to be quite viable, the customer was satisfied.
Conclusion
When protecting against account markups, we used the data of only one game and tried to catch it on internal inconsistencies. We proceeded from the fact that the game is completely vulnerable: all sources are available to the player, and the incoming data is unreliable. But the game has a part inaccessible to hackers - this is the data of all games. There is an assumption that using the database played by thousands of players correctly can improve the accuracy of hacking detection. Now we believe that the database of game results is the main advantage of developers over hackers.
And what do you think, Khabrovites? Share your thoughts and approaches to protecting online games.