DeepMind does not stop: AI now knows how to play Quake III Arena

The company DeepMind, formerly a division of Google, is engaged in the development of AI (its weak form) for various purposes. Now the DeepMind team is actively involved in creating various forms of AI, sharpened for games, both logical, desktop, and shooters. There are a lot of games - Go and StarCraft, and now Quake III Arena.

The developers have stated in their blog that they have trained the AI ​​system to play Quake III Arena just about the way a person does. That is, the computer system has learned to adapt to the rapidly changing game conditions, including switching levels and their elements. Traditionally, the training system was used with reinforcements .

During this type of training, the computer receives a reward or a fine depending on whether it is passing or not. Usually, the problem of the computer is that it cannot adapt to the changed conditions operatively enough - just as a person does. Despite the fact that neural networks have long been able to learn from their own mistakes, computer games are difficult for them if the system does not know the initial conditions.

The system was trained to play in Capture The Flag. In this case, the player must try to capture the opponent's flag, but in no case should he give his own. If any of the teams can capture and hold the opponent’s flag as many times as possible within five minutes, then such a team serves as the winner.

In order not to let the AI ​​simply learn the features of the level, including the location of rooms, buildings, etc., the neural network was forced to play at a new level each time. In this case, the AI ​​developed its own strategy of the game without "cramming". The computer watched the actions of other players, studied the “geography” of the level and acted according to the situation.

Moreover, the developers of Deepmind trained AI to play a whole team, which consists of different agents. The whole system is called For The Win (FTW).

So, For The Win (FTW) learned to manage his team, coordinating and directing the actions of each agent. The task, as mentioned above, the preservation of its own flag and the seizure of someone else. After the computer reached a certain level of skill, DeepMind offered to play with him the usual players in a special tournament.


All 40 people took part in it. The teams in the tournament were mixed - that is, there could be both people and AI agents on the same team. As a result of the game, it became clear that the AI ​​in pure form won more victories than teams of people. In mixed teams, the AI ​​showed a higher level of cooperation than people usually demonstrate. Thus, the computer, if necessary, served as a slave or was directly involved in the attack on the enemy base.

According to the developers, the principles of work that were used to create For The Win (FTW) can also be used to play other titles - for example, StarCraft II or Dota 2.

DeepMind demonstrated the process earlier this monthAI training for passing old school games - on Atari. The principle of learning with reinforcement was also used here, and teaching the AI ​​to play old games is quite difficult, since many of the actions of the protagonist are very implicit.

Montezuma's Revenge was taken as the basis. There is neither a clear task, nor the direction where to go, nor an understanding of what to gather or oppose. Two methods were used for learning by example: TDC (temporal distance classification) and CDC (cross-modal temporal distance classification).

The computer was trained to go through the game with the help of video walkthroughs from YouTube - there are a lot of them at the service. In the process of passing, frames of video recording of the passage of the AI ​​levels and his “teachers” from YouTube were compared. If the comparison produced a high level of similarity, the AI ​​received an award. As it turned out, after some time, the AI ​​performs the same sequence of actions as the person.

As for StarCraft, which was mentioned above, in 2017, the man still defeated the car , and dry, with a score of 4: 0. StarCraft pro player Son Byung-gu (Song Byung-gu) then fought four different StarCraft bots.