Gaming AI champion from Elon Mask killed people in video games? Not so simple

Original author: James Vincent
  • Transfer

You might not have noticed this, but in the first half of August 2017, a slight upheaval took place over the weekend. On Friday evening, in front of a crowd of thousands, the AI ​​bot defeated a professional player in Dota 2, one of the most popular video games in the world. Human champion, polite guy Danil “Dandy” Ishutin surrendered after he was killed three times, and said that he could not defeat the unstoppable bot. “He's a bit like a man,” said Dandy. “But at the same time it looks like something else.”

The bot’s father was none other than the techno-billionaire Ilon Musk, who helped finance and found the organization that developed it, OpenAI. He was not at the event, but he expressed his attitude on Twitter.


OpenAI was the first to defeat the best players in the world in competitive e-sports. This is a much more complex form than traditional board games like chess or go.

More interestingly, OpenAI itself learned everything that it knows. He studied, constantly playing with himself, accumulating numerous "careers" in the gaming experience in just two weeks.

What follows from this? Was the Friday show more impressive than the Google AI victory in the go board game? In short, probably not, but still this represents a significant step forward - both for e-sports and the world of AI.

image

Yes, video games are harder than chess


First, consider Mask's claim that Dota is “a much more complex form than traditional board games like chess or go.” It really is. Real-time battles and strategy games such as Dota and Starcraft II are complex challenges that computers still cannot handle. These games require strategic thinking, and, unlike desktops, withhold important information from players. You see everything that happens on a chessboard, but not in a video game. This means you need to predict and anticipate everything your opponent can do. It requires imagination and intuition.

In Dota, this complexity increases when people work in teams of five people, coordinating strategies that change in the course of action depending on the characters used. To further complicate the task, the game has more than 100 different characters, each of which has a unique set of skills; characters can be equipped with various unique objects, each of which, when applied at the right time, can lead to a win. All this means that it is impossible to program winning strategies in a bot for Dota.

But the game OpenAI played was not so complicated. Instead of “5 on 5” he played with people in “1 on 1”; instead of choosing a character, the person and the computer had the same hero - a comrade named Shadow Fiend, whose attack set is fairly straightforward. My colleague Vlad Savov, addicted to Dota, who also described his impression of the Friday game, said that the 1-on-1 match represents “only a small fraction of the difficulty of complete competitions”. So - probably not as difficult as go.

image

Cannot be considered better than a calculator


The second big catch is the benefits that OpenAI had over man. One of the main debates in the AI ​​community was the discussion of whether the bot has access to the API for Dota bots - this would allow it to connect directly to the flow of information from the game, to such parameters as, for example, the distance between players. Greg Brockman of OpenAI confirmed to our publication that the AI ​​did use the API and that certain techniques were hardwired into the agent, including the items that he used in the game. He was also taught certain strategies using the trial and error technique called “stimulated learning”. In general, he trained a little.

Andreas Theodorow, an AI researcher in games at Batu University and an experienced Dota player, explains why this matters. “One of the main features of Dota is that you need to calculate distances to know how far some attacks will spread,” he says. The API allows bots to estimate distances. So we can say: “If someone is 500 meters away, do it”, but a person has to calculate everything on his own, learning by trial and error. If bots have access to information that a person does not have, this gives them an advantage. ” This is especially important for a 1 on 1 game with a hero like Shadow Fiend, where players have to concentrate on choosing the right time for attacks, rather than on a common strategy.

Brockman says that learning this skill for AI is a trivial task, and it has never been fundamental to research at OpenAI. He says that the institute bot would have done without the information from the API, but "he would have simply spent much more time acquiring the skills of vision, which already works, so what's the point?"

Some skills can be learned, but not taught


With all this in mind, is it possible to sweep away the OpenAI victory? No way, says Brockman. He points out that the way he studied independently was more important than victory itself. Previous AI-type champions like AlphaGo learned to play games, processing past matches of human champions, and the OpenAI bot itself learned (almost) everything that it knows.

“You have a system that just played against itself and developed reliable strategies to defeat professionals. This cannot be taken for granted, ”Brockman says. - And this is a big question for any machine learning system: how does complexity get into the model? Where does it come from? ”

According to him, the OpenAI bot shows that we do not need to teach computers complex things: they can do it themselves. And although some bot behavior was pre-programmed, he developed some strategies himself. For example, he learned how to cheat opponents, pretending to start an attack, but canceling it at the last moment, and forcing a person to repel an attack that was not there - just like a feint in boxing.


Others evaluate it more skeptically. AI researcher Danny Britz, who wrote a popular blog entry on the subject, told us that it’s quite difficult to gauge the degree of achievement without knowing the technical details. Brockman says that they will follow, but when exactly, could not say. “Before the work was released, it was unclear what the achievements were from a technical point of view,” says Britz.

Theodorow points out that although the OpenAI bot defeated Dandy in the competition, when the players looked at his tactics, they were able to outwit him. “If you study their strategies, you can see that they played, not like everyone else, and won,” he says. Players used non-standard strategies - they would not surprise a person, but the AI ​​have not seen them yet. “The bot was not flexible enough,” says Theodorow. Brockman objects that after exploring new strategies, the bot would no longer succumb to them again.

All experts agree that this was a major achievement, but the real difficulties are only just beginning. This will be a 5 by 5 match, where OpenAI agents will not only have to duel in the middle of the map, but also work on a sprawling, chaotic battlefield, with many heroes, dozens of support units and unexpected turns. Brockman says OpenAI is now aiming for a Dota tournament next year, which will take place in 12 months. And during this time, you need to conduct much more training.

Also popular now: