Participants in the contest will learn several games on the Minecraft platform. Left: Build Battle, where players need to recreate the structure (in this case, the structure is located on the ground). Right: The Pig Chase game, where AI agents must work together to corner a pig.

Microsoft Research’s research and development unit completes the selection round of the MarLÖ 2018 (Multi-Agent Reinforcement Learning in MalmÖ) competition. Competitors presented AI agents capable of reinforcement training who would be able to play several 3D games as defined in the MalmO platform.

The purpose of the competition is to promote research in the field of general intelligence. AI agents are trained not on a particular specific game, but on several. In addition, they must cooperate, which requires an understanding of each other’s intentions and goals (this is an important property of human consciousness). So the system will be better suited for survival in the real world.

In order to stimulate a more general approach in teaching a universal AI agent, the task consists not of one, but of several games, each of which has several tasks of varying complexity and settings. Some of these tasks are public in nature, and participants can learn from them. Others, however, remained closed, they will be used only to determine the final ranking of the competition.

The competition is organized by Microsoft, Queen Mary University of London and crowdAI platform . The competition began on July 27, 2018. The qualifying round ends on December 31, 2018. The final tournament will be held offline a week after the end of the qualifying round.

Games and tasks

One of the main features of the competition is that agents play several games. Therefore, the competition is offered several tasks. The tasks in the game may differ from each other in the arrangement of levels, size, complexity, and other parameters depending on the game. The figure shows how the games and tasks in the competition are organized.



As you can see, each game has four tasks, of which two are published in open access, and two are secret.

To participate in the competition, you need to register on crowdAI, and then just clone the starter kit of the competition on GitHub .

Malmö Platformprovides an API that provides access to actions, observations (i.e. location, environment, video frames, game statistics) and other common data that are available on the Minecraft platform. Marlo, on the other hand, is a shell for Malmö that provides a higher level of API and a more standardized learning environment with reinforcement for research.

The framework is written to complement the OpenAI Gym framework , which is a tool for developing and comparing reinforcement learning algorithms, thereby providing a standard and familiar platform for scientists, developers, and popular frameworks.

The Malmö project started in 2015 by AI researcher Katya Hofmann at Microsoft Research Cambridge, UK. Although modern AI agents have shown a lot of achievements in different games, Katya Search for a game that would allow the AI to master a wide range of skills: "In the moment when we began to talk about of Minecraft, it was obvious that this is the ideal environment for AI research, - says she is. “This is a world to which people join without a definite goal.” Thus, the Malmö project is a platform built on top of Minecraft, where researchers can perform many different experiments with AI, as well as compare their results in a standardized way.

When testing the Marlo competition in 2017, participants were offered only one game: catch a pig. The 2018 competition is much more complicated: three missions have now been developed, each of which requires cooperation. Agents need to understand how to recognize another AI agent in the environment, and then find a way to work together to achieve their common goal.

If an AI agent hypothesizes another agent’s goals, it can be called a rudimentary form of what psychologists call the " mental state model. "“- the human ability to understand the mental states and intentions of other people. Katya Hoffman hopes that AI agents will ultimately hone this ability by collaborating with human gamers in Minecraft. “Then the algorithms will learn to work with people and find out what people want,” she says.