Libratus strategic poker bot adapted for military simulators

    In 2017, the Libratus poker bot hit the media headlines when it beat four professionals in no-limit hold'em at a distance of 120,000 hands. Now the technology has been adapted for military use .

    Developers of weak AI systems often compare the effectiveness of their programs in a game confrontation against a person. In games with full information at any time of the game, all players have full information about the state of the game, that is, about the position and all possible moves of any of the players. Unlike such deterministic situations, in games with incomplete informationSome information about the state of the game is hidden from the player - for example, the opponent’s cards. No Limit Hold'em is one of these games. In addition to the closed cards of the opponent, an element of uncertainty is added here due to the arbitrary size of each bet. With this in mind, the number of possible outcomes is estimated at 10,161 . Developing an optimal strategy in the face of uncertainty, taking into account the opponent’s tactics, is exactly what the military needs.

    Libratus ("balanced" from the Latin language) was created by researchers from Carnegie Mellon University to test the ideas of automated decision-making based on game theory. At the beginning of last year, after a loud victory over the professionals, the lead author of the program, Professor Tuomas Sandholm, founded a startup called Strategy Robot to adapt gaming technology for use by the government. For example, in war games and simulations to study military strategy and planning. At the end of August, according to public records, the company entered into a two-year contract with the US Army worth up to $ 10 million. Funding was provided by the Defense Innovation Unit, which the Pentagon formed in 2015 to improve contacts with Silicon Valley companies and more actively introduce new technologies.

    Libratus strategy at various stages of distribution. According to the results of the distribution, the model of a game against a specific opponent is specified

    Libratus is built on algorithmic (computational) game theory. In the headspace with the professionals, the bot won more than 1.8 million conditional dollars: during the tournament he developed powerful betting strategies and even demonstrated the possibility of bluffing.

    Sandholm is confident that this approach is applicable to many other games, as well as military simulators. Now military gaming exercises usually test only a small number of strategies for imaginary adversaries: “This opens up many opportunities for exploitation, because a real adversary may not play according to your assumptions,” says the professor.

    Sandholm refuses to discuss the specifics of Strategy Robot projects, although the company has signed at least one more government contract. He says that the program helps in imitational tasks that include making decisions in a simulated physical space, for example, where to place military units.

    The Libratus poker technique suggests that a strategic bot could give the military some amazing recommendations. Professional players who opposed the bot in the course of the tournament were surprised to find that he unexpectedly switched from tight tactics to hyper-aggressive tactics, so his actions are almost impossible to predict - but all actions of the bot are strictly aimed at winning. “It's strange, because there is no impression that he plays better than you, but then you look at the score and understand what happened,” says Sandholm.

    According to experts, such technologies can make war games and simulations more useful, although the results will still remain only one component of strategic planning. The real world is much more complicated and confusing than the scenarios that can master even the best AI technologies.

    The strategic bot Libratus is not the only attempt by the Pentagon to apply modern game theory and weak AI in military affairs. The research agency DARPA launched a special program "Sequential interaction in games with incomplete information in relation to the adoption of complex military decisions (SI3-CMD)". Michael Wellman, a professor at the University of Michigan, says that Libratus’s example shows that the technology is almost ripe for real use: “The breakthrough in poker was simply amazing, and things are fast with other games. It's time to try it in more realistic areas, ”he said.

    In addition to Strategy Robot, Professor Sandholm founded another startup Strategic Machine, where he adapted the strategic bot for commercial tasks such as electricity, sports and computer games.

    AI technologies interest the military not only in the United States, but also in Russia and China , Wired writes . Professor Sandholm does not share the concerns about this. He believes that AI "will make the world a lot safer."

    Also popular now: