August 30, 2011 at 19:19The story of one poker bot
Now the possibility of writing a poker bot is no longer in doubt (there is even a series of articles in the Hacker magazine), but in 2008 everything was not so obvious and all that could be found was one single article in English that described only general ideas, which never got accustomed to our project. But even now, few will be sharing successful stories.
In this article I will describe our experience in developing a bot that was really tested in real poker rooms and even learned to play and not lose. Those who are going to write their bot will find interesting things and ways to make their life easier in the article. Who cares - welcome to cat.
The idea of writing a bot was visited by my friend, who shared it with me and my dorm neighbor, and in fact the friend himself was also with the dorm.
The roles were divided by the following plan:
• The developer of the logic (author of the idea)
• The developer of the parser (neighbor)
• The developer of the eyes and hands (I)
It was decided to write a bot for limit hold'em (the author of the idea at that time was making some money playing this type of game)
In such a composition and with such roles, we moved into battle. As a result, we got 3 iterations of development with breaks between them in 3-4 months, which I will try to remember without forgetting more in detail.
What a time it was. Three green students began to think. We needed tools, the initial choice was this:
• C ++ - Logic of the game
• VB.Net - Everything else
But after a few hours of development, we decided that we needed to learn C # and began to use it instead of BASIC.
Unfortunately, I can’t tell anything about the brains that were in the C ++ DLL, and no one can (no one just remembers), I only remember that there was an absolute hell in which even the author got confused and sometimes wondered that Something accidentally works / does not work.
The poker room we chose for testing had an excellent chat, from which it was decided to take information to analyze what was happening. This analysis was taken up by my neighbor.
I needed to learn how to give him a chat on time, it remained to understand when it comes on time.
So, the first questions that needed to be addressed were:
• How to find the right ones among all the windows of the system?
• How to understand that it is time to walk?
• How to walk?
• How to get a chat?
More about solving problems below (there is a feeling that it was in that order that I solved them then).
I heard something about WinApi then, but what kind of animal it is and how to train it - I was not in the know. Very soon I learned that by the mysterious handle you can learn a lot about the window and do a lot of things, I also learned that the window can be well monitored. Armed with Spy ++, I plunged into the world of then still recognizing the handle by pointing a cross on the window)))
Smoking manulov allowed me to find out that you can ask for the color of a point on the desktop and that you can tell the system that the mouse clicked a button.
The solution to the problem became obvious:
1. look for points by the color of which you can find out what the buttons appeared (i.e. it's time to walk)
2. get the coordinates of the window,
3. add them to the coordinates of the point to determine
4. constantly make the window active and check the color of the point
5. If color has become necessary - it's time to walk
Already knowing how to use Spy ++, it was easy to find a handle. Having smoked again, I understood how to send Ctrl + A, Ctrl + C to the window. Next, using C #, I received the contents of the buffer. Chat is.
While determining how to understand that it was time to walk, it became clear how to walk. We calculate the coordinates of the button, put the cursor on the point, Send the left mouse button to the system. They were like.
This task turned out to be the most difficult - I could not solve it for a long time until I turned to my teacher (thanks to A. Chernosvitov) on C # / C ++ and he remembered the EnumWindowsProc function or something like that. Using Spy ++, I looked for window features, but in the end the title turned out to be such a feature, which, by the way, can also be obtained from the window handle, after which using EnumChildWindowsProc (enumeration of all child windows), a chat was found.
After the first iteration, we got a bot that played limit hold'em, knew how to go, knew how to walk. He had some logic. He could only play alone and completely blocked the computer (due to constant switching to the window before each action) on which he was tested.
A few months later, we decided to write again, to speed up the process, we decided to bring computers into one room and work together there.
During this iteration, the following was done:
• Moving into one language - we moved completely to C #.
• Transition to anlim tournaments
• A lot of tables are screwed up
• We tried using system hooks to determine what time to go
• Improved chat reception
• Improved the progress mechanism
The logic was rewritten in C # and start tables were used to play tournaments from pokerstrategy.com, which allowed us to make reasonably clean and understandable logic. There were also attempts to make a logic configurator (we wanted to sell the bot to the side so that everyone wrote their own logic), but we decided that it was a utopia and stopped developing the idea.
Together with the new transparent logic, they added the ability to play on several tables. The realization of such an opportunity was crooked - in turn, each table is in the foreground, checking the color for the need to move - if it is time to walk, then we go. However, such a scheme worked, and it was a victory. But now the bot blocked the computer completely and tightly.
By this time, I was already strong enough to start digging into the Windows system hooks. Since you can’t hang system hooks from C # to the message coming to the window, we used a project with CodeProject (http://www.codeproject.com/KB/system/WilsonSystemGlobalHooks.aspx), which consists of a DLL and C # logic that runs this DLL and receiving information from her. Hooks made it possible not to make a lot of switching between windows to determine the course, but to catch the exact order of messages, after which it was necessary to go after it, plus hooks slowed down the system and as a result refused hooks.
During the next sleepless night, I watched in Spy ++ the commands that came into the chat, and saw that certain commands corresponded to the selection of everything and copying. The code for these commands was recorded, the copy of the chat was rewritten to use the new commands, after that the copy was completely done using WinApi commands.
It turned out that you can send a message to the window, that they clicked on it with the mouse - this allowed us to rewrite the circulation to send messages directly to the click window. Now the cursor could no longer twitch around the screen. The computer gradually returned from the bot to the host =)
After this iteration, we got a bot playing transparent tournament logic on many tables in tournament hold'em (by the way, the table offered at PokerStrategy.com is losing if you follow it completely and don’t think anymore). The bot could already leave a small chance to somehow work for the host computer.
Two of us reached the third iteration, my neighbor dropped out of the project and his part of the code was rewritten from scratch (it was difficult to maintain someone else's code, especially since it was necessary to significantly develop it). During the third iteration, we achieved:
• Using the new mechanism for determining the time of the move
• Using the new mechanism for receiving chat
• Using the new logic
I still wonder why I didn’t guess before that the point color can be determined not only on the desktop, but also in the window specifically. Realizing this, I rewrote the definitions of the course in a new way. Now it was not necessary to constantly push the windows to the forefront to understand that it was time to walk. Such a mechanism works stably and crashed only in one case - when minimizing the game table to tray.
The poker room chat had the InetnetExplolerClass class or something like that. For this iteration, I found a way to extract text from such objects and VB.Net code that worked. The code was rewritten in C # and now receiving the chat was without any copying to the buffer and was more stable.
It became clear that for tournaments we need an equator and calculation of the ICM model, and we began to try to get it. The equator written by my partner could calculate 5000 games per second, while the Equiliator from PokerStrategy.com could have 1500000 on the same configuration - it was a failure and you had to do something about it.
Once, sitting at night, I went into the PokerStrategy.com software file folder (Equiliator, ICM Trainer) and found that they were written in .Net. It made me incredibly happy and the next 2 days I spent to run this software under my debug, but in the end we could fully watch how to use the libraries of this company and successfully learned how to use them.
As a result, our bot before the move considered its chances to win, and at the extreme stage of the tournaments used pure ICM, which gave it certain advantages.
As a result, we got a bot that played zero or a very easy plus at micro limits in tournaments on many tables (20 he played easily), using a lot of mathematics.
The next step will be screwing to the bot statistics on the players will lead to an improvement in the game.
Unfortunately, at the moment the development is and has been standing for a long time. We have left a lot of small jambs that can occur once in many hours and it is very difficult to debug such things.
If you have a desire to write your bot - I hope that the solutions we found allow you to write it faster.
In this article I will describe our experience in developing a bot that was really tested in real poker rooms and even learned to play and not lose. Those who are going to write their bot will find interesting things and ways to make their life easier in the article. Who cares - welcome to cat.
How it all began
The idea of writing a bot was visited by my friend, who shared it with me and my dorm neighbor, and in fact the friend himself was also with the dorm.
The roles were divided by the following plan:
• The developer of the logic (author of the idea)
• The developer of the parser (neighbor)
• The developer of the eyes and hands (I)
It was decided to write a bot for limit hold'em (the author of the idea at that time was making some money playing this type of game)
In such a composition and with such roles, we moved into battle. As a result, we got 3 iterations of development with breaks between them in 3-4 months, which I will try to remember without forgetting more in detail.
Iteration 1
What a time it was. Three green students began to think. We needed tools, the initial choice was this:
• C ++ - Logic of the game
• VB.Net - Everything else
But after a few hours of development, we decided that we needed to learn C # and began to use it instead of BASIC.
Unfortunately, I can’t tell anything about the brains that were in the C ++ DLL, and no one can (no one just remembers), I only remember that there was an absolute hell in which even the author got confused and sometimes wondered that Something accidentally works / does not work.
The poker room we chose for testing had an excellent chat, from which it was decided to take information to analyze what was happening. This analysis was taken up by my neighbor.
I needed to learn how to give him a chat on time, it remained to understand when it comes on time.
So, the first questions that needed to be addressed were:
• How to find the right ones among all the windows of the system?
• How to understand that it is time to walk?
• How to walk?
• How to get a chat?
More about solving problems below (there is a feeling that it was in that order that I solved them then).
How to understand that it’s time to walk?
I heard something about WinApi then, but what kind of animal it is and how to train it - I was not in the know. Very soon I learned that by the mysterious handle you can learn a lot about the window and do a lot of things, I also learned that the window can be well monitored. Armed with Spy ++, I plunged into the world of then still recognizing the handle by pointing a cross on the window)))
Smoking manulov allowed me to find out that you can ask for the color of a point on the desktop and that you can tell the system that the mouse clicked a button.
The solution to the problem became obvious:
1. look for points by the color of which you can find out what the buttons appeared (i.e. it's time to walk)
2. get the coordinates of the window,
3. add them to the coordinates of the point to determine
4. constantly make the window active and check the color of the point
5. If color has become necessary - it's time to walk
How to get a chat?
Already knowing how to use Spy ++, it was easy to find a handle. Having smoked again, I understood how to send Ctrl + A, Ctrl + C to the window. Next, using C #, I received the contents of the buffer. Chat is.
How to walk
While determining how to understand that it was time to walk, it became clear how to walk. We calculate the coordinates of the button, put the cursor on the point, Send the left mouse button to the system. They were like.
How to find the systems you need among all the windows?
This task turned out to be the most difficult - I could not solve it for a long time until I turned to my teacher (thanks to A. Chernosvitov) on C # / C ++ and he remembered the EnumWindowsProc function or something like that. Using Spy ++, I looked for window features, but in the end the title turned out to be such a feature, which, by the way, can also be obtained from the window handle, after which using EnumChildWindowsProc (enumeration of all child windows), a chat was found.
Iteration Results 1
After the first iteration, we got a bot that played limit hold'em, knew how to go, knew how to walk. He had some logic. He could only play alone and completely blocked the computer (due to constant switching to the window before each action) on which he was tested.
Iteration 2
A few months later, we decided to write again, to speed up the process, we decided to bring computers into one room and work together there.
During this iteration, the following was done:
• Moving into one language - we moved completely to C #.
• Transition to anlim tournaments
• A lot of tables are screwed up
• We tried using system hooks to determine what time to go
• Improved chat reception
• Improved the progress mechanism
Transition to anlim tournaments
The logic was rewritten in C # and start tables were used to play tournaments from pokerstrategy.com, which allowed us to make reasonably clean and understandable logic. There were also attempts to make a logic configurator (we wanted to sell the bot to the side so that everyone wrote their own logic), but we decided that it was a utopia and stopped developing the idea.
Many tables screwed
Together with the new transparent logic, they added the ability to play on several tables. The realization of such an opportunity was crooked - in turn, each table is in the foreground, checking the color for the need to move - if it is time to walk, then we go. However, such a scheme worked, and it was a victory. But now the bot blocked the computer completely and tightly.
Tried using system hooks to determine if it's time to walk
By this time, I was already strong enough to start digging into the Windows system hooks. Since you can’t hang system hooks from C # to the message coming to the window, we used a project with CodeProject (http://www.codeproject.com/KB/system/WilsonSystemGlobalHooks.aspx), which consists of a DLL and C # logic that runs this DLL and receiving information from her. Hooks made it possible not to make a lot of switching between windows to determine the course, but to catch the exact order of messages, after which it was necessary to go after it, plus hooks slowed down the system and as a result refused hooks.
Improved chat reception
During the next sleepless night, I watched in Spy ++ the commands that came into the chat, and saw that certain commands corresponded to the selection of everything and copying. The code for these commands was recorded, the copy of the chat was rewritten to use the new commands, after that the copy was completely done using WinApi commands.
Improved move mechanism
It turned out that you can send a message to the window, that they clicked on it with the mouse - this allowed us to rewrite the circulation to send messages directly to the click window. Now the cursor could no longer twitch around the screen. The computer gradually returned from the bot to the host =)
Iteration 2 Results
After this iteration, we got a bot playing transparent tournament logic on many tables in tournament hold'em (by the way, the table offered at PokerStrategy.com is losing if you follow it completely and don’t think anymore). The bot could already leave a small chance to somehow work for the host computer.
Iteration 3
Two of us reached the third iteration, my neighbor dropped out of the project and his part of the code was rewritten from scratch (it was difficult to maintain someone else's code, especially since it was necessary to significantly develop it). During the third iteration, we achieved:
• Using the new mechanism for determining the time of the move
• Using the new mechanism for receiving chat
• Using the new logic
Using the new mechanism for determining the time
I still wonder why I didn’t guess before that the point color can be determined not only on the desktop, but also in the window specifically. Realizing this, I rewrote the definitions of the course in a new way. Now it was not necessary to constantly push the windows to the forefront to understand that it was time to walk. Such a mechanism works stably and crashed only in one case - when minimizing the game table to tray.
Using the new chat engine
The poker room chat had the InetnetExplolerClass class or something like that. For this iteration, I found a way to extract text from such objects and VB.Net code that worked. The code was rewritten in C # and now receiving the chat was without any copying to the buffer and was more stable.
Using new logic
It became clear that for tournaments we need an equator and calculation of the ICM model, and we began to try to get it. The equator written by my partner could calculate 5000 games per second, while the Equiliator from PokerStrategy.com could have 1500000 on the same configuration - it was a failure and you had to do something about it.
Once, sitting at night, I went into the PokerStrategy.com software file folder (Equiliator, ICM Trainer) and found that they were written in .Net. It made me incredibly happy and the next 2 days I spent to run this software under my debug, but in the end we could fully watch how to use the libraries of this company and successfully learned how to use them.
As a result, our bot before the move considered its chances to win, and at the extreme stage of the tournaments used pure ICM, which gave it certain advantages.
Iteration 3 Results
As a result, we got a bot that played zero or a very easy plus at micro limits in tournaments on many tables (20 he played easily), using a lot of mathematics.
Summary
The next step will be screwing to the bot statistics on the players will lead to an improvement in the game.
Unfortunately, at the moment the development is and has been standing for a long time. We have left a lot of small jambs that can occur once in many hours and it is very difficult to debug such things.
If you have a desire to write your bot - I hope that the solutions we found allow you to write it faster.