Writing a bot for playing Balls 2.0

Recently I came across a simple toy where you need to shoot the ball into groups of the same color. Although I rarely play games, I sat with her for about 30 minutes.
I wanted to automate this process. Knowledge for the game is not required, but there are a lot of games.
I describe the process of writing a bot for this game.



The idea was as follows:
Step 1. get a picture from the screen
Step 2. transfer the colors of the balls into the matrix
Step 3. calculate the position and click with the mouse to complete the move

To write the program, I hesitated a bit between Qt and Delphi, but as I wanted to do everything quickly I decided to do it on Delphi.

Now you can implement Step 1.
To get a picture, we set the approximate position of the window with the game and copy this section to TImage, previously located on the form.

procedure MakeFrame();
var
  bmp:TBitmap;
  rect:TRect;
begin
    rect.Left := 50;
    rect.Top:= 150;
    rect.Right:=550; // Screen.Width
    rect.Bottom:=550;    // Screen.Height
    bmp := TBitmap.Create;
    bmp.Width := Screen.Width;
    bmp.Height := Screen.Height;
    BitBlt(bmp.Canvas.Handle, 0, 0, rect.Right, rect.Bottom,
      GetDC(0), rect.Left, rect.Top, SRCCOPY);
    Form1.Image1.Width := rect.Right;
    Form1.Image1.Height := rect.Bottom;
    Form1.Image1.Picture.Assign(bmp);
    bmp.Free;
end;


If you put this code in the timer, moving the window with the game, you can position it so that the image of the game will be copied.

Step 2. You need to know the colors of the balls. As you can see there are only 3 options, straight RGB.
The height between the rows and the diameters of the balls are constant, so we can compose a grid of the position of the balls.

for I:=0 to rows do
  begin
    offset:=10; // сдвиг
    if (I mod 2) <> 0 then offset:=offset+(36 div 2);
    for J:=0 to cols do
    begin
      mat[i,j].Y := 10+Round(I*36); // позиция шарика
      mat[i,j].X := offset+Round(J*36);
      // пулятель (было не охото создавать ещё переменную и повторять код)
      if (I=rows) and (J=cols) then begin
        mat[i,j].Y := 519;
        mat[i,j].X := 262;
      end;
		// обработка полученного пикселя
    end;
  end;


To find out the color of a ball, compare its pixel with the threshold value of each of the RGB colors.

      rgb:=Form1.Image1.Canvas.Pixels[ mat[i,j].X, mat[i,j].Y-4 ];
      mat[i,j].color := 0;
      mat[i,j].summa := 0;
      if GetRValue(rgb) > 230 then mat[i,j].color:=1;
      if GetGValue(rgb) > 230 then mat[i,j].color:=2;
      if GetBValue(rgb) > 230 then mat[i,j].color:=3;


And to make sure everything is in order, draw a small rectangle with the resulting color on the ball
      if mat[i,j].color = 0 then  Form1.Image1.Canvas.Brush.Color := clWhite;
      if mat[i,j].color = 1 then  Form1.Image1.Canvas.Brush.Color := clRed;
      if mat[i,j].color = 2 then  Form1.Image1.Canvas.Brush.Color := clGreen;
      if mat[i,j].color = 3 then  Form1.Image1.Canvas.Brush.Color := clBlue;
      Form1.Image1.Canvas.Rectangle(
        mat[i,j].X-5, mat[i,j].Y-5, mat[i,j].X+5, mat[i,j].Y+5
      );


It turns out such a pretty picture.


This completes the work with the grid and begins work with the resulting matrix.

Step 3.
We check all the balls for the possibility of firing at it, for this we need to check that we will not encounter another ball on the way to it.
Verification occurs through nested cycles (by balls and for each we check the others)
The verification condition itself is obtained with a system of three inequalities.

{A = 1, B = (x0-y0) / (y1-y0), C = -x0-By0}
| Ax3 + By3 + C | / sqrt (A ^ 2 + b ^ 2) <= radius

  /// теперь найдём шар куда пулять
  for I:=0 to rows-1 do
  for J:=0 to cols do
  begin
    if (mat[I,J].color = 0) and (I <> 0) then continue;
    mat[i,j].allow:=1;
     // куда хотим пулять
    LineMaxX:=mat[I,J].X;
    LineMaxY:=mat[I,J].Y;
    // откуда хотим пулять
    LineMinX:=mat[rows,cols].X;
    LineMinY:=mat[rows,cols].Y;
    mat[I,J].dist := sqrt(
    sqr(mat[I,J].X-mat[rows,cols].X)+
    sqr(mat[I,J].Y-mat[rows,cols].Y));
    for II:=I+1 to rows-1 do
    for JJ:=0 to cols do
    begin
      if mat[II,JJ].color = 0 then continue; // если шарика нет
      LineMiddleX:=mat[II,JJ].X;
      LineMiddleY:=mat[II,JJ].Y;
			// не множко не красивый код но переписывалось 
			// с решенных уравнений на бумажке
      ka:=1;
      kb:=(LineMinX-LineMaxX)/(LineMaxY-LineMinY);
      kc:=-LineMinX-kb*LineMinY;
      kz:=
      abs(ka*LineMiddleX + kb*LineMiddleY+kc)/
      sqrt(sqr(ka)+sqr(kb));
      if kz < 39 then mat[i,j].allow:=0;
      //Form1.Memo1.Lines.Add(FloatToStr(kz));
    end;


For clarity, draw a line to the balls to which you can bullet.

if mat[i,j].allow = 1 then begin
Form1.Image1.Canvas.Pen.Width:= 2;
Form1.Image1.Canvas.Pen.Color:= clWhite;
Form1.Image1.Canvas.MoveTo(Round(LineMinX),Round(LineMinY));
Form1.Image1.Canvas.LineTo(Round(LineMaxX),Round(LineMaxY));
end;


It is necessary to choose a ball with the same color from the allowed ones.

  for I:=0 to rows-1 do
  for J:=0 to cols do
  begin
   if mat[i,j].allow = 1 then
   begin
    Form1.Image1.Canvas.Pen.Width:= 2;
    Form1.Image1.Canvas.Pen.Color:= clWhite;
    Form1.Image1.Canvas.MoveTo(Round(LineMinX),Round(LineMinY));
    Form1.Image1.Canvas.LineTo(Round(LineMaxX),Round(LineMaxY)); // линия
    if mat[i,j].color=mat[rows,cols].color // если цвета совпали
    then begin
      MouseX:=mat[i,j].X+50;
      MouseY:=mat[i,j].Y+250; // шарик + смещение по экрану
      break;
    end;
  end;
  end;


And make a mouse click on the screen through WinApi

  if Form1.CheckBox2.Checked then
  begin
    GetCursorPos(MouseL);
    SetCursorPos(MouseX,MouseY);
    mouse_event(MOUSEEVENTF_LEFTDOWN,MouseX,MouseY,0,0);// - нажать левой кнопки
    mouse_event(MOUSEEVENTF_LEFTUP,MouseX,MouseY,0,0);// - отпустить левую кнопку
    SetCursorPos(MouseL.X,MouseL.Y);
  end;


It remains to do everything in the timer and go drink tea.

Program video


You can improve the algorithm, for example, find groups with a large number of balls or look for lines that cut off parts of the field.

Function names and variables may not be named nicely, but this can be fixed.
If you need the source , then they are here .

Also popular now: