December 25, 2012 at 17:52

Read more about Sikuli in test automation

Introduction


Sikuli is an API that allows you to write automation scripts in Jython based on the visual component of any program / site, etc. Especially nice for Flash automation.
Few articles have been written about Sikuli, and most of them are overview articles. Even less Russian-language help, and even less code examples. And the lack of the latter is perhaps the most tragic for a software tester who faced the need to automate a flash in work. This is exactly what prompted me to write a more capacious article on Sikuli and to describe in some detail some features of use.


Installation


For installation and proper operation of Sikuli you will need:
1. Java of the sixth version and necessarily 32-bit.
2. Distribution
3. Service Pack.

Due to the fact that Sikuli is only available in 32-bit configuration, for correct operation on 64-bit systems we must install Java 6u38 32x (it does not work with version 7).
It is also necessary to put the service pack. Easy to install. It simply copies the contents of the “Sikuli-IDE” folder to the folder with the installed program “C: \ Program Files (x86) \ Sikuli X” from the archive (step 3), by the way, do not change the default installation folder.
If suddenly something is wrong, here is a link to read.

Functionality and features.


Screenshotimage
It is used to select a specific area of ​​the screen and then create a pattern. image
When you click on a pattern, the “Template Settings” window appears. image
With tabs, everything is primitive and clear. The only thing I note is that on the “target offset” tab, the goal is the place where the cursor moves and makes a click / enter. Moreover, the place of the click can lie far beyond the limits of the pattern.

Now about the features.
*****. png is the name of the desired image that should be in the folder of the previously saved project. image
similar = similarity, where 1 = 100% (it’s better not to use it because it is not always found), for example, 0.85 (through a point) = 85%, which in my opinion is the most optimal value.
targetOffset (x, y)) and here it is important that the coordinates “x” and “y” are the offset of the target from the center of the pattern.

Highlighting the area / wait () / Click () An

image
excellent function and, most importantly, it is extremely useful.
Many say that Sikuli does not think well at work and is still mistaken. I fundamentally disagree with this and I think that a competent approach significantly increases the performance of the program compared to similar IDEs.

This function sets the active area of ​​the screen.
image
in which the search for the desired pattern will be performed.
You can also like this:
Region(x,y,w,h).find(Pattern("*****.png".similar(0.85)targetOffset(x,y)))

It is extremely useful if we need the script to respond to changes quickly, not to slow down the system.
As an example:
from our entire large screen we select only a small area where the treasured button appears each time. image
Next, we describe what we need to look for in this area and what to do next.
image
In order to optimize the process, we need to run this script and so that it clicks for us 1 time. After we go to the “Message” tab where we see the coordinates of the click.
image
Next, do a loop and click on the coordinates (this is provided that the button will always appear in the same place).
We do about 70 percent similarity in the search, find it faster, wait for 999 ...
image
As a result, we have an almost instantaneous response.


Find ()

The find () function searches for a pattern. But why? Everything is simple. Once found, and then he will know where the found object is always located. How to put it into practice:
We have a lot of something… image
And it’s necessary to press the “anything” program for a long time and hard ...
In order for the program to know where to press, set targetOffset ().
image
Repeat for each goal, assign variables, optimize if the field with a lot is in the same place.
deploy here
image

Further we can use the found zones as we want. Moreover, the click will occur without delays in the search, etc.
image

I will not consider the rest of the functionality; it is primitive and simple.

Examples


The script should be sane!
image

Creating the yan () function. The first call to the function will call find (), subsequent calls will click ().
image

Through the dialog box, set the number of cycles.
image

try / except function and how good it is.
image

And finally, a bot for sending gifts to friends (of which there may be 5k) in the DOTD game
Settings.MoveMouseDelay = 0.02
def f5():
    while "f5":
        click(Pattern("KyddinsmapsD-1.png").targetOffset(-41,0))
        sleep(5)
        if exists(Pattern("1344731699750.png").similar(0.90),10):
            continue
        else:
            pass
        try:
            wait(Pattern("CLOSElah.png").similar(0.85),25)
        except:
            continue
        try:
            click(Pattern("CLOSElah.png").similar(0.85).targetOffset(1,-6))
            dragDrop(Location(1430,388), Location(1431,435)) # 435
            break
        except:
            pass
def send_frend():
    click(Location(849,495))
    sleep(1)
    click(Location(578,695))
    sleep(1)
def pre_click_frend():
    click(Location(713,300))
    click(Location(713,315))
def click_frend():
    click(Location(713,332))
    click(Location(714,348))
    click(Location(714,364))
    click(Location(715,380))
    click(Location(714,396))
    click(Location(713,412))
    click(Location(713,427))
    click(Location(713,444))
    click(Location(713,460))
    click(Location(713,474))
    click(Location(713,490))
    click(Location(715,506))
    click(Location(714,522))
    click(Location(714,537))
    click(Location(713,554))
    click(Location(923,541)), click(Location(923,541)), click(Location(923,541)), click(Location(923,541)), click(Location(923,541)), click(Location(923,541)), 
    click(Location(923,541)), click(Location(923,541)), click(Location(923,541)), click(Location(923,541)), click(Location(923,541)), click(Location(923,541)), 
    click(Location(923,541)), click(Location(923,541))
    sleep(1)
    click(Location(713,288))
    click(Location(713,304))
    click(Location(713,320))
    click(Location(713,336))
    click(Location(713,352))
    click(Location(713,368))
    click(Location(713,384))
    click(Location(713,400))
    click(Location(713,416))
    click(Location(713,432))
    click(Location(713,448))
    click(Location(713,464))
    click(Location(713,480))
    click(Location(713,496))
    click(Location(713,512))
    click(Location(713,528))
    click(Location(713,544))
    click(Location(923,541)), click(Location(923,541)), click(Location(923,541)), click(Location(923,541)), click(Location(923,541)), click(Location(923,541)), 
    click(Location(923,541)), click(Location(923,541)), click(Location(923,541)), click(Location(923,541)), click(Location(923,541)), click(Location(923,541)), 
    click(Location(923,541))
    sleep(1)
    click(Location(713,300)),
    click(Location(713,315))
    click(Location(713,332))
    click(Location(714,348))
    click(Location(714,364))
    click(Location(715,380))
    click(Location(714,396))
    click(Location(713,412))
    click(Location(713,427))
    click(Location(713,444))
    click(Location(713,460))
    click(Location(713,474))
    click(Location(713,490))
    click(Location(715,506))
    click(Location(714,522))
    click(Location(714,537))
    sleep(1)
def send_fb():
    click(Location(756,760))
    sleep(2)
    try:
        click(Pattern("0TIIp3BHTb33.png").targetOffset(-2,1))
    except:
        pass
def off (event):
    popup("The program is completed.\nCreator of this miracle: Grumo Van Blum.\nIf you have questions about using this program, please contact us by e-mail: grumovanblum@gmail.com\nGood Luck 8c)")
    exit()
popup("If you want to stop the script, press: 'Ctrl' + 'Alt' + 'Space'")
Env.addHotkey(Key.SPACE, KeyModifier.ALT+KeyModifier.CTRL, off)
click(Pattern("1344836792562.png").similar(0.88))
n = 0
n = int(input("How many cycles you want to run?"))
while n > 0:
    f5()
    while "cycle":
        if n == 0:
            break
        else:
            send_frend()
            if exists(Pattern("Jd.png").similar(0.96),1):
                popup("Friends come to an end :)")
                n=0
                continue
            else:
                pre_click_frend()
                sleep(1)
                if exists(Pattern("Youcanontysd.png").similar(0.95),1):
                    f5()
                    send_frend()
                    pre_click_frend()
                else:
                    pass
            click_frend()
            send_fb()
            n -= 1
            if n == 0:
                break
            else:
                sleep(3)

And a video about how it works (shot in debug mode).


Utilities Used


Alt + Shift + c will stop script execution and return an IDE.

Set the cursor movement time:
Settings.MoveMouseDelay = 0.02

We assign any events to the keyboard shortcuts:
def off (event):
    exit()
Env.addHotkey(Key.SPACE, KeyModifier.ALT+KeyModifier.CTRL, off)

Dialog box invocation (integer):
n = int(input("А сколько раз будем проверять?"))

Inaction in 1 sec:
sleep(1)

If, after pasting the copied text, the code of the pattern or region is replaced by a picture, just press 1 or 2 times the key combination Ctrl + z.
image

This is far from all and will certainly be more. But at this stage I think that I expressed the main idea.
I will be glad to feedback.
If you have questions about Sikuli, then write.

Next time I’ll probably tell you how to teach Sikuli to recognize text and write a log of errors to a separate file.

Thanks for attention.
Tags:

Also popular now: