My experience of dating and working with the Robot Framework
Just over a year ago I tried Robot Framework for the first time. During my participation in a fairly large-scale project, I have experienced, on my own skin, two different approaches to test automation with this tool: writing tests on a pure DSL Robot Framework and working in conjunction with Python. If the first path has a low entry threshold, then the second, in my opinion, is more convenient in terms of supporting large projects. Although there is no fundamental difference between the approaches. Anyway, it all comes down to finding libraries.
However, the features of the approaches should be discussed.
Perhaps we should start with the presentation of this powerful tool.
Robot Framework is a framework for keyword-driven testing. It is used to automate acceptance testing and ATDD (development approach through acceptance testing). This system has easy-to-use test data syntax and allows you to automate tests using keywords. In addition, the Robot Framework has excellent built-in and third-party libraries that allow you to quickly begin and integrate automation into workflows without the invention of your own bikes - the very low entry threshold I mentioned above.
The main structure of the Robot Framework is implemented using Python and also works on Jython (JVM) and IronPython (.NET).
Examples of use, as well as full documentation on the framework and internal libraries can be found on the official project website: http://robotframework.org/ .
I first encountered the Robot Framework a year ago, after changing jobs. Before that, I had to automate only in Java and C #.
Choosing for myself a toolkit with which I would have to understand existing tests, I asked new colleagues about their preferences. They did not have a consensus about the best IDE for working with the Robot Framework. Mostly work with the Robot Framework allow plugins for various text editors and IDE, such as PyCharm. And the recommendations I collected divided 50/50 between Atom and PyCharm. Of course, there is RIDE, but this is not a panacea. At that time (a year ago) I did not find normal documentation for it, and in the one I found, I didn’t see any big advantages for my task. Therefore, for the beginning I decided to try Atom with plugins. Cloning the repository, I began to slowly study what happens in our tests and in the Robot Framework itself.
I was hooked to the project for almost everything. A bunch of Jython + Robot Framework was already working there, a huge codebase (written on the DSL Robot Framework itself) was assembled from more than 1,000 tests and several thousand lines of code of auxiliary libraries in Java.
As I understand it, when the project started, i.e. even before my arrival at the department, Java specialists mainly worked on it, and the product itself was tested in Java, therefore, choosing an approach, they were guided by the available resources. In general, the calculation was something like this: having solved some problems related to the integration of Robot Framework and Java (mainly due to the fact that Java is a compiled language, and the same Python and tests in the Python + RF bundle are interpretable), it was possible later It is easy to attract third-party professionals who know only the DSL Robot Framework, and quietly write tests on keywords. True, the colleagues had to spend quite a lot of effort in creating libraries in Java, so without the conditions on the part of the customer, they would not recommend such a way.
Having done a little refactoring, as part of my first task, I launched tests for the first time. Since the Jython + RF bundle was used, everything was going to be maven, and the robot files were simply copied to the target directory for subsequent execution.
Tests were launched by scripts (.bat or .sh files) that took the path either to a separate test case (a separate .robot file) or to a test plan (a file with a list of relative paths to test cases).
Refactoring touched on a large number of tests, so the first run took 15 minutes. After its completion, it was time to look at the reports provided by the Robot Framework.
The standard report (in the screenshot above) consists of the report.html and log.html files:
Frankly speaking, at first glance at the report, the Robot Framework starts to twitch a little: a huge amount of information is displayed and it takes some time to understand the through-structure of tests and develop the skill of reading such a log. But this thing is not so tricky. After a couple of months I could quote “The Matrix”: “Your brain makes the translation itself. I don’t even see the code. I see a blonde, brunette and red. " So I - saw all the necessary information in the file without additional tools.
The advantage is that the output can be controlled: there are different levels of logging that determine which information will be displayed and which one will not. The level can be adjusted at least for each line separately through the method of the built-in library. At the same time, knowledge of the order of making calls within the test and auxiliary libraries is not superfluous - it is easier to catch the moment of error.
Using the DSL Robot Framework as our primary tool, we worked for about six months. During this time, I switched from Atom to VSCode from personal preferences, but this did not change the essence of the approach.
However, the project has evolved. In the final iteration, the auxiliary library for working with databases consisted of 6,700 lines of code on a pure Robot Framework. With such a codebase scale, it became difficult to maintain it — refactoring required resources that were not allocated.
The final word in the application of the first approach belonged to the business. The customer of our project conducted work with other teams on related tasks. In one of the parallel tracks, he saw, from his point of view, the more effective and vivid for management version of the use of the Robot Framework, which they began to implement.
The second approach was the development of tests in Python in conjunction with the Robot Framework. Instead of creating everything on the DSL Robot Framework syntax, we started writing helper libraries and other low-level interactions with the tested product in Python. And the Robot Framework, in fact, became just a runner.
Since Python is a pure high-level language, not DSL, there are more options for structuring, it is easier to understand all this. At a minimum, you can use an IDE for Python, which will help you find the same methods (they do the same thing, but are called differently), or even write a part of the code for you. Some data could be wrapped in generators, decorators for functions, etc. Against this background, the toolkit of the first approach (pure Robot Framework) looks rather stern - in fact, it was a Notepad with syntax highlighting. No setters or getters that IntelliJ writes for you. So I was glad to return to the high-level language. Working with this approach is more like a regular development. True, there is a fly in the ointment. Without additional dancing, it is impossible to understand what fell inside Python, called from RF.
The first tests and auxiliary libraries for our subsystem began to be written slowly. During the time that I happened to work with the new approach, I felt that there were indeed more possibilities with another tool. But as a matter of fact, the writing of tests did not change much. In this particular project, inside the Python-code, the methods of the built-in libraries of the Robot Framework were still called. This was due to the peculiarity of customer requirements for test development. We simply separated the executable part from the test case algorithm.
Personally, I like the second approach. However, choosing the path of your project, it is worth starting from the task - who writes the tests and how.
As I said above, Python (the second approach) gives you more options, but in this case people who are familiar with this language are needed on the project. By itself, the Robot Framework (and the first approach) is less demanding - you can approach it by reading the official documentation on its DSL. After studying the user guide, you can create any tests - they will look pretty clean.
As a result, the Robot Framework and the way we used it at first will be more suitable for yesterday’s manual testers without explicit programming experience in high-level languages. Even a person who is not heavily involved in programming can simply write a test, simply by triggering the necessary keywords.
However, if you want to follow the example of our project, keep the executable part separate, and at the same time refactor the code in friendly IDEs, the second approach is for you.
The author of the article: Dmitry Masters
PS: We publish our articles on several sites of the Runet. Subscribe to our pages on the VK , FB or Telegram channel to find out about all our publications and other news from Maxilect.
However, the features of the approaches should be discussed.
Who is Robot and what does it eat?
Perhaps we should start with the presentation of this powerful tool.
Robot Framework is a framework for keyword-driven testing. It is used to automate acceptance testing and ATDD (development approach through acceptance testing). This system has easy-to-use test data syntax and allows you to automate tests using keywords. In addition, the Robot Framework has excellent built-in and third-party libraries that allow you to quickly begin and integrate automation into workflows without the invention of your own bikes - the very low entry threshold I mentioned above.
The main structure of the Robot Framework is implemented using Python and also works on Jython (JVM) and IronPython (.NET).
Examples of use, as well as full documentation on the framework and internal libraries can be found on the official project website: http://robotframework.org/ .
My first steps. First approach
I first encountered the Robot Framework a year ago, after changing jobs. Before that, I had to automate only in Java and C #.
Choosing for myself a toolkit with which I would have to understand existing tests, I asked new colleagues about their preferences. They did not have a consensus about the best IDE for working with the Robot Framework. Mostly work with the Robot Framework allow plugins for various text editors and IDE, such as PyCharm. And the recommendations I collected divided 50/50 between Atom and PyCharm. Of course, there is RIDE, but this is not a panacea. At that time (a year ago) I did not find normal documentation for it, and in the one I found, I didn’t see any big advantages for my task. Therefore, for the beginning I decided to try Atom with plugins. Cloning the repository, I began to slowly study what happens in our tests and in the Robot Framework itself.
I was hooked to the project for almost everything. A bunch of Jython + Robot Framework was already working there, a huge codebase (written on the DSL Robot Framework itself) was assembled from more than 1,000 tests and several thousand lines of code of auxiliary libraries in Java.
As I understand it, when the project started, i.e. even before my arrival at the department, Java specialists mainly worked on it, and the product itself was tested in Java, therefore, choosing an approach, they were guided by the available resources. In general, the calculation was something like this: having solved some problems related to the integration of Robot Framework and Java (mainly due to the fact that Java is a compiled language, and the same Python and tests in the Python + RF bundle are interpretable), it was possible later It is easy to attract third-party professionals who know only the DSL Robot Framework, and quietly write tests on keywords. True, the colleagues had to spend quite a lot of effort in creating libraries in Java, so without the conditions on the part of the customer, they would not recommend such a way.
Having done a little refactoring, as part of my first task, I launched tests for the first time. Since the Jython + RF bundle was used, everything was going to be maven, and the robot files were simply copied to the target directory for subsequent execution.
Tests were launched by scripts (.bat or .sh files) that took the path either to a separate test case (a separate .robot file) or to a test plan (a file with a list of relative paths to test cases).
Refactoring touched on a large number of tests, so the first run took 15 minutes. After its completion, it was time to look at the reports provided by the Robot Framework.
The standard report (in the screenshot above) consists of the report.html and log.html files:
- report contains a general summary of the last run, where you can see the superficial results of all tests (Passed or Failed);
- In the log file you can see more detailed information - the execution of each test step by step. You can also display everything you need for debugging tests.
Frankly speaking, at first glance at the report, the Robot Framework starts to twitch a little: a huge amount of information is displayed and it takes some time to understand the through-structure of tests and develop the skill of reading such a log. But this thing is not so tricky. After a couple of months I could quote “The Matrix”: “Your brain makes the translation itself. I don’t even see the code. I see a blonde, brunette and red. " So I - saw all the necessary information in the file without additional tools.
The advantage is that the output can be controlled: there are different levels of logging that determine which information will be displayed and which one will not. The level can be adjusted at least for each line separately through the method of the built-in library. At the same time, knowledge of the order of making calls within the test and auxiliary libraries is not superfluous - it is easier to catch the moment of error.
Using the DSL Robot Framework as our primary tool, we worked for about six months. During this time, I switched from Atom to VSCode from personal preferences, but this did not change the essence of the approach.
However, the project has evolved. In the final iteration, the auxiliary library for working with databases consisted of 6,700 lines of code on a pure Robot Framework. With such a codebase scale, it became difficult to maintain it — refactoring required resources that were not allocated.
The final word in the application of the first approach belonged to the business. The customer of our project conducted work with other teams on related tasks. In one of the parallel tracks, he saw, from his point of view, the more effective and vivid for management version of the use of the Robot Framework, which they began to implement.
Second approach
The second approach was the development of tests in Python in conjunction with the Robot Framework. Instead of creating everything on the DSL Robot Framework syntax, we started writing helper libraries and other low-level interactions with the tested product in Python. And the Robot Framework, in fact, became just a runner.
Since Python is a pure high-level language, not DSL, there are more options for structuring, it is easier to understand all this. At a minimum, you can use an IDE for Python, which will help you find the same methods (they do the same thing, but are called differently), or even write a part of the code for you. Some data could be wrapped in generators, decorators for functions, etc. Against this background, the toolkit of the first approach (pure Robot Framework) looks rather stern - in fact, it was a Notepad with syntax highlighting. No setters or getters that IntelliJ writes for you. So I was glad to return to the high-level language. Working with this approach is more like a regular development. True, there is a fly in the ointment. Without additional dancing, it is impossible to understand what fell inside Python, called from RF.
The first tests and auxiliary libraries for our subsystem began to be written slowly. During the time that I happened to work with the new approach, I felt that there were indeed more possibilities with another tool. But as a matter of fact, the writing of tests did not change much. In this particular project, inside the Python-code, the methods of the built-in libraries of the Robot Framework were still called. This was due to the peculiarity of customer requirements for test development. We simply separated the executable part from the test case algorithm.
What is better?
Personally, I like the second approach. However, choosing the path of your project, it is worth starting from the task - who writes the tests and how.
As I said above, Python (the second approach) gives you more options, but in this case people who are familiar with this language are needed on the project. By itself, the Robot Framework (and the first approach) is less demanding - you can approach it by reading the official documentation on its DSL. After studying the user guide, you can create any tests - they will look pretty clean.
As a result, the Robot Framework and the way we used it at first will be more suitable for yesterday’s manual testers without explicit programming experience in high-level languages. Even a person who is not heavily involved in programming can simply write a test, simply by triggering the necessary keywords.
However, if you want to follow the example of our project, keep the executable part separate, and at the same time refactor the code in friendly IDEs, the second approach is for you.
The author of the article: Dmitry Masters
PS: We publish our articles on several sites of the Runet. Subscribe to our pages on the VK , FB or Telegram channel to find out about all our publications and other news from Maxilect.