HeisenBug through the eyes of SberTech employee
In this post I want to share an overview of 15 reports from the Heisenbug conference, tell you what interesting things were on the stands of companies, and also share video material from Artem Eroshenko's report on creating actions plug-ins for IntelliJ IDEA, which will help you quickly change the code of the test project.
Heisenbug is a technical conference for testing professionals. Basically, it is interesting to testers, software development engineers in test, specialists in automated and load testing. The organizers are the JUG.RU team , behind which are conferences such as Joker, HolyJS, SmartData, DevOops, DotNext, Mobius.
The fifth conference was held at the Slavyanskaya Radisson Hotel, Kievskaya Metro, Moscow. When approaching the hotel, an electronic banner with the conference logo was visible. Further, at the entrance to the hotel lobby, the participant was accompanied by signs to the check-in counters and the wardrobe. It's impossible to get lost. Registration of participants and speakers was held on the ground floor, in the main conference hall there were partners' booths, conference rooms, rooms with a coffee break and lunches. The level of the guards pleased me, so that the "hawkless hares" could not be reached. In total, the event was attended by about 500+ participants, most of whom registered two weeks before the start.
The program can be found here . I want to note that JUG.RU always strives to make the program practical, without any excess water, and with well-known foreign experts. The reports are divided into conventions:
If you are planning to become a speaker at Heisenbag for the first time, look at the speaker’s memo sheet — it contains quite a lot of useful information.
To prevent participants from missing something important at the conference, a telegram channel and a telegram chat were organized. By the way, in the latter they were looking for volunteers to watch videos, for which they gave a free ticket to the conference.
If you decide to become a volunteer, label your decision to the organizers as soon as possible. Preparation for the event is carried out long before the conference. Subject to availability, you will be recorded in the team.
This time, more than a dozen stands of major IT companies, such as Deutsche Bank, Alfa Bank, Raiffeisen, Badoo, Luxoft, SKB Kontur, Gett, etc. were presented at the conference. Each of the companies approached the organization of interactions with the participants therefore, it was not necessary to be bored in the intervals between performances. It was possible to solve tasks for a small souvenir, play video and board games, participate in a lottery.
Raiffeisen offered to go online with a quest to solve all sorts of puzzles, and lovers of timeless classics could play Mortal Kombat 3 on Sega. Colleagues from Alfa-Bank made a telegram-bot with tasks, plus in the lottery played a large set of Lego Mindstorms. I want to praise the stand of Deutsche Bank, where the developers in live communication issued tasks and took part in the discussion, unlike all other stands, where they only checked the correctness of the answers.
Many companies are unobtrusively engaged in hunting specialists, handing out booklets describing vacancies (not everyone is involved in charity work).
1. We have DevOps. Let's dismiss all testers - Baruh Sadogursky.
The conference was opened by Baruh Sadogursky, developer advocate from JFrog, a well-known speaker at conferences related to Java and Devops. In the report, Baruch touched upon software quality problems in the context of frequent releases, which are caused by the ever-growing demands of the business, the complexity of code testing due to poor architecture, the modernization of the role of the tester in modern Agile teams. The report was interesting to listen from beginning to end, a lot of humor, interesting facts and comparisons.
Well, the main ideas of the report can be put in one slide, which illustrates that the automation of testing and the practice of DevOps reduces time costs for routine processes and allows you to devote more time to implement new tasks.
2. Do I need to refactor a project? There is an IDEA! - Artem Eroshenko.
Artem in his report talked about creating actions plug-ins for IntelliJ IDEA, which will help you in quickly changing the code of the test project.
He gave an explanation of the main classes (AnAction, PsiClass, PsiAnnotation, AnActionEvent, JavaCodeStyleManager), which are the entry point of the Plugin Actions.
I considered how to solve the following tasks:
a) What is automated on the project and what is not. Automatic synchronization with Jira.
According to the annotation text, @DisplayName went to Jira, got the necessary tickets and made all the necessary connections using @TmsLink.
b) Automatic insertion of @Tags from the context of Jira.
There are certain labels on the favorite , regress ticket . We go to the test, there are annotations @TmsLink, we apply the plugin and we have new annotations @Tags, and then using Junit we can run tests on these tags.
c) What is checked in the tests?
There is an autotest, there are steps, we do export and we have steps in the tests. Also in the video there is an example for two tests.
Artem also demonstrated how to quickly switch from Allure 1 to Allure 2.
A very practical report for those who are thinking about optimizing the process of writing code. The source code of the plugins can be found here .
3. Java project and Reactor? - What about the tests? - Kirill Merkushev.
Kirill shared his experience in developing an international medical startup Vivy. He told how the problems of monolithic system scalability were solved using microservices and Reactor library. Much attention was paid to this library, its basic principles, approaches to testing reactive systems, killer features (such as checkpoints and logs in the query chain, Hooks and repeated requests (retry).
I personally did not deal with testing reactive systems, therefore for It was new and interesting to me.
4. As we wrote the Sealant framework for searching for memory leaks in JS - Sergey Dokuchaev.
This report deals with memory leaks (objects accessible from code, but no longer needed) on single-page web applications. Sergey set the task to automatically find all critical memory leaks in the released version of the product. He talked about the difficulties of finding such errors and how you can do it manually using Google Chrome tools. Further I considered the tool SeaLant , the author of which he is. SeaLant allows you to automate the detection of leaks by interacting with the Chrome process using the Chrome DevTools protocol. I ended the report by saying that if you can perform cyclical scripts on the page (from one device, with one session, without reloading the page), then most likely, by testing them, you can get rid of most of the memory leaks.
5. Features of visual interface testing - Anton Usmansky, leading developer of Gemini tools (visual testing tool) and Hermione (next-generation, more general-purpose tool that can perform asserts with screenshots).
The report is useful to those who are thinking about the implementation of visual testing tools in their projects. Those who already use Gemini and Hermione may also be interested in it - it gives an understanding of how the instruments are arranged inside. In some places the author is concerned with enough basic things.
6. Troubles in Selenium WebDriver - Alexey Barantsev.
Alexey talked about the problems of the Selenium project and the reasons for their occurrence. Shared the details of the monorepository assembly using the Bazel collector. He touched on the topic of visibility of elements (in the W3C WebDriver standard there is no operation that checks whether an element is visible or not) and explained how the functions getProperty, getDomAttriubute, getAttribute differ.
Knowledge of the problems will allow users to distinguish features from bugs and develop effective “crutches” in their test projects.
7. Recipes for creating from scratch and development of a load testing system - Anatoly Plaskovsky.
Anatoly represents the Yandex.Money performance research team. In his report, he shared with practitioners how to determine the need for performance research, where to find people to conduct these activities and how to build a research process. The author focuses on the fact that performance research! = Load testing, and that only production testing can show a relevant result. At the same time, it is possible to carry out experiments without problems for customers by choosing test routes and data, monitoring and forecasting possible outcomes of the scenario.
Anatoly created a loadland telegram chat in which load testing is discussed, and where you can knock for help and advice.
8. Epic Fails device manufacturers - Valentine Wylsacom Petukhov.
The first day was closed by the report of the notorious Wylsacom, which was ambiguously perceived by the public (judging by the chat of the participants of the Heisenbag in the telegram :)).
For myself, I did not learn anything interesting from the report about beta testing of devices and applications, but maybe the fans liked it. On the discussion area lined up for the photo, so that the PR was a success.
9. Elegant integration testing of the microservice zoo using TestContainers and JUnit 5 using the example of a global SMS platform - Andrey Markelov.
The author told how they test microservices in their company using the TestContainers + Junit 5 + MockServer bundle. The report sounded interesting, but such a scheme is not for a large number of microservices. Link to source code.
10. Voyeurism tester, or How to monitor users will help you - Antonina Hisametdinova.
A very interesting report, albeit more related to web-design than with testing. It describes the UX practices and pitfalls that should be avoided when designing the client interface.
Antonina highlighted the main mistakes in observing users in such UI-solutions as disabling buttons, using loaders, hints, familiar components, etc.
Also worth noting are the graphic design of the report with intuitive examples from real projects (as befits reports about UX).
11. Testing of systems with external dependencies: problems, solutions, Mountebank - Andrey Glazkov.
The author spoke about the evolution of the stages of mocking on their projects. Considered the pros and cons of mocking on the code level. He shared experience, in what cases it is good to create a fake service implementation. But if you want fake services to be rebuilt on the fly, proxied, and logs of incoming requests are added , then Andrey recommends you to look at the Mountebank tool .
Main advantages:
Identified limitations of the tool:
12. Why profile mobile app end-to-end tests - Vyacheslav Frolov.
In his report, Vyacheslav talked about the state of autotests at Badoo, namely, how they solved the paralleling problem of 1500 mobile UI tests, which take 30 machine hours to run on one simulator. As a result, they managed to achieve a result of 30 minutes, which allowed them to run 80,000 tests per day. In addition to a linear increase in power, we applied optimization for clearing the application cache instead of restarting the simulator. Also, the profilers were mentioned in the report, and with their help they found special cases of bottlenecks in the tests: for example, the long_press () method in ios 11.0 ran for 5 seconds, and after optimization it was performed in a second, or as with the http-keep -alive "you can avoid re-installing the connection, thereby significantly speeding up all tests at once. Also the author told
13. Unit tests: from theory to practice - Vadim Pushtaev.
The author shares his experience in developing UnitTests. The report consisted of three parts.
14. We test and cry together with Spring Boot Test - Kirill Tolkachev
Kirill decided to take us back to the world of IoC, DI and Spring and tell us how to write unit and component tests using JUnit 5 and SpringBoot.
The big plus of this report is that the speaker wrote most of the tests in real-time, i.e. clearly showed the phased process of wrapping tests with Spring annotations. However, due to the large amount of code, it was difficult to put all the techniques shown in my head. Kirill examined Spring features for working with profiles, Mock & Spy Beans, as well as setting context configurations. As for me, Spring is a rather heavy framework for such tasks and only experienced users will not be able to step on the rake when developing unit tests.
15. We pump mobile autotests- Ekaterina Bateeva.
In the report, the author reviewed the XCTest tool for automating IOS applications. It was inconvenient to replicate this tool for other projects.
Namely, the following disadvantages were identified:
Then Ekaterina talked about the open-source framework Fastlane , which solved their problems.
In general, I liked the conference. Got a charge of emotions and answers to your questions. At the BoF session, most of the participants removed their “masks” and had heated discussions within the framework of the BoF topics. Coffee breaks and dinners were perfect, there was always something to eat. I also want to note the organization of the event - the JUG.RU team each time makes Heisenbug better and better. Finally, go to conferences, share your knowledge and improve your skills!
PS: I thank Alexey Rumyantsev for participating in the preparation of the article and Artem Yeroshenko for the video material.
Information about HeisenBug
Heisenbug is a technical conference for testing professionals. Basically, it is interesting to testers, software development engineers in test, specialists in automated and load testing. The organizers are the JUG.RU team , behind which are conferences such as Joker, HolyJS, SmartData, DevOops, DotNext, Mobius.
Location
The fifth conference was held at the Slavyanskaya Radisson Hotel, Kievskaya Metro, Moscow. When approaching the hotel, an electronic banner with the conference logo was visible. Further, at the entrance to the hotel lobby, the participant was accompanied by signs to the check-in counters and the wardrobe. It's impossible to get lost. Registration of participants and speakers was held on the ground floor, in the main conference hall there were partners' booths, conference rooms, rooms with a coffee break and lunches. The level of the guards pleased me, so that the "hawkless hares" could not be reached. In total, the event was attended by about 500+ participants, most of whom registered two weeks before the start.
Program
The program can be found here . I want to note that JUG.RU always strives to make the program practical, without any excess water, and with well-known foreign experts. The reports are divided into conventions:
If you are planning to become a speaker at Heisenbag for the first time, look at the speaker’s memo sheet — it contains quite a lot of useful information.
Communication and volunteering
To prevent participants from missing something important at the conference, a telegram channel and a telegram chat were organized. By the way, in the latter they were looking for volunteers to watch videos, for which they gave a free ticket to the conference.
If you decide to become a volunteer, label your decision to the organizers as soon as possible. Preparation for the event is carried out long before the conference. Subject to availability, you will be recorded in the team.
Company stands
This time, more than a dozen stands of major IT companies, such as Deutsche Bank, Alfa Bank, Raiffeisen, Badoo, Luxoft, SKB Kontur, Gett, etc. were presented at the conference. Each of the companies approached the organization of interactions with the participants therefore, it was not necessary to be bored in the intervals between performances. It was possible to solve tasks for a small souvenir, play video and board games, participate in a lottery.
Raiffeisen offered to go online with a quest to solve all sorts of puzzles, and lovers of timeless classics could play Mortal Kombat 3 on Sega. Colleagues from Alfa-Bank made a telegram-bot with tasks, plus in the lottery played a large set of Lego Mindstorms. I want to praise the stand of Deutsche Bank, where the developers in live communication issued tasks and took part in the discussion, unlike all other stands, where they only checked the correctness of the answers.
Many companies are unobtrusively engaged in hunting specialists, handing out booklets describing vacancies (not everyone is involved in charity work).
Review reports
1. We have DevOps. Let's dismiss all testers - Baruh Sadogursky.
The conference was opened by Baruh Sadogursky, developer advocate from JFrog, a well-known speaker at conferences related to Java and Devops. In the report, Baruch touched upon software quality problems in the context of frequent releases, which are caused by the ever-growing demands of the business, the complexity of code testing due to poor architecture, the modernization of the role of the tester in modern Agile teams. The report was interesting to listen from beginning to end, a lot of humor, interesting facts and comparisons.
Well, the main ideas of the report can be put in one slide, which illustrates that the automation of testing and the practice of DevOps reduces time costs for routine processes and allows you to devote more time to implement new tasks.
2. Do I need to refactor a project? There is an IDEA! - Artem Eroshenko.
Artem in his report talked about creating actions plug-ins for IntelliJ IDEA, which will help you in quickly changing the code of the test project.
He gave an explanation of the main classes (AnAction, PsiClass, PsiAnnotation, AnActionEvent, JavaCodeStyleManager), which are the entry point of the Plugin Actions.
I considered how to solve the following tasks:
a) What is automated on the project and what is not. Automatic synchronization with Jira.
According to the annotation text, @DisplayName went to Jira, got the necessary tickets and made all the necessary connections using @TmsLink.
b) Automatic insertion of @Tags from the context of Jira.
There are certain labels on the favorite , regress ticket . We go to the test, there are annotations @TmsLink, we apply the plugin and we have new annotations @Tags, and then using Junit we can run tests on these tags.
c) What is checked in the tests?
There is an autotest, there are steps, we do export and we have steps in the tests. Also in the video there is an example for two tests.
Artem also demonstrated how to quickly switch from Allure 1 to Allure 2.
A very practical report for those who are thinking about optimizing the process of writing code. The source code of the plugins can be found here .
3. Java project and Reactor? - What about the tests? - Kirill Merkushev.
Kirill shared his experience in developing an international medical startup Vivy. He told how the problems of monolithic system scalability were solved using microservices and Reactor library. Much attention was paid to this library, its basic principles, approaches to testing reactive systems, killer features (such as checkpoints and logs in the query chain, Hooks and repeated requests (retry).
I personally did not deal with testing reactive systems, therefore for It was new and interesting to me.
4. As we wrote the Sealant framework for searching for memory leaks in JS - Sergey Dokuchaev.
This report deals with memory leaks (objects accessible from code, but no longer needed) on single-page web applications. Sergey set the task to automatically find all critical memory leaks in the released version of the product. He talked about the difficulties of finding such errors and how you can do it manually using Google Chrome tools. Further I considered the tool SeaLant , the author of which he is. SeaLant allows you to automate the detection of leaks by interacting with the Chrome process using the Chrome DevTools protocol. I ended the report by saying that if you can perform cyclical scripts on the page (from one device, with one session, without reloading the page), then most likely, by testing them, you can get rid of most of the memory leaks.
5. Features of visual interface testing - Anton Usmansky, leading developer of Gemini tools (visual testing tool) and Hermione (next-generation, more general-purpose tool that can perform asserts with screenshots).
The report is useful to those who are thinking about the implementation of visual testing tools in their projects. Those who already use Gemini and Hermione may also be interested in it - it gives an understanding of how the instruments are arranged inside. In some places the author is concerned with enough basic things.
6. Troubles in Selenium WebDriver - Alexey Barantsev.
Alexey talked about the problems of the Selenium project and the reasons for their occurrence. Shared the details of the monorepository assembly using the Bazel collector. He touched on the topic of visibility of elements (in the W3C WebDriver standard there is no operation that checks whether an element is visible or not) and explained how the functions getProperty, getDomAttriubute, getAttribute differ.
Knowledge of the problems will allow users to distinguish features from bugs and develop effective “crutches” in their test projects.
7. Recipes for creating from scratch and development of a load testing system - Anatoly Plaskovsky.
Anatoly represents the Yandex.Money performance research team. In his report, he shared with practitioners how to determine the need for performance research, where to find people to conduct these activities and how to build a research process. The author focuses on the fact that performance research! = Load testing, and that only production testing can show a relevant result. At the same time, it is possible to carry out experiments without problems for customers by choosing test routes and data, monitoring and forecasting possible outcomes of the scenario.
Anatoly created a loadland telegram chat in which load testing is discussed, and where you can knock for help and advice.
8. Epic Fails device manufacturers - Valentine Wylsacom Petukhov.
The first day was closed by the report of the notorious Wylsacom, which was ambiguously perceived by the public (judging by the chat of the participants of the Heisenbag in the telegram :)).
For myself, I did not learn anything interesting from the report about beta testing of devices and applications, but maybe the fans liked it. On the discussion area lined up for the photo, so that the PR was a success.
9. Elegant integration testing of the microservice zoo using TestContainers and JUnit 5 using the example of a global SMS platform - Andrey Markelov.
The author told how they test microservices in their company using the TestContainers + Junit 5 + MockServer bundle. The report sounded interesting, but such a scheme is not for a large number of microservices. Link to source code.
10. Voyeurism tester, or How to monitor users will help you - Antonina Hisametdinova.
A very interesting report, albeit more related to web-design than with testing. It describes the UX practices and pitfalls that should be avoided when designing the client interface.
Antonina highlighted the main mistakes in observing users in such UI-solutions as disabling buttons, using loaders, hints, familiar components, etc.
Also worth noting are the graphic design of the report with intuitive examples from real projects (as befits reports about UX).
11. Testing of systems with external dependencies: problems, solutions, Mountebank - Andrey Glazkov.
The author spoke about the evolution of the stages of mocking on their projects. Considered the pros and cons of mocking on the code level. He shared experience, in what cases it is good to create a fake service implementation. But if you want fake services to be rebuilt on the fly, proxied, and logs of incoming requests are added , then Andrey recommends you to look at the Mountebank tool .
Main advantages:
- mounteBank - works at the TCP level;
- JavaScript injection;
- reliability and speed out of the box;
- excellent documentation;
- support docker containerization.
Identified limitations of the tool:
- external systems with data storage;
- WebSocket - not supported;
- load tests (> 150 rps), but you can use a balancer.
12. Why profile mobile app end-to-end tests - Vyacheslav Frolov.
In his report, Vyacheslav talked about the state of autotests at Badoo, namely, how they solved the paralleling problem of 1500 mobile UI tests, which take 30 machine hours to run on one simulator. As a result, they managed to achieve a result of 30 minutes, which allowed them to run 80,000 tests per day. In addition to a linear increase in power, we applied optimization for clearing the application cache instead of restarting the simulator. Also, the profilers were mentioned in the report, and with their help they found special cases of bottlenecks in the tests: for example, the long_press () method in ios 11.0 ran for 5 seconds, and after optimization it was performed in a second, or as with the http-keep -alive "you can avoid re-installing the connection, thereby significantly speeding up all tests at once. Also the author told
13. Unit tests: from theory to practice - Vadim Pushtaev.
The author shares his experience in developing UnitTests. The report consisted of three parts.
- What we want from UT → Regression (Changed code, checked), Influence on architecture (When developers write tests, the code becomes better), Understanding (to deal with legacy code);
- Principles → Vadim recalls that with theory is not as good as with practice. Rejected the principle of "test the interface, not the implementation," because the implementation can be a lot of logic. Plus, the principle of "unit tests is not a testing mechanism." This is considered a kind of architectural device, powerful and very useful, but it is not a mechanism for checking and correctness of the code.
- Implementation → A story about the techniques that the author uses in his work (for each class there is one test class, creating his own matchrs, applying external dependencies, if required, etc.).
14. We test and cry together with Spring Boot Test - Kirill Tolkachev
Kirill decided to take us back to the world of IoC, DI and Spring and tell us how to write unit and component tests using JUnit 5 and SpringBoot.
The big plus of this report is that the speaker wrote most of the tests in real-time, i.e. clearly showed the phased process of wrapping tests with Spring annotations. However, due to the large amount of code, it was difficult to put all the techniques shown in my head. Kirill examined Spring features for working with profiles, Mock & Spy Beans, as well as setting context configurations. As for me, Spring is a rather heavy framework for such tasks and only experienced users will not be able to step on the rake when developing unit tests.
15. We pump mobile autotests- Ekaterina Bateeva.
In the report, the author reviewed the XCTest tool for automating IOS applications. It was inconvenient to replicate this tool for other projects.
Namely, the following disadvantages were identified:
- hard to read launch settings;
- difficult to configure builds;
- difficult to integrate with various services;
- inconvenient to manage screenshots;
- inconvenient to configure test restarts.
Then Ekaterina talked about the open-source framework Fastlane , which solved their problems.
Results
In general, I liked the conference. Got a charge of emotions and answers to your questions. At the BoF session, most of the participants removed their “masks” and had heated discussions within the framework of the BoF topics. Coffee breaks and dinners were perfect, there was always something to eat. I also want to note the organization of the event - the JUG.RU team each time makes Heisenbug better and better. Finally, go to conferences, share your knowledge and improve your skills!
PS: I thank Alexey Rumyantsev for participating in the preparation of the article and Artem Yeroshenko for the video material.