June 11, 2019 at 09:51Why did we conduct a hackathon for testers
This article will be of interest to those who, just as we are faced with the problem of selecting a suitable specialist in the field of testing.
Oddly enough, but with the increase in the number of IT companies in our republic, only the number of worthy programmers, but not testers, is increasing. Many are eager for this profession, but not many understand its meaning.
I can’t say for all IT companies, but we have assigned the role of QA / QC to our quality specialists. They are part of the development team and participate at all stages of development, from research to the release of a new version.
The tester in the team, even at the planning stage, should consider all the functional and non-functional requirements for accepting the user story. He should be no worse than programmers, or even better, to understand the operational characteristics of the product and help the team not to make the wrong decisions at the planning stage. The tester should have a clear idea of how the implemented functionality will work and what pitfalls can be. Our testers draw up test plans and test cases themselves, as well as prepare all the necessary stands for testing. Testing according to the finished specification as a monkey clicker is not our option. Working within the team, he should help release a decent product and sound the alarm in time if something went wrong.
At the stage of studying a lot of resumes, it seemed that there were specialists with suitable experience for us and there would be no problems with choosing a tester in our team. But, in face-to-face meetings, we were increasingly confronted with candidates who in reality were quite far from the world of information technology (for example, they could not tell the principles of browser and web server interaction, security basics, relational and non-relational databases, had no idea about virtualization and containerization), but at the same time they evaluated themselves at the level of Senior QA. After more than a dozen interviews, we came to the conclusion that the number of specialists suitable for us in the region is negligible.
Next, I will tell you what steps we took and what rake we stepped on to find the very long-awaited fighters for quality.
Having suffered with sourcing of ready-made specialists, we began to shoot at the nearby areas:
To begin with, they tried to understand what kind of "beast" this is and how it is usually carried out. As it turned out, events of this kind in the Russian Federation have been held not so many times, and there is no place to borrow ideas. Secondly, I did not want to immediately, in a dubious event at first glance, invest a lot of resources. Therefore, we decided that we would do short mini hackathons, not for the entire QA work cycle, but for separate stages.
Our main headache is a lack of local testers practice in the formation of intelligible testing maps. They do not devote time to research at the stage before the implementation of user stories and the creation of acceptance criteria understandable by developers for functional and non-functional requirements, UI / UX, security, work and peak loads. Therefore, we decided, for the first time, to go through the most interesting and creative part of their work - analysis and formation of requirements for pre-project research.
They estimated the potential number of participants and decided that we need at least 5 backlogs for MVP releases, 5 products and 5 people who will act as product owners, decipher business needs and make decisions on restrictions.
Here's what we got:backlogs for a hackathon .
The main idea was to come up with topics that are as far from all the participants ’everyday work and give them room for creative imagination.
The application of assessment practices, which are so popular in the field of receiving salespeople and lower-level managers, took a huge amount of energy, but did not allow us to pay sufficient attention to each participant and evaluate his abilities. In general, this selection option creates a negative image of the company, since quite a lot of people receive insufficient feedback and, in the future, form the effect of tyranny of the employer (the communications in the IT communities are very developed). As a result, we have literally two potential candidates with a very distant prospect.
Here mitapas are a good thing. An extensive base for study is being created, the overall level of participants is being raised. The company is becoming increasingly recognizable in the market. But, the complexity of such undertakings is not small. It must be clearly understood that approximately 700-800 man-hours will take to hold meetings in a year.
As for the hackathon testers. Such events have not yet had time to get bored, because, unlike hackathons for developers, they are held much less frequently. The advantage of this venture is that in an unambiguous manner you can exchange a large amount of practical knowledge and fairly accurately determine the level of each participant.
After analyzing the results of the event, we realized that heaps made mistakes:
We partially solved our task and now we have 4 brave handsome men covering the rear of 4 development teams. A significant pool of potential strong candidates and tangible changes in the level of QA-community of the city have not yet been noticed. But, there are some progresses and this cannot but rejoice.
Oddly enough, but with the increase in the number of IT companies in our republic, only the number of worthy programmers, but not testers, is increasing. Many are eager for this profession, but not many understand its meaning.
I can’t say for all IT companies, but we have assigned the role of QA / QC to our quality specialists. They are part of the development team and participate at all stages of development, from research to the release of a new version.
The tester in the team, even at the planning stage, should consider all the functional and non-functional requirements for accepting the user story. He should be no worse than programmers, or even better, to understand the operational characteristics of the product and help the team not to make the wrong decisions at the planning stage. The tester should have a clear idea of how the implemented functionality will work and what pitfalls can be. Our testers draw up test plans and test cases themselves, as well as prepare all the necessary stands for testing. Testing according to the finished specification as a monkey clicker is not our option. Working within the team, he should help release a decent product and sound the alarm in time if something went wrong.
What we encountered when looking for testers
At the stage of studying a lot of resumes, it seemed that there were specialists with suitable experience for us and there would be no problems with choosing a tester in our team. But, in face-to-face meetings, we were increasingly confronted with candidates who in reality were quite far from the world of information technology (for example, they could not tell the principles of browser and web server interaction, security basics, relational and non-relational databases, had no idea about virtualization and containerization), but at the same time they evaluated themselves at the level of Senior QA. After more than a dozen interviews, we came to the conclusion that the number of specialists suitable for us in the region is negligible.
Next, I will tell you what steps we took and what rake we stepped on to find the very long-awaited fighters for quality.
How we tried to fix the situation
Having suffered with sourcing of ready-made specialists, we began to shoot at the nearby areas:
- We tried to apply assessment practice to identify among the great number of “get away people”, those who we can grow strong specialists from.
A group of potential candidates, with approximately the same level of knowledge, we were offered to complete tasks. Observing their thought process, they tried to single out the most promising candidate.
In particular, we came up with tasks for checking attentiveness, understanding the capabilities of technologies and the features of multiculturalism:
- Meetings were held for testers to expand the boundaries of understanding the profession for the existing contingent.
I’ll tell you a little about each of them.
Ufa Software QA and Testing Meetup # 1 is our first attempt to gather those who are not indifferent to the profession and at the same time to understand whether the public will be interested in what we want to convey to them. Basically, our reports were about where to start, if you decided to become a tester. Help beginners open their eyes and take a look at adult testing. We talked about the steps that beginner testers need to take in order to join the profession. About what quality is and how to achieve it in real conditions. And also what automatic testing is and where it is more appropriate to apply it.
Further, with an interval of 1-2 months, we held two more mitaps. There were already two times more participants. At Ufa Software QA and Testing Meetup # 2, we plunged deeper into the subject area. We talked about bug tracking systems, about testing UI / UX, touched upon Docker, Ansible, and also talked about possible conflicts between the developer and the tester and how to resolve them.
Our third mitap “Ufa Software QA and Testing Meetup # 3” indirectly related to the work of testers, but was useful in order to remind programmers in time about their technical and organizational duty: load testing, e2e testing, Selenium in self-testing, and web application vulnerabilities.
All this time we learned to make normal light and sound in broadcasts from our events:
→ First steps in testing - Ufa Software QA and Testing Meetup # 1
→ UI / UX testing - Ufa Software QA and Testing Meetup # 2
→ Security testing, stress testing and autotesting - Ufa QA and Testing Meetup # 3 - And in the end we decided to try a hackathon for testers
How hackathon was prepared and carried out for testers
To begin with, they tried to understand what kind of "beast" this is and how it is usually carried out. As it turned out, events of this kind in the Russian Federation have been held not so many times, and there is no place to borrow ideas. Secondly, I did not want to immediately, in a dubious event at first glance, invest a lot of resources. Therefore, we decided that we would do short mini hackathons, not for the entire QA work cycle, but for separate stages.
Our main headache is a lack of local testers practice in the formation of intelligible testing maps. They do not devote time to research at the stage before the implementation of user stories and the creation of acceptance criteria understandable by developers for functional and non-functional requirements, UI / UX, security, work and peak loads. Therefore, we decided, for the first time, to go through the most interesting and creative part of their work - analysis and formation of requirements for pre-project research.
They estimated the potential number of participants and decided that we need at least 5 backlogs for MVP releases, 5 products and 5 people who will act as product owners, decipher business needs and make decisions on restrictions.
Here's what we got:backlogs for a hackathon .
The main idea was to come up with topics that are as far from all the participants ’everyday work and give them room for creative imagination.
What mistakes have we made and what can be done better
The application of assessment practices, which are so popular in the field of receiving salespeople and lower-level managers, took a huge amount of energy, but did not allow us to pay sufficient attention to each participant and evaluate his abilities. In general, this selection option creates a negative image of the company, since quite a lot of people receive insufficient feedback and, in the future, form the effect of tyranny of the employer (the communications in the IT communities are very developed). As a result, we have literally two potential candidates with a very distant prospect.
Here mitapas are a good thing. An extensive base for study is being created, the overall level of participants is being raised. The company is becoming increasingly recognizable in the market. But, the complexity of such undertakings is not small. It must be clearly understood that approximately 700-800 man-hours will take to hold meetings in a year.
As for the hackathon testers. Such events have not yet had time to get bored, because, unlike hackathons for developers, they are held much less frequently. The advantage of this venture is that in an unambiguous manner you can exchange a large amount of practical knowledge and fairly accurately determine the level of each participant.
After analyzing the results of the event, we realized that heaps made mistakes:
- The most unforgivable mistake was to believe that 4-5 hours would be enough for us. As a result, only introduction and acquaintance with backlogs took almost 2 hours.
Work with the owners of the products at the initial stage and the time to immerse themselves in the subject area took as much. So, the remaining time was clearly not enough for a comprehensive study of testing cards. - There was not enough time and energy for a detailed feedback on each card, since it was already night. Therefore, this part was clearly failed by us, and was originally supposed to be the most valuable in the hackathon.
- We decided to evaluate the quality of the study by a simple vote of all participants, allocating 3 votes for each team that they could give for the highest quality work. Perhaps it would be better to organize a jury.
What have you achieved
We partially solved our task and now we have 4 brave handsome men covering the rear of 4 development teams. A significant pool of potential strong candidates and tangible changes in the level of QA-community of the city have not yet been noticed. But, there are some progresses and this cannot but rejoice.