Testing at Airbnb



    In a small startup, changing team behavior is relatively easy. You can sit with the team at one table, discuss a new model of work, and then implement everything. And when you have dozens of engineers scattered across different projects, transformation will require a more thorough strategy. One approach is to manage the process through direct “go and do” orders, but in the context of the relative independence of employees inherent in many startups, this approach does not work well. Another way is to personally set an example and indicate the direction of development.

    From the first days, what happened to many startups was happening at Airbnb: we wrote a lot of dubious code and spent little time on testing. And the more we grew, the bigger it became a problem. Today, we process the data of millions of users daily. Just imagine how much data you have to deal with, considering payments alone, when transactions occur every minute in dozens of different currencies. A small minor error can have huge consequences for our users. Now, thanks to our tests, we can be sure that the final product will come out really high-quality.

    We will tell you how we managed to change the testing culture and what tools we used to write and execute tests to simplify the life of our team.



    For us, a new testing culture began with pull requests (“requests for inclusion of changes”). Once, several employees decided to tell others with the help of pull requests about the changes that were made. This did not become a mandatory practice, but over time entailed a number of qualitative changes: firstly, the quality of the code increased, and secondly, it was considered simply old-fashioned.

    This was facilitated by the growth of the team - each new engineer was informed about the importance of peer reviews (hereinafter referred to as PR), the percentage of using PR in teams grew, regardless of whether the "old guard" switched to a new model of work or not. As a result, even the "old men" began to feel awkward, making changes directly to the master.

    We began to take care of testing, actively promoted the development methodology through testing (Test-Driven development, TDD) and followed the three laws of TDD:

    1. New working code is written only after the “falling” unit test is written.
    2. You write exactly the amount of code of the unit test that is needed so that this test crashes or the code does not compile
    3. You write exactly the amount of working code that is needed to pass the “falling” unit test.

    A year later, significant progress was made. We started with a few weak tests, and came to a large kit. Most importantly, we moved away from a vicious practice in which most of the new code gave up without a single test. We not only became a team that believed in tests, but also became those who really wrote them.

    Using PR gives a number of advantages: we have a platform for discussing the structure of the code and architectural solutions, the likelihood that typos and logical errors will be found before they reach users has increased. PR allows all team members to better understand what we are actually working on and, in turn, leads to a further transformation of the entire testing culture.



    Airbnb has installed the basic Rails testing environment. A copy of the applications on the “rails” was stored inside the / spec directory. Testing started rather slowly, and most employees did not know how to organize testing in their own local directory. New employees agreed that testing was crucial, but when they saw that no one was paying attention to the tests, they quickly forgot about them. It was clear that it was necessary to make serious adjustments to the testing strategy.

    First, we needed a lesson on how to deploy PR, including concrete examples of work, whether it was testing a new function, fixing bugs, or refactoring. Secondly, we had to begin to educate the team, arrange meetings of engineers, show teammates how to write tests. Exchange of references and recommendations on educational literature were welcomed. So the company appeared a weekly newsletter with testing news, a demonstration of well-written specifications and an explanation of the main points of PR.

    Thirdly, we had to use our development as a tool to turn employees into real testing champions. This question is solved "in words", emphasizing the importance of testing on the example of the progress that has already been achieved. When the volume of testing increased, other inertia officers joined the process and began to take the tests for granted.

    Obviously, all these changes would not have cost anything if writing and running the tests became too painful and difficult. In the next section, we will see how we managed to make the “cultural shift” less difficult for employees.



    We needed to have a simple and fast testing environment, so we chose Vagrant. This utility allows you to create and configure a virtual development environment and supports several virtualization systems right out of the box. In conjunction with Chef - utilities for managing configurations and much more - can be used to create VirtualHost and folders for a new project. We like the speed of such a solution, and most of the people in our team also use Zeus , which preloads the application on Rails in order to speed up its launch. Vagrant and Zeus have their problems, but in practice, we have found that they save a huge amount of time.

    There are many great solutions to speed up Rails, but given the size of our code base and the speed with which development is progressing, most of them could not be applied immediately. We needed a build system that would allow us to parallelize our tests in real time. Our team used several different solutions before settling on Solano. Each of the previous systems had some problems: instability, memory problems, poor parallelization, a "scary" web interface, etc. Solano supports testing in multiple threads, running them in parallel, and then collecting the results. The product has impressive performance, excellent web interface, there is support for the command line. After the start of its use, the number of screaming and suffering GIFs from disappointed engineers became record low.

    Implementing continuous integration (after each commit) was a huge first step, but we all wanted our testing to go perfectly. The reality of automated testing has shown that compromises in clean testing are needed. This is normal. It is important that testing goes on all the time. Even when it's hard. Especially when it's hard.

    With the growth of Airbnb, we began to add many more critical features to the service. Obviously, there are still many opportunities for improving automated testing, but now we are pleased with the result. We could never have achieved this without the efforts shown “from below,” and these efforts would have been futile if we had not been combined into a single strategy to improve testing tools. All the initiatives, combined with the educational program, have led us to the point where testing is ubiquitous and relatively painless for the system. The trick is that you need a team open to the idea of ​​developing good testing habits.

    Also popular now: