About the sad test

    It seems that everything is necessary, important and useful to know what to test. In this post, I would like to go over the points that make our testing necessary, important and useful.

    Due to the fact that we are working with a developing web application that has many partners, updates and new functionality have to be rolled out often and in many ways. This implies the basic principle of our testing - to catch what is broken as soon as possible, without compromising the speed of development .

    Tester at workShort:
    • Unit-tests code coverage
    • Selenium
    • Automated Testing
    • Detailed declaration of each error
    • Weekly analysis and evaluation of detected bugs
    • Sync with Acunote

    We seek and find

    To search for bugs, we use unit-tests, the Selenium plug-in for FireFox, manual testing of the interface.

    I have already seen the need for unit tests to track bugs more than once and still believe in their strength. At the moment, we have written several dozen unit tests covering, of course, not all the functionality, but a substantial part of the typical user actions: registration, marking, creating a plan, and so on.

    Unlike the “standard” unit-tests, we do not cover the test with a specific function in the code, but with a certain user action. There is a test that checks the creation of friendship between users. When it starts, it successively passes through all possible cases - users are already friends, users cannot become friends, because some conditions are not met and so on.The test contains mini-TK on the logic of the application. The number of tests is growing with the introduction of new functionality. They wrote the binding for the tests themselves, only third-party PHPUnit_assert gladly used it from third-party sources.

    After writing major edits, each of the development team on its local server should test the code, check whether the basic algorithms of the social network violate its edits, and so on. This approach shortens the development time for new features, and, of course, minimizes the appearance of new random errors.

    We took for granted that intercept all the bugs that track unit-tests can not be- there are times when it is urgent to post an update to the site. To do this, tests are run automatically on the main server every 6 hours in automatic mode. The result of the execution goes to the log file, failed tests are sent to me and the development manager by e-mail, SMS are sent.

    Among the tools for front-end testing, we chose the Selenium IDE plug-in for FireFox. The plugin is used to test input forms, the appearance of spinners, the appearance of pop-ups, and the like. As an example, manually registering a user is very inconvenient to test, and using the Selenium test, verification takes only a few seconds.

    The new functionality is primarily tested manually. I try to try all possible options. For example, the interface for adding a place should work out not only the phased filling in of all information about the place (on our website this interface is divided into 4 screens), but also the situations when the user at one moment changed his mind to add a place, did not completely fill out the necessary form, or tried to skip one of steps. In this case, you must at least show the error message. Later, according to the results of manual testing, Selenium tests are written.


    Most often, the error cannot be fixed at the same moment after its discovery, which means they need to be stored somewhere. For these purposes, we selected Google documents, where we created a table with the following fields:
    • Bug number (it’s more convenient to operate when communicating with developers)
    • Who discovered him (the author, the one to whom you can ask additional questions)
    • The place where they found
    • A type
    • Condition of occurrence (if someone wants to repeat)
    • Description (what specifically does not suit)
    • Should appear (which you must see as a result)
    • Comments (just in case, attach a screenshot for example)
    • Dates (opening, closing)
    • Contractor (who corrected)
    • A priority
    We already found them
    The picture is clickable.

    The priority field has several options for filling:
    • P0 - high priority bugs that need to be solved right now. Most often these are errors concerning our partners or errors of the main functionality of the service
    • P1 - must be solved today
    • P2 - decide for the current sprint
    • P3 - decide for the current sprint, if time remains
    • P4 - minor bugs, typos
    Priority is fully synchronized with task priorities in Acunote.

    Most of the time I work with this table and the head of the development department. There was an attempt to connect developers to add new records and edit old ones, but programmers are not comfortable working in a table . If any of the team finds an error, it informs me of it in Skype or by email.

    As a result, the life cycle of the error found looks something like this:
    1. A bug is found and entered into the table;
    2. If the bug is assigned priority P0 or P1, it goes into the current sprint as an unplanned task;
    3. The programmer fixes the bug, when I close it I will find out about it;
    4. I check the correction - you cannot skip this step - and update the table.

    Are planning

    Before the start of each sprint, I and Product Owner, make a list of errors that should be fixed in the next sprint. As a rule, the list includes no more than 10 bugs requiring analysis, problem identification and code editing. Most often, the list includes open P2. It would be more convenient to immediately add all the new bugs to Aconote - then the need for a document with bugs disappears, but to see the pool of unclosed tasks for bugs as a result of the sprint is not the best option.


    The analysis allows you to draw the attention of the development team to repeated or typical code errors, typesetting, in order to prevent them and save time on debugging.

    In my experience, I note that bugs more often arise due to the fact that the task was not described in sufficient detail, or was not divided into subtasks . Let me give an example, it was necessary to make a list of friends from the social network Facebook, to realize the opportunity to invite them to register in our service. The task was completed, but the list page itself was not brought to mind, because it also contained a search for friends. The programmer did not take the initiative and the task of debugging the search did not stand, as a result of a bug.

    The second type of bugs in popularity in our AlterGeo team isbugs at the junction of the functional . For example, adding a new type of event. In this case, the bug may well appear in the event feeds on one of the platforms, be it a site, a mobile version, or one of the mobile applications.

    Convenient statistics, which we build in the same table, allow us to evaluate the speed of occurrence and correction of bugs, only for convenience it is placed in a separate tab. Our chart not only reflects the general trend of errors, but also trends for each of the priorities.

    Summing up, I repeat. Our approach to fixing bugs in the application is not aimed at trying to fix all the errors. Instead, we want to find bugs as early as possible without losing development pace. What is symbolic - after we admitted that developers will always make mistakes and take this into account in their work, the quality of the application has increased dramatically.

    PS Thank you for reading the article to the end. Have a question? Ask, it will be interesting to answer!

    Also popular now: