September 30, 2014 at 13:54How we practice corridor testing
Developers want to make clear and convenient software products.
But for us, both the console and hot keys are a completely understandable and convenient interface:
If a plug-in can be written for a program, then it is convenient and extensible. If you can change the background color of the message box in the config, it’s flexible. The normal user is neither warm nor cold from such flexibility; he needs to solve his tasks, not write plugins.
Previously, we tested the user interface during trial operation (this is when a new version is installed in our company). This method had disadvantages:
We decided to deal with these shortcomings using corridor testing. Here we want to share our experience.
Prepare scenarios that are close to real. It is important to prepare scenarios with real data: contracts and acts with real names, and not a “test document”, real users and organizational structure in the system. If you just let the system poke, there will be no such effect.
Adapt scripts during testing. If the first two users have problems, change the script. It is possible directly in the process to invent new situations of interaction with the system.
Test on different users.For us, the most interesting users who gave the most useful feedback are beginners or those who are not at all familiar with the system. They have not yet blurred their eyes and no addictions from working with previous versions. From the old people, especially the developers, they mostly get comments on the fact that their favorite hotkey does not work, but you can’t ignore these users either.
Exclude bureaucracy from the process. Almost anyone in the company is ready to allocate 15–20 minutes, no harsh approvals are needed from the leaders and people’s choice, it’s enough to write in a communicator and arrange the time. Almost everyone responds without problems; one didn’t respond - there is always someone to call instead of a dodger.
Limit the number of subjects.To find the main problems, 8-10 people with different levels of training and functional responsibilities are enough. According to the results of one of the corridor tests, we calculated the number of detected bugs and the number of new bugs for each user. It can be seen that the latest users are not saying almost anything new:
Get a natural environment. The keyboard, monitor, system, languages and other environment should be standard and familiar, so as not to disturb or distract people.
Test in beta. Testing on a raw product is normal. It’s not scary if a couple of bugs come out. We do corridor testing at the end of the sprint. Here, almost everything works and there is time to quickly fix something or analyze and plan for correction in the next sprint.
You can create scripts before development, they will come in handy both on design and planning (understand what needs to be done), and during development for testing and testing, and on a demo.
Record everything. Fix not only problems, but also that with which there were no problems. This is necessary so as not to spoil what works well now.
But for us, both the console and hot keys are a completely understandable and convenient interface:
If a plug-in can be written for a program, then it is convenient and extensible. If you can change the background color of the message box in the config, it’s flexible. The normal user is neither warm nor cold from such flexibility; he needs to solve his tasks, not write plugins.
Previously, we tested the user interface during trial operation (this is when a new version is installed in our company). This method had disadvantages:
- Users have no time to make comments on the product. Especially if it concerns the usability of the program, and not the functionality.
- It is not clear how intuitive the interface is. You can, of course, conduct surveys, but they are usually uninformative.
- Comments come too late. The developer has less time to fix them.
We decided to deal with these shortcomings using corridor testing. Here we want to share our experience.
Our recommendations for CT
Prepare scenarios that are close to real. It is important to prepare scenarios with real data: contracts and acts with real names, and not a “test document”, real users and organizational structure in the system. If you just let the system poke, there will be no such effect.
Adapt scripts during testing. If the first two users have problems, change the script. It is possible directly in the process to invent new situations of interaction with the system.
Test on different users.For us, the most interesting users who gave the most useful feedback are beginners or those who are not at all familiar with the system. They have not yet blurred their eyes and no addictions from working with previous versions. From the old people, especially the developers, they mostly get comments on the fact that their favorite hotkey does not work, but you can’t ignore these users either.
Exclude bureaucracy from the process. Almost anyone in the company is ready to allocate 15–20 minutes, no harsh approvals are needed from the leaders and people’s choice, it’s enough to write in a communicator and arrange the time. Almost everyone responds without problems; one didn’t respond - there is always someone to call instead of a dodger.
Limit the number of subjects.To find the main problems, 8-10 people with different levels of training and functional responsibilities are enough. According to the results of one of the corridor tests, we calculated the number of detected bugs and the number of new bugs for each user. It can be seen that the latest users are not saying almost anything new:
Get a natural environment. The keyboard, monitor, system, languages and other environment should be standard and familiar, so as not to disturb or distract people.
Test in beta. Testing on a raw product is normal. It’s not scary if a couple of bugs come out. We do corridor testing at the end of the sprint. Here, almost everything works and there is time to quickly fix something or analyze and plan for correction in the next sprint.
You can create scripts before development, they will come in handy both on design and planning (understand what needs to be done), and during development for testing and testing, and on a demo.
Record everything. Fix not only problems, but also that with which there were no problems. This is necessary so as not to spoil what works well now.