December 16, 2007 at 13:33How do usability
Small cozy room. Obviously not an apartment, but it doesn’t look like an office either. A sofa, some funny pictures on the wall, a mirror, a desktop ... On the table there is a regular computer, an ordinary LCD screen, an ordinary webcam, and ordinary speakers. There are less ordinary objects: a video camera on a tripod, a speakerphone on the table. This is UsabilityLab's usability lab. Here you may be surprised to find that it is sometimes nice to be a guinea pig.
Posted by Ilya Schurov Voyager
Published in the journal "Computerra" No. 45 dated December 12, 2007
Most of us work with awkward programs every day. We have adapted to some and don’t notice any inconvenience, while others continue to cause a storm of emotions whenever we don’t find the desired button in the “obvious” place or in response to the usual action we get a strange result. For us, these emotions are a by-product of the work that we release into the environment for free. For usability professionals, they are a source of valuable information. Those who ended up in this room will be paid for all their sighs, cursing and cursing the developers.
“It is clear that the test participants can get nervous and worried at first,” says Tatyana Kravchuk, the leading usability specialist, “and we are trying to calm them down and make the situation quite comfortable.” This is understandable: the participants know that after the start of testing, each of their actions, mouse movement, head rotation, change in facial expression or a carelessly torn out word will be recorded and then examined almost under a microscope. We are all afraid of these complex and incomprehensible programs, but even more we are afraid of people who will see our fear of programs. Nobody wants to feel like a fool. “We have to explain that it is not a person who is being tested, but a program and its interface; that if any problems arise or something doesn’t work out, that’s just very good, that’s what we need, ”says Tatyana.
At first, I wanted to look at a relatively recent development in the field of usability testing - a user-tracking system (eye-tracking; we wrote about Tobii, a company that produces such solutions, in a report from last year's CeBIT) - there is such a thing in UsabilityLab, but it appeared she had recently and not yet embarked on "combat duty." “Maybe it's for the best,” I think, taking a seat at the desktop. Tatyana sets the settings in the Morae Recorder program, opens our editorial blog inside.computerra.ru in the browser and clicks on the big red button, then leaves for the next room and leaves me almost alone with the computer. I know that the mirror to my left is transparent on the other hand, Tatyana sees me through him and through a webcam, and that I can communicate with her through the speakerphone system,
Actually, I’m not a good candidate for the role of a test participant - despite all the oddities of our blog (for example, a half-English interface), I perform most of the standard actions “on the machine”, and in order to run into some kind of problem worthy of analysis, you have to come up with unobvious tasks. “The participant probably should not be familiar with the product to obtain adequate results?” - I specify. Tatyana replies that everything depends on the task: if you need to find out how new users will react, then the participants are selected inexperienced; and if, say, a new version of a popular program is released or two interface designs are compared, the test results of experienced users will show how painful (or, conversely, pleasant) changes will turn out for them.
However, in my case, I didn’t have to go too far for the problem - trying to log in using OpenID via LiveJournal, I quickly get an error message that makes me think for a few seconds. “What happened, Ilya?” Asks Tatyana, who constantly maintains a conversation with me in the testing process so that I comment on the actions, and not become isolated. (This technique is called “speaking” and is often used in such tests.) Returning to the page back (and cursing the MSIE developers, in which, apparently, the data entered into the web form is still not saved when returning to the page), I understand that instead of my login in LiveJournal, I entered the full path to the journal. In another situation, I would say that it was my mistake (I didn’t notice that the OpenID type “LiveJournal account” was selected),
Having dealt with OpenID, I am pleased to add a comment to my own post. The first test problem is solved.
I notice on the table a strange device resembling a miniature lamppost. “This is a system for testing mobile applications,” Tatyana explains. “The phone is placed on the line, the camera on top records what is happening on the screen and the buttons pressed.” Generally speaking, there are similar devices in industrial design, but until recently, UsabilityLab was quite comfortable with their “home” development. “Usability laboratories generally have to invent a lot themselves - the area is very young, and the market does not always have the necessary solutions; moreover, this way we can 100% take into account our needs. ” However, in the near future, "homemade" still give way to an industrial model.
The next room looks very different. The mirror is no longer a mirror, but a window. Two dual-monitor systems that display the current actions of the participant. Behind one is an employee who directly conducts testing and communicates with the “subject” (he is called the facilitator or moderator), after the other there is an assistant who monitors what is happening and puts notes, which then simplify the analysis of the record - for example, about the beginning or end of the assignment, about the occurrence difficulty, etc.
A number of chairs are intended for representatives of the customer; an ordinary TV, which broadcasts a picture from a video camera, is for them. “It is very good when the customer is involved in the testing process. Often this makes a very strong impression - says Tatiana. “It’s one thing - when during the expert assessment some shortcomings in the interface are reported, and quite another - when you can see what kind of difficulties real living people experience when working with the program.” According to her, once after such a "session", the customer literally ran away to redo the interface with the words "Well, you, of course, will also send the report, but we already understood our mistakes."
Facilitator intervention in the testing process should be very accurate. On the one hand, it is often required to direct the user's actions in the right direction in order to obtain the necessary information (for example, if a specific dialogue is being tested, and the user tries to come up with a solution to the task assigned to him without using this dialogue), on the other hand, the testing participant must have sufficient freedom in their actions to make the situation realistic - therefore, there are no clear instructions on what to do.
After the completion of the righteous works of the user, a detailed record of all his actions remains, already provided with notes and ready for analysis - it is reviewed, certain points are provided with additional comments, etc. At the “exit” of the analyst’s work, a thick written report is obtained that can be provided with a video application — for example, by cutting out the expression on the faces of participants when they encountered difficulties in performing an action — so that the customer would literally face his problems. (We hope to lay out an example of such a report on the same long-suffering inside.computerra.ru.)
“At the end of testing, participants are usually satisfied,” says Tatyana. - They are pleased to feel like experts; usually people ask us to invite them the next time. ”
Posted by Ilya Schurov Voyager
Published in the journal "Computerra" No. 45 dated December 12, 2007
Under the microscope
Most of us work with awkward programs every day. We have adapted to some and don’t notice any inconvenience, while others continue to cause a storm of emotions whenever we don’t find the desired button in the “obvious” place or in response to the usual action we get a strange result. For us, these emotions are a by-product of the work that we release into the environment for free. For usability professionals, they are a source of valuable information. Those who ended up in this room will be paid for all their sighs, cursing and cursing the developers.
“It is clear that the test participants can get nervous and worried at first,” says Tatyana Kravchuk, the leading usability specialist, “and we are trying to calm them down and make the situation quite comfortable.” This is understandable: the participants know that after the start of testing, each of their actions, mouse movement, head rotation, change in facial expression or a carelessly torn out word will be recorded and then examined almost under a microscope. We are all afraid of these complex and incomprehensible programs, but even more we are afraid of people who will see our fear of programs. Nobody wants to feel like a fool. “We have to explain that it is not a person who is being tested, but a program and its interface; that if any problems arise or something doesn’t work out, that’s just very good, that’s what we need, ”says Tatyana.
At first, I wanted to look at a relatively recent development in the field of usability testing - a user-tracking system (eye-tracking; we wrote about Tobii, a company that produces such solutions, in a report from last year's CeBIT) - there is such a thing in UsabilityLab, but it appeared she had recently and not yet embarked on "combat duty." “Maybe it's for the best,” I think, taking a seat at the desktop. Tatyana sets the settings in the Morae Recorder program, opens our editorial blog inside.computerra.ru in the browser and clicks on the big red button, then leaves for the next room and leaves me almost alone with the computer. I know that the mirror to my left is transparent on the other hand, Tatyana sees me through him and through a webcam, and that I can communicate with her through the speakerphone system,
Actually, I’m not a good candidate for the role of a test participant - despite all the oddities of our blog (for example, a half-English interface), I perform most of the standard actions “on the machine”, and in order to run into some kind of problem worthy of analysis, you have to come up with unobvious tasks. “The participant probably should not be familiar with the product to obtain adequate results?” - I specify. Tatyana replies that everything depends on the task: if you need to find out how new users will react, then the participants are selected inexperienced; and if, say, a new version of a popular program is released or two interface designs are compared, the test results of experienced users will show how painful (or, conversely, pleasant) changes will turn out for them.
However, in my case, I didn’t have to go too far for the problem - trying to log in using OpenID via LiveJournal, I quickly get an error message that makes me think for a few seconds. “What happened, Ilya?” Asks Tatyana, who constantly maintains a conversation with me in the testing process so that I comment on the actions, and not become isolated. (This technique is called “speaking” and is often used in such tests.) Returning to the page back (and cursing the MSIE developers, in which, apparently, the data entered into the web form is still not saved when returning to the page), I understand that instead of my login in LiveJournal, I entered the full path to the journal. In another situation, I would say that it was my mistake (I didn’t notice that the OpenID type “LiveJournal account” was selected),
Having dealt with OpenID, I am pleased to add a comment to my own post. The first test problem is solved.
Do it yourself
I notice on the table a strange device resembling a miniature lamppost. “This is a system for testing mobile applications,” Tatyana explains. “The phone is placed on the line, the camera on top records what is happening on the screen and the buttons pressed.” Generally speaking, there are similar devices in industrial design, but until recently, UsabilityLab was quite comfortable with their “home” development. “Usability laboratories generally have to invent a lot themselves - the area is very young, and the market does not always have the necessary solutions; moreover, this way we can 100% take into account our needs. ” However, in the near future, "homemade" still give way to an industrial model.
On the other side of the glass
The next room looks very different. The mirror is no longer a mirror, but a window. Two dual-monitor systems that display the current actions of the participant. Behind one is an employee who directly conducts testing and communicates with the “subject” (he is called the facilitator or moderator), after the other there is an assistant who monitors what is happening and puts notes, which then simplify the analysis of the record - for example, about the beginning or end of the assignment, about the occurrence difficulty, etc.
A number of chairs are intended for representatives of the customer; an ordinary TV, which broadcasts a picture from a video camera, is for them. “It is very good when the customer is involved in the testing process. Often this makes a very strong impression - says Tatiana. “It’s one thing - when during the expert assessment some shortcomings in the interface are reported, and quite another - when you can see what kind of difficulties real living people experience when working with the program.” According to her, once after such a "session", the customer literally ran away to redo the interface with the words "Well, you, of course, will also send the report, but we already understood our mistakes."
Facilitator intervention in the testing process should be very accurate. On the one hand, it is often required to direct the user's actions in the right direction in order to obtain the necessary information (for example, if a specific dialogue is being tested, and the user tries to come up with a solution to the task assigned to him without using this dialogue), on the other hand, the testing participant must have sufficient freedom in their actions to make the situation realistic - therefore, there are no clear instructions on what to do.
After the completion of the righteous works of the user, a detailed record of all his actions remains, already provided with notes and ready for analysis - it is reviewed, certain points are provided with additional comments, etc. At the “exit” of the analyst’s work, a thick written report is obtained that can be provided with a video application — for example, by cutting out the expression on the faces of participants when they encountered difficulties in performing an action — so that the customer would literally face his problems. (We hope to lay out an example of such a report on the same long-suffering inside.computerra.ru.)
“At the end of testing, participants are usually satisfied,” says Tatyana. - They are pleased to feel like experts; usually people ask us to invite them the next time. ”