About registration of eye movements in the process of usability testing
The first Russian usability company to use eye-tracking technology in its work was the USABILITYLAB laboratory , which was announced in mid-June.
So far, the eye-tracking technology, which is not very well known to Russian site owners, is based on recording the user's eye movements. According to research, it is quite useful for usability testing of user interfaces and, as representatives of USABILITYLAB report, allows developers to "see their product through the eyes of the user." With its help, you can find out exactly where the user is looking when looking at the screen, how his gaze moves, how long he searches for this or that information on the site, and what he does not notice for some reason.
Dmitry Satin, the head of the USABILITYLAB laboratory and its scientific adviser, senior researcher at the Institute of Psychology of the Russian Academy of Sciences Anatoly Kostin, tell us more about eye-tracking technology and the prospects for its use in our country in TelNews interviews . - Recently, USABILITYLAB announced the acquisition and "introduction into regular practice" of eye-tracking technology. What does this really mean? What specific new services can you offer your customers in this regard?
- Let's first say a few words about what usability testing is, in order to better understand the value of the eye movement analysis technique. Usability testing is an experimental method for evaluating user interfaces of a product (for example, a site, but not only), based on attracting its real or potential users. Users (profiles that necessarily match the portrait of the target audience) perform tasks related to the purpose of the product. For example, for a tourist site such a task could be the selection and ordering a tour to any country. Tasks are taken from life, that is, they are realistic.
We do not tell the user exactly how to solve the problem, but watch how he will cope with it. Most usability tests performed at the end of the project reveal problems that are sometimes so critical to the product that you have to conclude that the product is not suitable for use. True, there are sometimes pleasant exceptions .
For our client, the problems found are of relative value. First of all, he needs to know how to fix a detected defect. In order to answer this question, you have to contact the user. To do this, they often use the “thinking out loud” method - they ask the user to comment on what he thinks about, explain how he makes decisions and how he understands certain elements of the user interface with which he interacts.
This method, although it gives a lot of hints, is not protected from distortion. Firstly, not everyone knows how to talk about what they think - this requires special skills. Secondly, studies show that a person’s behavior changes when he is asked to comment on what he is doing. The user turns into a guide describing the interfaces. He begins to pay attention to what he would not pay attention to if he worked silently.
It turns out that we need such objective methods that would allow us to understand the behavior of the user without asking him to explain what he is doing. One such method is the analysis of eye movements. Due to the high cost of technology that allows you to see where the user is looking, this method is not available to all companies that provide similar services. We decided that the quality of the results of our work is above all, and began to work on the issue of acquiring the appropriate equipment.
With the purchase of a special camera, the essence of the service has not changed. This is all the same usability testing. But the result we give is qualitatively and quantitatively different. The client sees firsthand where his users are looking. But we get the opportunity to answer new questions, the solution of which was not available before. For example, to calculate how much time the user spent looking at a specific element of user interfaces, how much time was needed and in what way the user's gaze got to the target (needed) object. The reliability of the conclusions drawn on this data is much higher than on the presentation by the user of what he was thinking.
- Describe how the process of usability testing of user interfaces using eye-tracking technology in your company?
- The process of structure, as I said, remained the same. But now in front of the user under the monitor (if we are testing a site or desktop application) is a black object that looks like a box. It is this device that performs the registration of eye movements.
During testing, the facilitator (the person who organizes it) communicates with the user from another room and sees him through a translucent mirror. On its monitor, the facilitator sees the user's screen and his eye movements in real time. This helps him understand why the user, for example, hesitated.
The data recorded during testing are analyzed. Heat maps of user attention distribution, eye movement diagrams, statistics on “zones of interest”, etc. are built.
- How expensive will it be to use such technology to test the usability of a site?
- I think my answer will surprise you (hereinafter in the first person words of Dmitry Satin - approx. Ed.). We did not change our prices after purchasing new equipment. There are two reasons for this: on the one hand, the technique is quite new and you need to give time to ensure that customers feel its value; on the other hand, internal ethics does not allow raising prices only because we have become more professional.
Of course, we will come to the need to raise prices for usability testing using eye registration techniques, but for now we are delaying this point.
- Why was USABILITYLAB chosen specifically Tobii eye-tracking equipment?
- At the time of the choice, the X120 Tobii camera seemed to us to be a universal solution, allowing us to use it to test not only computer interfaces, but also products with which the user can interact via TV, mobile devices, even examine how the eyes “work” when reading printed materials . For example, we adapted the camera for testing payment terminals. Even Tobii employees praised us for such a witty decision.
But now we are studying other types of equipment for recording eye movements. For example, special glasses made by the German company ManGold International .
- To whom, besides online stores, eye-tracking technologies can be useful first of all for doing business on the Web?
- I have to destroy the myth that the customers of usability services are online stores. Most of these companies save money so much that they delay their own development. It would seem that it is for them that improving user interfaces should bring money the next day, and therefore they should pay a lot of attention to this issue.
Our practice shows that this is surprisingly not the case. A company must reach a certain level of maturity in order to understand simple things: there is nothing more important for a business than the efficiency (effectiveness) of its customers.
Therefore, usability testing is now more popular among software developers, mobile operators, developing their non-voice services, banks promoting their self-service systems (Internet banking and payment terminals, ATMs).
- According to your observations and research - what most often distracts users from the information that the site owner would like to convey to them?
- Distracted banners - this is obvious. But users learn to deal with them. They begin to force them out of their consciousness, and this leads to new problems. If the interface element looks like a banner, then the user cannotfind him. Sometimes horizontal banners in the middle of the page “break” it so much that the user does not perceive either that is above the banner or lower, depending on which edge of the page the banner is closer to.
- Tell us about your testing of a dating site using eye-tracking technology and its results.
- I can’t disclose the details, because we are associated with customers with non-disclosure agreements. I’ll tell you about a funny experiment that my colleagues made a joke. They selected a photo depicting several attractive and not very dressed girls and showed it on a computer to one of the employees. The employee had nothing to do but look at the girls. Then the observers informed the subject which girl he liked more than others. The result of this joke declined staggering, hitting the target was very accurate. A person may hide his preferences, but his gaze betrays them.
- In the case of eye-tracking, is there a problem with the fact that the user can, roughly speaking, look at one thing and click on a completely different and different one in the case of different users?
- No, it does not. Conversely, users working with complex interfaces often have mouse movements following their gaze. It's like moving your finger along the lines of a book when you are still reading poorly - it helps to concentrate. The same effect is observed in inexperienced or overloaded users.
- It is known that the so-called “faceless interfaces” have already been invented for people with disabilities - when actions usually performed with the mouse are performed only with the help of the eyes. Do you think this technology has distribution prospects among healthy users?
- You need to understand that the equipment is not cheap yet. Not everyone can afford it. Russian disabled people, I think, are deprived of the opportunity to use such technologies, and in the West they can receive it (sometimes for free) from funds involved in the rehabilitation of people with disabilities.
For healthy people, the appropriateness of using such technologies is not yet clear, although numerous studies are underway on the problems of controlling the gaze control technique, and not only computer control. For example, for control with busy hands or remote technical devices, although it seems that it is better to use conventional remote controls for this. The fact is that the eye is constantly moving and people cannot completely control these movements, which means that either the control element must be large enough and stable (like a virtual keyboard for the disabled), or serious errors will occur. For example, a pilot of a military aircraft, whose hands are occupied with piloting, directs the weapon at the target not with his eyes, but with turns of his head, having a special optical sight on his helmet.
So far, the eye-tracking technology, which is not very well known to Russian site owners, is based on recording the user's eye movements. According to research, it is quite useful for usability testing of user interfaces and, as representatives of USABILITYLAB report, allows developers to "see their product through the eyes of the user." With its help, you can find out exactly where the user is looking when looking at the screen, how his gaze moves, how long he searches for this or that information on the site, and what he does not notice for some reason.
Dmitry Satin, the head of the USABILITYLAB laboratory and its scientific adviser, senior researcher at the Institute of Psychology of the Russian Academy of Sciences Anatoly Kostin, tell us more about eye-tracking technology and the prospects for its use in our country in TelNews interviews . - Recently, USABILITYLAB announced the acquisition and "introduction into regular practice" of eye-tracking technology. What does this really mean? What specific new services can you offer your customers in this regard?
- Let's first say a few words about what usability testing is, in order to better understand the value of the eye movement analysis technique. Usability testing is an experimental method for evaluating user interfaces of a product (for example, a site, but not only), based on attracting its real or potential users. Users (profiles that necessarily match the portrait of the target audience) perform tasks related to the purpose of the product. For example, for a tourist site such a task could be the selection and ordering a tour to any country. Tasks are taken from life, that is, they are realistic.
We do not tell the user exactly how to solve the problem, but watch how he will cope with it. Most usability tests performed at the end of the project reveal problems that are sometimes so critical to the product that you have to conclude that the product is not suitable for use. True, there are sometimes pleasant exceptions .
For our client, the problems found are of relative value. First of all, he needs to know how to fix a detected defect. In order to answer this question, you have to contact the user. To do this, they often use the “thinking out loud” method - they ask the user to comment on what he thinks about, explain how he makes decisions and how he understands certain elements of the user interface with which he interacts.
This method, although it gives a lot of hints, is not protected from distortion. Firstly, not everyone knows how to talk about what they think - this requires special skills. Secondly, studies show that a person’s behavior changes when he is asked to comment on what he is doing. The user turns into a guide describing the interfaces. He begins to pay attention to what he would not pay attention to if he worked silently.
It turns out that we need such objective methods that would allow us to understand the behavior of the user without asking him to explain what he is doing. One such method is the analysis of eye movements. Due to the high cost of technology that allows you to see where the user is looking, this method is not available to all companies that provide similar services. We decided that the quality of the results of our work is above all, and began to work on the issue of acquiring the appropriate equipment.
With the purchase of a special camera, the essence of the service has not changed. This is all the same usability testing. But the result we give is qualitatively and quantitatively different. The client sees firsthand where his users are looking. But we get the opportunity to answer new questions, the solution of which was not available before. For example, to calculate how much time the user spent looking at a specific element of user interfaces, how much time was needed and in what way the user's gaze got to the target (needed) object. The reliability of the conclusions drawn on this data is much higher than on the presentation by the user of what he was thinking.
- Describe how the process of usability testing of user interfaces using eye-tracking technology in your company?
- The process of structure, as I said, remained the same. But now in front of the user under the monitor (if we are testing a site or desktop application) is a black object that looks like a box. It is this device that performs the registration of eye movements.
During testing, the facilitator (the person who organizes it) communicates with the user from another room and sees him through a translucent mirror. On its monitor, the facilitator sees the user's screen and his eye movements in real time. This helps him understand why the user, for example, hesitated.
The data recorded during testing are analyzed. Heat maps of user attention distribution, eye movement diagrams, statistics on “zones of interest”, etc. are built.
- How expensive will it be to use such technology to test the usability of a site?
- I think my answer will surprise you (hereinafter in the first person words of Dmitry Satin - approx. Ed.). We did not change our prices after purchasing new equipment. There are two reasons for this: on the one hand, the technique is quite new and you need to give time to ensure that customers feel its value; on the other hand, internal ethics does not allow raising prices only because we have become more professional.
Of course, we will come to the need to raise prices for usability testing using eye registration techniques, but for now we are delaying this point.
- Why was USABILITYLAB chosen specifically Tobii eye-tracking equipment?
- At the time of the choice, the X120 Tobii camera seemed to us to be a universal solution, allowing us to use it to test not only computer interfaces, but also products with which the user can interact via TV, mobile devices, even examine how the eyes “work” when reading printed materials . For example, we adapted the camera for testing payment terminals. Even Tobii employees praised us for such a witty decision.
But now we are studying other types of equipment for recording eye movements. For example, special glasses made by the German company ManGold International .
- To whom, besides online stores, eye-tracking technologies can be useful first of all for doing business on the Web?
- I have to destroy the myth that the customers of usability services are online stores. Most of these companies save money so much that they delay their own development. It would seem that it is for them that improving user interfaces should bring money the next day, and therefore they should pay a lot of attention to this issue.
Our practice shows that this is surprisingly not the case. A company must reach a certain level of maturity in order to understand simple things: there is nothing more important for a business than the efficiency (effectiveness) of its customers.
Therefore, usability testing is now more popular among software developers, mobile operators, developing their non-voice services, banks promoting their self-service systems (Internet banking and payment terminals, ATMs).
- According to your observations and research - what most often distracts users from the information that the site owner would like to convey to them?
- Distracted banners - this is obvious. But users learn to deal with them. They begin to force them out of their consciousness, and this leads to new problems. If the interface element looks like a banner, then the user cannotfind him. Sometimes horizontal banners in the middle of the page “break” it so much that the user does not perceive either that is above the banner or lower, depending on which edge of the page the banner is closer to.
- Tell us about your testing of a dating site using eye-tracking technology and its results.
- I can’t disclose the details, because we are associated with customers with non-disclosure agreements. I’ll tell you about a funny experiment that my colleagues made a joke. They selected a photo depicting several attractive and not very dressed girls and showed it on a computer to one of the employees. The employee had nothing to do but look at the girls. Then the observers informed the subject which girl he liked more than others. The result of this joke declined staggering, hitting the target was very accurate. A person may hide his preferences, but his gaze betrays them.
- In the case of eye-tracking, is there a problem with the fact that the user can, roughly speaking, look at one thing and click on a completely different and different one in the case of different users?
- No, it does not. Conversely, users working with complex interfaces often have mouse movements following their gaze. It's like moving your finger along the lines of a book when you are still reading poorly - it helps to concentrate. The same effect is observed in inexperienced or overloaded users.
- It is known that the so-called “faceless interfaces” have already been invented for people with disabilities - when actions usually performed with the mouse are performed only with the help of the eyes. Do you think this technology has distribution prospects among healthy users?
- You need to understand that the equipment is not cheap yet. Not everyone can afford it. Russian disabled people, I think, are deprived of the opportunity to use such technologies, and in the West they can receive it (sometimes for free) from funds involved in the rehabilitation of people with disabilities.
For healthy people, the appropriateness of using such technologies is not yet clear, although numerous studies are underway on the problems of controlling the gaze control technique, and not only computer control. For example, for control with busy hands or remote technical devices, although it seems that it is better to use conventional remote controls for this. The fact is that the eye is constantly moving and people cannot completely control these movements, which means that either the control element must be large enough and stable (like a virtual keyboard for the disabled), or serious errors will occur. For example, a pilot of a military aircraft, whose hands are occupied with piloting, directs the weapon at the target not with his eyes, but with turns of his head, having a special optical sight on his helmet.