Through thorns to the stars: why access to users is necessary, and how to do it
We have already written a lot about the role of qualitative methodology in user research (for example, http://bit.ly/25hnAMQ and http://bit.ly/2jmfkcM ). In one of the articles ( http://bit.ly/2ay3Nzg ) we talked about how qualitative methods are used in UX research to develop IT products in foreign companies. What is happening in Russia?
In this article, we will talk about our experience in applying a high-quality methodology for developing IT products in the Russian market. We will talk not only about the specifics of the work, but also about how to get started. How to explain to the customer the value of a quality methodology? What can I tell him to give the green light to communicate with users? And why does the need arise to explain?
In this article, we will discuss three key points.
Let's start with the problems of understanding a quality methodology. In fact, it will be about what objections to conducting a qualitative study are raised by the customer. Organizing a study is a job. The main thing is to start it. And for this we need to reach out to the customer. What problems can arise here?
A still popular question from Russian customers of different calibers. The situation is changing, but not as fast as we would like. Let's figure out what's the matter.
The customer fears that time and money will be wasted.
The concerns are generally clear: for the customer, quality methods and user analytics remain a black box. Recall that in Western countries the situation is different. And we have a black box, where we are asked to add money and the time of the project. Do not forget about it when you are talking with the customer. The situation is complicated by the fact that the outcome is unknown.
The customer does not understand what he will receive as a result of the study and how they will work with it.
This is a serious problem associated with the presentation of the results of qualitative research. For example, the numbers in quantitative studies are understandable. Web analytics is clear. The work of the copywriter and the designer is visible to the naked eye. The results of a qualitative study, packed in formats of user portraits, a description of the environment of use, life situations and scenarios are incomprehensible to the customer without a detailed explanation. What do these results give? How does the team work with them? What benefits do they bring? You must be able to answer all these customer questions.
There is another situation where the customer approximately represents ( or thinks what he represents ) what user analytics is. Perhaps I heard about quality methods. Why do you need to conduct a study if I can tell everything myself?
The customer believes in his expert knowledge of users.
What remains for us? It is necessary to understand the sources of this knowledge and its depth. For example, this is many years of experience in the industry. It's good. To analyze this experience, we conduct an interview with the customer about users. Who are they? What groups can be distinguished? What problems do they face? What are their questions? How do they even live? And many more questions that qualitative research covers. We do not deny the knowledge of the customer, but consider it as hypotheses - we verify and supplement it with the help of research.
Another possible source of expert knowledge is own research. This is very good, but you need to understand how the study was conducted? What can we learn about users, their life situations and behavior? Not all studies are equally useful for UX tasks.
There is another facet of faith in your expert knowledge - disbelief in user knowledge.
The customer doubts user competencies and knowledge.
What can they tell you? They will offer a wagon of “features”, but they don’t need them. The problem in this case is a misunderstanding of the essence of user research. The researcher studies the daily life of the user, his problems, the environment of use, and does not ask about ready-made solutions. Solution development is our task. Users do not need competencies in the development of IT products.
And now let's talk about the concerns of the customer on the organization of the study.
The customer is convinced that we cannot afford to organize a study.
Organizing a study may seem like a complicated process: you need to find users, agree with the company employees, process the collected data, etc. In practice, the complexity of organizing research depends on many factors. Often, active customer involvement greatly simplifies the process.
The customer is afraid that users will refuse to participate in the study.
Various methods of motivation help to attract users to participate in the research - from delayed benefits, personal interest in the research topic to stimulation with money. Sometimes such problems do not arise at all: people are ready to help, if it is clear to explain why we are addressing them.
Before analyzing the cases, we’ll once again list the problems and concerns that the customer has regarding user research using a high-quality methodology:
Now let's move on to the cases. The main task of this part:
We specially selected cases with different project objectives: content preparation, audit and prototype development of a professional interface, research of self-service kiosks. The customers are also different - small business (Set.Design), medium business (Solar Staff) and large business (Sportmaster).
Cases Set.Design and Solar Staff can be viewed on our website. The Sportmaster case, unfortunately, is not available - strict NDA conditions. The main emphasis in the analysis of cases is on:
Company Set. Design produces and sells wooden eco-mosaics, decorative 3D panels, coffered ceilings and other decor items for interior decoration. The company has its own designers and its own carpentry workshop. They make products according to the catalog, adapt someone else's solution or create an original product according to the sketch.
The company came to the new site for texts. During the discussion, we found out that the company has questions about positioning - whether it is a manufacturer of designer materials, or a supplier of solutions based on these materials, or even individual production. It’s not quite a text task, but without this, texts cannot be approached.
The project had severe financial constraints, so the best method was to use a semi-structured interview. The time allotted was enough for only three interviews (taking into account the preparation of the text). But even three interviews are better than not one.
To determine the target audience, we talked with the customer and found out: we are interested in interior designers. Based on the results of this communication, an interview guide was compiled and discussed with the customer. The customer can offer additional questions or topics - if possible, we add them.
One respondent was offered by the customer - the buyer of his store. Another is the first designer: "a friend who does everything differently." Snowball method in action. The third respondent was found on their own. Interviews were conducted in person and on Skype.
We conducted three interviews - we could have done more, but there was no more time in the project. For each interview, a summary was compiled, which the customer could familiarize themselves with.
Based on the interview, user portraits were created - two designers and one “independent customer”. A portrait of an independent buyer was made according to information shared by designers and the customer. It would be nice to talk with such a buyer, but the limitations of the project did not allow.
The interview texts helped formulate the designer’s questions about the company, its work, materials, etc. These questions formed the basis of the content plan and the object-information model - a structured list of questions on which the customer was interviewed. After the documents are taken to work, there is no doubt - now they will not forget about the users. The text will answer their questions. Questions of real people.
The results of the interview helped with positioning. It turned out that own production is a huge plus in the eyes of designers, and this should be borne in mind when forming the image. Until now, the client was afraid that his company would look like an ordinary carpentry workshop, and on the old site he was silent about the workshop. Qualitative research results often go beyond UX tasks.
Solar Staff is a service for automating workflow and payments to distributed teams and freelancers. The employer is registered in the system and invites freelancers involved in the project there. The employer sets goals, negotiates a price, accepts or rejects the results, pays for the work. And the whole bureaucracy takes care of the service.
Conduct user research and audit of the interface to identify the main problems of freelancers and managers.
Before the audit, it was decided to conduct a study of freelancers and managers who had experience working with the service, using a semi-structured interview.
To develop the guide, we conducted a blitz interview inside the company - we were interested in those who worked as a freelancer or had experience in hiring them. The created guide was approved by the customer, and we began to research.
The customer was ready to help with the search for respondents - given the specifics of the project, help was very necessary. Before starting the study, customers were afraid that people would refuse to talk about hiring freelancers, since from the point of view of labor legislation this is a rather sensitive issue. Of particular concern was the topic of paying for freelancers.
Problems with the selection of respondents arose, but they were resolved - the customer approached the search for the right people seriously. Despite concerns, people agreed to discuss paying for freelancers when they explained the purpose of the study. We were interested in the payment process and emerging problems, rather than the specific amounts and destinations of transfers.
As a result, we managed to conduct ten interviews - five each with freelancers and managers.
We discussed with respondents not only work with the service, but also work experience before the service. How were the freelancer-manager relationships organized before? These are valuable insights that can be reflected in the service. In quality research, we always go beyond the boundaries of product use.
Interviews revealed problems in working with the service from freelancers and managers. The customer guessed about many problems of the service, but wanted to test his hypotheses. He did not abandon the study: as a result, the list of problems and difficulties was expanded.
The results of the study were framed in portraits of users. This helped to understand how the users of the service live and what problems they face. The received information can be used for further development of the service.
The interview allowed us to develop realistic scenarios for using the service. These scenarios were combined into two tasks for the audit: the story of a freelancer and the story of a leader. For these tasks, the interface designer conducted an audit of the service. As a result, we received a list of user problems and an expert assessment of the service interface according to real scenarios.
The customer received a list of problems and recommendations: he decided to implement the part on his own (main page), and we left the redesign of the service to us. The results are available on the site.
Sportmaster does not need long introductions - this large retailer specializing in sports goods has stores in Russia, Belarus, Ukraine, Kazakhstan and even in China.
We had to learn how shoppers and store employees work with self-service kiosks. We had to identify the problems faced by kiosk users. And not only problems with the interface.
After an interview with customers, we compiled a list of hypotheses that needed to be verified during the study.
Since the task was difficult, and the questions concerned not only interfaces, we decided to use several methods:
The study turned out to be large-scale, but the tasks were not easy. Problems with the organization of the customer were not confused. Field work was carried out in two stages: first, they communicated with customers, and then with employees. The study involved four stores in different areas of the city - the researcher visited two stores a day. Employees were interviewed directly at the workplace, and they talked with customers while they made purchases.
Buyers were motivated to participate in usability testing and interviews using loyalty card bonuses. There were practically no failures.
Despite the large volume of research procedures, problems with the organization did not arise. Field work took four days.
As a result, we conducted and recorded 20 sessions of moderated usability testing on video. In addition, they conducted about 40 interviews of varying lengths with customers and store employees.
The observation results were recorded in the diary and with the help of photo / video.
The customer received a 50-page report, which included a list of confirmed, partially confirmed and refuted hypotheses. Each conclusion was supported by a reference to the study: photographs, video recordings of the sessions.
An expert assessment of the interfaces and recommendations on how to improve these interfaces are given. Based on analytics and brainstorming, they put forward a number of suggestions on how to draw visitors' attention to the terminals. The report included seven ideas.
Based on the results of observations and interviews, we collected data on how self-service kiosks fit into the daily lives of employees and customers. This information is especially important for the development of a self-service system.
Let us return to the objections and doubts of the customer and try to answer them using case materials.
Unfortunately, high-quality research is still exotic on the Russian market for the development of IT products. Be prepared for the fact that the customer does not understand the essence of the methodology. And you need to be able to persuade him.
In this article, we will talk about our experience in applying a high-quality methodology for developing IT products in the Russian market. We will talk not only about the specifics of the work, but also about how to get started. How to explain to the customer the value of a quality methodology? What can I tell him to give the green light to communicate with users? And why does the need arise to explain?
In this article, we will discuss three key points.
- What does the customer think about quality methods in user analytics? Consider the main problems in understanding the qualitative methodology and the reasons for their appearance.
- How is a qualitative study conducted? We show on the example of our cases the organization of research, problems and their solutions, communication with the customer during the work.
- What do we get as a result of the study? How and who works with these results? On the example of cases, we consider the work of the team with the results of the study, as well as how the customer can use these results outside of UX-tasks.
Let's start with the problems of understanding a quality methodology. In fact, it will be about what objections to conducting a qualitative study are raised by the customer. Organizing a study is a job. The main thing is to start it. And for this we need to reach out to the customer. What problems can arise here?
Is it possible without research and user analytics?
A still popular question from Russian customers of different calibers. The situation is changing, but not as fast as we would like. Let's figure out what's the matter.
The customer fears that time and money will be wasted.
The concerns are generally clear: for the customer, quality methods and user analytics remain a black box. Recall that in Western countries the situation is different. And we have a black box, where we are asked to add money and the time of the project. Do not forget about it when you are talking with the customer. The situation is complicated by the fact that the outcome is unknown.
The customer does not understand what he will receive as a result of the study and how they will work with it.
This is a serious problem associated with the presentation of the results of qualitative research. For example, the numbers in quantitative studies are understandable. Web analytics is clear. The work of the copywriter and the designer is visible to the naked eye. The results of a qualitative study, packed in formats of user portraits, a description of the environment of use, life situations and scenarios are incomprehensible to the customer without a detailed explanation. What do these results give? How does the team work with them? What benefits do they bring? You must be able to answer all these customer questions.
Belief in own expert knowledge
There is another situation where the customer approximately represents ( or thinks what he represents ) what user analytics is. Perhaps I heard about quality methods. Why do you need to conduct a study if I can tell everything myself?
The customer believes in his expert knowledge of users.
What remains for us? It is necessary to understand the sources of this knowledge and its depth. For example, this is many years of experience in the industry. It's good. To analyze this experience, we conduct an interview with the customer about users. Who are they? What groups can be distinguished? What problems do they face? What are their questions? How do they even live? And many more questions that qualitative research covers. We do not deny the knowledge of the customer, but consider it as hypotheses - we verify and supplement it with the help of research.
Another possible source of expert knowledge is own research. This is very good, but you need to understand how the study was conducted? What can we learn about users, their life situations and behavior? Not all studies are equally useful for UX tasks.
There is another facet of faith in your expert knowledge - disbelief in user knowledge.
The customer doubts user competencies and knowledge.
What can they tell you? They will offer a wagon of “features”, but they don’t need them. The problem in this case is a misunderstanding of the essence of user research. The researcher studies the daily life of the user, his problems, the environment of use, and does not ask about ready-made solutions. Solution development is our task. Users do not need competencies in the development of IT products.
We will not be able to organize a study
And now let's talk about the concerns of the customer on the organization of the study.
The customer is convinced that we cannot afford to organize a study.
Organizing a study may seem like a complicated process: you need to find users, agree with the company employees, process the collected data, etc. In practice, the complexity of organizing research depends on many factors. Often, active customer involvement greatly simplifies the process.
The customer is afraid that users will refuse to participate in the study.
Various methods of motivation help to attract users to participate in the research - from delayed benefits, personal interest in the research topic to stimulation with money. Sometimes such problems do not arise at all: people are ready to help, if it is clear to explain why we are addressing them.
Before moving on to cases
Before analyzing the cases, we’ll once again list the problems and concerns that the customer has regarding user research using a high-quality methodology:
- the customer fears that time and money will be wasted;
- the customer does not understand what he will receive as a result of the study and how they will work with it;
- the customer believes in his expert knowledge of users;
- the customer doubts user competencies and knowledge;
- the customer is convinced that the study will not be able to organize;
- the customer is afraid that users will not participate in the study.
Now let's move on to the cases. The main task of this part:
- show how a qualitative methodology is used to study users in specific projects;
- show what arguments in favor of qualitative research can lead the customer.
We specially selected cases with different project objectives: content preparation, audit and prototype development of a professional interface, research of self-service kiosks. The customers are also different - small business (Set.Design), medium business (Solar Staff) and large business (Sportmaster).
Cases Set.Design and Solar Staff can be viewed on our website. The Sportmaster case, unfortunately, is not available - strict NDA conditions. The main emphasis in the analysis of cases is on:
- project objective;
- organization of research and customer participation in the work;
- collected materials;
- research results and their use.
Set. Design: texts on a site and positioning for small business
Company Set. Design produces and sells wooden eco-mosaics, decorative 3D panels, coffered ceilings and other decor items for interior decoration. The company has its own designers and its own carpentry workshop. They make products according to the catalog, adapt someone else's solution or create an original product according to the sketch.
Project objective
The company came to the new site for texts. During the discussion, we found out that the company has questions about positioning - whether it is a manufacturer of designer materials, or a supplier of solutions based on these materials, or even individual production. It’s not quite a text task, but without this, texts cannot be approached.
Research organization
The project had severe financial constraints, so the best method was to use a semi-structured interview. The time allotted was enough for only three interviews (taking into account the preparation of the text). But even three interviews are better than not one.
To determine the target audience, we talked with the customer and found out: we are interested in interior designers. Based on the results of this communication, an interview guide was compiled and discussed with the customer. The customer can offer additional questions or topics - if possible, we add them.
One respondent was offered by the customer - the buyer of his store. Another is the first designer: "a friend who does everything differently." Snowball method in action. The third respondent was found on their own. Interviews were conducted in person and on Skype.
Materials collected
We conducted three interviews - we could have done more, but there was no more time in the project. For each interview, a summary was compiled, which the customer could familiarize themselves with.
Research results and their use
Based on the interview, user portraits were created - two designers and one “independent customer”. A portrait of an independent buyer was made according to information shared by designers and the customer. It would be nice to talk with such a buyer, but the limitations of the project did not allow.
The interview texts helped formulate the designer’s questions about the company, its work, materials, etc. These questions formed the basis of the content plan and the object-information model - a structured list of questions on which the customer was interviewed. After the documents are taken to work, there is no doubt - now they will not forget about the users. The text will answer their questions. Questions of real people.
The results of the interview helped with positioning. It turned out that own production is a huge plus in the eyes of designers, and this should be borne in mind when forming the image. Until now, the client was afraid that his company would look like an ordinary carpentry workshop, and on the old site he was silent about the workshop. Qualitative research results often go beyond UX tasks.
Solar Staff: A Professional Interface for Freelancers and Leaders
Solar Staff is a service for automating workflow and payments to distributed teams and freelancers. The employer is registered in the system and invites freelancers involved in the project there. The employer sets goals, negotiates a price, accepts or rejects the results, pays for the work. And the whole bureaucracy takes care of the service.
Project objective
Conduct user research and audit of the interface to identify the main problems of freelancers and managers.
Research organization
Before the audit, it was decided to conduct a study of freelancers and managers who had experience working with the service, using a semi-structured interview.
To develop the guide, we conducted a blitz interview inside the company - we were interested in those who worked as a freelancer or had experience in hiring them. The created guide was approved by the customer, and we began to research.
The customer was ready to help with the search for respondents - given the specifics of the project, help was very necessary. Before starting the study, customers were afraid that people would refuse to talk about hiring freelancers, since from the point of view of labor legislation this is a rather sensitive issue. Of particular concern was the topic of paying for freelancers.
Problems with the selection of respondents arose, but they were resolved - the customer approached the search for the right people seriously. Despite concerns, people agreed to discuss paying for freelancers when they explained the purpose of the study. We were interested in the payment process and emerging problems, rather than the specific amounts and destinations of transfers.
Materials collected
As a result, we managed to conduct ten interviews - five each with freelancers and managers.
We discussed with respondents not only work with the service, but also work experience before the service. How were the freelancer-manager relationships organized before? These are valuable insights that can be reflected in the service. In quality research, we always go beyond the boundaries of product use.
Research results and their use
Interviews revealed problems in working with the service from freelancers and managers. The customer guessed about many problems of the service, but wanted to test his hypotheses. He did not abandon the study: as a result, the list of problems and difficulties was expanded.
The results of the study were framed in portraits of users. This helped to understand how the users of the service live and what problems they face. The received information can be used for further development of the service.
The interview allowed us to develop realistic scenarios for using the service. These scenarios were combined into two tasks for the audit: the story of a freelancer and the story of a leader. For these tasks, the interface designer conducted an audit of the service. As a result, we received a list of user problems and an expert assessment of the service interface according to real scenarios.
The customer received a list of problems and recommendations: he decided to implement the part on his own (main page), and we left the redesign of the service to us. The results are available on the site.
Sportmaster: a study of self-service stalls
Sportmaster does not need long introductions - this large retailer specializing in sports goods has stores in Russia, Belarus, Ukraine, Kazakhstan and even in China.
Project objective
We had to learn how shoppers and store employees work with self-service kiosks. We had to identify the problems faced by kiosk users. And not only problems with the interface.
Research organization
After an interview with customers, we compiled a list of hypotheses that needed to be verified during the study.
Since the task was difficult, and the questions concerned not only interfaces, we decided to use several methods:
- semi-structured interviews with customers and store employees;
- monitoring the interaction of customers and employees with self-service kiosks;
- usability testing of self-service kiosks with customers and employees.
The study turned out to be large-scale, but the tasks were not easy. Problems with the organization of the customer were not confused. Field work was carried out in two stages: first, they communicated with customers, and then with employees. The study involved four stores in different areas of the city - the researcher visited two stores a day. Employees were interviewed directly at the workplace, and they talked with customers while they made purchases.
Buyers were motivated to participate in usability testing and interviews using loyalty card bonuses. There were practically no failures.
Despite the large volume of research procedures, problems with the organization did not arise. Field work took four days.
Materials collected
As a result, we conducted and recorded 20 sessions of moderated usability testing on video. In addition, they conducted about 40 interviews of varying lengths with customers and store employees.
The observation results were recorded in the diary and with the help of photo / video.
Research results and their use
The customer received a 50-page report, which included a list of confirmed, partially confirmed and refuted hypotheses. Each conclusion was supported by a reference to the study: photographs, video recordings of the sessions.
An expert assessment of the interfaces and recommendations on how to improve these interfaces are given. Based on analytics and brainstorming, they put forward a number of suggestions on how to draw visitors' attention to the terminals. The report included seven ideas.
Based on the results of observations and interviews, we collected data on how self-service kiosks fit into the daily lives of employees and customers. This information is especially important for the development of a self-service system.
Conclusion: how to achieve access to users?
Let us return to the objections and doubts of the customer and try to answer them using case materials.
- The customer fears that time and money will be wasted. Qualitative research allows you to get detailed information about users and their behavior. Even three interviews help gather valuable data about your users.
- The customer does not understand what he will receive as a result of the study and how they will work with it. The research results are used by the project team throughout the work. Custom questions help write texts on the site. Life situations and scenarios are needed when auditing an interface. The results of the monitoring will allow you to go beyond the boundaries of the interface and see new problems that interfere with the effective work with your IT product. The important thing is that users will not be forgotten. Sometimes the results of qualitative research go beyond solving UX problems. For example, they help determine the positioning of a company or service. These results can be used including by marketers of the company.
- The customer believes in his expert knowledge of users. Hypotheses that are at the beginning of the work of the customer or the project team are verified during research. Some hypotheses can be confirmed, some are disproved. The project team will be able to work with confirmed hypotheses in the future. We do not give up the knowledge of the customer, but we check it and supplement it.
- The customer doubts user competencies and knowledge. In research, we study user experience. We collect this experience in order to make product decisions based on it. At the same time, in the study, we specifically go beyond the scope of the product in order to get valuable insights from the user's daily life that can be reflected in the product.
- The customer is convinced that the study will not be able to organize. Any research can be organized. Of course, there are problems, but they can be solved. Customer involvement in research helps speed up many processes.
- The customer is afraid that users will refuse to participate in the study. Users are ready to talk. There are many ways to motivate them. Of course, financial incentives simplify the data collection process, but you can do without it.
Unfortunately, high-quality research is still exotic on the Russian market for the development of IT products. Be prepared for the fact that the customer does not understand the essence of the methodology. And you need to be able to persuade him.