November 14, 2014 at 17:45Personalization for search services
In this paper, the principles of the application of personalization technology that takes into account the psychological characteristics of users in the formation of search results for information retrieval are described.
The purpose of this technology is to create for the user a personal comfortable space on the Internet in general, and a positive experience in interacting with the search service in particular. As a result, search services receive tools to optimize the use of their resources.
Currently, for each user's information request, the search engine finds thousands of resources. How to determine which resources will interest the user in order to assign them a higher rank among all others? Solving the relevance of releasing an information request is one of the priority tasks for search services, and not because they want to save user time, but because of the resource-intensive process. Since current systems need to periodically scan and index all pages on the Internet, determine their popularity with reference to users' search queries, store information about all requests from all users, so that sometimes they can refer to the query history, the question of optimizing the use of the resource becomes very relevant.
One of the optimization options is a personalized search, designed to save the user from the need to spend time turning over the pages of the issue, and the service - from “pulling up” new pages.
Until recently, personal search used certain data about the user (whether it was a short-term or long-term search history, interests or something else) to increase the relevance of the search or did not use it.
This technology is universal for use in various fields of Internet activity, combined by the two concepts of "man - digital information." This may include referral systems, advertising, information retrieval, etc.
The described technology of personalization is based on 2 basic principles. First - the needs of a person determine the vector of his activity (in other words - behavior), conscious or unconscious. The exception is some mental disorders, but we will not take them into account. The second principle is the reactivity of the psyche, that is, when the stimulus is exposed, the psyche gives out a certain definite reaction to it.
With regard to human behavior on the Internet, these principles form a chain - "the search for relevant information - the consumption of information." (It should be noted that the first - in this case, the word "relevant" reflects the situational nature of the phenomenon. A person has a number of needs, updating one or more of which determines the situational vector of behavior; the second - even if a person accesses the Internet for communication, this does not change essence - a certain need is satisfied, and communication itself acts as a way to achieve a result). In other words, the user purposefully performs Internet surfing (the sequence of transitions from page to page, but in fact - a search), and the psyche gives a response to each content unit - positive (suitable) or negative (not suitable).
Initially, the problem of personalized search lies at the junction of two areas of science - psychology and information technology. In this case, psychology should answer the questions - what exactly are the elements of the content and how to take into account when forming the issuance of an information request so that the issuance can be called personalized, that is, satisfying the needs of this user personally. Information technology, in turn, should provide tools (algorithms) for the isolation and interpretation of these elements.
The proposed approach is to build a model of the user's psyche, which will allow a high probability to predict the response of the user to the proposed unit of content. (In this case, “high probability” means a probability equal to or greater than 80%).
The construction of the psyche model of a specific user is carried out according to the following algorithm:
1) Registration of the user’s reaction to the content unit. As a positive reaction, both the fact of a click on the link (or the indication of the resource in the browser address bar by the user himself) and the time spent on the page are taken into account.
2) The selection in the link and its description (if there was a link) or in the subject matter of the resource (if there was a use of the address bar of the browser) of semantic units (hereinafter referred to as “semantic units” we mean some ideas) to which the user's psyche could react . For this purpose, a semantic network has been developed that describes the relationship between certain phenomena of the external world and their reflection in the internal (mental) plane of a person, up to the identification of personal characteristics that are inherent or formed during ontogenesis that determine just such a variant of reflection.
3) The construction of a model that contains not only categories that are directly determined by the reaction to certain content elements, but also, basically, the underlying personality characteristics (innate, acquired during ontogenesis), but also [categories] related to relevant cultural and information space (accepted social norms, belonging to one or another social class, etc.).
However, as it turned out in practice, the use of the psyche model in this form for solving applied problems was difficult. Therefore, it was proposed to form a user profile that will contain a number of user characteristics that reflect his reaction to a limited number of semantic units that are significant for search behavior. The user profile is a simplified version of the psyche model that is easy to apply for solving applied problems.
After analyzing, using the expert method, queries in the users ’search activity provided by Yandex for free access (due to the competition), it was possible to distinguish 9 main groups of queries, such as a request for instructions, receiving on the Internet, choice, etc.
The analysis took into account the “idea” of the request, ignoring the expression form of the given idea.
For each of the groups, an expert method has compiled a glossary. Words in the glossaries of the various query groups did not overlap. Using this glossary, you can automatically determine a group by query keywords.
Also, for each of the groups, an expert method developed a specific system of scales. These scales reflect the invariance of the significant characteristics of the information that will be contained in the responses to queries related to this group. For example, for the group "Request for instructions" the following scales were defined: level of intelligence, theory-practice, superficiality-in-depth, imagery, randomness-structurality and others. Different groups include a different number of scales.
Each user, due to personal characteristics, forms specific expectations for him from a response to a request, that is, each user "expects" to receive information with a certain set of characteristics. An example for a request for a new smartphone is if a young girl makes such a request, then she, with a high probability, expects to see resources with a large number of photos in order to evaluate the design, color scheme, etc., since a smartphone is fashionable for her accessory. If this request is made by a geek, then he will expect information from the technical plan in order to evaluate the capabilities of the new product and compare it with analogues, if any.
The exception is situations when using the search system as an analogue of a calculator, slide rule or any other device, the output of information in which is maximally unified in terms of interpretation and is usually made in the form of symbols (numbers and / or conventional signs).
Based on the foregoing, we want to emphasize 2 points: first - the scales reflect the characteristics of the information; the second - the personal characteristics of the user determine his expectations.
Therefore, personal search, in fact, boils down to determining the expectations of a particular user and offering him information with the necessary characteristics. This procedure does not affect the subject of the search.
The proposed personalization technology, on the one hand, makes it possible to determine the expectations of a particular user. For this purpose, the user's query history is analyzed and his profile is formed, where all scales relevant to this user for each group of queries will be displayed. On the other hand, technology offers tools for characterizing information.
Both the formation of the initial user profile and its subsequent recalculations, and the determination of the characteristics of information can be performed offline, which will reduce the operation in real time to the calculation of the current index of the [necessary] group of queries in the user profile and ranking resources according to the maximum correspondence to the calculated index.
An experiment was conducted to assess the feasibility of using the proposed technology for personalized search. As a working hypothesis, the assumption was chosen that the use of this personalization technology in a search service increases the relevance of search results for an information request, which means that the rank of the resource selected by the user will be less.
Description of the sample The
experiment was conducted on a sample of data provided for public access by Yandex as part of an open competition for personalized network search (Personalized Wed Search Challenge). This "base" contained information about search activity ~ 41 thousand. users for a period of 60 days. The following data was offered:
- User
ID - Request ID
- the text of the user's search query
- the unified text of the search engine’s query
- the list of resources on the first page of the search (or the first 10 resources) and the corresponding rank
- the rank of the resource that the user clicked on.
Due to limited production capacity, the sample was reduced to 4 500 people selected by the system randomly.
As a result, the sampling parameters were as follows:
- number of users - 4,500 people.
- accounting period of activity - 60 days (from 19-09-13 to 17-11-13)
- total number of requests - 1 104 347
Description of the experiment
As a method for the experiment, a modified A / B test was chosen. The modification consisted in the fact that the division into the experimental and control group was carried out not among the subjects, but among the objects. In other words, the search activity of users, not the users themselves, was divided into groups. This was due to the fact that this approach implies the formation of a profile for each user with his personal characteristics.
At the 1st stage of the experiment, the system categorized the queries, that is, assigned to one of the above 9 groups.
In the sample, the selected groups covered 48.7% (538,441) of all requests. The remaining 51.3% (565 906) included queries that the system could not identify due to grammatical errors, the use of transliteration, and the use of words that were not included in any of the glossaries.
At the 2nd stage, an assessment was made of the possibility of forming a personal profile for users of the sample. Since there is a lower limit for user search activity (in our case 40 queries) to form a functional profile within the framework of the proposed approach, the system discarded users who do not meet this criterion and their requests. As a result, the sample decreased to 3,826 people. with a total number of requests - 523 007.
This stage is due solely to limited data. Search services should not have this problem.
At the 3rd stage, the requests were divided into an experimental and a control group.
It was decided to divide into groups in the ratio of 80:20, that is, 80% of the activity of each user fall into the experimental group, 20% - in the control.
As a result, in the experimental group 418 406 requests were received, and in the control group - 104 601.
At the 4th stage, requests from the experimental group (418 406 requests) were processed by the system according to the scheme: determining the request group -> assessing the severity of significant scales for the resource selected by user.
The query group was defined by comparing the query keywords with the glossaries of the query groups. Then the system determined significant scales for this group of requests and evaluated their severity. We cannot disclose the principles and mechanism for assessing severity, since this information is a trade secret, but we can say that they are accessible to machine learning.
According to the results of processing requests in the experimental group, user profiles were formed. In the profile for each of the groups of requests, the names of the scales that are significant for a given user and calculated coefficients (the level of priority of the scale for the user) were indicated.
It should be noted that, due to the limited design capacity, the system processed the resources as a whole, rather than specific resource pages selected by users. For example, if the user selected slovo.ws/resh/007 , then the system analyzed slovo.ws . This fact, firstly, led to the fact that all content aggregators, such as social networks, Youtube, etc., dropped out of processing due to the wide variety of content on them; secondly, this could not but affect the results of the experiment in the direction of their deterioration.
At the 5th stage, requests from the control group (104 601 requests) were processed by the system according to the following scheme: determining the request group -> determining the severity of significant scales -> calculating the index of compliance with a personal profile -> ranking resources in the search results by correspondence index -> determining the rank of a resource selected by the user.
The definition of the query group was carried out, as in the previous step, by comparing the query keywords with the glossaries of the query groups. Then the system determined significant scales for this group of requests and evaluated their severity. After that, the correspondence index was calculated for each resource from the search results (first 10 resources). This index reflected how much the scales themselves and their severity of the resource in the output correspond to the scales that are significant for the user for this group of requests in his personal profile. Based on the calculated compliance indices, the system compiled a ranked list of resources in the search results and determined the rank of the resource selected by the user.
At the final stage, there was an analysis of requests by the criterion of information content. After filtering out non-informative queries, the average ranks of the selected resources were calculated and compared in the search engine results and in the lists ranked by personal correspondence indices.
When analyzing queries regarding their information content, the following were eliminated:
a) queries when the user selected resources in the search results strictly in a rank sequence, for example, (resource ranks) 1 *; 12; 1, 2, 3; 1, 2, 3, 4; 1, 2, 3, 4, 5, etc., - because, as we believe, such behavior does not reflect personal preferences, but rather is a result of stereotyping perceptions of ranked information - from the most significant (top) to less significant (lower in to the list). It would be possible to assume that in this case the last resource is the most relevant for the user (after reading it, the user has stopped / changed the search activity for this request). In this case, it was necessary to analyze (compare) the following time parameters, - the time between the presentation of the search results and the first choice of the user, the time of interaction with the content, the time of the next request - and the uniqueness of the content, - the presence of disjoint information, presentation style, etc. The specified analysis we didn’t
* - to determine that the user has chosen the resource No. 1 because of meeting his expectations (that is, the user's personal characteristics determined the choice), and not because this is a template action, it is necessary to analyze the time between presenting the search results and clicking on the link. The time period should be sufficient to compare the parameters of resource No. 1 (wording of the link phrase, snippet content, address of the resource) with the corresponding parameters of at least the following in the list of resource No. 2. As mentioned above, there were no temporary parameters in the database provided by the search service.
The presence of full information on search history in search services can be used to determine the presence of template actions in the user's search behavior and then optimize the use of personal search technology.
b) requests if at least one of the resources in the relevant search results did not have a compliance index. Despite the fact that the lack of an index could not increase the effectiveness of the proposed approach, since it put resources down the list, however, the analysis should be based on the choices determined by the personal characteristics of users, and not the imperfection of the equipment.
c) requests where in the search results there were links to different pages of one resource. Since the system did not analyze specific pages, but the resources as a whole, when compiling a ranked list it was not possible to separate different pages of the same resource.
As a result, the calculation of average ranks was carried out on 74,279 queries (~ 71% of the total number of queries in the control group). The average rank of selected resources in the search engine results was 3.6. The average rank in the lists ranked by personal correspondence indices was 2.9. That is, the indicator improved by ~ 19.4%, which for the control group as a whole (taking into account users for whom the profile was not formed) gave a result of ~ 16.1%.
Interpretation of Results
The results of the experiment showed that the application of personalization technology in the field of information retrieval increases the relevance of search results, which can indirectly be judged by the decrease in the rank of the resource selected by the user. In our case, the efficiency in the control group increased by ~ 16.1%.
We look at the result quite realistically. So, for example, having more data on user activity (specifically, time parameters), you would not have to filter out all requests where the user selected resources in a rank sequence, and only a part - from users with template search behavior. This would undoubtedly reduce the final result. On the other hand, if we had more productive resources, we could analyze specific pages in the search results, and not the resources as a whole. This would slightly increase efficiency, since it would allow taking into account a number of factors that, in our opinion, are important when choosing a source of information (uniqueness of information, presentation style, etc.).
In our opinion, the proposed personalization technology for search services has a number of advantages designed to optimize the operation of search services. First, building a model of the user's psyche provides a high degree of predictability of this approach. Secondly, part of the operations can be taken offline, which allows you to optimally distribute resources and reduce the load during periods of online activity, for example, determining the severity of scales for resources and / or individual pages.
In the case of users with a stereotypical perception of ranked information, the application of this approach will not affect the relevance of the issue directly (with or without personalized search, such users browse resources according to the rank sequence). However, if we assume that such users stop / change the search activity when a resource is found that meets their expectations, then the use of personal preferences will allow to “raise” this resource higher in the search results, thereby forming a positive experience of interaction with the search service.
The article was co-written by Sergey Lepikhov, Alexander Golovan.
The purpose of this technology is to create for the user a personal comfortable space on the Internet in general, and a positive experience in interacting with the search service in particular. As a result, search services receive tools to optimize the use of their resources.
Currently, for each user's information request, the search engine finds thousands of resources. How to determine which resources will interest the user in order to assign them a higher rank among all others? Solving the relevance of releasing an information request is one of the priority tasks for search services, and not because they want to save user time, but because of the resource-intensive process. Since current systems need to periodically scan and index all pages on the Internet, determine their popularity with reference to users' search queries, store information about all requests from all users, so that sometimes they can refer to the query history, the question of optimizing the use of the resource becomes very relevant.
One of the optimization options is a personalized search, designed to save the user from the need to spend time turning over the pages of the issue, and the service - from “pulling up” new pages.
Until recently, personal search used certain data about the user (whether it was a short-term or long-term search history, interests or something else) to increase the relevance of the search or did not use it.
This technology is universal for use in various fields of Internet activity, combined by the two concepts of "man - digital information." This may include referral systems, advertising, information retrieval, etc.
The described technology of personalization is based on 2 basic principles. First - the needs of a person determine the vector of his activity (in other words - behavior), conscious or unconscious. The exception is some mental disorders, but we will not take them into account. The second principle is the reactivity of the psyche, that is, when the stimulus is exposed, the psyche gives out a certain definite reaction to it.
With regard to human behavior on the Internet, these principles form a chain - "the search for relevant information - the consumption of information." (It should be noted that the first - in this case, the word "relevant" reflects the situational nature of the phenomenon. A person has a number of needs, updating one or more of which determines the situational vector of behavior; the second - even if a person accesses the Internet for communication, this does not change essence - a certain need is satisfied, and communication itself acts as a way to achieve a result). In other words, the user purposefully performs Internet surfing (the sequence of transitions from page to page, but in fact - a search), and the psyche gives a response to each content unit - positive (suitable) or negative (not suitable).
Initially, the problem of personalized search lies at the junction of two areas of science - psychology and information technology. In this case, psychology should answer the questions - what exactly are the elements of the content and how to take into account when forming the issuance of an information request so that the issuance can be called personalized, that is, satisfying the needs of this user personally. Information technology, in turn, should provide tools (algorithms) for the isolation and interpretation of these elements.
The proposed approach is to build a model of the user's psyche, which will allow a high probability to predict the response of the user to the proposed unit of content. (In this case, “high probability” means a probability equal to or greater than 80%).
The construction of the psyche model of a specific user is carried out according to the following algorithm:
1) Registration of the user’s reaction to the content unit. As a positive reaction, both the fact of a click on the link (or the indication of the resource in the browser address bar by the user himself) and the time spent on the page are taken into account.
2) The selection in the link and its description (if there was a link) or in the subject matter of the resource (if there was a use of the address bar of the browser) of semantic units (hereinafter referred to as “semantic units” we mean some ideas) to which the user's psyche could react . For this purpose, a semantic network has been developed that describes the relationship between certain phenomena of the external world and their reflection in the internal (mental) plane of a person, up to the identification of personal characteristics that are inherent or formed during ontogenesis that determine just such a variant of reflection.
3) The construction of a model that contains not only categories that are directly determined by the reaction to certain content elements, but also, basically, the underlying personality characteristics (innate, acquired during ontogenesis), but also [categories] related to relevant cultural and information space (accepted social norms, belonging to one or another social class, etc.).
However, as it turned out in practice, the use of the psyche model in this form for solving applied problems was difficult. Therefore, it was proposed to form a user profile that will contain a number of user characteristics that reflect his reaction to a limited number of semantic units that are significant for search behavior. The user profile is a simplified version of the psyche model that is easy to apply for solving applied problems.
After analyzing, using the expert method, queries in the users ’search activity provided by Yandex for free access (due to the competition), it was possible to distinguish 9 main groups of queries, such as a request for instructions, receiving on the Internet, choice, etc.
The analysis took into account the “idea” of the request, ignoring the expression form of the given idea.
For each of the groups, an expert method has compiled a glossary. Words in the glossaries of the various query groups did not overlap. Using this glossary, you can automatically determine a group by query keywords.
Also, for each of the groups, an expert method developed a specific system of scales. These scales reflect the invariance of the significant characteristics of the information that will be contained in the responses to queries related to this group. For example, for the group "Request for instructions" the following scales were defined: level of intelligence, theory-practice, superficiality-in-depth, imagery, randomness-structurality and others. Different groups include a different number of scales.
Each user, due to personal characteristics, forms specific expectations for him from a response to a request, that is, each user "expects" to receive information with a certain set of characteristics. An example for a request for a new smartphone is if a young girl makes such a request, then she, with a high probability, expects to see resources with a large number of photos in order to evaluate the design, color scheme, etc., since a smartphone is fashionable for her accessory. If this request is made by a geek, then he will expect information from the technical plan in order to evaluate the capabilities of the new product and compare it with analogues, if any.
The exception is situations when using the search system as an analogue of a calculator, slide rule or any other device, the output of information in which is maximally unified in terms of interpretation and is usually made in the form of symbols (numbers and / or conventional signs).
Based on the foregoing, we want to emphasize 2 points: first - the scales reflect the characteristics of the information; the second - the personal characteristics of the user determine his expectations.
Therefore, personal search, in fact, boils down to determining the expectations of a particular user and offering him information with the necessary characteristics. This procedure does not affect the subject of the search.
The proposed personalization technology, on the one hand, makes it possible to determine the expectations of a particular user. For this purpose, the user's query history is analyzed and his profile is formed, where all scales relevant to this user for each group of queries will be displayed. On the other hand, technology offers tools for characterizing information.
Both the formation of the initial user profile and its subsequent recalculations, and the determination of the characteristics of information can be performed offline, which will reduce the operation in real time to the calculation of the current index of the [necessary] group of queries in the user profile and ranking resources according to the maximum correspondence to the calculated index.
An experiment was conducted to assess the feasibility of using the proposed technology for personalized search. As a working hypothesis, the assumption was chosen that the use of this personalization technology in a search service increases the relevance of search results for an information request, which means that the rank of the resource selected by the user will be less.
Description of the sample The
experiment was conducted on a sample of data provided for public access by Yandex as part of an open competition for personalized network search (Personalized Wed Search Challenge). This "base" contained information about search activity ~ 41 thousand. users for a period of 60 days. The following data was offered:
- User
ID - Request ID
- the text of the user's search query
- the unified text of the search engine’s query
- the list of resources on the first page of the search (or the first 10 resources) and the corresponding rank
- the rank of the resource that the user clicked on.
Due to limited production capacity, the sample was reduced to 4 500 people selected by the system randomly.
As a result, the sampling parameters were as follows:
- number of users - 4,500 people.
- accounting period of activity - 60 days (from 19-09-13 to 17-11-13)
- total number of requests - 1 104 347
Description of the experiment
As a method for the experiment, a modified A / B test was chosen. The modification consisted in the fact that the division into the experimental and control group was carried out not among the subjects, but among the objects. In other words, the search activity of users, not the users themselves, was divided into groups. This was due to the fact that this approach implies the formation of a profile for each user with his personal characteristics.
At the 1st stage of the experiment, the system categorized the queries, that is, assigned to one of the above 9 groups.
In the sample, the selected groups covered 48.7% (538,441) of all requests. The remaining 51.3% (565 906) included queries that the system could not identify due to grammatical errors, the use of transliteration, and the use of words that were not included in any of the glossaries.
At the 2nd stage, an assessment was made of the possibility of forming a personal profile for users of the sample. Since there is a lower limit for user search activity (in our case 40 queries) to form a functional profile within the framework of the proposed approach, the system discarded users who do not meet this criterion and their requests. As a result, the sample decreased to 3,826 people. with a total number of requests - 523 007.
This stage is due solely to limited data. Search services should not have this problem.
At the 3rd stage, the requests were divided into an experimental and a control group.
It was decided to divide into groups in the ratio of 80:20, that is, 80% of the activity of each user fall into the experimental group, 20% - in the control.
As a result, in the experimental group 418 406 requests were received, and in the control group - 104 601.
At the 4th stage, requests from the experimental group (418 406 requests) were processed by the system according to the scheme: determining the request group -> assessing the severity of significant scales for the resource selected by user.
The query group was defined by comparing the query keywords with the glossaries of the query groups. Then the system determined significant scales for this group of requests and evaluated their severity. We cannot disclose the principles and mechanism for assessing severity, since this information is a trade secret, but we can say that they are accessible to machine learning.
According to the results of processing requests in the experimental group, user profiles were formed. In the profile for each of the groups of requests, the names of the scales that are significant for a given user and calculated coefficients (the level of priority of the scale for the user) were indicated.
It should be noted that, due to the limited design capacity, the system processed the resources as a whole, rather than specific resource pages selected by users. For example, if the user selected slovo.ws/resh/007 , then the system analyzed slovo.ws . This fact, firstly, led to the fact that all content aggregators, such as social networks, Youtube, etc., dropped out of processing due to the wide variety of content on them; secondly, this could not but affect the results of the experiment in the direction of their deterioration.
At the 5th stage, requests from the control group (104 601 requests) were processed by the system according to the following scheme: determining the request group -> determining the severity of significant scales -> calculating the index of compliance with a personal profile -> ranking resources in the search results by correspondence index -> determining the rank of a resource selected by the user.
The definition of the query group was carried out, as in the previous step, by comparing the query keywords with the glossaries of the query groups. Then the system determined significant scales for this group of requests and evaluated their severity. After that, the correspondence index was calculated for each resource from the search results (first 10 resources). This index reflected how much the scales themselves and their severity of the resource in the output correspond to the scales that are significant for the user for this group of requests in his personal profile. Based on the calculated compliance indices, the system compiled a ranked list of resources in the search results and determined the rank of the resource selected by the user.
At the final stage, there was an analysis of requests by the criterion of information content. After filtering out non-informative queries, the average ranks of the selected resources were calculated and compared in the search engine results and in the lists ranked by personal correspondence indices.
When analyzing queries regarding their information content, the following were eliminated:
a) queries when the user selected resources in the search results strictly in a rank sequence, for example, (resource ranks) 1 *; 12; 1, 2, 3; 1, 2, 3, 4; 1, 2, 3, 4, 5, etc., - because, as we believe, such behavior does not reflect personal preferences, but rather is a result of stereotyping perceptions of ranked information - from the most significant (top) to less significant (lower in to the list). It would be possible to assume that in this case the last resource is the most relevant for the user (after reading it, the user has stopped / changed the search activity for this request). In this case, it was necessary to analyze (compare) the following time parameters, - the time between the presentation of the search results and the first choice of the user, the time of interaction with the content, the time of the next request - and the uniqueness of the content, - the presence of disjoint information, presentation style, etc. The specified analysis we didn’t
* - to determine that the user has chosen the resource No. 1 because of meeting his expectations (that is, the user's personal characteristics determined the choice), and not because this is a template action, it is necessary to analyze the time between presenting the search results and clicking on the link. The time period should be sufficient to compare the parameters of resource No. 1 (wording of the link phrase, snippet content, address of the resource) with the corresponding parameters of at least the following in the list of resource No. 2. As mentioned above, there were no temporary parameters in the database provided by the search service.
The presence of full information on search history in search services can be used to determine the presence of template actions in the user's search behavior and then optimize the use of personal search technology.
b) requests if at least one of the resources in the relevant search results did not have a compliance index. Despite the fact that the lack of an index could not increase the effectiveness of the proposed approach, since it put resources down the list, however, the analysis should be based on the choices determined by the personal characteristics of users, and not the imperfection of the equipment.
c) requests where in the search results there were links to different pages of one resource. Since the system did not analyze specific pages, but the resources as a whole, when compiling a ranked list it was not possible to separate different pages of the same resource.
As a result, the calculation of average ranks was carried out on 74,279 queries (~ 71% of the total number of queries in the control group). The average rank of selected resources in the search engine results was 3.6. The average rank in the lists ranked by personal correspondence indices was 2.9. That is, the indicator improved by ~ 19.4%, which for the control group as a whole (taking into account users for whom the profile was not formed) gave a result of ~ 16.1%.
Interpretation of Results
The results of the experiment showed that the application of personalization technology in the field of information retrieval increases the relevance of search results, which can indirectly be judged by the decrease in the rank of the resource selected by the user. In our case, the efficiency in the control group increased by ~ 16.1%.
We look at the result quite realistically. So, for example, having more data on user activity (specifically, time parameters), you would not have to filter out all requests where the user selected resources in a rank sequence, and only a part - from users with template search behavior. This would undoubtedly reduce the final result. On the other hand, if we had more productive resources, we could analyze specific pages in the search results, and not the resources as a whole. This would slightly increase efficiency, since it would allow taking into account a number of factors that, in our opinion, are important when choosing a source of information (uniqueness of information, presentation style, etc.).
In our opinion, the proposed personalization technology for search services has a number of advantages designed to optimize the operation of search services. First, building a model of the user's psyche provides a high degree of predictability of this approach. Secondly, part of the operations can be taken offline, which allows you to optimally distribute resources and reduce the load during periods of online activity, for example, determining the severity of scales for resources and / or individual pages.
In the case of users with a stereotypical perception of ranked information, the application of this approach will not affect the relevance of the issue directly (with or without personalized search, such users browse resources according to the rank sequence). However, if we assume that such users stop / change the search activity when a resource is found that meets their expectations, then the use of personal preferences will allow to “raise” this resource higher in the search results, thereby forming a positive experience of interaction with the search service.
The article was co-written by Sergey Lepikhov, Alexander Golovan.