August 5, 2014 at 13:11Evolution of the list of recommendations in SmartProgress
The selection of the most interesting user-generated content for users is an urgent task for many projects, and we are no exception. In this article I want to talk about how we solved this problem from the moment the project was launched until today using the example of the list of goals in SmartProgress .
Stage 1: Moderation.
This is the easiest way to organize the process of selecting user-generated content. Therefore, at the start of the project, we resorted to this particular method. Its main advantage is the speed and ease of implementation. What could be easier is to go through the list of new content and mark the most interesting ones, and then take this mark into account in the selection. But the disadvantages of this decision are significant:
1) A person is required to constantly monitor new goals. And this person needs to constantly pay a salary. Or spend your time on this operation.
2) The moderator selects goals on a subjective basis - as he thinks, and not the fact that other users share his opinion.
3) With this selection, a single list of goals is created, which is shown to all users, and all users have different tastes and preferences, and the fact that some users may be interested, others are completely indifferent.
Stage 2: We follow preferences of users.
To implement the consideration of user preferences, we needed to think much better about the scheme of combining goals than just a list of categories. For this, we have introduced the concept of a group of goals. Goal group is a list of all user goals united by one specific topic, for example, “Promotion and development on the Internet” or “Moving abroad”. We have already created more than a hundred of such groups. You can read more about groups, and how we did them in our articles:
Similar goals. A new tool for searching like-minded
ElasticSearch and vice versa. Percolate API
Now most of our goals are in groups. And knowing what goals the user sets and what goals he subscribes to, we know his preferences. We get a list of groups in which these goals are composed - and show other goals from these groups.
As a result, the list of recommendations has become more personalized, taking into account the preferences and interests of each user.
But tracking this recommendation system, we found several problems that are still relevant for it:
1) Uneven distribution of goals among groups. Those. if the user has subscribed to 1 goal in group A, in which there are much more goals than in other groups, then the goals from group A will prevail in the recommendations.
2) With an insufficient flow of new goals for its relevant groups - recommendations stagnate. Because with such a goal list system, we artificially limit the selection of goals only to the user's preferences, then we reduce the flow of new content for the user, and the user constantly sees the same goals in his recommendation.
Stage 3: Do not show the goal. Which the user has already watched.
In order for the user not to blur the eyes with the same goals, we decided to hide from the display the goals that the user had already viewed. To do this, we added the table `goals_ignore`, which contains goals that the user looked at, as well as those that he persistently ignores. Those. if the user was shown the goal 10 times in the list of recommendations and he never entered it, then it is not interesting to him and this user does not need to show it anymore - put it in `goals_ignore`.
Thus, it was possible to achieve that the list of recommendations began to clear off goals that the user had already seen or did not want to watch.
Stage 4: Probabilistic sampling of goals. We show first of all the most interesting.
To do this, we introduced the CTR for the target, which was calculated from the ratio of the target’s opening to its views in the list of recommendations. And they began to sort the list of recommendations for this indicator. Thus, the user first of all sees the most interesting goals. But with this scheme, less interesting goals fall lower and lower and the ability to reach them is constantly decreasing. Therefore, we decided to introduce a certain randomness in the selection of goals. That would be less interesting goals still had the chance to be on the first lines of the list and thereby adjust your ctr, in case of an erroneous understatement.
To do this, we run the crown script to calculate a random goal coefficient based on their ctr.
And already by the order_random parameter we sort the output.
Due to this, the list of recommendations began to rotate and the goals that long ago plunged deep down began to appear on the first positions of the list. In addition to improving the list of recommendations for users, this step gave a new impetus to old goals. Users went into forgotten goals, brought activity and the authors returned to their goals again.
Step 5. We monitor the degree of user preference.
If the user sets more goals in one group or subscribes more to the goals of a certain group, then this group of goals is more interesting to him than others, so in the recommendations he needs to show more goals from this group accordingly.
And so, using steps 3-5, we were able to solve the problems described for the list of recommendations from the second stage, namely:
1) Not uniform distribution of goals into groups
2) Stagnation of the list of goals with an insufficient flow of new ones.
But a new problem was discovered. We do not know the preferences of new users who have just registered and have not set more than one goal and have not subscribed to others.
Step 6. If you don’t know what your preferences are, ask them.
After registration, we show users a guide to the site, which helps them quickly figure out the interface. A small questionnaire was added to it, in which they asked to mark the categories that are most interesting to the user. And the user’s responses began to build his list of recommendations until a certain number of subscriptions were recruited, after which we switched the user to our standard recommendation system.
As a result, the average viewing depth increased by 15% from 3.9 to 4.5. And the average length of stay is 36% from 5.5 minutes. up to 7.5 minutes
In one and the next article we will tell you how it all works quickly for us.
SmartProgress - Achieving Goals
Stage 1: Moderation.
This is the easiest way to organize the process of selecting user-generated content. Therefore, at the start of the project, we resorted to this particular method. Its main advantage is the speed and ease of implementation. What could be easier is to go through the list of new content and mark the most interesting ones, and then take this mark into account in the selection. But the disadvantages of this decision are significant:
1) A person is required to constantly monitor new goals. And this person needs to constantly pay a salary. Or spend your time on this operation.
2) The moderator selects goals on a subjective basis - as he thinks, and not the fact that other users share his opinion.
3) With this selection, a single list of goals is created, which is shown to all users, and all users have different tastes and preferences, and the fact that some users may be interested, others are completely indifferent.
Stage 2: We follow preferences of users.
To implement the consideration of user preferences, we needed to think much better about the scheme of combining goals than just a list of categories. For this, we have introduced the concept of a group of goals. Goal group is a list of all user goals united by one specific topic, for example, “Promotion and development on the Internet” or “Moving abroad”. We have already created more than a hundred of such groups. You can read more about groups, and how we did them in our articles:
Similar goals. A new tool for searching like-minded
ElasticSearch and vice versa. Percolate API
Now most of our goals are in groups. And knowing what goals the user sets and what goals he subscribes to, we know his preferences. We get a list of groups in which these goals are composed - and show other goals from these groups.
As a result, the list of recommendations has become more personalized, taking into account the preferences and interests of each user.
But tracking this recommendation system, we found several problems that are still relevant for it:
1) Uneven distribution of goals among groups. Those. if the user has subscribed to 1 goal in group A, in which there are much more goals than in other groups, then the goals from group A will prevail in the recommendations.
2) With an insufficient flow of new goals for its relevant groups - recommendations stagnate. Because with such a goal list system, we artificially limit the selection of goals only to the user's preferences, then we reduce the flow of new content for the user, and the user constantly sees the same goals in his recommendation.
Stage 3: Do not show the goal. Which the user has already watched.
In order for the user not to blur the eyes with the same goals, we decided to hide from the display the goals that the user had already viewed. To do this, we added the table `goals_ignore`, which contains goals that the user looked at, as well as those that he persistently ignores. Those. if the user was shown the goal 10 times in the list of recommendations and he never entered it, then it is not interesting to him and this user does not need to show it anymore - put it in `goals_ignore`.
Thus, it was possible to achieve that the list of recommendations began to clear off goals that the user had already seen or did not want to watch.
Stage 4: Probabilistic sampling of goals. We show first of all the most interesting.
To do this, we introduced the CTR for the target, which was calculated from the ratio of the target’s opening to its views in the list of recommendations. And they began to sort the list of recommendations for this indicator. Thus, the user first of all sees the most interesting goals. But with this scheme, less interesting goals fall lower and lower and the ability to reach them is constantly decreasing. Therefore, we decided to introduce a certain randomness in the selection of goals. That would be less interesting goals still had the chance to be on the first lines of the list and thereby adjust your ctr, in case of an erroneous understatement.
To do this, we run the crown script to calculate a random goal coefficient based on their ctr.
UPDATE goals SET order_random = ((list_click/list_view)*RAND())And already by the order_random parameter we sort the output.
Due to this, the list of recommendations began to rotate and the goals that long ago plunged deep down began to appear on the first positions of the list. In addition to improving the list of recommendations for users, this step gave a new impetus to old goals. Users went into forgotten goals, brought activity and the authors returned to their goals again.
Step 5. We monitor the degree of user preference.
If the user sets more goals in one group or subscribes more to the goals of a certain group, then this group of goals is more interesting to him than others, so in the recommendations he needs to show more goals from this group accordingly.
And so, using steps 3-5, we were able to solve the problems described for the list of recommendations from the second stage, namely:
1) Not uniform distribution of goals into groups
2) Stagnation of the list of goals with an insufficient flow of new ones.
But a new problem was discovered. We do not know the preferences of new users who have just registered and have not set more than one goal and have not subscribed to others.
Step 6. If you don’t know what your preferences are, ask them.
After registration, we show users a guide to the site, which helps them quickly figure out the interface. A small questionnaire was added to it, in which they asked to mark the categories that are most interesting to the user. And the user’s responses began to build his list of recommendations until a certain number of subscriptions were recruited, after which we switched the user to our standard recommendation system.
As a result, the average viewing depth increased by 15% from 3.9 to 4.5. And the average length of stay is 36% from 5.5 minutes. up to 7.5 minutes
In one and the next article we will tell you how it all works quickly for us.
SmartProgress - Achieving Goals