Who and why is spying on Yandex?

We continue to reveal the secrets of the secret cuisine of search engines. Today we look in the looking glass for how there is a “leak” of information about new updates - in particular, for example, we will open the cards and honestly admit how we were the first to know that rotation appeared in Yandex.


Any search engine provides updates to the search results, search algorithm and other aspects. One thing unites them all: they try not to disclose updates and even, if possible, not to advertise just so that all kinds of black SEOs do not sharpen their methods under the new algorithm. For a simple user, this is not very noticeable - an invisible struggle, as a rule, is conducted according to the so-called "commercial requests" - for example, "plastic windows" - will you often type this in a search? Once, twice in your life - you are unlikely to notice that something has changed. Unless sometimes it is impossible to find a site that used to be easily searched and was “at hand” - the only way you can notice is that, most likely, this was another “update”.

All that is known about updates to a wide audience is their certain supposed frequency. For example, Yandex search results are updated on average once every 6 days. Guess who expects this every time with trepidation and why.

This time, the “looking glass” of the Internet industry “mirrored” an interesting picture: over the last two updates, on June 12 and 21, a number of sites disappeared from issuing for commercial requests and a young new generation took their place. Good riddance, not too bad - you think. However, we are talking about information leakage, the analysis of which led to the conclusion that Yandex has included a site rotation system in the new algorithm - and this will soon become a well-known sensation (in narrow circles).

How does a "leak" occur?

Information rules the world. A search engine is a klondike of information about users and sites. Therefore, whoever has this information is Yandex (or Google). However, the collection of this information is not prohibited by law, therefore, there are tools that scan the search engine for changes. In some ways, it looks like stock and currency quotes - the same interesting “landscape” is obtained. Search engines "close" from such initiatives, for example, forcing you to enter captcha (Nobody noticed? - But this is a sure sign that somewhere near you there was a seoshnik. Well, or that the company has one IP and a couple of dozen employees at the same time google something).

However, for every tricky maze, there is a new way with a screw: the so-called SERP trackers that collect information about the ranking of sites by requests use captcha bypass tools, which are often very effective, because they use cheap labor of hired droids and Karebeliziya and Krablelyan (for numerous requests of schoolchildren of these countries, their names are changed).

Something like this works:



Creating such a database of queries in dynamics is not a cheap pleasure: even despite the low cost of solving captcha. In the case of a large number of large projects, the collection of information of this kind alone can result in a monthly budget of 9-12 thousand rubles.

There are not so many large systems, but since we have access to the colossal MegaIndex database, it was with the help of it that we were able to conduct fairly simple and accurate observations (remember - does the information rule the world?). Consider an example with a change in the issuance of “plastic windows” in the Moscow region for June 21 and 12. Here is what the system gives out: You



can see the full analysis using MegaIndex here , however, so that you don’t have to look at the small letters for a long time, we have already made a comparison and here’s what we noticed:

1. The sites that were in the first positions remained in their places
2 Some of the sites that were in the middle of the first hundred have risen markedly. This happened as a result of the loss of the TOP100 sites in front of them.
3. Sites that were below a certain position (average below 40th place) dropped out of the TOP100. In their place, new projects appeared.

Thus, updating the algorithm that made such a massive change clearly contains "something else." This "something" is the introduction of site rotation in the SERP .

If you want to make sure of what is happening, here are statistics on requests " freight transportation " and " tours ".

What is site rotation in the SERP?

In fact, a harmless innovation that gives sites that users have never found before a chance to appear in the search. It would seem, what does the word "behavioral" have to do with it? And despite the fact that since this is a kind of "trend of the year", it is quite acceptable to assume that such a step will make it possible to show the user sites that are potentially not "crowded" with links and keys and allow the user to wander around such a site and find out the degree of it (the user) activity on it. Based on this, the site can either be left in the issue, or again “throw” it out and try further, trying to remove sites from the issue that one of the MOZ speakers responds like this:

Dear websites that look obscene, but promise a lot in words - here you are:

Thus, there is a chance that, thanks to this rotation, "foul" sites will disappear somewhere, and more decent ones will take their place - this is the probable essence of the changes.

Although Yandex does not advertise what it is doing this summer, nevertheless, analytics of data on the current situation can provide first-hand information about the whole industry. How to use this information, everyone decides for himself. We hope that it fell into your hands to create and improve smart sites. If so, remember: yes, you have a chance. But the captain does not tire of recalling evidence: always keep in mind that people will go to your site - it is up to them to decide who you are and where you will be in the near future. At least, the new changes fit well with the Yandex policy announced at the beginning of the year regarding the consideration of behavioral factors .

Business intelligence itself for the Internet industry is widely available, and the ability to draw the right conclusions allows you to see the whole picture, even where information from other sources is not disclosed. We will continue to keep you informed about what is happening this summer - stay with us.