How to stop Rambler?

Published on August 26, 2008

How to stop Rambler?

    The most outdated search engine of the Russian Internet seems to be in earnest in love with us.

    Rambler requests documents from habrahabr at a speed of up to 170 requests per second, which doesn’t suit us somewhat and according to our observations is an absolute record among all indexers visiting our project.

    Several incoherent paranoid answers were received from the technical support service of the search engine, advising you to read the page with the description of the robots.txt file or “put a delay on the response to the robot”.

    Apparently, Rambler does not support the Crawl-delay directive and finding a solution to the problem without blocking access to Rambler will not be as easy as we would like (nginx is a step away from solving this problem for the desired user-agent value).

    I wonder if there are those among the readers who can boast an even greater degree of attention from Rambler?