Black lists of runet in action. How to reduce the risk for your site

Bill No. 89417-6, adopted on July 10, 2012 by the State Duma of the Russian Federation, marked a new era in the development of Runet. The authorities finally got a mechanism to block unwanted sites at the hands of Internet access providers. Unfortunately, as with most laws adopted in the last 4 years, the implementation was thought out a little less than completely, which led to the blocking of completely innocent sites. Today, after a little over two years, according to the site rublacklist.net, access to 58940 domain names that do not contain any illegal content is illegally blocked. The question of why this happens, and how to avoid this situation, we will consider in this article.

Your site has been blocked - who is to blame?

Providers get the initial information about which site should be blocked from several sources at once:

1. Unified Register of Prohibited Information eais.rkn.gov.ru
2. Register of Copyright Infringers nap.rkn.gov.ru
3. Register of information prohibited by law 398-ФЗ 398-fz.rkn.gov.ru
4. Federal List of Extremist Materials minjust.ru/ru/extremist-materials
5. Court decisions regarding individual providers (times rospravosudie.com/court-proletarskij-rajonnyj-sud-g-rostova-na-donu-rostovskaya-oblast-s/act-101271803 two rospravosudie.com/court-novgorodskij-rajonnyj-sud- novgorodskaya-oblast-s / act-107327437 )
6. Prosecutor’s instructions regarding individual providers forum.nag.ru/forum/index.php?showtopic=95062

Your site does not contain prohibited information and, nevertheless, it turned out to be blocked? Welcome to the company of illegally blocked sites. Most likely the reason for this is blocking by IP address. The fact is that this is the only way for the provider to implement blocking without high costs. But hosting theory involves placing multiple sites on the same IP address. And not only hosting - the well-known “cloud” services (issuing an IP address for temporary use) and CDN networks (using geo-distribution by one or several IP addresses) have already suffered from such locks.



Is the IP address of your site missing from the registry? It doesn’t matter - the prosecutor’s office may well find banned material on one of the sites that are on the same IP with you and write an order to a separate transit provider.



Moreover, even the formal absence on the hosting of sites that do not contain prohibited content does not guarantee against being blocked. The fact is that in accordance with the recommendations of the ILV eais.rkn.gov.ru/docs/Recomendation.pdf, in order to combat the frequent change of IP addresses of violators' websites, providers themselves must resolve the domain name from the registry to the IP address. And this allows the owner of such a domain name to block any server. ILV looks at this opportunity through the fingers ( www.dropbox.com/s/l4lk0uafordlvsi/%D0%98%D1%81%D1%85%D0%BE%D0%B4%D1%8F%D1%89%D0%B8 % D0% B9% 08/30/20133.pdf ).

As you can see, there are many sources. And if it is still possible to independently determine whether your site got into them using the three registries and the list of materials of the Ministry of Justice, then it’s not technically possible to track court decisions and provincial orders of the prosecutor’s office.

How does this happen at the provider level

In order to understand how to effectively maneuver in a turbulent stream of prohibitions, you need to understand what both the provider and the regulator are doing and why. The provider doesn't care. The provider always follows the path of least resistance. The regulator, on the contrary, wants to create the appearance of violent activity, and, perhaps, is not out of harmony lays out solitaire games from the standards. Dependence on irresponsible, uncontrolled, constantly changing standards is the main risk in terms of site accessibility.

In the general case, the established blocking technique at the provider level is as follows:

1. The provider creates lists of IP addresses, access to which should be limited.
2. If the registries do not contain the IP address of the site, the provider receives it by resolving the domain name. Some providers follow the recommendations of the ILV and always resolve the domain name to IP address, even if it is explicitly specified in the registry.
3. Traffic to this IP is sent to a separate server in the provider's network.
4. If the provider has DPI in its network, traffic is analyzed, only requests to prohibited URLs are blocked. If there is no DPI, all traffic is blocked.
5. For the HTTPS protocol - if a URL with HTTPS is explicitly specified in the registry, they try to block only traffic to a given IP on port 443 (since URLs cannot be extracted from encrypted traffic). If a URL with HTTP appears in the registry, HTTPS traffic will not be blocked.

How to protect yourself from all this?

Moderate your sites and follow them

While the practice of "setup" is not very common, however, it is quite possible to block your site for material of illegal content left on the forum or published using the vulnerability of your site.
In the case of extremist content, your hosting provider will not even receive prior notice - the site will be entered into the registry and you will receive a notification after the fact.

If possible, host websites with Russian hosting providers.

This statement only looks "advertising." Russian hosters are aware of the realities of the law and are making some efforts to protect their customers - they respond to ILV requests in a timely manner, have access to registries and track their changes. After the very first blockages, we split user sites into hundreds of different IP addresses, trying to minimize the number of clients that could “fly” under the block at the same time as one of the illegal sites. Most likely, none of the foreign hosting providers will make such an effort.

Dedicated IPv4 Address.

Try to move your site to a separate IP address. This will protect you from blocking "for the company." Many hosting providers offer a dedicated IP address service for a fee. Unfortunately, the shortage of IPv4 addresses affected its size, but blocking losses can be much larger.

Separate IPv6 Address

Hosts that have implemented an IPv6 stack on their network today already issue a dedicated IPv6 address for each site. Unfortunately, their number is still small, but due to the shortage of IPv4 addresses, the number of such providers is increasing. This means that the chance to get blocked by an innocent site is reduced. Separately, it is worth mentioning that most of the inspection bodies have not yet mastered access to sites by IPv6 addresses.

Using encryption and SSL certificates

Unlike the usual HTTP protocol, HTTPS is carried out via a secure SSL channel and it is impossible to view it, which means that a specific URL can be extracted from encrypted traffic. Due to a lack of understanding of the protocols, the complicated issue of the secure channel is still being tried. In most cases, unless explicitly indicated, providers try to ignore the blocking of the HTTPS protocol. This will continue until the use of HTTPS becomes widespread.

Something seems to have gone wrong ...

The story about blocking Alexey Navalny’s blog deserves special attention.

Surely many of you have heard that Alexey Navalny’s blog on the LiveJournal platform is in the registry (despite the fact that the department has still not decided on the reasons for the block). Supporters of Alexei did not put up with blocking and created a whole system of dynamic domain names and IP addresses to organize free access to the blog. For weeks, ILV employees only did that they blocked these domains. And at one point, the blocked domains began to point to the IP addresses of the registry itself. Providers that followed the ILH recommendations on self-resolving domain names automatically blocked access to the registry. And then they could no longer turn to him. After that, the ILV recommended nevertheless not to resolve domain names on their own, but to block the IP addresses specified in the registry.
As there was a struggle with supporters of Alexei Roskomnadzor can read the blog Ruslan Leviev: ruslanleviev.livejournal.com/34401.html
way, the result of this struggle was the appearance of their registry - a registry of state bodies subnets: github.com/AntiZapret/AntiZapret

We had another one similar case. The owner of a certain domain name, which is on the list of extremist materials of the Justice Ministry, decided to transfer his website to us and changed the A-record of his domain to the IP address of our server. As a result, access to this server during the day was cut off by one of the transit operators. Moreover, only one, so that only a certain number of users lost access to the server. We were able to identify the problem only by escalating the request through a chain of operators.

What are the prospects?

Unfortunately, the prospects are rather bleak. Having gotten into the hands of the coveted censorship mechanism, the lawmakers do not want to stop and almost daily put forward new ideas for prohibitions. In the near future, we are waiting for a ban on access to sites that store personal data outside the territory of the Russian Federation, a ban on access to large blogs (more than 3,000 visitors) that are not registered with the Ministry of Communications, as well as the possibility of blocking not individual IP addresses, but immediately subnets ( habrahabr.ru / post / 229431 ). The situation is aggravated by the fact that Internet access providers have enough of their problems, and they will not fight censorship - they will go along the easiest path - the path of non-resistance to "innovation." We will write about how hosting providers will deal with this scourge in the next article.