March 11, 2019 at 10:23How Protonmail is blocked in Russia
A completely routine Trouble Ticket to our tech support revealed another strange blocking of the Protonmail service community, which respects its Internet freedoms, in some Russian networks. I would not want to exploit the “yellow headline,” but the story is strange and somewhat outrageous.
TL; DR
Important note: parsing continues and so far everything is in process. Maybe there’s no boy, but most likely there is. Will be supplemented as new information becomes available.
The largest Russian telecom operators MTS and Rostelecom are blocking traffic to the Protonmail secure email service SMTP servers by letter from the FSB. Apparently, it’s been long enough, but no one has paid much attention so far. And here we are.
WTF and burning continues, all participants received relevant requests and must provide reasoned answers.
UPD: MTS provided a scan of the FSB letter, which is used for blocking. Motivation: Universiade and “telephone terrorism”. So that letters from ProtonMail do not fall into the disturbing addresses of spa services and schools.
UPD: Protonmail was surprised at the methods of dealing with fraud in "these strange Russians" and advised a more effective type of struggle through abuse mailbox.
UPD: The gallant concept of the FSB’s fight against false addresses did not stand up to criticism: it was a letter that broke incoming mail to ProtonMail, and not outgoing.
UPD: Protonmail shrugged their shoulders and changed the IP addresses of their MXs, thus removing them from blocking this specific email. The question is what will be further open.
UPD: Apparently, such a letter is not one and there is still a set of IP addresses of VOIP services that are blocked non-registerly.
UPD: As the story began to spread outside the Runet, we prepared a translation into English, the link at the top.
We love our Habra users for the fact that they are technically savvy. Technically savvy users know what “computer hygiene” is. Some of our users decided to use Protonmail's “secure email” service . We will leave aside the discussion of the service itself, their product and business model, we will discuss only significant technical points.
Every day we send a lot of emails to our users, and since we worry about our independence and their privacy, we do not use third-party email distribution services (ESP) to send most types of messages. To do this, we use our capacities, from bare-metal servers and MX-servers controlled by us, to encryption of connections and ownership of our independent IP addresses.
Last week we received a lot of calls from Protonmail users to our ticket support system that our letters did not reach them.
Of course, the basic response of our tech support was the suggestion to look for other typical solutions to common problems in spam, but the volume of requests prompted me to understand the issue in more detail. And then wrap it all up ...
For most modern Internet users, the use of e-mail consists of a browser entering the “personal box” on the website of the email service provider, composing and sending a letter to the recipient through the same web interface. With the help of some magic, in a moment the letter appears in the web interface of the recipient's email service.
So: the magic is called SMTP (esmtp, to be precise). The sender server extracts the domain part (after @) from the recipient's mailbox address and makes a DNS query to obtain a list of MX servers of the recipient's domain. It looks something like this for support@habr.team:
MX-server, literally "mail exchange server" (mail exchange). It indicates which email service the recipient’s domain email is on. To be more precise, through which email servers the hosting domain of the recipient receives mail. That is, the example above shows that the mail servers for our habr.team domain are located on Google servers (g. Suite).
After establishing the list of recipient MX-servers, esmtp (s) is accessed to the server with the lowest priority number in order to transfer mail to the user. More than one, the number of servers in the list is made to ensure fault tolerance, since Internet connectivity is often conditional.
The message transfer looks something like this:
An important note: mail from a certain domain is not obliged to leave recipients from MX servers specified in DNS, this mechanism is used only for incoming mail. Outgoing mail from a domain can be transmitted through other servers, an approximate list of which is indicated, as a rule, in a different SPF record.
We began to scoop up mail logs and found that the connections of our servers with Protonmail MX-servers (185.70.40.40.101, 185.70.40.40.102) ended with network timeouts. This looked strange for a number of reasons and was similar to using the blocking mechanism practiced in Russia.
In general, the term “Internet” is literally translated roughly as “Inter-Network”, it can be translated as “Network of networks” and “Networking”, as anyone likes. In fact, the Internet does not have a “technical center” (as opposed to an “organizational center”), it is an association of various networks that are as if equal to each other, although there are networks “more equal” than others, but this is another story. Networks are called "Autonomous Systems" (AS) and are interconnected by joints (peers). Each autonomous system has a unique number by which it is identified by another autonomous system on the Internet. It looks like IP addresses, but with more “big brushstrokes”. Each network receives data from the neighboring one about the topology of its connections with its neighbors, the connection of its neighbors with their neighbors, and so on. That is, a map of the connections of autonomous systems to each other in terms of this junction.
For example, we have an Autonomous System Number (ASN) 204671 , Protonmail servers are located in the network of one of the largest American network corporation Neustar, which has ASN 19905 . We have two interfaces with various Internet service providers, that is, two possible AS paths from us to the Neustar network. For a number of reasons, we have a priority with one of the MGTS operators, so the AS-path is as follows: 204671 (We) - 57681 (MGTS) - 8359 (MTS) - 22822 (Limelight) - 19905 (Neustar)
The map looks something like this:
Traceroute to either of the two Protonmail MX servers ended up on the MTS network and looked something like this:
GW-Core-R3#traceroute ip 185.70.40.101 probe 1 timeout 3
Type escape sequence to abort.
Tracing the route to 185.70.40.101
VRF info: (vrf in name/id, vrf out name/id)
1 185.2.126.73 [AS 57681] 2 msec
2 212.188.12.73 [AS 8359] 2 msec
3 195.34.50.73 [AS 8359] 3 msec
4 212.188.55.2 [AS 8359] 3 msec
5 *
6 *
7 *
8 * An alternative host was found on the Neustar network and used as a reference to eliminate possible connectivity problems between MTS and Limelight:
GW-Core-R3#traceroute ip 156.154.208.234 probe 1 timeout 3
Type escape sequence to abort.
Tracing the route to 156.154.208.234
VRF info: (vrf in name/id, vrf out name/id)
1 185.2.126.73 [AS 57681] 2 msec
2 212.188.12.73 [AS 8359] 2 msec
3 212.188.2.37 [AS 8359] 14 msec
4 212.188.54.2 [AS 8359] 20 msec
5 195.34.50.146 [AS 8359] 27 msec
6 195.34.38.54 [AS 8359] 37 msec
7 68.142.82.159 [AS 22822] 26 msec
8 *
9 156.154.208.234 [AS 19905] 26 msecA successful trail was conducted through an alternative operator to the Protonmail MX server (it breaks off already in the Neustar network, but there it is planned, the connection to the host works):
$ traceroute -a 185.70.40.101
traceroute to 185.70.40.101 (185.70.40.101), 64 hops max, 52 byte packets
1 [AS49063] hidden (hidden) 5.149 ms 268.571 ms 6.707 ms
2 [AS49063] 185.99.11.146 (185.99.11.146) 5.161 ms 6.317 ms 5.476 ms
3 [AS0] 10.200.16.128 (10.200.16.128) 5.588 ms
[AS0] 10.200.16.176 (10.200.16.176) 5.225 ms
[AS0] 10.200.16.130 (10.200.16.130) 5.001 ms
4 [AS0] 10.200.16.49 (10.200.16.49) 6.480 ms
[AS0] 10.200.16.156 (10.200.16.156) 5.439 ms 7.469 ms
5 [AS20764] 80-64-98-234.rascom.as20764.net (80.64.98.234) 6.208 ms 9.301 ms 6.348 ms
6 [AS20764] 80-64-100-102.rascom.as20764.net (80.64.100.102) 24.281 ms
[AS20764] 80-64-100-86.rascom.as20764.net (80.64.100.86) 54.632 ms 23.936 ms
7 [AS20764] 81-27-254-223.rascom.as20764.net (81.27.254.223) 27.589 ms 116.438 ms 27.348 ms
8 [AS22822] siteprotect.security.neustar (68.142.82.153) 28.683 ms 25.376 ms 41.489 msIn general, often a traceroute thing is not very revealing, but a number of additional studies were carried out, for example, through the looking glass MTS service:
It became obvious that the case smells like kerosene and looks like blocking a service at an address on the MTS network. However, a request to a special ILV service showed that both addresses are not listed in the well-known registry either by domain name or IP address:
We leave out the specifics of Internet regulation in Russia and recall that the binding order to block a particular resource by the operator is the so-called “unloading from the registry”, in which there is a resource blocked for one reason or another. Practically, sometimes blocking resources without legal placement in the register received the community term “non-register blocking”; they often have dubious grounds that do not withstand normal legal examination.
But at that time there was a suspicion simply of a technical misunderstanding or unsuccessful unlocking of a previously carried out lock. Yes, we are such, we check for a long time and we don’t hype without thorough fact-checking.
An appeal was sent to MGTS technical support with the fixation of this fact and a request to sort it out. The answer was not received immediately: "it is not with us, but with MTS, go there." Of course, we did not go to MTS for clarifications, but forced MGTS to do its job and sort it out on its own. The most interesting answer was received: according to the employees of the authorized MTS department, they were contacted by respected people from the well-known FSB organization with a letter number 12 / T / 3 / 1-94 dated 02.25.2019, which says something urging to block these hosts urgently.
Here, everyone involved in the investigation of the unfortunate trouble ticket from such impudence burned completely and further study went at a double pace.
A request was sent to the FSB with a question about the reality of the existence of such a letter and the grounds for ordering it:
A request was sent to MGTS to provide a basis for blocking:
After which I did not want to sit still and questions were asked in the profile chats of one well-known and not used in Russia by virtue of the Law of the messenger. In the telecom community, the question was perceived rather sluggishly in the style of “I had practice communicating with similar situations and none of them want to use ILV tools. Firstly, it’s difficult, secondly, bringing the problem to another department is not a solution to the problem by the current department ”and“ in short, they need a scheme, details, an extract, contacts, everything seems to be ... and then gears will turn around and all offenses will be taken into account, and give birth to a plan to be followed. They can write "need puller" = popos on a bunch of dough), or they may not write if the operator helps in illegal blocking of resources). " Well, it’s hard to blame, in telecom you have to work with this smut often
One way or another, in the chat of the Internet Protection Society, this case was received with enthusiasm, and a more extensive study of the problem was conducted.
They suggested an interesting idea - to check how much MX-server data is available from different telecom operators in Russia and the world through the RIPE Atlas service, which gave the expected, even more interesting answer: in Russia, MTS and Rostelecom, as well as networks, block connections. having a connection through these providers ( study of the availability of the main Protonmail MX server from Russia , the spare Protonmail MX server from Russia ). A global study showed problems in Russia, Ukraine and Iran (a study of the availability of the main Protonmail MX server from around the world , the Protonmail MX backup server from around the world )
A detailed study showed that in the Rostelecom network, blocking is carried out in a similar way:
After the holidays, MTS provided a scan of the letter, on the basis of which blocking measures are carried out. Of course, “mother's phone terrorists” are to blame, and the Universiade 2019 is the rubber base:
There was a reaction from representatives of Protonmail on reddit , on Twitter and on TechCrunch , as expected, they were somewhat surprised at the methods of dealing with fraud and advised us to work more closely with them in catching public order violators:
Hmm, the Protons said that the case may have a double bottom. Well, it turned out to be, although slightly leaky. So, under the first spoiler it is written how SMTP interaction between servers works and it is said that MX records are a mechanism for incoming mail. In general, yes, in fact, the FSB decided to break the incoming mail on Proton, and not the outgoing one, so the selfless passage about preserving the nerve cells of school principals does not hold water. There are two options:
- what fell into their hands was blocked (an option for the lazy, but according to the principle of Occam’s razor, the most likely one, someone read the article “nslookup for dummies”);
- dragged under the guise of heroism the impossibility of delivering letters with compromising materials from concerned citizens to interesting boxes of enemies of the people on the proton (it is very difficult and does not work in the vast majority of cases);
- Option about UFO, which insists Vartan Khachaturov.
The proof is in the screenshot below: a directed letter from ProtonMail to another service showed that outgoing letters on this service use other IP addresses that are not completely blocked. Whoever sees 185.70.40.40.101 and 185.70.40.102 here, raise your hand:
The head of ProtonMail confirmed the findings to the TechCrunch network publication , advising him to fight terrorist activities in collaboration with foreign law enforcement agencies, rather than sticking his head in the sand, adding that they decided to get away from blocking:
ProtonMail chief executive Andy Yen called the block “particularly sneaky,” in an email to TechCrunch.
“ProtonMail is not blocked in the normal way, it's actually a bit more subtle,” said Yen. “They are blocking access to ProtonMail mail servers. So Mail.ru - and most other Russian mail servers - for example, is no longer able to deliver email to ProtonMail, but a Russian user has no problem getting to their inbox, ”he said.
“The wholesale blocking of ProtonMail in a way that hurts all Russian citizens who want greater online security seems like a poor approach,” said Yen. He said his service offers superior security and encryption to other mail providing rivals in the country.
“We have also implemented technical measures to ensure continued service for our users in Russia and we have been making good progress in this regard,” he explained. “If there is indeed a legitimate legal complaint, we encourage the Russian government to reconsider their position and solve problems by following established international law and legal procedures.”
- ProtonMail chief executive Andy Yen @ TechCrunch
So, Protonmail changed the IP addresses of one of their MXs from 185.70.40.101 to 185.70.40.103, which excludes it from the list of blocked resources for this particular letter. What will happen next is a question.
As it turned out, there is some evidence that a similar blocking scheme applies to a specific list of other foreign email services and popular IP-telephony services. Case Studies Analytics.
Actually, why is this outraged? In general, a colleague from the ZaTelecom channel explained the point quite clearly and with a characteristic emotionality:
Attention, everything here is generally true, but there are technical inaccuracies: AS Neustar has not been blacked out, only / 32 of the filters are in the filters.
Moreover, it’s surprisingly close: only SMTP service servers for incoming mail are blocked by this letter. Web access and SMTP servers for outgoing are very much alive, which casts doubt on the effectiveness of these events. Moreover, ProtonMail went to the meeting and changed the address of one of their MX.
We repeat, leaving behind the entire legislative initiative related to the regulation of the Internet in a single 1 / (8+) part of the land, we note that there are game rules. They are unreliable, constantly changing and not obvious to everyone. But they are rules, although they give a severe handicap to the maintainers of these rules. But even in this case, there are those who want to bypass these rules and pull the foul-smelling ace from the sleeve. Just this case shows in action the mechanism of “no trial,” even Kafka has nothing to profit from here. Because in the trial, whatever it may be, you can at least attract an expert or industry specialist, in contrast to someone’s personal decision, taken solely on the basis of a personal worldview.
So, this is the whole factology for this case at the moment. All requests have been sent, answers so far only partially received. There was, of course, little hope that this was all due to crooked hands in MTS and RT, but to be honest, this version did not look consistent from the very beginning and, logically, was not confirmed.
As for our users, who are part-time users of Protonmail, they can continue to use their mailboxes on Protonmail in conjunction with Habr, as we re-routed traffic from us to Protonmail servers through another Russian operator that does not deal with such things. And, apparently, MGTS will soon also have one less client.
UFO Care Minute
This material could cause conflicting feelings, so before writing a comment, refresh something important in your memory:How to write a comment and survive
- Do not write offensive comments, do not get personal.
- Refrain from obscene language and toxic behavior (even in a veiled form).
- To report comments that violate the rules of the site, use the "Report" button (if available) or the feedback form .
What to do if: minus karma | blocked an account
→ Code of authors Habr and habraetiket
→ Full version of site rules
