March 13, 2014 at 14:28The bright future of IPv6: when the new world order will finally come
Yandex.Mail now knows how to exchange letters with other mail systems over IPv6. Thus, it becomes the second mass mail system in the world that supports IPv6.
At first glance, this may not look very important, but in fact right now, the lack of IPv6 support is hindering many people. In this post, I want to talk about how things are now going with the adaptation of v6 on the Internet, as well as what we have done right now and why. IPv4 addresses, of which 4 billion, have already ended. Switching to IPv6 will bring online 3.4 × 10 38
devices. And we have a future ahead, when more and more devices for each person will be connected to the Internet and when all the inhabitants of Asia and Africa will finally go online after Europe and America. Therefore, the worldwide network is gradually moving to a new communication protocol, the support of which is necessary for any service that is going to work in the future on the Internet.
As you know, already in September 1981, when the US Department of Defense Advanced Defense Research Projects Agency published the IPv4 specification, it was clear that the number of addresses is certainly equal to 2 32 . At first glance, the number seems large. In 1995, IP address space was only 25% used, but scientists and engineers have already formulated and published the first specification for the next version of the basic Internet protocol, which was called IPv6. The new protocol laid the possibility of using approximately 3.4 * 10 38 (340 undecillion) addresses, which is almost 10 29 (100 octillion) times more than in the old version. In 2008, IPv4 address space was already 86% full.
In a first approximation, the plan for converting the Internet to IPv6 was described almost 20 years ago in RFC 1933 , when the specification of the new protocol was published. He meant a full switch in 2007. This turned out to be a very optimistic assessment. Only in the year 2000, the first provider, the Japanese NTT, announced its readiness to support IPv6. The current version is described in RFC 4213 , and its tone is somewhat less optimistic.
For almost the entire first decade of the twenty-first century, there has been an unhurried discussion between Internet service providers and owners of large Internet resources about who exactly is responsible for promoting IPv6. Providers said that there was no point in implementing, because IPv6 resources are too few. Resources answered that since there are almost no users with IPv6 connectivity, there is no need to include this support on sites. In general, the real problem is chicken and eggs.
However, time passed, IPv4 addresses were allocated, and in February 2011, ICANN distributed the last “fourth” addresses to regional registries. There are five of them - APNIC in the countries of the Asia-Pacific region, ARIN in North America, AfrNIC in Africa, LACNICin Central and South America and RIPE in Europe. Already in April 2011, APNIC distributed all the IPv4 addresses allocated to the region except the last network / 8 (i.e. 2 24 or 16777216 addresses) and switched to a special mode for distributing the last addresses . Similar events in the European Internet segment took place in September 2012 , and RIPE also switched to the mode of allocating the latest addresses. By the way, Yandex accidentally got the last block 5.255.192.0/18 (and, accordingly, the address 5.255.255.255) before switching to this mode.
To break the vicious circle of shifting problems from providers to Internet resources and back, World IPv6 Day was held in 2011. On this day, several of the largest world sites included IPv6 for a day. The purpose of this inclusion was to test how hardware and software from users, providers and Internet sites will respond to the advent of IPv6. And we raised a copy of the Yandex homepage on ipv6.yandex.ru. We talked about this launch, preparation for it and the results in detail in one of the YaC 2012 presentations . There were a lot of results and there are a lot of tasks on these results.
For example, we experienced the problems of traffic loss when connecting IPv6. In particular, in one of the experiments, we opened the suggest.yandex.net service (serving the prompt in the search bar) over IPv6 to the entire Internet. As a result, it turned out that we lost up to 2% of requests, which was unacceptable. Theoretically, the mechanism of operation of systems with support for IPv6 and IPv4 suggests that the program (for example, a browser) first tries to connect to the IPv6 address, and if it fails, reconnects via IPv4. The problem is that in modern operating systems, the connection timeout can be tens of seconds and, of course, the user will not wait so long.
We had no choice but to implement DNS AAAA whitelisting - to start showing IPv6 addresses only to those providers whose IPv6 connectivity with Yandex we were sure of. We are aware of the administrative complexity of supporting white lists (the problem, in particular, is described here ) and poor scaling in the long term, but we do not see good alternatives for Runet.
A year later, on June 6, 2012, in the world of World IPv6 Launch Day, we launched “now for real” several of our IPv6 services. Then the websites yandex.com, mail.yandex.com and passport.yandex.com became available. This launch was presented on IPv6 Russian Day.. It seems that in Russia we became the first major site to launch IPv6. VKontakte joined in a few days or weeks, and after some time Mail.ru. Classmates still do not have an IPv6 address.
In 2013, we turned off whitelisting support for services in the yandex.com zone, since the situation with IPv6 support in the world is on average better than in Runet. The services in the yandex.ru zone for the most part still work through white lists, but we hope that we can abandon this practice already in 2014.
Having dealt with websites and working with users, we realized that now we need to pay attention to interserver interaction. One of the most prominent services corresponding to this category was Mail.
Yandex.Mail consists of hundreds of components, many of which work on Open Source open source software such as Postfix or MongoDB . Support for sending / receiving emails over IPv6 appeared in Postfix more than 10 years ago and has been tested by many thousands of mail administrators before us. Postfix works with us on send servers, and also as a universal queue system. Very reliable and versatile product. Yandex receives mail using a special high-performance NwSMTP server created entirely within the company. Asynchronous and very fast, NwSMTP did not support IPv6 for a long time and required serious development.
Another important component of mail, which required a lot of improvements, is Spam Defense- A set of programs and databases that protect our (and yours, if you receive mail) users from spam and other unwanted emails. Our mail anti-spam algorithms combine statistical and heuristic methods, as well as machine learning, which Yandex is so famous for.
A number of factors in deciding whether or not spam is using the IP addresses of the computers involved. The so-called reputation methods accumulate and use historical information about such objects as domains, IP addresses, individual email addresses. All this in order to make reasonable assumptions about their future behavior. Very simplified, this scheme looks like a hash or associative array, where the key is, for example, the IP address, and the value is information about how much spam or non-spam we received from this address. Once the storage and use of such a hash even for 2 32of various possible key values represented certain difficulties in programming. Imagine what happens when you need to maintain a reputation for IPv6 addresses, the total number of which greatly exceeds even the number of individual bits in the RAM of all Yandex servers.
Fortunately, IPv6 address space is very sparse. Huge sequences, as they say, by design are not used. An additional important optimization factor is the standardization of the / 64 network as a quantum of address allocation in IPv6. In general, in the end, everything worked as it should.
About a year ago, we began to send letters to external servers via IPv6, if they supported it. This part of the work required only fine-tuning of Postfix and thorough testing. Reception of mail turned out to be a more difficult task primarily because of anti-spam algorithms. Finally, this work has been completed - in February 2014, we began receiving emails over IPv6. See:
Now all Yandex.Mail cross-server traffic can be received and transmitted using IPv6 protocol. We did not use DNS whitelists here, because IPv6 connectivity between servers is generally better than between servers and users. In addition, working with multiple MXs in SMTP allows you to essentially automatically degrade to IPv4 if IPv6 is down. Finally, server-side SMTP is not an interactive protocol, so possible connection timeouts do not directly affect user experience.
This change went unnoticed by users, and this invisibility is a special pride for the entire team. A big step has been taken forward, guarantees have been created that your mail will be reliably delivered by Yandex in 2, 5, and 10 years, when the old protocols completely cease to satisfy the needs of mankind and become obsolete.
It is interesting that all modern anti-spam protection working in the framework of large mail systems such as Yahoo !, Gmail, Yandex, AOL, Mail.ru, Outlook.com have been working as part of a kind of common ecosystem for several years. There is a cautious exchange of information about the general waves of spam, certain settings are made for reliable mutual exchange of letters, and so on. We all work closely even though we are competitors. This, by the way, is one of the reasons why no stand-alone anti-spam solutions can match filtering with large mails. They simply do not have enough information and nowhere to get it. So, this ecosystem will require some time to self-configure for the widespread use of IPv6. The sooner you start this work, the faster the overall result will be achieved. It is no longer possible to put off.
When the problem of the lack of IPv4 addresses from theoretical became more and more real, providers began to use NAT. But in the end, this did not become a solution. When there are too many devices hiding behind NAT, sites may decide that robots are loading them and take measures to protect them from such actions.
Such a problem, in particular, has become widespread in Belarus . This happened relatively recently, after the transition of RIPE to the phase of address allocation from the last block / 8. Belarusian providers did not have a sufficient supply of IP addresses and, finding themselves in conditions of their shortage, went by increasing the number of users for NAT. As a result, some Internet services took the activity of NAT-users for the activity of robots and someof them completely blocked the IP address (and all NAT users) or increased the captcha display intensity. Thus, the unavailability of providers to exhaust IPv4 addresses has led to a decrease in the quality of Internet access for users. A similar problem exists for mail systems: there, large NAT pools mean that letters sent by a user to NAT can be mistaken for spam and not reach the addressee.
Schedule of IPv6 traffic at www.yandex.ru
Currently, according to various estimates, only about 3% of traffic on the Internet uses IPv6, and 97% use old IPv4. But when such major players as Yandex, Google, VKontakte, Facebook support IPv6 technically and ideologically, they push providers to support sixth addresses, they push the entire Internet into the future. Of course, it’s important for us and ourselves to understand that our services have enough addresses. But even in total, there are much fewer of them compared to how many sites may appear on the Internet in the future and how many people will get access to it.
At first glance, this may not look very important, but in fact right now, the lack of IPv6 support is hindering many people. In this post, I want to talk about how things are now going with the adaptation of v6 on the Internet, as well as what we have done right now and why. IPv4 addresses, of which 4 billion, have already ended. Switching to IPv6 will bring online 3.4 × 10 38
devices. And we have a future ahead, when more and more devices for each person will be connected to the Internet and when all the inhabitants of Asia and Africa will finally go online after Europe and America. Therefore, the worldwide network is gradually moving to a new communication protocol, the support of which is necessary for any service that is going to work in the future on the Internet.
From IPv4 to IPv6: Transition History
As you know, already in September 1981, when the US Department of Defense Advanced Defense Research Projects Agency published the IPv4 specification, it was clear that the number of addresses is certainly equal to 2 32 . At first glance, the number seems large. In 1995, IP address space was only 25% used, but scientists and engineers have already formulated and published the first specification for the next version of the basic Internet protocol, which was called IPv6. The new protocol laid the possibility of using approximately 3.4 * 10 38 (340 undecillion) addresses, which is almost 10 29 (100 octillion) times more than in the old version. In 2008, IPv4 address space was already 86% full.
In a first approximation, the plan for converting the Internet to IPv6 was described almost 20 years ago in RFC 1933 , when the specification of the new protocol was published. He meant a full switch in 2007. This turned out to be a very optimistic assessment. Only in the year 2000, the first provider, the Japanese NTT, announced its readiness to support IPv6. The current version is described in RFC 4213 , and its tone is somewhat less optimistic.
For almost the entire first decade of the twenty-first century, there has been an unhurried discussion between Internet service providers and owners of large Internet resources about who exactly is responsible for promoting IPv6. Providers said that there was no point in implementing, because IPv6 resources are too few. Resources answered that since there are almost no users with IPv6 connectivity, there is no need to include this support on sites. In general, the real problem is chicken and eggs.
However, time passed, IPv4 addresses were allocated, and in February 2011, ICANN distributed the last “fourth” addresses to regional registries. There are five of them - APNIC in the countries of the Asia-Pacific region, ARIN in North America, AfrNIC in Africa, LACNICin Central and South America and RIPE in Europe. Already in April 2011, APNIC distributed all the IPv4 addresses allocated to the region except the last network / 8 (i.e. 2 24 or 16777216 addresses) and switched to a special mode for distributing the last addresses . Similar events in the European Internet segment took place in September 2012 , and RIPE also switched to the mode of allocating the latest addresses. By the way, Yandex accidentally got the last block 5.255.192.0/18 (and, accordingly, the address 5.255.255.255) before switching to this mode.
To break the vicious circle of shifting problems from providers to Internet resources and back, World IPv6 Day was held in 2011. On this day, several of the largest world sites included IPv6 for a day. The purpose of this inclusion was to test how hardware and software from users, providers and Internet sites will respond to the advent of IPv6. And we raised a copy of the Yandex homepage on ipv6.yandex.ru. We talked about this launch, preparation for it and the results in detail in one of the YaC 2012 presentations . There were a lot of results and there are a lot of tasks on these results.
For example, we experienced the problems of traffic loss when connecting IPv6. In particular, in one of the experiments, we opened the suggest.yandex.net service (serving the prompt in the search bar) over IPv6 to the entire Internet. As a result, it turned out that we lost up to 2% of requests, which was unacceptable. Theoretically, the mechanism of operation of systems with support for IPv6 and IPv4 suggests that the program (for example, a browser) first tries to connect to the IPv6 address, and if it fails, reconnects via IPv4. The problem is that in modern operating systems, the connection timeout can be tens of seconds and, of course, the user will not wait so long.
We had no choice but to implement DNS AAAA whitelisting - to start showing IPv6 addresses only to those providers whose IPv6 connectivity with Yandex we were sure of. We are aware of the administrative complexity of supporting white lists (the problem, in particular, is described here ) and poor scaling in the long term, but we do not see good alternatives for Runet.
A year later, on June 6, 2012, in the world of World IPv6 Launch Day, we launched “now for real” several of our IPv6 services. Then the websites yandex.com, mail.yandex.com and passport.yandex.com became available. This launch was presented on IPv6 Russian Day.. It seems that in Russia we became the first major site to launch IPv6. VKontakte joined in a few days or weeks, and after some time Mail.ru. Classmates still do not have an IPv6 address. In 2013, we turned off whitelisting support for services in the yandex.com zone, since the situation with IPv6 support in the world is on average better than in Runet. The services in the yandex.ru zone for the most part still work through white lists, but we hope that we can abandon this practice already in 2014.
Having dealt with websites and working with users, we realized that now we need to pay attention to interserver interaction. One of the most prominent services corresponding to this category was Mail.
How did the transition to IPv6 in Yandex.Mail
Yandex.Mail consists of hundreds of components, many of which work on Open Source open source software such as Postfix or MongoDB . Support for sending / receiving emails over IPv6 appeared in Postfix more than 10 years ago and has been tested by many thousands of mail administrators before us. Postfix works with us on send servers, and also as a universal queue system. Very reliable and versatile product. Yandex receives mail using a special high-performance NwSMTP server created entirely within the company. Asynchronous and very fast, NwSMTP did not support IPv6 for a long time and required serious development.
Another important component of mail, which required a lot of improvements, is Spam Defense- A set of programs and databases that protect our (and yours, if you receive mail) users from spam and other unwanted emails. Our mail anti-spam algorithms combine statistical and heuristic methods, as well as machine learning, which Yandex is so famous for.
A number of factors in deciding whether or not spam is using the IP addresses of the computers involved. The so-called reputation methods accumulate and use historical information about such objects as domains, IP addresses, individual email addresses. All this in order to make reasonable assumptions about their future behavior. Very simplified, this scheme looks like a hash or associative array, where the key is, for example, the IP address, and the value is information about how much spam or non-spam we received from this address. Once the storage and use of such a hash even for 2 32of various possible key values represented certain difficulties in programming. Imagine what happens when you need to maintain a reputation for IPv6 addresses, the total number of which greatly exceeds even the number of individual bits in the RAM of all Yandex servers.
Fortunately, IPv6 address space is very sparse. Huge sequences, as they say, by design are not used. An additional important optimization factor is the standardization of the / 64 network as a quantum of address allocation in IPv6. In general, in the end, everything worked as it should.
About a year ago, we began to send letters to external servers via IPv6, if they supported it. This part of the work required only fine-tuning of Postfix and thorough testing. Reception of mail turned out to be a more difficult task primarily because of anti-spam algorithms. Finally, this work has been completed - in February 2014, we began receiving emails over IPv6. See:
% host -t any mx.yandex.ru
mx.yandex.ru has IPv6 address 2a02: 6b8 :: 89
mx.yandex.ru has address 87.250.250.89
mx.yandex.ru has address 93.158.134.89
mx.yandex.ru has address 213.180.193.89
mx.yandex.ru has address 213.180.204.89
mx.yandex.ru has address 77.88.21.89
%
Now all Yandex.Mail cross-server traffic can be received and transmitted using IPv6 protocol. We did not use DNS whitelists here, because IPv6 connectivity between servers is generally better than between servers and users. In addition, working with multiple MXs in SMTP allows you to essentially automatically degrade to IPv4 if IPv6 is down. Finally, server-side SMTP is not an interactive protocol, so possible connection timeouts do not directly affect user experience.
This change went unnoticed by users, and this invisibility is a special pride for the entire team. A big step has been taken forward, guarantees have been created that your mail will be reliably delivered by Yandex in 2, 5, and 10 years, when the old protocols completely cease to satisfy the needs of mankind and become obsolete.
It is interesting that all modern anti-spam protection working in the framework of large mail systems such as Yahoo !, Gmail, Yandex, AOL, Mail.ru, Outlook.com have been working as part of a kind of common ecosystem for several years. There is a cautious exchange of information about the general waves of spam, certain settings are made for reliable mutual exchange of letters, and so on. We all work closely even though we are competitors. This, by the way, is one of the reasons why no stand-alone anti-spam solutions can match filtering with large mails. They simply do not have enough information and nowhere to get it. So, this ecosystem will require some time to self-configure for the widespread use of IPv6. The sooner you start this work, the faster the overall result will be achieved. It is no longer possible to put off.
Why is it important for providers and big players to support IPv6?
When the problem of the lack of IPv4 addresses from theoretical became more and more real, providers began to use NAT. But in the end, this did not become a solution. When there are too many devices hiding behind NAT, sites may decide that robots are loading them and take measures to protect them from such actions.
Such a problem, in particular, has become widespread in Belarus . This happened relatively recently, after the transition of RIPE to the phase of address allocation from the last block / 8. Belarusian providers did not have a sufficient supply of IP addresses and, finding themselves in conditions of their shortage, went by increasing the number of users for NAT. As a result, some Internet services took the activity of NAT-users for the activity of robots and someof them completely blocked the IP address (and all NAT users) or increased the captcha display intensity. Thus, the unavailability of providers to exhaust IPv4 addresses has led to a decrease in the quality of Internet access for users. A similar problem exists for mail systems: there, large NAT pools mean that letters sent by a user to NAT can be mistaken for spam and not reach the addressee.
Schedule of IPv6 traffic at www.yandex.ru
Currently, according to various estimates, only about 3% of traffic on the Internet uses IPv6, and 97% use old IPv4. But when such major players as Yandex, Google, VKontakte, Facebook support IPv6 technically and ideologically, they push providers to support sixth addresses, they push the entire Internet into the future. Of course, it’s important for us and ourselves to understand that our services have enough addresses. But even in total, there are much fewer of them compared to how many sites may appear on the Internet in the future and how many people will get access to it.