Study of the sustainability of the national segments of the Internet for 2018
This study explains how the failure of one autonomous system (AS) affects the global connectivity of a particular region, especially when it comes to the country's largest Internet provider (ISP). The connectivity of the Internet at the network level is due to the interaction between autonomous systems. As the number of alternative routes increases between ASs, resistance to failures arises and the stability of the Internet in a given country increases. However, some paths become more important than others, and having as many alternative routes as possible is ultimately the only viable way to ensure system reliability (in the sense of AS).
The global connectivity of any AS, regardless of whether it represents a secondary Internet provider or an international giant with millions of service consumers, depends on the quantity and quality of its paths to Tier-1 providers. As a rule, Tier-1 implies an international company offering a global IP transit service and connecting to other Tier-1 operators. However, within this elite club there is no obligation to maintain such a connection. Only the market can motivate such companies to unconditionally connect with each other, providing high quality service. Is this a sufficient incentive? We will answer this question below - in the section on IPv6 connectivity.
If an Internet provider loses connection with at least one of its own Tier-1 connections, it will most likely be unavailable in some parts of the Earth.
Brief facts TL; DR :
Romania and Luxembourg out of the top 20 from 11th and 20th place, respectively, at the end of 2017;
Singapore jumped 18 places to 5th position;
Hong Kong fell 13 places to 15 position;
The Netherlands entered the top 20 in the 17th position;
18 out of 20 countries remained in the top 20 compared with last year.
Measuring Internet Reliability
Imagine that AS is experiencing significant network degradation. We are looking for the answer to the following question: “What percentage of ASs in this region may lose touch with Tier-1 operators, thereby losing global accessibility?”
Why simulate a similar situation? Strictly speaking, when BGP and the world of inter-domain routing were at the design stage, the creators assumed that each non-transit AS would have at least two upstream vendors (upstream) to ensure fault tolerance in case one of them falls. However, in reality, everything is completely different - more than 45% of ISPs have only one connection with a transit upstream. A set of unconventional relationships between transit ISPs further reduces overall reliability. So do transit ISPs fall? The answer is yes, and it happens quite often. The correct question in this case is: “When will a particular ISP experience a degradation of connectivity?”. If such problems seem remote to someone, it is worth remembering Murphy's law: “Everything that can go wrong will go wrong.”
To simulate a similar scenario, we use the same model for the third year in a row. In the same year, we didn’t just repeat the previous calculations - we significantly expanded the field of study. The following steps were completed to assess the reliability of the AS:
For every AS in the world, we obtain all alternative paths to Tier-1 operators using the AS relationship model, which is the core of the Qrator.Radar product;
Using the IPIP geodatabase, we compared countries with the provided address of each AS;
For each AS, we calculated the share of its address space corresponding to the selected region. This helped to filter out situations where an Internet provider may be present at the exchange point in a particular country, but does not have a presence in the region as a whole. An illustrative example is Hong Kong, where hundreds of participants of the largest Asian Internet exchange HKIX exchange traffic, having zero presence in the Internet segment of Hong Kong;
Having obtained clear results for AS in the region, we assess the impact of the possible failure of this AS on other AS and the countries in which they are represented;
In the end, for each country we found a specific AS, affecting the largest percentage of other AS in the region. Foreign AS are not considered.
IPv4 Reliability
Below you can see the top 20 countries in terms of reliability, in terms of resiliency in case of a single AS failure. In practice, this means that the country has good Internet connectivity, and the percentage reflects the share of AS, which will lose global connectivity in the event of failure of the largest AS.
While individual countries can move up and down by rating, the overall picture has not changed significantly since 2017. Last year, the average failure rate in the world was 41%, in 2018 it decreased by 3% to 38% with a small one. The number of countries that reduced their dependence on one AS to less than 10% (which is a sign of high fault tolerance) increased by one to 30.
Another noticeable change was a significant increase in Internet reliability in small countries in South Asia and Africa. These regions are still developing, but a significant improvement in diversity in the IP transit market is a sign of accelerated progress.
IPv6 game
It is believed that if the technology works well in IPv4, it can be easily ported to IPv6. This erroneous assumption may be a structural problem for the entire IPv6 development process.
Measuring global reliability between two protocol versions is not as easy as it may seem. To maintain global connectivity in IPv4, any and only path to the Tier-1 provider will be sufficient. But in IPv6 this may not be the case. Because of the ongoing peering wars between several top-level IPv6 providers, they are not all related to each other. At least two pairs of providers decided to break the peer-to-peer relationship (“de-peer”) on IPv6: Cogent (AS174) and Hurricane Electric (AS6939), Deutsche Telekom (AS3320) and Verizon US (AS701). These telecommunications companies may have a variety of reasons for their conflicts, but if the network is connected to only one side of them, it will not have full-fledged connectivity in IPv6.
Wanting to fix these problems, we adjusted the measurement process to see if full IPv6 connectivity is maintained during a failure. In other words, to ensure full connectivity and the highest reliability, the path to Tier-1 operators must be constantly present. We also calculated the percentage of ASs in the country that have only partial connectivity in IPv6 due to peer-to-peer wars. Here are the results:
A general comparison of IPv4 and IPv6 in the case of a single failure shows that 86% of the national IPv4 segments have a much higher connectivity. An important discovery in the world of IPv6 is that many Internet service providers do not have proper connectivity even under normal conditions - without any failures. For example, in the US, this refers to about 10% of all ASs that support IPv6, and in China, the situation is even worse, because China Telecom (AS4134) receives global connectivity from only one provider - Hurricane Electric.
As stated earlier, no one can force Tier-1 providers to connect with each other other than their customers. Recent data clearly shows that consumer demand is not a sufficient incentive for them to connect to each other and achieve 100% visibility of networks. The only way to improve this situation seems to be calls for an appropriate level of IPv6 service. The Qrator.Radar team is considering various options for how to make this information more apparent to every Internet provider in the world, thereby increasing community awareness on this issue.
As for partial connectivity, several countries “do not see” more than 20% of the IPv6 address space. It:
The finding of the two former island colonies, St. Maarten and St. Barthelemy, are explicable by looking at the overall IPv4 rating; microstates, in most cases, depend on a single Internet service provider. The remaining 9 states with values higher than 20%, and especially the United Arab Emirates with 25% of partially connected (respectively, without a full-fledged global IPv6 peering) networks, are surprising.
Broadband Internet and PTR Records
Repeating the question we asked ourselves from last year: “Is it true that the leading provider in the country always affects regional reliability more than everyone else or some other?”, We developed an additional metric for further study.
Perhaps the most significant (in terms of the client base) Internet provider in this area will not necessarily be the autonomous system that will become most important in ensuring global connectivity. An example can be quickly found in the topmost position in the IPv4 rating - Germany. Last year it was clear that German transit was at least 2.29% dependent on the provider Versatel. This year Versatel was replaced by DTAG (Deutsche Telekom), which improved connectivity in the country by 0.03%.
But knowing the position of DTAG did not let us sleep for a long time and we tried to develop a metric that would sometimes include invisible connections (ASN can change affiliation or geography, the number can be released) to show what really happens where we know for sure about the existence of a dominant local player. Russia with Rostelecom, the United States with the all-powerful Comcast, who own the lion's share of the broadband user base in their own countries. So what will happen in the region if such a provider suddenly fails?
After numerous experiments, we determined that the most accurate indicator of the actual value of the provider can be based on the analysis of PTR records. Typically, they are used for reverse DNS lookup: using the IP address you can identify the associated hostname or domain name.
This means that the PTR can allow the measurement of specific equipment in the address space of an individual operator. Since we already know the fattest AS for every country in the world, we could calculate the PTR records in the networks of these suppliers, determining their share among all the PTR records of the region. You should immediately make a disclaimer: we counted ONLY PTR records and did not calculate the ratio of IP addresses without PTR records to IP addresses with PTR records.
So, then we talk exclusively about IP addresses with PTR records present. Creating them is not a general rule, so some providers enter PTR, while others do not.
We showed how many of these IP addresses with the specified PTR records will be disconnected in case of disconnection from / together with the largest (by PTR) autonomous system in the specified country. The figure reflects the percentage of all IP addresses with PTR support in the region.
Let's compare the 20 most stable countries in IPv4 with a PTR rating:
Obviously, the approach of examining PTR records gives completely different results. In most cases, not only the central AS in the region changes, but the percentage of instability for the specified AS is completely different. In all reliable, from the point of view of global availability, the number of IP-addresses with PTR support that will turn off due to the fall of AS is ten times higher.
This may mean that the leading national Internet provider always owns the end users. Thus, we must assume that this percentage is part of the user and customer base of the ISP that will be disabled (in case you cannot switch to an alternative provider) in the event of a failure. From this point of view, countries no longer seem as reliable as they appear in terms of transit. We leave the reader to possible conclusions from comparing the top 20 IPv44 with PTR rating values.
Detail changes in the regions
Last year, we clearly pointed out the significant influence of the AS174 operator Cogent. This year, given the static 90% of the rating year-on-year, when Cogent served regions such as France, the United Kingdom, the United States and Ireland, we see all these countries in all positions with the same values depending on one upstream. However, changes occurred in the US, where AS209, Centurylink, dismissed Cogent. In response, Cogent has added Spain and Belgium to the list of countries where he is the dominant supplier of the compound.
Although this means that disabling Cogent represents a risk for several regions at once, it is also necessary to recognize that the market position of this particular provider is primarily the result of its good IPv4 connectivity. Despite the fact that Cogent added two more countries to its strategic portfolio, the denial of service will not lead to complete inaccessibility in these highly diversified national segments of the Internet.
But the big news is exactly what happened in the United States. For two years in a row - 2016 and 2017 - we have defined AS174, Cogent, as decisive in this market. This is no longer the case - in 2018, AS 209 CenturyLink replaced it, sending the United States up three positions to 7th place in the IPv4 rating.
Turning to the region of the former USSR, we see small changes compared to last year. Rostelecom remains the main Internet provider in Russia (AS 12389), whose market share is quite significant (reflected in PTR values). However, in 2018, the shutdown of Rostelecom will lead to a lack of global connectivity in only 5.27% of autonomous systems in the country, which puts Russia in 13th place in terms of reliability.
As in the past year, the transit market in Russia is represented mainly by medium Tier-2 networks, which leads to high availability values. The same cannot be said about Uzbekistan, where for the third year in a row we observe 99.9% dependence on one supplier (AS 28910). In Turkmenistan, there is only one upstream supplier: Rostelecom. The Tajikistan index is “much better” with a value of 78.14% of instability, which puts the country in 202nd place in the overall rating. The fourth country in the region with a high instability is Azerbaijan, although its 47% dependence on AS 29049 is not so bad compared to the rest.
The downgrade of Ukraine by four positions can be explained by the ongoing legislative processes inside the country, where ways are being sought to determine an acceptable level of applicable Internet regulation. Such intervention inevitably leads to some loss of reliability, although the position of Ukraine in the top 20 is quite stable for three years in a row.
There are “only” 83 countries with more than 40% dependence on a single supplier, 65 of them have values above 50%. There are 39, for the most part, small countries with instability indexes of 90% or more. There are exceptions: North Korea’s index is 100% for AS 131729, which does not need to be explained; the same applies to Eritrea, Greenland and New Caledonia, having their own reasons. 99% of Ethiopia’s only AS 24757 (EthioNet), in a country of 100 million people, is an unexpected fact in 2018. Syria is 99.5% dependent on AS 29256, which represents a significant loss compared to last year's 88.75%, caused by the shift of last year's AS owned by German telecom. Cuba has a 97% dependence on AS 11960. The Jamaica index is 91.3% for AS 34520 owned by Columbus Networks - a company
Compared with Luxembourg, which is on the 30th place of the classic rating with a value of 9.8% depending on AS6661, the 66% indicator of Monaco for AS6758 is not typical of a country of such a standard of living and geography.
Over the past two years, significant changes have taken place in the Asian market of Internet providers.
The increased reliability of Singapore connectivity is due to its position among Asian tigers, the region’s fastest growing economies. This competition is fierce at the top of the sustainability rating, where the positions are quite tight in percentage terms. Hong Kong and Singapore swapped on the basis of a 2% change in the index.
According to independent observers interviewed by us, Singtel positions in Singapore are weakening. Last year, the key AS was owned by StarHub, a Singapore broadband Internet access provider. In 2018, the main AS in Singapore is already becoming SingNet AS3758 (SingTel's broadband division). We were told that lately, SingTel has been experiencing a lack of capital, since their main business in Singapore has stopped active growth (which can be further confirmed by looking at the dynamics of the company's shares over the past two years). The difficulties of SingTel are exacerbated by the mutual integration process with the Philippine ISP Globe - this is probably another reason why SingNet lost consumers, thereby losing market share and real connections. These events together dramatically and rather sharply decentralized the ISP market of Singapore,
Speaking of Globe ISP, owning AS4775, last year was leading for the Philippines, in 2018 it ceased to be - AS9299 surpassed it. AS9299 is owned by PLDT, which is reportedly becoming increasingly aggressive in the Philippine market. In addition, the position of the region is restored. The Philippines fell from 20th place in the reliability rating to 31st in 2017, and rose to 27th now. This fact alone clearly shows that competition is good, especially when it comes to the reliability of the Internet in a particular region. However, we should not forget that Globe represents SingTel, and PLDT is, in fact, an NTT branch in the Philippines.
Hong Kong's loss of reliability rating may be related to PCCW activities in the region. Local observers tell us that in 2018, PCCW chose an active customer acquisition strategy, gaining market share and active customers in the economic zone itself.
Last year, the main AS of Hong Kong belonged to Level3 (AS3356). After the merger with CenturyLink at the end of 2017, the position of Level3 could not fail to change to a larger, but at the same time less stable from a regional point of view.
With the growing risks of cyber security and, in fact, the constant flow of news about attacks on the Internet infrastructure, it is time for all governments, private and public companies, but first of all for ordinary users to carefully assess their positions. The risks associated with the connectivity of the regions must be studied carefully and honestly, analyzing the true levels of reliability. Even low values in the rating of instability can cause real problems with accessibility in the event of a massive attack on a large, nationwide, provider of a critical service, say, DNS. Do not forget also that the outside world will disconnect from services and data located within the region, in case of complete loss of connectivity.
Our study clearly shows that the markets of Internet providers and telecom operators based on competition ultimately develop more dynamically in order to become much more stable and fault tolerant with respect to risks inside and even outside of a specific region. Without a competitive market, the failure of one AS can and will lead to the loss of a network connection from a significant part of users from a country or a wider region.