The secret world of microwave networks (RRL) communication

Between London and Frankfurt stretched a mysterious private communications network, which operates twice as fast as regular Internet. The connection is provided by a chain of towers with microwave transceivers (we have referred to as radio relay or RRL - note. Trans.). Once the network was completely secret: it was used by a single, very rich company, and no one knew about it.

However, a couple of years later, a competitor built his own microwave link between the two cities. And then the first company, in order not to lose an advantage in business, revealed information about its line. If it were not for the efforts of a competitor, probably this secret would still not have been revealed.

Similar stories happened around the world. But due to the fact that these are private networks and they are often used by financial groups, trying to gain an advantage in the stock market and earn more than billions, it is necessary to make efforts to find information about such communication networks.

For example, in 2013, when the high-frequency trading (HFT) bubble began to burst, researchers from Californiastudied a large sample of stock trading and found that since March 2011, the network latency between Chicago and New York has dropped sharply by 2.5 ms. Prior to this, the best result was 7.5 ms, so the reduction to 5 ms turned out to be significant. Next, the researchers used the FCC and other records, on the basis of which they concluded that as of 2013 (when the study was conducted), there were 15 (!) Microwave communication networks between the two cities. Probably, the marked decrease in the delay was due to the commissioning of the new network.



^{A note from a 1949 newspaper about the AT & T microwave network, “just 34 jumps” connecting New York to Chicago}

What about British weather?

By themselves, microwave networks have been known for a very long time. In 1949, between New York and Chicago (about 1,140 km in a straight line), AT & T launched a communication line of 34 retransmission segments between communication towers that are in line of sight. In the UK, from the mid-1950s to the 1980s, a national communications backbone network was created from microwave networks. She served everything from television and telephones to the needs of the military and intelligence.



^{The basis of the British microwave network in 1956}
Expanding the microwave network was cheaper than pulling cables, and its bandwidth was higher than that of copper-conducting technologies of the time. But the disadvantage of microwaves (usually the frequencies from 6 to 30 GHz) is the need for very accurate positioning of the antennas and providing a clear view between the towers. Moreover, rain, clouds, or simply cloudy air weaken the microwave signal — the so-called rain fade effect.

You can partially compensate for attenuation by enhancing transmit power by installing larger antennas with a hydrophobic coating, as well as implementing fault-tolerant protocols that use radio frequency attenuation to improve communication stability in inclement weather. Also since the 1980s, microwave communications have been replaced or supplemented by fiber as an additional transmission channel.

To date, optical fiber has practically superseded the microwave in the main channels. Nevertheless, microwave networks are widely used so far, and the underlying technologies are being actively developed. Nowadays, microwave is mostly used to connect remote, hard-to-reach areas to the Internet, as well as for special tasks such as servicing private financial institutions. Just a few years ago, almost all cell phone towers in Europe provided microwave transit, but now many sections have been replaced with fiber to reduce the load on LTE and LTE Advanced base stations.



^{Map of some microwave networks in the south of England and in continental Europe. Networks in the Cornwall area are used to connect to underwater trunk cables that go to the surface here.}

How long will they last?

Given that the world is already tightly covered with high-speed fiber optic cables , a logical question arises: why do we need our own microwave networks?

The first reason is obvious: it is easier to ensure security, quality of service, bandwidth and other things that are important for business in your network.

The second reason that has already been mentioned is that, surprisingly, microwave networks have a lower level of delay compared to fiber-optic channels. At short distances, fiber works quickly, but on long routes, for example, between overseas offices and the stock exchange, lags often occur.

Unfortunately, long fiber optic networks are very expensive: it is necessary to dig a trench hundreds or thousands of kilometers long (or pay for access to networks already laid by other companies). You also need to take into account the relief: how to cross mountains and rivers - directly (more expensive) or make a hook and pull the cable through the nearest bridge or tunnel? Actually, therefore, most terrestrial fiber optic lines are located close to highways and railways.

Every time when network designers have to make such important decisions when planning the route, the total delay increases slightly. In total, a few extra milliseconds are typed. And as a result, microwave networks begin to win on this parameter.



<sup>Map of land and underwater European lines, 2001. More modern maps are hard to find, but most of the lines are still in use, simply with updated equipment.


Landline



maps in Europe, 2002 A modern map of submarine cables between the UK and the continent</sup>
Take for example London and Frankfurt. Between them in a straight line about 644 km. The light will cover this distance (299 700 km / s) in 2.126 ms. In optical fiber, the light beam does not move in a straight line, but along a broken line, so the speed drops to 200,000 km / s, which gives us a theoretical minimum of 3.186 ms.

In addition, in reality, the cable between London and Frankfurt does not consist of a single piece. For example, a data packet may be forwarded several times within London itself before reaching the desired router and starting a journey through Europe. On the way he meets other routers and repeaters. Upon arrival in Frankfurt, a packet can also be transmitted several times from one network node to another, before reaching the destination. Add to this a variety of terrain and infrastructure, passing through the submarine cable (through the French Calais or the Belgian Oostende), as well as the likelihood that the London router will decide to send the package through Paris. The result is an average delay between London and Frankfurt at 17 ms.

Now compare: the delay of the private microwave network between them is about 4.2 ms. Around - because the parameters of the newest networks are kept secret. Therefore, companies for which the time delay is critical, make a choice in favor of the microwave, and not fiber.

Creating a point-to-point network

Microwave networks have two key benefits:

• the radio signal goes through the air about 1.5 times faster than the light in the fiber,
• most often between two points you can build a network close to a straight line.

The latter means that you can greatly reduce the physical distance of the package. In addition, the number of intermediate routers decreases when a packet is transferred from point A to point B.



^{BridgeWave FP80-3000 transceiver capable of transmitting up to 3 Gbit / s at 80 GHz. Antenna diameter - 30 cm.}
If we talk about infrastructure, the microwave network is a chain of towers with two transceivers directed in opposite directions. The distance between the towers depends on the conditions on the ground, but on average - from 40 to 65 km. The maximum distance is determined by the height of the antennas above the ground, frequency limitations and terrain. If you hoist the mast to the top of the hill and put a powerful transmitter, you can easily overcome more than 80 kilometers, resting only on the radio horizon.

The cost of the transceiver is around 10–20 thousand pounds sterling for a device like the aforementioned BridgeWave plus 100–200 thousand pounds for each tower (this includes material delivery, erection, backup power, and so on). Returning to the example of London-Frankfurt: this network consists of about 20 towers, that is, the cost of construction is 2.5-5 million pounds sterling. This does not include personnel costs, operating expenses (food, land rent, technical support) and much more. Companies that manage such networks refuse to disclose total costs, but about London-Frankfurt it is known that we are talking about an amount of 10 to 20 million euros.

If you compare the cost of deploying a microwave network and cellular, in the second case, you need to purchase a lot of equipment and combine the towers with a network, fiber-optic or microwave. According to 2013 data, AT & T sold 9,700 towers for $ 4.85 billion, each with $ 500,000.

Finally, for the sake of completeness, let's see how much a direct fiber-optic network between London and Frankfurt will cost. Suppose we have a 644-kilometer cable, which costs about £ 1 per meter per year to rent. The cost of laying your cable depends on a number of parameters, including the number of nodes and shielding, but in general it is 0.5-1 pound per meter for a normal ground cable. In addition, as in the case of a microwave network, every 30–50 km we need signal amplifiers, which, although expensive, are still cheaper than microwave transceivers.

Laying a new fiber is cheaper than deploying its own microwave network, but this is true only if there is already a trunk between the two points to which you can connect on both sides. But if you need to build your own 640 km line ... good luck!

Fast networks need fast software.

Reducing the delay by several microseconds when brokers connect to the exchange system is impressive, but it will not be of any use if the software does not keep up with the speed of the network.

For the most part, financial institutions use microwave networks for algorithmic high-frequency trading. We are talking about computer algorithms that process data on securities markets and make quick decisions (sell, buy, long positions, short) in order to outrun other traders. In the early 2000s, the first algorithms competed with people, but today there is a continuous dominance of algorithms on the market.

As you understand, the details of the algorithms are a strictly guarded secret. But we can tell about a curious case. On the morning of August 1, 2012, Knight Capital, which occupied one of the leading positions in the HFT field, rolled out a new trading software. Starting at approximately 9:30, the application went out of control and began to buy and sell randomly. For 45 minutes, losses amounted to $ 7 billion. In the end, the company managed to roll back part of the deals and reduce the damage to $ 460 million, at that time - about half the market value of Knight Capital itself. Since then, this case is called the “Knightmare” (pun: nightmare - a bad dream).



<sup>The graph shows the volatile behavior of the trading application Knight Capital. At 9:52 there was a failure, probably, someone interfered with the software, then it bucked up again. By 10 am, it was fixed. The


graph shows the trading volume of the application compared to other exchanges. Probably it was here that the most money was lost. The more


detailed behavior of the algorithm over the 27-second segment. Blue shows the bid and ask prices on the New York Stock Exchange (NYSE) An


even more detailed chart. The app sells for 38.60, then buys for 38.75, losing 15 cents each time. The bot carried out transactions about 40 times per second, 2400 times per minute.


And here you can see the behavior of other trading software (Nokia). Bot sells and buys again and again</sup>

In addition to the microwave

In the past couple of years, some network companies have been experimenting with laser coupling as a substitute for point-to-point microwave networks.

Today, laser networks do not work much faster: the delay is slightly lower, the throughput is almost the same. And the signal transmitted at microwave frequencies (about 200,000 GHz) is extremely sensitive to precipitation and cloudiness. In part, this can be compensated by adaptive optics (when the receiver tries to change its shape to compensate for signal distortion), but in case of bad weather, you still can not do without a backup channel - microwave, millimeter communications, or optical fiber.



^{Equipment for creating a laser network manufactured by AOptix}
Now the popularity of point-to-point millimeter networks, which use part of the spectrum over the microwave range (~ 30-300 GHz), is growing. But in many cases they play the role of an alternative link, their bandwidth is slightly higher compared to the microwave and the weather resistance is even worse. The use of the millimeter range for 5G deployment is also being discussed now, and a number of WiGig devices (802.11ad) are already using it. But all this is very different from directional point-to-point networks.

The future of private networks

^{One of the microwave towers}
Companies that create communication networks do not particularly spread about their current and future projects. However, curious information about the creation of the transatlantic microwave networkwas voiced. In theory, transceivers can be placed on anchored barges (like small oil platforms) or on tethered balloons. The benefits are the same as for terrestrial networks: low latency. Today, when transmitting data over fiber from the west coast of England to the east coast of Long Island (4,960 km), the delay is 25 ms. On microwave it will be about 16 ms.

The difference of 9 ms does not look very impressive, but if you add delays in the transmission of microwave from London to Cornwall (such a network already exists), as well as from Long Island to New York (also exists), then we get a total on the London line - New York delay of about 25 ms. And this is MUCH faster than the current 60 ms fiber.

But, alas, no one has yet invested in the creation of 80 floating microwave towers crossing the Atlantic. As far as is known ...