June 28, 2019 at 17:01RDMA inside the data center in the implementation of Huawei
Under the cat about what approach Huawei offers when organizing remote direct memory access using AI Fabric technology and how it differs from InfiniBand and pure Ethernet-based RDMA.
Distributed computing is used in a wide variety of industries. These are scientific research, and technical developments like face recognition or autopilot recognition tools, and industry. In general, data analysis is finding more and more applications, and we can say with confidence that in the near future it will not lose popularity. In fact, now we are experiencing a transition from the era of cloud computing, where the most important factors were applications and the speed of deployment of services, to the era of data monetization, including through the use of artificial intelligence algorithms. According to our internal data (reportGIV 2025: Unfolding the Industry Blueprint of an Intelligent World ) By 2025, 86% of companies will use AI in their work. Many of them consider this area as the main for the modernization of activities and, possibly, the basic tool for making business decisions in the future. And this means that each of these companies will need some kind of processing of raw data - most likely through distributed clusters.
With the growing popularity of distributed computing, the volume of traffic exchanged between individual data center machines increases. Traditionally, when discussing networks, attention is focused on the growth of traffic between the data center and end users on the Internet, and it is really growing. But the increase in horizontal traffic within distributed systems far exceeds everything that users generate. According to Facebook, traffic between their internal systems doubles in less than a year.
In an attempt to cope with this traffic, you can increase the clusters, but you can not do this indefinitely. Therefore, predicting the growth of the computing load on the clusters, it is necessary to increase the processing efficiency - first of all, to find and eliminate bottlenecks inside these distributed networks.
If earlier the resources of each of these systems separately were the “weak link” of distributed systems, while constantly evolving data transmission networks even outstripped the needs, today it is network communications that are the main source of the problem. The familiar TCP / IP protocol stack and tree topology no longer correspond to the assigned tasks. Therefore, more and more data centers are abandoning the centralized one and are moving to a new CLOS architecture that provides greater bandwidth and better cluster scalability, as, for example, Facebook did several years ago.
At the same time, it is necessary to optimize the process at a different level - at the level of interaction of two separate systems. In this article we want to talk about what optimization tools the Huawei Ai Fabric data center provides. This is our proprietary technology that accelerates the exchange of data between nodes.
The main “trick” of Huawei Ai Fabric is to reduce the overhead when transferring data packets between systems within the cluster by implementing RDMA (Remote Direct Memory Access) - direct access to the memory of systems in the cluster.
RDMA is not a new idea. The technology provides direct data exchange between memory and the network interface, reducing latency and eliminating unnecessary copying of data to buffers. Its roots go back to the 1990s by Compaq, Intel and Microsoft.
There are three types of delays in transmitting a packet from one system to another:
To reduce losses throughout this chain, as early as the 1990s, it was proposed to use direct access to the memory of interacting systems - an abstract model of Virtual Interface Architecture. Its main idea is that applications running on two interacting systems completely fill their local memory and establish a P2P connection for data transfer without affecting the OS. In this way, packet transmission delays can be significantly reduced. In addition, since the VIA model did not imply placing the transmitted data in intermediate buffers, it saved the resources needed for the copy operation.
Regarding the abstract model, VIA RDMA, as a technology, has stepped further towards optimal resource utilization. In particular, it does not wait for the buffer to be filled to establish a connection and allows connections to several computers simultaneously. Due to this, the technology can reduce transmission delays up to 1 ms, reducing the load on the processor.
The two main RDMA implementations on the market - the proprietary InfiniBand transport protocol and the “pure” Ethernet-based RDMA, are unfortunately not without drawbacks.
The InfiniBand transport protocol has a built-in packet delivery control mechanism (data loss protection), but is supported by specific equipment and is not compatible with Ethernet. In fact, the use of this protocol closes the data center at one supplier of equipment, which carries certain risks and promises difficulties in terms of service (since InfiniBand has a small market share, it will not be so easy to find specialists). Well, of course, when implementing the protocol, you cannot use existing IP-network equipment.
RDMA over Ethernet allows you to use existing equipment on the network, supports Ethernet networks, which means it will be easier to find service specialists. Compared to Infiniband, this significantly reduces the cost of ownership of the infrastructure and simplifies its deployment.
The only serious drawback that prevented the widespread adoption of RDMA over Ethernet is the lack of protection against packet loss, which limits the bandwidth of the entire network. Third-party mechanisms must be used to reduce packet loss or prevent network congestion. We went just this way, offering our own intelligent algorithms for compensating for the disadvantages of RDMA over Ethernet while maintaining its advantages in the new tool - Huawei Ai Fabric.
AI Fabric implements RDMA over Ethernet, supplemented by its own intelligent network congestion management algorithm, which provides zero packet loss, high network bandwidth and low transmission delay for RDMA streams.
Huawei Ai Fabric is built on open standards and supports a range of different equipment, which optimizes the implementation process. However, some additional tools - add-ons over open standards, allowing to increase the efficiency of data exchange, which we will discuss in subsequent publications - are available only for devices manufactured by Huawei. The CloudEngine series switches that support the solution have an integrated chip that analyzes traffic characteristics and dynamically adjusts network parameters, which allows more efficient use of the switch buffer. The collected characteristics are also used to predict future traffic patterns.
Huawei Ai Fabric allows you to get profit on two levels.
On the one hand, the solution allows optimizing the data center architecture - reducing the number of nodes (due to more optimal utilization of resources), creating a converged environment without the traditional separation into separate subnets, which are difficult and expensive to maintain in parts. Using the tool, you do not have to select separate subnets for each type of service in the domain controller (with its own network requirements). You can create a single environment that provides all services.
On the other hand, AI Fabric allows you to increase the speed of distributed computing, especially where you often need to access the memory of remote systems. For example, the introduction of AI in any field implies a period of learning the algorithm, which can include millions of operations, so the gain in delay on each such operation will result in a serious acceleration of the process.
The effect of introducing a specialized tool, such as Huawei Ai Fabric, will be noticeable in a data center with six or more switches. But the larger the data center, the higher the profit - due to the optimal utilization of resources, a cluster of the same scale with Ai Fabric will provide higher performance. For example, a cluster of 384 nodes can achieve the performance of a “regular” cluster of 512 nodes. Moreover, the solution has no restrictions on the number of physical switches within the infrastructure. There can be tens of thousands (if you forget that projects are usually limited to the size of the administrative domain).
Distributed computing is used in a wide variety of industries. These are scientific research, and technical developments like face recognition or autopilot recognition tools, and industry. In general, data analysis is finding more and more applications, and we can say with confidence that in the near future it will not lose popularity. In fact, now we are experiencing a transition from the era of cloud computing, where the most important factors were applications and the speed of deployment of services, to the era of data monetization, including through the use of artificial intelligence algorithms. According to our internal data (reportGIV 2025: Unfolding the Industry Blueprint of an Intelligent World ) By 2025, 86% of companies will use AI in their work. Many of them consider this area as the main for the modernization of activities and, possibly, the basic tool for making business decisions in the future. And this means that each of these companies will need some kind of processing of raw data - most likely through distributed clusters.
The evolution of architecture
With the growing popularity of distributed computing, the volume of traffic exchanged between individual data center machines increases. Traditionally, when discussing networks, attention is focused on the growth of traffic between the data center and end users on the Internet, and it is really growing. But the increase in horizontal traffic within distributed systems far exceeds everything that users generate. According to Facebook, traffic between their internal systems doubles in less than a year.
In an attempt to cope with this traffic, you can increase the clusters, but you can not do this indefinitely. Therefore, predicting the growth of the computing load on the clusters, it is necessary to increase the processing efficiency - first of all, to find and eliminate bottlenecks inside these distributed networks.
If earlier the resources of each of these systems separately were the “weak link” of distributed systems, while constantly evolving data transmission networks even outstripped the needs, today it is network communications that are the main source of the problem. The familiar TCP / IP protocol stack and tree topology no longer correspond to the assigned tasks. Therefore, more and more data centers are abandoning the centralized one and are moving to a new CLOS architecture that provides greater bandwidth and better cluster scalability, as, for example, Facebook did several years ago.
At the same time, it is necessary to optimize the process at a different level - at the level of interaction of two separate systems. In this article we want to talk about what optimization tools the Huawei Ai Fabric data center provides. This is our proprietary technology that accelerates the exchange of data between nodes.
Networking Changes
The main “trick” of Huawei Ai Fabric is to reduce the overhead when transferring data packets between systems within the cluster by implementing RDMA (Remote Direct Memory Access) - direct access to the memory of systems in the cluster.
RDMA - a way to reduce transmission delays
RDMA is not a new idea. The technology provides direct data exchange between memory and the network interface, reducing latency and eliminating unnecessary copying of data to buffers. Its roots go back to the 1990s by Compaq, Intel and Microsoft.
There are three types of delays in transmitting a packet from one system to another:
- due to processor processing necessary, for example, for buffering data in the OS and calculating check sums;
- due to buses and data transmission channels (it is technically impossible to significantly increase the bandwidth);
- due to network equipment.
To reduce losses throughout this chain, as early as the 1990s, it was proposed to use direct access to the memory of interacting systems - an abstract model of Virtual Interface Architecture. Its main idea is that applications running on two interacting systems completely fill their local memory and establish a P2P connection for data transfer without affecting the OS. In this way, packet transmission delays can be significantly reduced. In addition, since the VIA model did not imply placing the transmitted data in intermediate buffers, it saved the resources needed for the copy operation.
Regarding the abstract model, VIA RDMA, as a technology, has stepped further towards optimal resource utilization. In particular, it does not wait for the buffer to be filled to establish a connection and allows connections to several computers simultaneously. Due to this, the technology can reduce transmission delays up to 1 ms, reducing the load on the processor.
InfiniBand vs Ethernet
The two main RDMA implementations on the market - the proprietary InfiniBand transport protocol and the “pure” Ethernet-based RDMA, are unfortunately not without drawbacks.
The InfiniBand transport protocol has a built-in packet delivery control mechanism (data loss protection), but is supported by specific equipment and is not compatible with Ethernet. In fact, the use of this protocol closes the data center at one supplier of equipment, which carries certain risks and promises difficulties in terms of service (since InfiniBand has a small market share, it will not be so easy to find specialists). Well, of course, when implementing the protocol, you cannot use existing IP-network equipment.
RDMA over Ethernet allows you to use existing equipment on the network, supports Ethernet networks, which means it will be easier to find service specialists. Compared to Infiniband, this significantly reduces the cost of ownership of the infrastructure and simplifies its deployment.
The only serious drawback that prevented the widespread adoption of RDMA over Ethernet is the lack of protection against packet loss, which limits the bandwidth of the entire network. Third-party mechanisms must be used to reduce packet loss or prevent network congestion. We went just this way, offering our own intelligent algorithms for compensating for the disadvantages of RDMA over Ethernet while maintaining its advantages in the new tool - Huawei Ai Fabric.
Huawei AI Fabric - its way
AI Fabric implements RDMA over Ethernet, supplemented by its own intelligent network congestion management algorithm, which provides zero packet loss, high network bandwidth and low transmission delay for RDMA streams.
Huawei Ai Fabric is built on open standards and supports a range of different equipment, which optimizes the implementation process. However, some additional tools - add-ons over open standards, allowing to increase the efficiency of data exchange, which we will discuss in subsequent publications - are available only for devices manufactured by Huawei. The CloudEngine series switches that support the solution have an integrated chip that analyzes traffic characteristics and dynamically adjusts network parameters, which allows more efficient use of the switch buffer. The collected characteristics are also used to predict future traffic patterns.
Who is this useful for?
Huawei Ai Fabric allows you to get profit on two levels.
On the one hand, the solution allows optimizing the data center architecture - reducing the number of nodes (due to more optimal utilization of resources), creating a converged environment without the traditional separation into separate subnets, which are difficult and expensive to maintain in parts. Using the tool, you do not have to select separate subnets for each type of service in the domain controller (with its own network requirements). You can create a single environment that provides all services.
On the other hand, AI Fabric allows you to increase the speed of distributed computing, especially where you often need to access the memory of remote systems. For example, the introduction of AI in any field implies a period of learning the algorithm, which can include millions of operations, so the gain in delay on each such operation will result in a serious acceleration of the process.
The effect of introducing a specialized tool, such as Huawei Ai Fabric, will be noticeable in a data center with six or more switches. But the larger the data center, the higher the profit - due to the optimal utilization of resources, a cluster of the same scale with Ai Fabric will provide higher performance. For example, a cluster of 384 nodes can achieve the performance of a “regular” cluster of 512 nodes. Moreover, the solution has no restrictions on the number of physical switches within the infrastructure. There can be tens of thousands (if you forget that projects are usually limited to the size of the administrative domain).
