GitHub discovered the code of its load balancer - how their solution works
The developers from GitHub last week laid out in open access the sources of their load balancer - GLB Director. The team worked on this project for several years.
What is remarkable about their decision, how it is arranged, and who has yet transferred the load distribution systems to open source, we describe below.
/ Flickr / theilr / CC
GitHub uses bare metal based cloud infrastructure to improve performance. In this case, the software works without additional levels of virtualization on the bare metal.
Previously, the company used haproxy with a special hardware configuration for load balancing , which provided fault tolerance for 10 Gigabit Ethernet connections. However, this approach did not scale well (it meant vertical scaling), and GitHub decided to write its own load balancer, which could still work on low-cost hardware.
The GitHub balancer provides uninterrupted TCP connections, manages the load of individual services, is resistant to DDoS attacks and is able to scale horizontally. It is “ sharpened ” for work in data centers, where a large number of servers announce one IP address via BGP , and routers use ECMP strategy .
Load balancing is performed at levels L4 and L7. Unlike solutions such as LVS , GLB Director does not direct all packets to the director node (routing node), in order to redistribute them between other nodes. Instead, it uses a rendezvous hashing variation (rendezvous hashing, HRW) to create a static table to select for each incoming connection a pair of proxy servers (primary and secondary). If one of them fails, the package is sent to the second. The system remembers this choice, and it does not need to be made for each package.
The "health" of the servers is monitored by the glb-healthcheck solution, which switches the primary and secondary systems in case of detection of problems. glb-healthcheck monitors the correct operation of each GUE tunnel (Generic UDP Encapsulation) and an arbitrary HTTP port of backend servers.
GLB also uses the Netfilter system and the iptables utility.. Netfilter solves a simple problem: it determines whether the internal TCP / IP packet in each GUE packet meets the requirements of the Linux kernel stack. If not, it redirects the packet to the secondary proxy server, rather than decapsulating it locally.
The scheme of interaction of components looks like this:
In GitHub they hope that their balancer will be useful to all companies that have their data centers.
How to install GLB and start working with it can be found in the quick start guide prepared by the developers .
In May, Facebook also shared the source code for its Katran load balancer library. The IT giant uses it to effectively distribute the load between backend servers.
The previous balancer of the company - L4LB - could not cope with the task, as it required dedicated servers for operation, which increased the load on the network. To solve this problem, the company and developed Katran. It is launched using the eXpress Data Path framework and the eBPF virtual machine. VM extends the overall functionality by running programs at specific points in the Linux kernel.
/ Flickr / Da Sal / CC
Updated balancer distributes more efficientlyinfrastructure load and increases packet processing speed. Sources developers "flooded" on GitHub.
The Katran system has a number of differences from the solution proposed in GitHub. For example, Facebook uses XDP and IPIP tunnels that work with the Linux kernel. GLB, by contrast, resorts to using DPDK to process packets from user space.
Theo Julienne, developer of GitHub, added that DPDK allows to handle large volumes of incoming traffic. This guarantees high performance (10 Gigabit connectivity) even in complex working environments and provides some protection against DDoS attacks.
The transfer of such powerful tools as GLB and Katran in open source will open up new opportunities for other IT companies and will contribute to the more rapid development of the IT ecosystem in the world.
PS A couple of additional articles from the First Corporate IaaS blog:
What is remarkable about their decision, how it is arranged, and who has yet transferred the load distribution systems to open source, we describe below.
/ Flickr / theilr / CC
Why does GitHub have a balancer?
GitHub uses bare metal based cloud infrastructure to improve performance. In this case, the software works without additional levels of virtualization on the bare metal.
Previously, the company used haproxy with a special hardware configuration for load balancing , which provided fault tolerance for 10 Gigabit Ethernet connections. However, this approach did not scale well (it meant vertical scaling), and GitHub decided to write its own load balancer, which could still work on low-cost hardware.
What GLB Director can do and how it works
The GitHub balancer provides uninterrupted TCP connections, manages the load of individual services, is resistant to DDoS attacks and is able to scale horizontally. It is “ sharpened ” for work in data centers, where a large number of servers announce one IP address via BGP , and routers use ECMP strategy .
Load balancing is performed at levels L4 and L7. Unlike solutions such as LVS , GLB Director does not direct all packets to the director node (routing node), in order to redistribute them between other nodes. Instead, it uses a rendezvous hashing variation (rendezvous hashing, HRW) to create a static table to select for each incoming connection a pair of proxy servers (primary and secondary). If one of them fails, the package is sent to the second. The system remembers this choice, and it does not need to be made for each package.
The "health" of the servers is monitored by the glb-healthcheck solution, which switches the primary and secondary systems in case of detection of problems. glb-healthcheck monitors the correct operation of each GUE tunnel (Generic UDP Encapsulation) and an arbitrary HTTP port of backend servers.
GLB also uses the Netfilter system and the iptables utility.. Netfilter solves a simple problem: it determines whether the internal TCP / IP packet in each GUE packet meets the requirements of the Linux kernel stack. If not, it redirects the packet to the secondary proxy server, rather than decapsulating it locally.
The scheme of interaction of components looks like this:
In GitHub they hope that their balancer will be useful to all companies that have their data centers.
How to install GLB and start working with it can be found in the quick start guide prepared by the developers .
Similar developments
In May, Facebook also shared the source code for its Katran load balancer library. The IT giant uses it to effectively distribute the load between backend servers.
The previous balancer of the company - L4LB - could not cope with the task, as it required dedicated servers for operation, which increased the load on the network. To solve this problem, the company and developed Katran. It is launched using the eXpress Data Path framework and the eBPF virtual machine. VM extends the overall functionality by running programs at specific points in the Linux kernel.
/ Flickr / Da Sal / CC
Updated balancer distributes more efficientlyinfrastructure load and increases packet processing speed. Sources developers "flooded" on GitHub.
The Katran system has a number of differences from the solution proposed in GitHub. For example, Facebook uses XDP and IPIP tunnels that work with the Linux kernel. GLB, by contrast, resorts to using DPDK to process packets from user space.
Theo Julienne, developer of GitHub, added that DPDK allows to handle large volumes of incoming traffic. This guarantees high performance (10 Gigabit connectivity) even in complex working environments and provides some protection against DDoS attacks.
The transfer of such powerful tools as GLB and Katran in open source will open up new opportunities for other IT companies and will contribute to the more rapid development of the IT ecosystem in the world.
PS A couple of additional articles from the First Corporate IaaS blog:
- How to place 100% of the infrastructure in the cloud IaaS-provider and not regret it
- Testing a disk system in the cloud: the basics
- IaaS Cloud Load Balancing: Why DRS is needed