March 15, 2013 at 15:11Packet balancing in linux aggregated interfaces (bond) at source or sender address
In a small telecom operator, Cisco SCE 2020 is used to control Internet access, which is installed between the Cisco 6500 and a debian server that performs NAT functions. Aggregated 2Gbit link is configured between the server and the switch. As the operator grew, the load of channels began to approach 1.8 Gbps, which is the SCE 2020 bandwidth limit at which it is able to adequately apply traffic processing policies. In this regard, it was decided to install a second SCE, expanding the link to 4Gbps.
The standard scheme for enabling multiple SCE 2020s, according to Cisco documentation, looks like this (1):
But, as usual, due to the lack of ports on the 6500, it was decided to include SCE between the server and the catalyst.
For the DPI functionality to work correctly (protocol definition, http redirection), it is necessary to ensure symmetric traffic flow through the SCE, i.e., packets of a specific user should always go both ways through the same port and, accordingly, the same SCE. From the Cisco 6500 side, the problem is solved by the “port-channel load-balance src-ip” (2) command, after which packets from the same source always go to the same port.
Accordingly, on the server side, it was required to enable traffic balancing by the recipient's IP address. It would seem that such a simple type of balancing should be implemented in the bonding module of the linux kernel, however, after studying the documentation, we found that this is not so. As a result, a small patch to the kernel 2.6.32 (3) was born, which adds balancing modes to the source or destination address.
After applying the patch, to use these types of balancing, you must enable bonding with the parameter bond_mode = 2 (Mode of operation: balance-xor), and in the xmit_hash_policy parameter select the necessary type of balancing: 3 for balancing by the recipient's IP address or 4 for balancing by the IP address sender.
1) “10 Gigabit and Multi-Gigabit Solution”www.cisco.com/en/US/prod/collateral/ps7045/ps6129/ps6133/ps6151/product_data_sheet0900aecd801d8574.html
2) Configuring EtherChannel Load Balancing www.cisco.com/en/US/docs/switches/lan/catalyst6500/ios /12.2SX/configuration/guide/channel.html#wp1020804
3) Patch for the kernel 2.6.32 pastebin.com/5YPNZBiy
The standard scheme for enabling multiple SCE 2020s, according to Cisco documentation, looks like this (1):
But, as usual, due to the lack of ports on the 6500, it was decided to include SCE between the server and the catalyst.
For the DPI functionality to work correctly (protocol definition, http redirection), it is necessary to ensure symmetric traffic flow through the SCE, i.e., packets of a specific user should always go both ways through the same port and, accordingly, the same SCE. From the Cisco 6500 side, the problem is solved by the “port-channel load-balance src-ip” (2) command, after which packets from the same source always go to the same port.
Accordingly, on the server side, it was required to enable traffic balancing by the recipient's IP address. It would seem that such a simple type of balancing should be implemented in the bonding module of the linux kernel, however, after studying the documentation, we found that this is not so. As a result, a small patch to the kernel 2.6.32 (3) was born, which adds balancing modes to the source or destination address.
After applying the patch, to use these types of balancing, you must enable bonding with the parameter bond_mode = 2 (Mode of operation: balance-xor), and in the xmit_hash_policy parameter select the necessary type of balancing: 3 for balancing by the recipient's IP address or 4 for balancing by the IP address sender.
1) “10 Gigabit and Multi-Gigabit Solution”www.cisco.com/en/US/prod/collateral/ps7045/ps6129/ps6133/ps6151/product_data_sheet0900aecd801d8574.html
2) Configuring EtherChannel Load Balancing www.cisco.com/en/US/docs/switches/lan/catalyst6500/ios /12.2SX/configuration/guide/channel.html#wp1020804
3) Patch for the kernel 2.6.32 pastebin.com/5YPNZBiy