Configuring server operation on CentOS with 2 gateways and balancing between them

Instead of joining

It is based on an earlier article I read on Habré , which personally was enough for me to understand the policy routing mechanism as a whole - and disastrously little to implement this type of routing on the company's server. There were 2 serious pitfalls that had to be worked on independently, and which cannot be ignored:

• Saving settings in general
• Interrupting Network Manager Utility Settings

I will write my article in the form of the instructions that I wrote for future generations of IT specialists in my company - so I will cite some points from the main article either unchanged or retold for myself. I will highlight italics . For a complete understanding of what is written here, I recommend that you familiarize yourself with it completely.

A little practice, or what we want

And we want to get routing, which:

• will send data to the same gateway from which the request came
• will be controlled to balance the load between the gateways, using the so-called Gateway weight

For these purposes, we will use the iproute2 utility, which is included in the standard distribution of all Linux distributions.

We give out IP addresses:

ip a a 11.11.11.11/22 dev eth6
ip a a 22.22.22.22/28 dev eth5

Defining tables:
To use more than one route, static routing is no longer enough. For OSPF, our scheme is too simple, so let's focus on dynamic routing using tables.

ip route add default via 22.22.22.17 table 101
ip route add default via 11.11.8.1 table 102

We wrap the outgoing traffic to the gateways from which the incoming one came:

ip rule add from 22.22.22.17 table 101
ip rule add from 11.11.8.1 table 102

I think now it’s not necessary to explain the meaning of these lines. Similarly, you can make the server available on more than two gateways.

Balancing traffic between uplinks It becomes
one elegant team:

ip route replace default scope global \
nexthop via 22.22.22.17 dev eth5 weight 3 \
nexthop via 11.11.8.1 dev eth6 weight 7

This entry will replace the existing default routing in the main table. In this case, the route will be selected depending on the weight of the gateway (weight). For example, if you specify weights 7 and 3, 70% of the connections will go through the first gateway, and 30% through the second. There is one thing to consider when doing this: the kernel caches the routes, and the route for any host through a certain gateway will hang in the table for some time after the last access to this record. And the route to frequently used hosts may not be in time to be reset and will be constantly updated in the cache, remaining on the same gateway. If this is a problem, you can sometimes clear the cache manually with the ip route flush cache command.

Result
After executing this command, you can check server availability. In principle, one could say that this is where our work ended. However, there is one problem - after rebooting, all settings will be reset.

Saving settings in general

Now you need to force the system to apply the settings after a reboot.

IP Addresses
First, find out which mac-address which interface belongs to. We execute the command:

ip a

Next, we create ifconfig configuration files using the obtained mac-addresses. To do this, create files with the ifcfg prefix and the interface name in the folder



Create routing tables
Next, you need to create persistent tables, give them names (to make it easier to navigate), and write down the rules for using these tables:

cd /etc/iproute2/ 
echo "101 int" >> rt_tables
echo "102 ext" >> rt_tables

Now we return to the catalog /etc/sysconfig/network-scriptsand continue to work there.
Create the contents of the tables:

# cat route-eth5
default via 22.22.22.17 table ext
# cat route-eth6
default via 11.11.8.1 table int

We create rules for processing these tables:

# cat rule-eth5
from 22.22.22.22 lookup ext
# cat rule-eth6
from 11.11.11.11 lookup int

Balancing
Next, we replace the statics with dynamics. Not without a file, because the startup scripts described above are bound to interfaces, and in our case the rule is written immediately about 2 interfaces. Therefore, it was decided to create a separate script and register it in startup, using the standard Linux startup mechanism - /etc/rc.local. The contents of the script:

# cat /etc/network.sh
#!/bin/bash
/sbin/ip route replace default scope global nexthop via 11.11.8.1 dev eth6 weight 7 nexthop via 22.22.22.17 dev eth5 weight 3
exit 0

His way to startup:

# cat /etc/rc.local
#!/bin/sh
touch /var/lock/subsys/local
/bin/bash /etc/network.sh

For those who have not run the rc.local file, I ask you to run the following commands:

chmod u+x /etc/rc.d/rc.local
systemctl start rc-local

As a result, after a reboot, we get a working system using 2 independent uplinks, the traffic between which is balanced using weight. And now the sad news: none of this will work.

Interrupting Network Manager Utility Settings

The smaller problem of those that were in my way, but which nonetheless, pretty much spoiled my nerves. The problem with this utility is that it saves and uses settings not from the main system configuration files, but from its internal ones. Since it starts with X11, that is, in the last turn, when the network is already running, it overwrites the network settings, and it will not work to build any complex configurations on it.

To disable it, you must run in the console:

sudo /etc/init.d/NetworkManager stop
chkconfig NetworkManager off

Also, you can go the other way and use systemctl:

systemctl disable NetworkManager
systemctl stop  NetworkManager

Result.
And after performing this action and rebooting, we get a working system using 2 independent uplinks, the traffic between which is balanced using weight.