IP Multipathing in Solaris 10

Hello dear habrosociety.

Having one fine day needed to configure aggregation of network interfaces on a working server, but (much to my surprise) not finding a single article on the hub about ipmp technology, I decided to write a short essay on installing and implementing IPMP (IP Multipathing) in Solaris 10. I want to immediately warn that the article carries mainly practical workload, so I apologize in advance if somewhere I did not fully reveal the theoretical component of this topic.
And so all the work was done on the Oracle Sun T3-1 server, with the patched ILOM, Solaris 10 on board and two network cards igb0 and igb1.

root @ s1-solaris1 # cat / etc / release
Oracle Solaris 10 8/11 s10s_u10wos_17b SPARC
Copyright 1983, 2011, Oracle and / or its affiliates. All rights reserved.
Assembled 23 August 2011

root @ s1-solaris1 # uname -a
SunOS s1-solaris1 5.10 Generic_147440-09 sun4v sparc sun4v

But first, a little theory:

IPMP is a technology for combining physical channels (interfaces) to balance the load of outgoing traffic and fault tolerance of incoming traffic. In solaris, ipmp has been present since version 8 and is controlled by the in.mpathd daemon.


IPMP can work in two modes

- Probe mode
- Link-state mode

Probe mode - this is the mode of operation in which the server sends icmp requests to the gateway or host on the network (up to several of these verification hosts can be configured in solaris) and on the basis of this checks link performance.

Link-state mode - checks the availability of the communication channel by the physical state of the network interface, i.e. What is the state of the interface, up / down.

Well, that’s all, theory is enough-), if I missed something, please forgive.

And so, now the most interesting

In my case, probe mode ipmp was configured, with the addresses:
igb0 - 10.0.0.1
igb1 - 10.0.0.2
igb0: 1 (ipmp virt-address) - 10.0.0.3

We will bring the configuration files of the interfaces to the following form:

root @ s1-solaris1 # cat /etc/hostname.igb0

s1-solaris1-igb0 netmask + broadcast + deprecated -failover group IPMP-1
addif s1-solaris1 netmask + broadcast +

root @ s1-solaris1 # cat /etc/hostname.igb1

s1-solaris1-igb1 netmask + broadcast + deprecated -failover group IPMP-1

In the IPMP-1 group value, enter any group name that is convenient for you. I will not paint all the values ​​(broadcast, deprecated, failover ...), because The article focuses on the practical use of ipmp, the definition of these values ​​can be seen in man ifconfig.

Also, we need to edit the / etc / hosts file:

root @ s1-solaris1 # cat / etc / hosts
#
# Internet host table
#
:: 1 localhost
127.0.0.1 localhost
10.0.0.3 s1-solaris1 s1-solaris1.name.domain .ru loghost
10.0.0.1 s1-solaris1-igb0
10.1.0.2 s1-solaris1-igb1

And reboot the system in order to make sure that the interfaces go up with the above settings

root @ s1-solaris1 # init 6

Let's check the output of ifconfig -a, we should have a virtual interface igb0: 1

root @ s1-solaris1 # ifconfig -a
lo0: flags = 2001000849mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
igb0: flags = 9040843mtu 1500 index 2
inet 10.0.0.1 netmask ffffff00 broadcast 10.0.0.255
groupname IPMP-1
ether 0: 21: 28: d2: 1c: c6
igb0: 1: flags = 1000842mtu 1500 index 2
inet 10.0.0.3 netmask ffffff00 broadcast 10.0.0.255
igb1: flags = 9040843mtu 1500 index 3
inet 10.0.0.2 netmask ffffff00 broadcast 10.0.0.255
groupname IPMP-1
ether 0: 21: 28: d2: 1c: c7

The goal has been reached. The ibg0: 1 virtual interface works by referencing the igb0 interface. We will verify this with if_mpadm:

root @ s1-solaris1 # if_mpadm -d igb0

the -d (detach) switch puts the interface offline.

We look at the output of ifconfig, igb1 will take over the virtual interface function by creating a sub interface igb1: 1

root @ s1-solaris1 # ifconfig -a
lo0: flags = 2001000849mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
igb0: flags = 9040843mtu 1500 index 2
inet 10.0.0.1 netmask ffffff00 broadcast 10.0.0.255
groupname IPMP-1
ether 0: 21: 28: d2: 1c: c6
igb1: flags = 9040842mtu 1500 index 3
inet 10.0.0.2 netmask ffffff00 broadcast 10.0.0.255
groupname IPMP-1
ether 0: 21: 28: d2: 1c: c7
igb1: 1: flags = 1000843mtu 1500 index 3
inet 10.0.0.3 netmask ffffff00 broadcast 10.0.0.255

Let's get everything back using the -r (reattach)

root @ s1-solaris1 # if_mpadm -r igb0 key

Finally, probe based mode pings default routes by default make sure the network card is active, and if it is not there, it pings multicast address 224.0.0.1, so depending on the specific situation, you need to add this route:

root @ s1-solaris1 # route add default 10.0.0.10

That's all. Thanks for attention.