Basics of computer networks. Theme number 8. Link Aggregation Protocol: Etherchannel

Hello again! After a short break, we continue to gnaw at the granite of network science. This article will focus on the Etherchannel protocol. As part of this topic, we’ll talk about what aggregation, fault tolerance, and load balancing are. Themes are important and interesting. I wish you a pleasant reading.


So, let's start with a simple one.

Etherchannel is a technology that allows you to combine (aggregate) several physical wires (channels, ports) into a single logical interface. Typically, this is used to increase fault tolerance and increase channel throughput. Usually, for connecting critical nodes (switch-switch, switch-server, etc.). The word Etherchannel itself was introduced by Cisco, and everything related to aggregation, it includes. Other vendors call aggregation differently. Huawei calls it Link Aggregation, D-Link calls LAG, and so on. But the essence of this does not change.

Let's analyze the aggregation work in more detail.

There are 2 switches interconnected by one wire. Networks of departments and groups are connected to both switches (size is not important). The main thing is that a certain number of users are sitting at the switches. These users actively work and exchange data with each other. Accordingly, they should in no case be left without communication. There are 2 questions:

1. If the link between the switches fails, communication will be lost. The work will rise, and the administrator will run in fear to understand what’s the matter.
2. The second question is not so critical, but with a foundation for the future. The company is growing, new employees are appearing, there is more traffic, and the channels are the same. It is necessary to somehow increase the throughput.

The first thing that comes to mind is to drop a few more wires between the switches. But this campaign is fundamentally wrong. Adding redundant links will lead to the appearance of loops in the network, as mentioned in a previous article. It can be argued that we have a wonderful family of STP protocols and they will solve everything. But this is also not entirely true. I show the example of the same Packet Tracer.

As you can see, from 2 channels, only one is active. The second will wait until the active fails. That is, we will achieve some fault tolerance, but we will not solve the problem with an increase in throughput. And the second channel will just stand idle. The rule of good form is such an approach that network elements do not stand idle. The optimal solution would be to create one large logical interface from several physical interfaces and drive traffic through it. And the Etherchannel comes to the rescue. There are 3 types of aggregation in Cisco OS:

1. 1) LACP or Link Aggregation Control Protocol is an open IEEE standard.
2. 2) PAgP or Port Aggregation Protocol - Cisco's proprietary protocol.
3. Manual aggregation.

All 3 types of aggregation will be performed only in the following cases:

• Same Duplex
• Same interface speed
• Identical VLANs and Native VLANs Allowed
• The same interface mode (access, trunk)

That is, the ports must be identical to each other.

Now about their differences. The first 2 allow you to dynamically agree and in case of failure of any of the links to notify about it.

Manual aggregation is done at the administrator’s own risk. The switches will not coordinate anything and will rely on what the administrator has foreseen. Despite this, many vendors recommend using manual aggregation, because in any case, the rules described above must be followed for proper operation, and the switches do not have to generate service messages to negotiate LACP or PAgP.

I'll start with the LACP protocol. To make it work, you need to put it into active or passive mode . The difference between the modes is that the active modeimmediately enables LACP, and passive mode enables LACP if it detects a LACP message from a neighbor. Accordingly, in order for aggregation with LACP to work, it is necessary that both be in active mode , either one in active and the other in passive . I'll make a tablet.

Now let's move on to the laboratory and fix in the practical part.

There are 2 switches connected by 2 wires. As you can see, one link is active (lit green), and the second backup (lit orange) due to the triggering of the STP protocol. This is good, the protocol is working out. But we want to combine both links together. Then the STP protocol will consider that it is one wire and will stop blocking.

We go to the switches and aggregate the ports.

SW1(config)#interface fastEthernet 0/1 - заходим на интерфейс
SW1(config-if)#shutdown  - выключаем его (чтобы не было проблем с тем, что STP вдруг его заблокирует)
%LINK-5-CHANGED: Interface FastEthernet0/1, changed state to administratively down
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
SW1(config-if)#channel-group 1 mode active - создаем интерфейс port-channel 1 (это и будет виртуальный интерфейс агрегированного канала) и переводим его в режим active.
Creating a port-channel interface Port-channel 1 - появляется служебное сообщение о его создании.
SW1(config-if)#no shutdown - включаем интерфейс.
%LINK-5-CHANGED: Interface FastEthernet0/1, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up
%LINK-5-CHANGED: Interface Port-channel 1, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel 1, changed state to up
SW1(config)#interface fastEthernet 0/2 - заходим на второй интерфейс
SW1(config-if)#shutdown - выключаем.
%LINK-5-CHANGED: Interface FastEthernet0/2, changed state to administratively down
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to down
SW1(config-if)#channel-group 1 mode active - определяем в port-channel 1
SW1(config-if)#no shutdown - включаем.
%LINK-5-CHANGED: Interface FastEthernet0/2, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to up

This completes the setup on the first switch. For reliability, you can type the show etherchannel port-channel command:

SW1#show etherchannel port-channel 
                Channel-group listing:
                ----------------------
Group: 1
----------
                Port-channels in the group:
                ---------------------------
Port-channel: Po1    (Primary Aggregator)
------------
Age of the Port-channel   = 00d:00h:08m:44s
Logical slot/port   = 2/1       Number of ports = 2
GC                  = 0x00000000      HotStandBy port = null
Port state          = Port-channel 
Protocol            =   LACP
Port Security       = Disabled
Ports in the Port-channel:
Index   Load   Port     EC state        No of bits
------+------+------+------------------+-----------
  0     00     Fa0/1    Active             0
  0     00     Fa0/2    Active             0
Time since last port bundled:    00d:00h:08m:43s    Fa0/2

We see that there is such a port-channel and both interfaces are present in it.

We pass to the second device.

SW2(config)#interface range fastEthernet 0/1-2 - переходим к настройке сразу нескольких интерфейсов.
SW2(config-if-range)#shutdown - выключаем их.
%LINK-5-CHANGED: Interface FastEthernet0/1, changed state to administratively down
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
%LINK-5-CHANGED: Interface FastEthernet0/2, changed state to administratively down
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to down
SW2(config-if-range)#channel-group 1 mode passive - создаем port-channel и переводим в режим passive (включится, когда получит LACP-сообщение).
Creating a port-channel interface Port-channel 1 - интерфейс создан.
SW2(config-if-range)#no shutdown - обратно включаем.
%LINK-5-CHANGED: Interface FastEthernet0/1, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up
%LINK-5-CHANGED: Interface Port-channel 1, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel 1, changed state to up
%LINK-5-CHANGED: Interface FastEthernet0/2, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to up

After that, the channel is consistent. You can look at this with the show etherchannel summary command:

SW1#show etherchannel summary 
Flags: D - down P - in port-channel
I - stand-alone s - suspended
H - Hot-standby (LACP only)
R - Layer3 S - Layer2
U - in use f - failed to allocate aggregator
u - unsuitable for bundling
w - waiting to be aggregated
d - default port
Number of channel-groups in use: 1
Number of aggregators: 1
Group Port-channel Protocol Ports
------+-------------+-----------+----------------------------------------------
1 Po1(SU) LACP Fa0/1(P) Fa0/2(P) 

Here you can see the port-channel group, the protocol used, the interfaces and their status. In this case, the SU parameter indicates that second-level aggregation has been performed and that this interface is being used. And the parameter P indicates that the interfaces are in port-channel state.

All links are green and active. STP does not work on them.

I’ll immediately warn you that there is a glitch in the packet tracer. The bottom line is that after the configuration, the interfaces can go into stand-alone (parameter I) and do not want to leave it in any way. At the time of writing, I had this glitch and decided to recreate the lab.

Now a little deeper into the work of LACP. We turn on the simulation mode and select only the LACP filter so that the rest are not distracting.



We see that SW1 sends a LACP message to the neighbor. We look at the Ethernet field. In Source, he writes his MAC address, and in Destination, the multicast address is 0180.C200.0002. This address is listening on LACP. Well and above is the "long footcloth" from LACP. I will not dwell on each field, but only note those that, in my opinion, are important. But before that a few words. This message is used by devices for many purposes. This is synchronization, collection, aggregation, verification of activity and so on. That is, it has several functions. And before this all starts to work, they choose a virtual MAC address for themselves. This is usually the smallest available.


And they will write these addresses in the LACP fields.



On the move, this may not immediately go into your head. With pictures, I think it will lie easier. In CPT, the LACP format is a little crooked, so I will give a screen of a real dump.



The highlighted line shows for what purpose the message was sent. This is the essence of his work. Now this is a single port-channel logical interface. You can go to it and see:

SW1(config)#interface port-channel 1
SW1(config-if)#?
arp Set arp type (arpa, probe, snap) or timeout
bandwidth Set bandwidth informational parameter
cdp Global CDP configuration subcommands
delay Specify interface throughput delay
description Interface specific description
duplex Configure duplex operation.
exit Exit from interface configuration mode
hold-queue Set hold queue depth
no Negate a command or set its defaults
service-policy Configure QoS Service Policy
shutdown Shutdown the selected interface
spanning-tree Spanning Tree Subsystem
speed Configure speed operation.
storm-control storm configuration
switchport Set switching mode characteristics
tx-ring-limit Configure PA level transmit ring limit

And all actions performed on this interface will automatically lead to changes on physical ports. Here is an example:

SW1(config-if)#switchport mode trunk 
%LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel 1, changed state to down
%LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel 1, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to down
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to up

When port-channel was put into trunk mode, it automatically pulled physical interfaces along with it. We type show running-config :

SW1#show running-config 
Building configuration...
Current configuration : 1254 bytes
!
version 12.2
no service timestamps log datetime msec
no service timestamps debug datetime msec
no service password-encryption
!
hostname SW1
!
!
!
!
!
spanning-tree mode pvst
!
interface FastEthernet0/1
channel-group 1 mode active
switchport mode trunk
!
interface FastEthernet0/2
channel-group 1 mode active
switchport mode trunk
!
***************************************
interface Port-channel 1
switchport mode trunk
!

And indeed it is.

Now I’ll tell you about a technology that deserves special attention, such as Load-Balance or “balancing” in Russian. When creating an aggregated channel, one must not forget that it is the physical interfaces inside it that pass traffic. There are cases when the channel is aggregated, everything works, but there is a situation that all traffic goes through one interface, and the rest are idle. I will explain how this happens with a typical example. Let's see how Load-Balance works in the current lab.

SW1#show etherchannel load-balance 
EtherChannel Load-Balancing Operational State (src-mac):
Non-IP: Source MAC address
  IPv4: Source MAC address
  IPv6: Source MAC address

At the moment, it performs balancing based on the value of the MAC address. By default, balancing is done. That is, it will pass the first MAC address through the first link, the 2nd MAC address through the second link, the 3rd MAC address again through the first link and so will alternate. But this approach is not always true. I explain why.

There is a certain conditional network. 2 computers are connected to SW1. Next, this switch connects to SW2 aggregated channel. A router is connected to SW2. By default, Load-Balance is set to src-mac. And this is what will happen. Frames with MAC address 111 will be transmitted on the first link, and with MAC address 222 on the second link. That's right here. We pass to SW2. Only one router with a MAC address of 333 is connected to it. And all frames from the router will be sent to SW1 by the first link. Accordingly, the second will always be idle. Therefore, it is more logical here to adjust the balancing not by Source MAC address, but by Destination MAC address. Then, for example, everything that is sent to the first computer will be sent to the first link, and the second to the second link.

This is a very simple example, but it reflects the essence of this technology. It changes as follows:

SW1(config)#port-channel load-balance ?
dst-ip Dst IP Addr
dst-mac Dst Mac Addr
src-dst-ip Src XOR Dst IP Addr
src-dst-mac Src XOR Dst Mac Addr
src-ip Src IP Addr
src-mac Src Mac Addr

I think it’s clear here. I note that this is an example of balancing not only for LACP, but also for other methods.

I am ending the conversation about LACP. In the end, I can only say that this protocol is used most often, due to its openness and can be used on most vendors.

Those who thought it was not enough can finish off LACP here , here and here . And in addition, a link to this laboratory.

Now about the colleague PAgP. As mentioned above, this is a purely “tsiskovskiy” protocol. It is used less frequently (since networks built exclusively on Cisco equipment are less than heterogeneous). It works and is configured similarly to LACP, but Cisco requires it to be known and we will pass to consideration.

PAgP also has 2 modes:

1. Desirable - includes PAgP.
2. Auto - turn on if a PAgP message arrives.

We collect a similar lab.

And go to SW1:

SW1(config)#interface range fastEthernet 0/1-2 - выбираем диапазон интерфейсов.
SW1(config)#shutdown - выключаем.
%LINK-5-CHANGED: Interface FastEthernet0/1, changed state to administratively down
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
%LINK-5-CHANGED: Interface FastEthernet0/2, changed state to administratively down
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to down
SW1(config-if-range)#channel-group 1 mode desirable - создаем port-channel и переключаем его в режим desirable (то есть включить).
Creating a port-channel interface Port-channel 1

Now we go on to configure SW2 (do not forget that the interfaces on SW1 are turned off and you should return to them later):

SW2(config)#interface range fastEthernet 0/1-2 - выбираем диапазон интерфейсов.
SW2(config-if-range)#channel-group 1 mode auto - создаем port-channel и переводим в auto (включиться, если получит PAgP-сообщение).
Creating a port-channel interface Port-channel 1

Go back to SW1 and turn on the interfaces:

SW1(config)#interface range fastEthernet 0/1-2
SW1(config-if-range)#no shutdown 
%LINK-5-CHANGED: Interface FastEthernet0/1, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up
%LINK-5-CHANGED: Interface Port-channel 1, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel 1, changed state to up
%LINK-5-CHANGED: Interface FastEthernet0/2, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to up
Вроде как все поднялось. Проверим. SW1:
SW1#show etherchannel summary 
Flags: D - down P - in port-channel
I - stand-alone s - suspended
H - Hot-standby (LACP only)
R - Layer3 S - Layer2
U - in use f - failed to allocate aggregator
u - unsuitable for bundling
w - waiting to be aggregated
d - default port
Number of channel-groups in use: 1
Number of aggregators: 1
Group Port-channel Protocol Ports
------+-------------+-----------+----------------------------------------------
1 Po1(SU) PAgP Fa0/1(P) Fa0/2(P) 

SW2:

SW2#show etherchannel summary 
Flags: D - down P - in port-channel
I - stand-alone s - suspended
H - Hot-standby (LACP only)
R - Layer3 S - Layer2
U - in use f - failed to allocate aggregator
u - unsuitable for bundling
w - waiting to be aggregated
d - default port
Number of channel-groups in use: 1
Number of aggregators: 1
Group Port-channel Protocol Ports
------+-------------+-----------+----------------------------------------------
1 Po1(SU) PAgP Fa0/1(P) Fa0/2(P) 

Now go to the simulation and tune in to the PAgP filter. We see a flown message from SW2. We look.



That is, we see in Source MAC-address of SW2. Destination is a multicast address for PAgP. Higher LLC and SNAP protocols. They in this case do not interest us and we pass to PAgP. In one of the fields, he writes the virtual MAC address of SW1 (it is selected according to the same principle as in LACP), and below it writes his name and the port from which this message came out.

In principle, there are practically no differences from LACP, except for the structure itself. Who wants to read more, a link to the laboratory. And so it looks real:



The last thing left is manual aggregation. Everything is simple with aggregation:

With other settings, the channel will not work.

As mentioned above, an additional protocol of coordination and verification is not used here. Therefore, before aggregating, you need to verify the identity of the interface settings. Or reset interface settings with the command:

Switch(config)#default interface faX/X

In the created lab, everything is initially default. Therefore, I go straight to the settings.

SW1(config)#interface range fastEthernet 0/1-2
SW1(config-if-range)#shutdown 
%LINK-5-CHANGED: Interface FastEthernet0/1, changed state to administratively down
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
%LINK-5-CHANGED: Interface FastEthernet0/2, changed state to administratively down
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to down
SW1(config-if-range)#channel-group 1 mode on - создается port-channel и сразу включается.
Creating a port-channel interface Port-channel 1
SW1(config-if-range)#no shutdown 
%LINK-5-CHANGED: Interface FastEthernet0/1, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up
%LINK-5-CHANGED: Interface Port-channel 1, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel 1, changed state to up
%LINK-5-CHANGED: Interface FastEthernet0/2, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to up

And similarly on SW2:

SW2(config)#interface range fastEthernet 0/1-2
SW2(config-if-range)#channel-group 1 mode on 
Creating a port-channel interface Port-channel 1
%LINK-5-CHANGED: Interface Port-channel 1, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel 1, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to down
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to up

Setup is complete. Check with the show etherchannel summary command:

SW1#show etherchannel summary 
Flags: D - down P - in port-channel
I - stand-alone s - suspended
H - Hot-standby (LACP only)
R - Layer3 S - Layer2
U - in use f - failed to allocate aggregator
u - unsuitable for bundling
w - waiting to be aggregated
d - default port
Number of channel-groups in use: 1
Number of aggregators: 1
Group Port-channel Protocol Ports
------+-------------+-----------+----------------------------------------------
1 Po1(SU) - Fa0/1(P) Fa0/2(P) 

Ports with the necessary parameters, and in the protocol field “-”. That is, nothing is additionally used.

As you can see, all the methods for setting aggregation do not cause any difficulties and differ only in a couple of commands.

To conclude this article, I’ll give you a little Best Practice on proper aggregation. All laboratories used 2 cables for aggregation. In fact, you can use both 3 and 4 (up to 8 interfaces in one port-channel). But it is better to use 2, 4 or 8 interfaces. And all because of the hashing algorithm that Cisco came up with. The algorithm calculates hash values ​​from 0 to 7.

This table displays 8 values ​​in binary and decimal.

Based on this value, the Etherchannel port is selected and a value is assigned. After that, the port receives a certain “mask”, which displays the values ​​for which that port is responsible. Here is an example. We have 2 physical interfaces that we combine into one port-channel.

The values ​​are scattered as follows:

1) 0x0 - fa0 / 1
2) 0x1 - fa0 / 2
3) 0x2 - fa0 / 1
4) 0x3 - fa0 / 2
5) 0x4 - fa0 / 1
6) 0x5 - fa0 / 2
7) 0x6 - fa0 / 1
8) 0x7 - fa0 / 2

As a result, we get that half of the values ​​or patterns will be taken over by fa0 / 1, and the second half by fa0 / 2. That is, we get 4: 4. In this case, the balancing will work correctly (50/50).

Now let's move on and explain why it is not recommended to use, for example, 3 interfaces. We make a similar comparison:

1) 0x0 - fa0 / 1
2) 0x1 - fa0 / 2
3) 0x2 - fa0 / 3
4) 0x3 - fa0 / 1
5) 0x4 - fa0 / 2
6) 0x5 - fa0 / 3
7) 0x6 - fa0 / 1
8) 0x7 - fa0 / 2

Here we get that fa0 / 1 will take on 3 patterns, fa0 / 2 also 3 patterns, and fa0 / 3 2 patterns. Accordingly, the load will not be distributed evenly. Get 3: 3: 2. That is, the first two links will always be more loaded than the third.

I will not consider all other options, as the article will stretch to even more characters. You can only estimate that if we have 8 values ​​and 8 links, then each link will take on a pattern and get 1: 1: 1: 1: 1: 1: 1: 1. This means that all interfaces will be loaded the same way. There is still some assertion that you need to aggregate only an even number of wires in order to achieve the correct balance. But this is not entirely true. For example, if you combine 6 wires, the balancing will not be uniform. Try to count yourself. I hope the algorithm is clear.

Cisco has a good article on the website for this case with a sign. You can read on this link . If you still have questions, write!

Since I’ve gone deeper, I’ll tell you about increasing throughput. I specifically touched on this topic at the end. There are cases that urgently need to increase the bandwidth of the channel. There is no money for equipment, but there are free ports that can be collected and put into one "thick" stream. Many sources (books, forums, sites) claim that by connecting eight 100-megabit ports, we get a stream of 800 Mbps or eight gigabit ports will give 8 Gbps. Here is a piece of text from the Tsiskov article .



Theoretically, this is possible, but in practice it is almost unattainable. At least I have not met. If there are people who have been able to achieve this, I will be glad to hear. That is, to get this, you need to consider a bunch of formalities. And those that I described are only part. This does not mean that there will be no increase at all. Of course it will, but not so much as possible.

This article came to an end. In this article, we learned how to aggregate channels manually, as well as using the LACP and PAgP protocols. We learned what balancing is, how it can be controlled and how to properly assemble the Etherchannel to get the maximum load distribution. See you in the next article!