March 12, 2015 at 16:05Virtual Link Trunking (VLT) Technology for Dell Network Factories
Today we want to tell you about Dell Virtual Link Trunking (VLT) technology, which allows you to combine two switches into a fault-tolerant configuration for connecting to other switches or servers using Link Aggregation Group (LAG) and 802.3ad LACP protocol.
Typically, LAGs from servers and access level switches can be connected to only one aggregation level switch. To ensure fault tolerance, you can use the stacking of several aggregation switches, which are a single logical device. But this solution has its drawbacks, since the stack has a single control plane - it becomes the weak link in the system. When upgrading the operating system, a reboot of all switches on the stack is required, which causes service interruptions. Now let's compare this scheme with the one that uses VLT technology: this configuration already has two control planes, so the network continues to work even when updating the operating system on one of the switches. Moreover, all physical links between the switches assembled in VLT,
All major manufacturers of network equipment have such technologies, each of which has its own proprietary implementation. For example, Cisco supports Virtual Port Channel (vPC), Brocade uses Multi-Chassis Trunking (MCT), and Juniper uses Virtual Chassis technology. Technologies with open standards are used that provide similar functionality: Transparent Interconnection of Lots of Links (TRILL) and Shortest Path Bridging (SPB).
Key Benefits of VLT
VLT Technology Description
VLT is a configuration of two switches that on Layer 2 are perceived by other network devices as a whole. Physically, these are two devices interconnected according to a special scheme, but for servers, switches and other network devices, they look like a single switch. However, each of them has its own control plane, is configured independently of the other and is controlled by its operating system. Despite the fact that one network device is connected to two switches using LAGs, VLT prevents loops from forming - thus, the Spanning Tree (STP) protocol fights. This connection provides fault tolerance and even load distribution between the switches. They can connect switches or switch stacks, servers or other devices,
Let's look at the example of a small scheme in the VLT terminology, which we will need in the future.
Periodic hello service messages are sent via the VLT backup connection, and VLT control messages are sent via VLTi interfaces. This is necessary to synchronize the L2 / L3 control plane between two VLT switches. MAC, ARP and IGMP tables are synchronized between them to provide joint switching and protection against failures if one of the switches or physical connection channels fails. VLT technology also provides forwarding of traffic along the shortest path, bypassing the VLTi channel, but which can be used in case of failure.
A backup channel for exchanging heartbeat packets is usually provided through the management ports of switches connected to a dedicated management network. Also, the backup channel can connect any ports of the VLT switches, but it is recommended to use the control port for heartbeat packets: it can work when the VLTi channel fails.
In VLT, one of the switches is selected as the main one, and it takes control of the LACP and STP protocols and sends service packets to the second VLT peer.
All VLANs configured for VLT connections are automatically tagged for the VLTi channel.
Each device connected to a VLT domain receives broadcast packets through VLT switches only once. Broadcast packets are blocked when sent to the port of the connected device, if they are received by the VLT peer through the VLTi link. But this only happens when the ports for the connected device are active on both peers. If the port at one of the peers does not work, then information about its status is immediately transmitted to the second peer. That, in turn, immediately removes the lock, and the device continues to receive packets.
Dell Switch Table with VLT Support The
table was updated in January 2016 due to the release of new products supporting VLT.
Default Gateway with VLT
To protect the default gateway, you can configure VRRP within the VLT domain. In this case, both switches will route traffic sent to the default gateway, including a switch that is not a VRRP master at the moment.
An alternative to VRRP is Peer-routing in the VLT domain. Let's see how it works.
Packets from the server intended for the default IP gateway 10.10.10.2 (as shown in the diagram on the left), due to the hashing algorithm for the LAG, can be transferred to the Peer 1 switch. Without Peer-routing functionality, the switch will have to forward these packets to the Peer 2 switch .
Peer-routing functionality allows the switch to forward packets destined for the default gateway located on the neighboring VLT peer (as shown in the diagram on the right). Thanks to this, optimal routing is achieved, packets are delivered faster, the VLTi link does not load, and there is no need to configure the VRRP protocol. Peer-routing functionality protects the default gateway for a larger number of VLAN interfaces compared to VRRP, which has a limit of 255 VLANs.
Routing with VLT
In data center networks, it is important to provide a “stretched” Layer 2 domain for cluster applications and free migration of virtual machines. VLT technology provides such “stretched” VLAN routing capabilities. This is achieved using the Peer-routing functionality, which, in addition to the default gateway, supports OSPF, IS-IS, and BGP routing protocols. One of these protocols can be configured inside a “stretched” VLAN on VLT switches. And then the neighbors will be able to exchange their routes using the routing protocol. In this case, only broadcast mode is supported for OSPF or IS-IS routing protocols, and not point-to-point.
In data centers with a high density of virtual machines for a given VLAN “stretched” across several racks, you can configure several default gateways and distribute them among the switches in VLT domains. This in turn will allow you to distribute ARP tables to VLT domains for greater scaling.
VLT supports IPv4 and IPv6 routing protocols, but Peer-routing to protect the default gateway works only for IPv4. To protect IPv6 default-gateway, it is recommended that you configure VRRP.
The diagram shows an example of using OSPF on a network with multiple VLT domains:
OSPF is configured on VLAN 10 interfaces in broadcast mode. For optimal route distribution, the core-level VLT switch, which acts as the main one (IP 10.10.10.1), is recommended to be assigned as OSPF DR and its neighboring VLT peer (IP 10.10.10.2) as BDR. At the main VLT peer for VLAN 10, you must set the highest OSPF priority so that this switch is selected as DR. At a neighboring VLT peer, a lower priority is set for selecting it as a BDR. After exchanging routes, all VLT switches have information through which next-hop addresses routes are available. At the same time, it remains possible to “stretch” VLANs across all network switches, as shown in the example of VLAN 20 used for virtual machines.
Proxy Gateway for neighboring VLT domain
Between two VLT domains, you can configure the Proxy Gateway functionality. It allows you to locally route traffic destined for the default gateway from a neighboring VLT domain. This functionality is well suited for the case when two VLT domains are located in different geographically remote data centers. Proxy Gateway allows you to optimize the flow of packets between the data centers, reduce the load on the channels between the data centers and provide protection in case of failure of one of the VLT peers, which act as the default gateway.
An example of the application of this feature is shown in the diagram. Remote VLT domains are interconnected by four physical links using dark fiber or DWDM. A virtual machine with a default gateway C configured will migrate to a geographically remote data center without stopping its operation. After moving to Data Center 2, the virtual machine will continue to work without reconfiguration. Packets sent by the virtual machine to gateway C will be routed locally by switches C1 and D1. If static routes are used on switches C and D, you must configure them on switches C1 and D1 for symmetric routing.
VLT setup example
Before starting the configuration, make sure that both switches in the VLT domain use the same version of Dell Networking OS and that the RSTP protocol is configured. It will provide protection against loops if configuration errors are made.
In the following example, VLT peer 1 uses the hostname S4810-1, VLT peer 2 uses the hostname S4810-2, and the Top-of-Rack switch is the hostname S60-1.
We configure VLTi between VLT peers. It is recommended to use a static LAG for this:
We set the same VLT domain number on VLT peers:
Add the VLTi link inside the VLT domain configuration:
Configure a backup connection between VLT peers. At VLT peer 1, we use the IP address specified on the VLT control interface of peer 2 for backup communication:
Assign VLT to peer 1 the main role by using a lower priority than the neighbor:
Set the default MAC address for interaction in the VLT domain:
Assign each unit a unique unit ID (0 and 1):
When you create a VLT domain on switches, Dell Networking OS can itself assign a MAC address for communication and give the switches a unique unit ID. Using the system-mac mac-address and unit-id commands minimizes peer synchronization time after rebooting one of the switches.
Configure VLT domain at peer 2:
We configure VLT connection of peers 1 and 2 with the Top-of-Rack switch. To do this, configure LACP LAG on peer ports 1 and 2 connected to the Top-of-Rack switch. Configure the same VLT identifier for this LACP LAG on VLT peers 1 and 2:
On the Top-of-Rack switch, configure the LACP LAG for the physical ports connected to peers 1 and 2:
Check if the VLT, VLTi connection, the backup channel, and the state of the neighboring VLT peer are working:
Check if the VLT LAG is active on both VLT peers:
Design Example
Consider an example of a data center network infrastructure design using VLT technology. Two VLT domains are connected by a single LAG consisting of four 40GE ports. VLT allows you to distribute traffic across all channels between the level of blade switches and network core switches.
This network infrastructure also provides LAN and SAN convergence by supporting iSCSI DCB technology on all components of the solution: Dell EqualLogic storage, Dell switches and converged adapters for Dell blade servers.
You can learn more about this design example and its configuration described in the Datacenter Reference Architecture - Deploying Active fabric for Datacenter document .
Typically, LAGs from servers and access level switches can be connected to only one aggregation level switch. To ensure fault tolerance, you can use the stacking of several aggregation switches, which are a single logical device. But this solution has its drawbacks, since the stack has a single control plane - it becomes the weak link in the system. When upgrading the operating system, a reboot of all switches on the stack is required, which causes service interruptions. Now let's compare this scheme with the one that uses VLT technology: this configuration already has two control planes, so the network continues to work even when updating the operating system on one of the switches. Moreover, all physical links between the switches assembled in VLT,
All major manufacturers of network equipment have such technologies, each of which has its own proprietary implementation. For example, Cisco supports Virtual Port Channel (vPC), Brocade uses Multi-Chassis Trunking (MCT), and Juniper uses Virtual Chassis technology. Technologies with open standards are used that provide similar functionality: Transparent Interconnection of Lots of Links (TRILL) and Shortest Path Bridging (SPB).
Key Benefits of VLT
- no loops form in the Layer 2 domain;
- availability and fault tolerance;
- fast convergence in case of network failures;
- all physical channels in the network are active and used more efficiently;
- Active load balancing on the default gateway using the VRRP protocol or Peer-routing functionality;
- fast convergence when rebooting one of the switches;
- virtual machine migration within the Layer 2 domain.
VLT Technology Description
VLT is a configuration of two switches that on Layer 2 are perceived by other network devices as a whole. Physically, these are two devices interconnected according to a special scheme, but for servers, switches and other network devices, they look like a single switch. However, each of them has its own control plane, is configured independently of the other and is controlled by its operating system. Despite the fact that one network device is connected to two switches using LAGs, VLT prevents loops from forming - thus, the Spanning Tree (STP) protocol fights. This connection provides fault tolerance and even load distribution between the switches. They can connect switches or switch stacks, servers or other devices,
Let's look at the example of a small scheme in the VLT terminology, which we will need in the future.
- Virtual link trunk (VLT) - a virtual network channel between connected devices and two VLT switches.
- VLT backup link - a backup connection that verifies the operation of VLT switches. Through it, periodic keep alive packets are transmitted between the VLT switches.
- VLT interconnect (VLTi) - a connection for transferring service information between VLT switches. These ports should be 10 Gb / s or 40 Gb / s.
- VLT domain - a domain of two VLT switches, VLTi ports and all VLT connections of devices connected to them. This domain is used for a single configuration of global VLT parameters.
- VLT peer is one of two VLT switches interconnected via VLT interconnect (VLTi).
- Non-VLT port - any port on the VLT switch to which a device is connected that is not connected to another VLT switch.
Periodic hello service messages are sent via the VLT backup connection, and VLT control messages are sent via VLTi interfaces. This is necessary to synchronize the L2 / L3 control plane between two VLT switches. MAC, ARP and IGMP tables are synchronized between them to provide joint switching and protection against failures if one of the switches or physical connection channels fails. VLT technology also provides forwarding of traffic along the shortest path, bypassing the VLTi channel, but which can be used in case of failure.
A backup channel for exchanging heartbeat packets is usually provided through the management ports of switches connected to a dedicated management network. Also, the backup channel can connect any ports of the VLT switches, but it is recommended to use the control port for heartbeat packets: it can work when the VLTi channel fails.
In VLT, one of the switches is selected as the main one, and it takes control of the LACP and STP protocols and sends service packets to the second VLT peer.
All VLANs configured for VLT connections are automatically tagged for the VLTi channel.
Each device connected to a VLT domain receives broadcast packets through VLT switches only once. Broadcast packets are blocked when sent to the port of the connected device, if they are received by the VLT peer through the VLTi link. But this only happens when the ports for the connected device are active on both peers. If the port at one of the peers does not work, then information about its status is immediately transmitted to the second peer. That, in turn, immediately removes the lock, and the device continues to receive packets.
Dell Switch Table with VLT Support The
table was updated in January 2016 due to the release of new products supporting VLT.
Default Gateway with VLT
To protect the default gateway, you can configure VRRP within the VLT domain. In this case, both switches will route traffic sent to the default gateway, including a switch that is not a VRRP master at the moment.
An alternative to VRRP is Peer-routing in the VLT domain. Let's see how it works.
Packets from the server intended for the default IP gateway 10.10.10.2 (as shown in the diagram on the left), due to the hashing algorithm for the LAG, can be transferred to the Peer 1 switch. Without Peer-routing functionality, the switch will have to forward these packets to the Peer 2 switch .
Peer-routing functionality allows the switch to forward packets destined for the default gateway located on the neighboring VLT peer (as shown in the diagram on the right). Thanks to this, optimal routing is achieved, packets are delivered faster, the VLTi link does not load, and there is no need to configure the VRRP protocol. Peer-routing functionality protects the default gateway for a larger number of VLAN interfaces compared to VRRP, which has a limit of 255 VLANs.
Routing with VLT
In data center networks, it is important to provide a “stretched” Layer 2 domain for cluster applications and free migration of virtual machines. VLT technology provides such “stretched” VLAN routing capabilities. This is achieved using the Peer-routing functionality, which, in addition to the default gateway, supports OSPF, IS-IS, and BGP routing protocols. One of these protocols can be configured inside a “stretched” VLAN on VLT switches. And then the neighbors will be able to exchange their routes using the routing protocol. In this case, only broadcast mode is supported for OSPF or IS-IS routing protocols, and not point-to-point.
In data centers with a high density of virtual machines for a given VLAN “stretched” across several racks, you can configure several default gateways and distribute them among the switches in VLT domains. This in turn will allow you to distribute ARP tables to VLT domains for greater scaling.
VLT supports IPv4 and IPv6 routing protocols, but Peer-routing to protect the default gateway works only for IPv4. To protect IPv6 default-gateway, it is recommended that you configure VRRP.
The diagram shows an example of using OSPF on a network with multiple VLT domains:
OSPF is configured on VLAN 10 interfaces in broadcast mode. For optimal route distribution, the core-level VLT switch, which acts as the main one (IP 10.10.10.1), is recommended to be assigned as OSPF DR and its neighboring VLT peer (IP 10.10.10.2) as BDR. At the main VLT peer for VLAN 10, you must set the highest OSPF priority so that this switch is selected as DR. At a neighboring VLT peer, a lower priority is set for selecting it as a BDR. After exchanging routes, all VLT switches have information through which next-hop addresses routes are available. At the same time, it remains possible to “stretch” VLANs across all network switches, as shown in the example of VLAN 20 used for virtual machines.
Proxy Gateway for neighboring VLT domain
Between two VLT domains, you can configure the Proxy Gateway functionality. It allows you to locally route traffic destined for the default gateway from a neighboring VLT domain. This functionality is well suited for the case when two VLT domains are located in different geographically remote data centers. Proxy Gateway allows you to optimize the flow of packets between the data centers, reduce the load on the channels between the data centers and provide protection in case of failure of one of the VLT peers, which act as the default gateway.
An example of the application of this feature is shown in the diagram. Remote VLT domains are interconnected by four physical links using dark fiber or DWDM. A virtual machine with a default gateway C configured will migrate to a geographically remote data center without stopping its operation. After moving to Data Center 2, the virtual machine will continue to work without reconfiguration. Packets sent by the virtual machine to gateway C will be routed locally by switches C1 and D1. If static routes are used on switches C and D, you must configure them on switches C1 and D1 for symmetric routing.
VLT setup example
Before starting the configuration, make sure that both switches in the VLT domain use the same version of Dell Networking OS and that the RSTP protocol is configured. It will provide protection against loops if configuration errors are made.
In the following example, VLT peer 1 uses the hostname S4810-1, VLT peer 2 uses the hostname S4810-2, and the Top-of-Rack switch is the hostname S60-1.
We configure VLTi between VLT peers. It is recommended to use a static LAG for this:
s4810-1 (conf) #interface port-channel 100 s4810-1 (conf-if-po-1) # channel-member TenGigabitEthernet 0 / 4-5 s4810-2 (conf) #interface port-channel 100 s4810-2 (conf-if-po-1) # channel-member TenGigabitEthernet 0 / 4-5
We set the same VLT domain number on VLT peers:
s4810-1 (conf) #vlt domain 1
Add the VLTi link inside the VLT domain configuration:
s4810-1 (conf-vlt-domain) # peer-link port-channel 100
Configure a backup connection between VLT peers. At VLT peer 1, we use the IP address specified on the VLT control interface of peer 2 for backup communication:
s4810-2 # show interfaces managementethernet 0/0 Internet address is 10.11.206.58/24 s4810-1 (conf-vlt-domain) # back-up destination 11/10/206.58
Assign VLT to peer 1 the main role by using a lower priority than the neighbor:
s4810-1 (conf-vlt-domain) # primary-priority 100
Set the default MAC address for interaction in the VLT domain:
s4810-1 (conf-vlt-domain) # system-mac mac-address 00: 11: 22: 33: 44: 55
Assign each unit a unique unit ID (0 and 1):
s4810-1 (conf-vlt-domain) # unit-id 0
When you create a VLT domain on switches, Dell Networking OS can itself assign a MAC address for communication and give the switches a unique unit ID. Using the system-mac mac-address and unit-id commands minimizes peer synchronization time after rebooting one of the switches.
Configure VLT domain at peer 2:
s4810-2 (conf) #vlt domain 1 s4810-2 (conf-vlt-domain) # peer-link port-channel 100 s4810-2 (conf-vlt-domain) # back-up destination 11/10/206.43 s4810-2 (conf-vlt-domain) # system-mac mac-address 00: 11: 22: 33: 44: 55 s4810-2 (conf-vlt-domain) # unit-id 1
We configure VLT connection of peers 1 and 2 with the Top-of-Rack switch. To do this, configure LACP LAG on peer ports 1 and 2 connected to the Top-of-Rack switch. Configure the same VLT identifier for this LACP LAG on VLT peers 1 and 2:
s4810-1 (conf) #interface TenGigabitEthernet 0/40 s4810-1 (conf-if-te-0/40) # port-channel-protocol lacp s4810-1 (conf-if-te-0/40-lacp) # port-channel 1 mode active s4810-1 (conf) #interface port-channel 1 s4810-1 (conf-if-po-1) #switchport s4810-1 (conf-if-po-1) # vlt-peer-lag port-channel 1
s4810-2 (conf) #interface TenGigabitEthernet 0/40 s4810-2 (conf-if-te-0/40) # port-channel-protocol lacp s4810-2 (conf-if-te-0/40-lacp) # port-channel 1 mode active s4810-2 (conf) #interface port-channel 1 s4810-2 (conf-if-po-1) #switchport s4810-2 (conf-if-po-1) # vlt-peer-lag port-channel 1
On the Top-of-Rack switch, configure the LACP LAG for the physical ports connected to peers 1 and 2:
S60-1 (conf) #interface range TenGigabitEthernet 0/48, TenGigabitEthernet 0/50 S60-1 (conf-if-range-te-0/48, te-0/50) # port-channel-protocol lacp S60-1 (conf-if-range-te-0/48, te-0/50-lacp) # port-channel 1 mode active
Check if the VLT, VLTi connection, the backup channel, and the state of the neighboring VLT peer are working:
s4810-1 # show vlt brief VLT Domain Brief ------------------ Domain ID: 1 Role: Primary Role Priority: 100 ICL Link Status: Up HeartBeat Status: Up VLT Peer Status: Up Local Unit Id: 0 Version: 6 (1) Local System MAC address: 00: 01: e8: 8b: 2d: 41 Remote System MAC address: 00: 01: e8: 8b: 28: a9 Configured System MAC address: 00: 11: 22: 33: 44: 55 Remote system version: 6 (1) Delay-Restore timer: 90 seconds
Check if the VLT LAG is active on both VLT peers:
s4810-1 # show vlt detail Local LAG Id Peer LAG Id Local Status Peer Status Active VLANs ------------ ----------- ------------ ----------- ---- --------- 1 1 UP UP 1
s4810-1 # sho interfaces port-channel 1 brief
Codes: L - LACP Port-channel
LAG Mode Status Uptime Ports
L 1 L2L3 up 00:34:00 Te 0/40 (Up)Design Example
Consider an example of a data center network infrastructure design using VLT technology. Two VLT domains are connected by a single LAG consisting of four 40GE ports. VLT allows you to distribute traffic across all channels between the level of blade switches and network core switches.
This network infrastructure also provides LAN and SAN convergence by supporting iSCSI DCB technology on all components of the solution: Dell EqualLogic storage, Dell switches and converged adapters for Dell blade servers.
You can learn more about this design example and its configuration described in the Datacenter Reference Architecture - Deploying Active fabric for Datacenter document .