Brocade Ethernet Switches

Brocade, known to most as a manufacturer of storage switches, also manufactures Ethernet switches. A certain set of such switches (ICX and VDX) was in our hands, and I would like to do a short review and note things that seemed interesting. I will try to avoid marketing language and numbers, but for those interested, I’ll indicate links to the manufacturer’s website. Let's get started :)

ICX

switches These switches are designed for use on the enterprise network (Enterprise Campus). The ICX family consists of access lines (7150/7250/7450), aggregation (7450) and aggregation / core (7750).


The ICX 7150 is the youngest and most recent addition to the ICX family. In addition to standard models with 24 and 48 ports, this line has a compact 12-port switch (7150-C12P) and a switch with 802.3bz Multigigabit Ethernet ports (7150-48ZP). Models for 12/24/48 ports have passive cooling, while models for 24/48 ports with PoE can work without using active cooling.


ICX 7250 - standard models for 24 and 48 ports. The main difference is 8 uplink / stacking ports.


ICX 7450 - standard models for 24 and 48 ports. The switches do not have built-in uplink / stacking ports, but have 3 slots for additional interface modules (1GE, 10GE, 40GE). Also in this lineup is a switch with 802.3bz Multigigabit Ethernet ports (7450-32ZP) and an aggregation switch 7450-48F (48x1GE SFP).


ICX 7750 - Aggregation / Core. 3 models are available (48x10BASE-T, 48x1 / 10GE SFP / SFP +, 26x40GE QSFP), all support the installation of an additional module on 6x40GE QSFP ports. All 40GE ports support breakout mode (splitting 40GE port into 4 10GE ports).

The switches of the ICX line support a standard set of technologies for such positioning; no obvious gaps in the functionality have been noticed. In the 7150/7250/7450 series, some models in addition to PoE + support Power over HDBaseT (up to 90 watts per port).

Switch management and monitoring software - Brocade Network Advisor (runs on Windows or Linux, on a separate server or as a virtual machine).

What I want to note from the features:

• Stack of switches. All ICX switches support stacking through uplink / stacking ports, you can build a stack within the line (i.e. 7250 with 7450, for example, you can’t build it). The maximum number of switches in the stack is 12. For stacking, the usual 10GE or 40GE ports are used, with a maximum throughput of 480 Gbit / s. The maximum length of a connection for a stack is 10 km (long distance stacking). Both ring and linear topologies are supported.
  
  
• For those who do not consider stacking as an option for providing fault tolerance, the 7750 offers an alternative - Multi-Chassis Trunking (aka MC-LAG). Those. a pair of switches is assembled in a cluster, which devices connected to the cluster see as one logical device.
  
  
• Licensing. For 7150/7250/7450, the main licenses are Port on Demand (license for 10GE ports) and the license for L3 functionality (dynamic routing protocols, PIM, PBR, VRRP, etc.). The 7750 has only one license - L3 - which does not require activation.
  
  
• In terms of switch OS images, there are two types: switch image and router image. Each switch has two flash (primary and secondary) and by default contains both types of image.
  
  
• Combinations of OC image and L3 licenses give us the following options for feature set: Layer 2 (switch image without license), basic Layer 3 (router image without license), full Layer 3 (router image with license). If you need, for example, static routing, then basic Layer 3 is enough.
  
  
• Supported features are conveniently listed in the Features and Standards Support Matrix . I would even say that this is a model document.
  
  
• Functionality of Time Domain Reflectometer, which Brocade calls Virtual Cable Test.
  
  
• The 7150 is a new line, and some things on the datasheet are labeled Feature to be supported in a future software release. In July, a software release is planned, in which, among other things, there will be stacking and L3-functionality.
  

Now let's see what can be assembled on the basis of ICX. In addition to the usual two / three-tier architecture (on standalone switches + Multi-Chassis Trunking) for Enterprise Campus, two solutions are offered.

The first, using stacking and, optionally, taking into account the long-distance stacking, may look like this:


The scheme, I think, does not require comments.

The second solution is called Campus Fabric. In fact, this is an implementation of the 802.1BR standard (Bridge Port Extension) on ICX switches, where the 7750 acts as a single point of control (Control Bridge - CB), and the rest of the switches as external line cards (Port Extenders - PE).


What are the benefits of such an architecture?

• This is a distributed chassis in which port capacity can easily be increased.
  
• All links are active.
  
• Centralized management plane and control plane.
• L3-license, if required, is needed only on Control Bridge, which allows saving on L3-licenses.

But there are controversial points:

• The centralized control plane can be considered a drawback, as the whole factory is one failure domain.
  
  
• All traffic goes through the core of the factory (Control Bridge). The exceptions are multicast and broadcast replication, which are done locally on Port Extender.
  
  
• Port Extenders do not support LAGs on access ports.
  

On scaling / redundancy, I note the following things:

• The Control Bridge based on the 7750 is a stack of 4 (maximum) switches; the recommended stack topology is a ring. We use at least 2 switches, as we definitely need a Control Bridge reservation.
  
  
• You can use 7450 and 7250 as Port Extenders. 7150 support will come later.
  
  
• PoD licenses are still needed.
  
  
• The maximum number of Port Extenders is 36. The switch stack is counted as separate switches.
  
  
• Ability to connect Port Extenders in a chain - up to 6 devices.
  

Whether it is worth building the entire network on such a solution is a moot point, but as an access solution, the option has the right to life.

VDX Switches

Let me remind you what the folded Clos network looks like:


Leaf switches provide server connectivity; Spine switches provide redundant, high-speed connectivity for leaf switches. The picture shows a 3-stage (leaf-spine-leaf) version of such a network, but Brocade also considers the 5-stage (leaf-spine-superspine-spine-leaf) version in the documentation.

In the VDX family, I would like to dwell on two lines: 6740 and 6940.


VDX 6740 - leaf switches with 48x10GE (10GBASE-T or SFP +) + 4x40GE QSFP ports.


VDX 6940 - leaf / spine switches. 6940-36Q - 36x40GE QSFP ports. 6940-144S - 96x10GE SFP + and 12x40GE QSFP ports (while 3 40GE ports can be combined into one 100GE port and use QSFP28 transceivers).

These switches are positioned for use in the data center, and have the corresponding characteristics:

• All 40GE ports support breakout mode (splitting 40GE port into 4 10GE ports).
  
• Both front-to-back and back-to-front cooling are supported.
  
• Two power supplies.
  
• Support for FCoE and DCB.
  
• Support for automation tools, the ability to integrate with OpenStack and VMware products.
  
• VXLAN support.
  

I will not deny that the data center network at 10GE / 40GE speeds is a bit outdated, but Brocade's 25GE / 100GE switches are in another family (SLX), and they didn’t get to us.

Among the features of VDX switches, I note the following:

• Licensing. There are two types of licenses - Port on Demand (for 10GE and 40GE ports) and a license for FCoE. PoD licenses automatically bind to active ports. Those. you took the switch out of the box, turned it on, and you do not need to look in the documentation which group of ports is activated by the license - you can just install transceivers. To return a license to the free license pool, you will need to run several commands in the CLI.
  
  
• The switches support VCS Fabric - a TRILL-based Ethernet factory (up to 48 switches). TRILL-based, because from TRILL only data-plane is used, and control-plane uses its own. The thing is unpretentious to the topology, convenient in both configuration and maintenance.
  
  Unusually, VDX switches are always in Logical Chassis mode, i.e. virtually every switch is a VCS factory from one switch. And, because within the VCS factory, the configuration is saved automatically, on VDX switches you will not find an analogue of copy run startup.
  
  VCS is the basic functionality, but using a VCS factory is optional and can be disabled.
  
  
• Two flash, but, unlike ICX, they are here for fault tolerance and the convenience of software updates. Separation of OS images by options is not here.
  
  
• Supported features are still listed in the Features and Standards Support Matrix . The approach to writing documentation in this regard is no different from ICX.
  
  
• Because Since it is Brocade, it is expected that some SAN switch technologies can be found on Ethernet switches. For example, these are MAPS (advanced monitoring) and part of the technology of the VCS factory.
  
  
• All branded DACs (copper direct attach) on 10GE and 40GE are active.
  

Architectures

In addition to designs based on the VCS factory, Brocade, of course, offers designs based on the IP factory. This is a Clos network (3-stage or 5-stage), where the L2 / L3 border is on leaf switches, and the connectivity of leaf-spine and ECMP provides a dynamic routing protocol (usually BGP).

IP factory with pervasive eBGP (use eBGP for all types of connectivity).


IP factory with iBGP (eBGP is used only for the super-spine <-> spine and super-spine <-> edge leaf connectivity, iBGP is used within the DC POD for spine <-> leaf).


In both cases, to connect the servers to two leaf switches (multihoming), a VCS factory of two switches (vLAG pair) is used.

And do not forget about VXLAN. Brocade supports virtualization with VXLAN as a data-plane, while EVPN can be used as a control plane. So, we take the designs of the IP factory, add the EVPN VXLAN and get the options:

c eBGP

and with iBGP

For multihoming, the same mechanism is used as in the IP factory.

Brocade does not insist on using iBGP or eBGP, both options are equivalent. Brocade supports both Integrated Routing and Bridging options for VXLAN EVPN: asymmetric and symmetric. Of those EVPN VXLAN implementations that I have seen, VDX is one of the most fully functional.

Of course, besides VDX, there are many other Brocade data center products, but they are not covered in this article.

In conclusion, I want to add that there were no significant claims to the quality of equipment and software. The CLI, like many other manufacturers, is very similar to Cisco, in some places it is even more convenient (and we know that part of Cisco's CLI is far from ideal). The documentation for each product is collected in one place (Document Library), and this documentation is written very correctly. Validated Design is generally a pleasure to read. Well, let me remind you again about the Features and Standards Support Matrix documents - it would be nice for some to adopt this approach. Developed and active community, which not every manufacturer has. Complexity.

And a little about the current status of Brocade.

Broadcom buys the entire Brocade business, with Fiber Channel solutions (i.e. everything for SAN) remaining in the Broadcom portfolio and the remaining Brocade products will be sold to other companies. The Broadcom-Brocade deal will end in fiscal 3 (ending July 30, 2017).

What will happen to the rest of the products:

• ARRIS buys Ruckus Wireless and its line of ICX switches (which are likely to be sold as Ruckus ICX). At the same time, it is stated that Ruckus Wireless will retain its structure and sales channel, i.e. there will be something like a Meraki-Cisco deal. The Broadcom-ARRIS deal is due to close by the end of August 2017.
  
  
• Extreme Networks buys data center solutions, namely VDX / MLX / SLX, Brocade Workflow Composer, and Network Visibility & Analytics. The Broadcom-Extreme deal is due to close by the end of September 2017.
  
  
• Pulse Secure buys the vADC family of products (virtual Traffic Manager and related products).
  
  
• AT&T buys Vyatta platform (Vyatta Network OS and related products).