iSCSI 2.0 with FAS2xxx or the small data center scaling path
Saving investments is in demand by any company. It is important to have a data center infrastructure concept that would allow easy scalability if necessary, while maximizing the disposal of existing equipment for new business needs. The transition from iSCSI to iSCSI 2.0 can become the basis for such a concept. iSCSI over DCB is often referred to as iSCSI 2.0 due to the optional DCB extensions for Ethernet.
Continuing the article " FC & Ethernet ".
DCB, consists of
Switches supporting DCB often also support Shortest Path Bridging IEEE 802.1AQ and / or IETF TRILL allowing you to choose the shortest path for Ethernet traffic, which positively affects the operation of iSCSI .
iSCSI managed to evolve by learning to work with Thin Provisioning (SCSI SBC-3) and Space Reclamation (T10 UNMAP), it has the ability to balance load and provide path resiliency (using MPIO and / or MCS ).
It is important to note that all NetApp FAS storages support iSCSI protocol on all of their Ethernet data ports.Starting with Data ONTAP 8.1.3, SCSI SBC-3 and UNMAP are supported ; starting with DATA Ontap 8.2.1, FAS storage systems support DCB .
Thus, iSCSI can be an ideal candidate for the growth of a small data center if it provides:
To do this, I propose to consider the advantages and disadvantages of several designs, observing a single concept - connecting NetApp FAS storage systems via iSCSI . These designs should be relatively simple to convert a small data center to a large one. Schematic designation of the proposed transition paths between configurations with the growth of the data center, a la Gartner square:
Since iSCSI lives on top of Ethernet, you need to configure Flow Control (not to confuse the “normal” FlowControl with PFC IEEE 802.1Qbb for DCB Losless Ethernet) and configure Jumbo Frames on host network adapters, on storage and on switches (if any). Using MPIO / MCS within a single link can increase its total throughput compared to using a single connection in a link. It is recommended to allocate a separate VLAN for iSCSI traffic if the switch is used for a mixed type of traffic.
The design of such a "small data center" consists of one or two servers directly connected to the storage system, each with at least two links for fault tolerance: one to one storage controller , the other to the other. At this point, it is logical to use the free iSCSI Initiator Software. In the case of increasing hosts, you must either add ports to the store or install switches. Depending on the storage model, there may be a different number of Ethernet ports. It is important to note that all Ethernet ports on NetApp FAS storages (there are no "dedicated" iSCSI ports) can be used for these types of traffic: iSCSI , NFS , CIFS ( SMB ), including at the same time.
In such a scheme, it is necessary to have at least two links from each node, each link connected through different switches. This will ensure complete fault tolerance when the controller, switch or path exits. At this point, you can still use the Software iSCSI Initiator or a combination with dedicated iSCSI HBA (which would load the load from the host CPU ).
Like Switched iSCSI , iSCSI 2.0 uses switches, but they must support DCB . This design will also require network adapters that support DCB on both the host and storage. Therefore, DCB must be supported throughout the iSCSI traffic. Also known as Lossless Ethernet , DCB provides guaranteed frame delivery while improving reliability, performance and response speed to FC . It is important to note that DCB supports all converged Ethernet ports on NetApp FAS storages.(there are no "dedicated" iSCSI or FCoE ports) and can be used for all types of traffic: iSCSI (2.0), FCoE , NFS , CIFS ( SMB ), including at the same time. The host supports QLogic 8300 CNA (HBA) adapters for working with DCB , this adapter allows offloading the load from the host CPU while providing low storage response speed. Some additional features of switches can improve the already low level of response for iSCSI , for example, CLEAR-Flow .
The benefits of iSCSI over DCB include:
The disadvantages of iSCSI over DCB:
There are several ways to grow NetApp FAS systems:
Thus, the iSCSI protocol with NetApp FAS storage can be the basis for starting growth from a "small data center" to a "large" one, step-by-step and simply updating the network design, observing a single concept of addressing and multipassing. During the upgrade, already purchased equipment is utilized as much as possible. Each upgrade step increases the scalability of the data center infrastructure , its speed and reduces the response for applications, while performing costs as needed. Thanks to the versatility of FAS storage systems , support for advanced data center development trends and the ability to mutually replace protocols (or use them all at the same time including on the same ports), easy scalability, it is a good investment investment for both small and “large data centers”.
Learn more about NetApp network topology and zoning guidelines in pictures .
It is important to follow best practices when configuring your infrastructure and checking the compatibility matrix for maximum performance and fault tolerance:
TR-3441 Windows Multipathing Options with Data ONTAP: Fiber Channel and iSCSI
WP-7071: "How Do I Get to Ethernet from Here?"
TR-3519: “The Road to 10-Gigabit Ethernet”
WP-7046: “Ethernet Storage”
TR-3441: “iSCSI Multipathing Possibilities on Windows with Data ONTAP”
TR-3163: “Data Protection and Recovery for Network-Attached Storage over IP / Ethernet Networks ”
WP-7052:“ Converged Enhanced Ethernet — Good for iSCSI SANs ”
TR-3519: “The Road to 10-Gigabit Ethernet”
iSCSI 10Gig Performance Tuning With Windows Server
iSCSI Configuration and Provisioning for Windows
TR-3802 Ethernet Storage Best Practices
TR-4182 Ethernet Storage Design Considerations and Best Practices
Best Practices for Network Configuration with NetApp Storage Systems
NetApp Verifying that DCB is configured correctly NetApp SAN Design and SAN Fundamentals on Data ONTAP
courses at NetApp Academy are available. Sources: sniaesfblog.org/?p=210&cpage=1#comment-82839 blogs.cisco.com/datacenter/the-napkin-dialogues-lossless-iscsi
www.storage-switzerland.com/Articles/Entries/2012/1/17_iSCSI_2.0_-_Using_Data_Center_Bridging_To_Enhance_iSCSI.html
www.snia.org/sites/default/education/tutorials/2010/spring/networking/GaryGumanow-JasonBlosil_iSCSI_Lossless_Ethernet_DCB.pdf
www. slideshare.net/extreme-muk/i-scsi-extremeintelnetappclearflow
I ask you to send comments on errors and suggestions for corrections in the text to the LAN .
Continuing the article " FC & Ethernet ".
DCB, consists of
- PFC (802.1Qbb) - Enables Ethernet lossless frames (Lossless Ethernet)
- ETS (802.1Qaz) - Assigns bandwidth to frames, allows low-priority traffic to use bandwidth if it is not affected
- CN (802.1Qau) - Pauses the source
- DCBX - Defines a DCB domain
Switches supporting DCB often also support Shortest Path Bridging IEEE 802.1AQ and / or IETF TRILL allowing you to choose the shortest path for Ethernet traffic, which positively affects the operation of iSCSI .
iSCSI managed to evolve by learning to work with Thin Provisioning (SCSI SBC-3) and Space Reclamation (T10 UNMAP), it has the ability to balance load and provide path resiliency (using MPIO and / or MCS ).
It is important to note that all NetApp FAS storages support iSCSI protocol on all of their Ethernet data ports.Starting with Data ONTAP 8.1.3, SCSI SBC-3 and UNMAP are supported ; starting with DATA Ontap 8.2.1, FAS storage systems support DCB .
Thus, iSCSI can be an ideal candidate for the growth of a small data center if it provides:
- High performance (can be achieved with no more tuning, as well as thanks to DCB ).
- A single concept of connecting storage in the event of growth from a "small" to a "large data center" to save investment.
- A relatively simple transition from the design of a "small data center" direct-attached or Switched (with cheap switches) storage connections to the design of a "large data center" for future growth.
To do this, I propose to consider the advantages and disadvantages of several designs, observing a single concept - connecting NetApp FAS storage systems via iSCSI . These designs should be relatively simple to convert a small data center to a large one. Schematic designation of the proposed transition paths between configurations with the growth of the data center, a la Gartner square:
iSCSI tuning
Since iSCSI lives on top of Ethernet, you need to configure Flow Control (not to confuse the “normal” FlowControl with PFC IEEE 802.1Qbb for DCB Losless Ethernet) and configure Jumbo Frames on host network adapters, on storage and on switches (if any). Using MPIO / MCS within a single link can increase its total throughput compared to using a single connection in a link. It is recommended to allocate a separate VLAN for iSCSI traffic if the switch is used for a mixed type of traffic.
Directly-Attached iSCSI
The design of such a "small data center" consists of one or two servers directly connected to the storage system, each with at least two links for fault tolerance: one to one storage controller , the other to the other. At this point, it is logical to use the free iSCSI Initiator Software. In the case of increasing hosts, you must either add ports to the store or install switches. Depending on the storage model, there may be a different number of Ethernet ports. It is important to note that all Ethernet ports on NetApp FAS storages (there are no "dedicated" iSCSI ports) can be used for these types of traffic: iSCSI , NFS , CIFS ( SMB ), including at the same time.
- The positive side is the relative cheapness (no specialized adapters for hosts are needed, no switches are needed) and simplicity.
- The negative side of this solution will be low scalability and additional load on the CPU of the host performing the function of Software iSCSI Initiator.
Attribute | Value |
---|---|
Fully redundant | Yes |
Type of network | None, direct-attached |
Different host operating systems | Yes, with multiple-host configurations |
Multipathing required | Yes |
Type of storage configuration | HA pair |
Switched iSCSI
In such a scheme, it is necessary to have at least two links from each node, each link connected through different switches. This will ensure complete fault tolerance when the controller, switch or path exits. At this point, you can still use the Software iSCSI Initiator or a combination with dedicated iSCSI HBA (which would load the load from the host CPU ).
- Updating from the Direct-Attached configuration is relatively simple and can be performed without disabling the service, you may need to add a couple of additional paths after connecting the switches. If you need to install the iSCSI HBA adapter, host shutdown is inevitable.
- The positive side of this design can be attributed to the relatively high scalability, ease of configuration and low cost of the network.
- The negative side of this solution is the loss of frames in the network, which leads to an increase in the response between the storage and the host. For smaller configurations, it will provide a more than acceptable level of response.
Attribute | Value |
---|---|
Fully redundant | Yes |
Type of network | Multi-network |
Different host operating systems | Yes, with multiple-host configurations |
Multipathing required | Yes |
Type of storage configuration | HA pair |
iSCSI 2.0
Like Switched iSCSI , iSCSI 2.0 uses switches, but they must support DCB . This design will also require network adapters that support DCB on both the host and storage. Therefore, DCB must be supported throughout the iSCSI traffic. Also known as Lossless Ethernet , DCB provides guaranteed frame delivery while improving reliability, performance and response speed to FC . It is important to note that DCB supports all converged Ethernet ports on NetApp FAS storages.(there are no "dedicated" iSCSI or FCoE ports) and can be used for all types of traffic: iSCSI (2.0), FCoE , NFS , CIFS ( SMB ), including at the same time. The host supports QLogic 8300 CNA (HBA) adapters for working with DCB , this adapter allows offloading the load from the host CPU while providing low storage response speed. Some additional features of switches can improve the already low level of response for iSCSI , for example, CLEAR-Flow .
- Thus, in order to switch from Switched iSCSI to iSCSI 2.0, it is necessary to replace network cards and switches with DCB compatible ones, completely leaving the network topology and multipasing scheme as it was. DCB is configured on network cards and switches. Remaining after the replacement, the old "ordinary" switches can be used to connect clients to servers. Upgrading from a Switched iSCSI configuration without interruption of service is possible provided that the iSCSI HBA adapter is already installed on the host. If you need to install the iSCSI HBA adapter, host shutdown is inevitable.
The benefits of iSCSI over DCB include:
- Relative ease of switch configuration
- Low response from storage to application
- Ability to use QoS in mixed traffic environments
- Prevent frame loss and increase Ethernet frame viability
- Increased performance of IP protocols including iSCSI .
The disadvantages of iSCSI over DCB:
- The architecture does not include a large number of hopes from the host to the storage; ideally, there should be one hop.
- The need to calculate the level of oversubscription and compliance with the Fan-In coefficient in case of an increase in the number of hopes.
Attribute | Value |
---|---|
Fully redundant | Yes |
Type of network | DCB , Dual fabric |
Different host operating systems | Yes, with multiple-host configurations |
Multipathing required | Yes |
Type of storage configuration | HA pair |
- Replacement option : All convergent NetApp FAS ports can be easily modified by simply replacing SFP + modules, instead of Ethernet switches with DCB for iSCSI , you can always buy FC switches or use the same Ethernet switches with DCB for FCoE connection , and since NetApp FAS supports access to to the same moon through all these protocols, switching between FC / FCoE / iSCSI protocols occurs without data loss or conversions. To the negative side of FC and FCoErelatively high complexity in configuring switches can be attributed. When choosing between FC and FCoE, pay attention to the article “ FCoE: The Future of Fiber Channel ” and the article “ Consolidation of LAN and SAN Data Center Networks Based on DCB and FCoE Protocols ”
FAS scaling
There are several ways to grow NetApp FAS systems:
- Add shelves with disks (no stop)
- Combine the system into a cluster with the same or higher Madeleine (no stop)
- Convert the FAS255x controller to a shelf, buy new controllers, switch the converted and all other shelves to new controllers (simple is not inevitable)
- Combine in a cluster, migrate data online, remove the old system from the cluster, convert the FAS255x to a shelf and switch the old shelves to the new system (without stopping)
conclusions
Thus, the iSCSI protocol with NetApp FAS storage can be the basis for starting growth from a "small data center" to a "large" one, step-by-step and simply updating the network design, observing a single concept of addressing and multipassing. During the upgrade, already purchased equipment is utilized as much as possible. Each upgrade step increases the scalability of the data center infrastructure , its speed and reduces the response for applications, while performing costs as needed. Thanks to the versatility of FAS storage systems , support for advanced data center development trends and the ability to mutually replace protocols (or use them all at the same time including on the same ports), easy scalability, it is a good investment investment for both small and “large data centers”.
Best practics
Learn more about NetApp network topology and zoning guidelines in pictures .
It is important to follow best practices when configuring your infrastructure and checking the compatibility matrix for maximum performance and fault tolerance:
TR-3441 Windows Multipathing Options with Data ONTAP: Fiber Channel and iSCSI
WP-7071: "How Do I Get to Ethernet from Here?"
TR-3519: “The Road to 10-Gigabit Ethernet”
WP-7046: “Ethernet Storage”
TR-3441: “iSCSI Multipathing Possibilities on Windows with Data ONTAP”
TR-3163: “Data Protection and Recovery for Network-Attached Storage over IP / Ethernet Networks ”
WP-7052:“ Converged Enhanced Ethernet — Good for iSCSI SANs ”
TR-3519: “The Road to 10-Gigabit Ethernet”
iSCSI 10Gig Performance Tuning With Windows Server
iSCSI Configuration and Provisioning for Windows
TR-3802 Ethernet Storage Best Practices
TR-4182 Ethernet Storage Design Considerations and Best Practices
Best Practices for Network Configuration with NetApp Storage Systems
NetApp Verifying that DCB is configured correctly NetApp SAN Design and SAN Fundamentals on Data ONTAP
courses at NetApp Academy are available. Sources: sniaesfblog.org/?p=210&cpage=1#comment-82839 blogs.cisco.com/datacenter/the-napkin-dialogues-lossless-iscsi
www.storage-switzerland.com/Articles/Entries/2012/1/17_iSCSI_2.0_-_Using_Data_Center_Bridging_To_Enhance_iSCSI.html
www.snia.org/sites/default/education/tutorials/2010/spring/networking/GaryGumanow-JasonBlosil_iSCSI_Lossless_Ethernet_DCB.pdf
www. slideshare.net/extreme-muk/i-scsi-extremeintelnetappclearflow
I ask you to send comments on errors and suggestions for corrections in the text to the LAN .