Interesting features of HP 3PAR storage systems. Part 1 - Persistent Ports
Today I will describe a very interesting, in my opinion, new feature of HP 3PAR storage systems. Her name is Persistent Ports (virtual ports).
It is a well-known fact that all manufacturers of data storage systems in a tough competition are constantly forced to add new functions to the microcodes of their storage systems and, of course, correct errors in microcodes. Storage administrators have to update microcodes at regular intervals. Sometimes a bug appears in the firmware, sometimes they want to get new functionality or remove the limitations of the previous version, and often they just need to migrate, because the firmware of the storage system is so old that it is about to be removed from support.
Those with hands can always solve the problem of updating the microcode, but there is one problem - the business and its requirements for the service to be provided 24 hours, 7 days a week and 365 days a year. And downtime is only planned and short, but better without them at all. The availability factor of equipment with a large number of nines after the decimal point must be worked out.
So, solving the problem of eliminating the need for temporary equipment downtime during maintenance work on storage systems, 3PAR engineers implemented an algorithm called Persistent Ports. The solution captivates with its ingenious simplicity. But before describing this solution, we recall what needed to be done using other storage systems for this. Further in the text, we mean storage systems connected via the FC protocol and updates that do not require the simultaneous restart of all storage system controllers (offline upgrade), because in this case, it is simply not possible to completely avoid even the slightest downtime. Consider only cases in which a sequential restart of the storage controllers is required.
The classic solution to this problem is to use software to switch paths on the server (multipath software). In this case, each volume from the storage system must be visible on the server through several independent paths. Paths must go through different storage network switches and different storage controllers.
This scheme is beautiful and with responsible implementation it most often works. The multipath-inga software does its job, but sometimes everything breaks up into harsh reality. Reality tells me that:
1. When I see several hundred servers, each of which runs under its own operating system, is administered by its system administrator with unknown qualifications, I don’t have any desire to check if all of them work switching paths during updating the microcode of storage systems.
2. I have known cases of errors in the operation of multipath-ing software. I do not mean the configuration errors made by the system administrator, but the software errors. Unfortunately, both of these cases have been discovered on productive systems. The manufacturers of this software subsequently released patches, but this is easier for anyone. Therefore, once again to check on a productive system whether this method works or not, there is little desire.
With the next update of 3PAR OS to version 3.1.2, a very useful Persistent Ports function (aka virtual ports) appeared. The meaning of this function is that each host port on the storage system controller is assigned a partner port on another storage system controller. That is, when updating the microcode on one controller of the storage system, the addresses (wwn) of this controller will be transferred to the working controller. At the same time, the host software does not mark any path to the storage system as offline, that is, this operation is transparent for multipath-ing software.
This feature is now activated by default and manifests itself when the storage system controller is restarted and when the microcode is updated online. This feature is implemented using the NPIV functionality (N_Port ID Virtualization). Let me remind you that NPIV allows several nodes of a storage network to use network access through one physical port, but each one has its own unique address (wwn).
It is a well-known fact that all manufacturers of data storage systems in a tough competition are constantly forced to add new functions to the microcodes of their storage systems and, of course, correct errors in microcodes. Storage administrators have to update microcodes at regular intervals. Sometimes a bug appears in the firmware, sometimes they want to get new functionality or remove the limitations of the previous version, and often they just need to migrate, because the firmware of the storage system is so old that it is about to be removed from support.
Those with hands can always solve the problem of updating the microcode, but there is one problem - the business and its requirements for the service to be provided 24 hours, 7 days a week and 365 days a year. And downtime is only planned and short, but better without them at all. The availability factor of equipment with a large number of nines after the decimal point must be worked out.
So, solving the problem of eliminating the need for temporary equipment downtime during maintenance work on storage systems, 3PAR engineers implemented an algorithm called Persistent Ports. The solution captivates with its ingenious simplicity. But before describing this solution, we recall what needed to be done using other storage systems for this. Further in the text, we mean storage systems connected via the FC protocol and updates that do not require the simultaneous restart of all storage system controllers (offline upgrade), because in this case, it is simply not possible to completely avoid even the slightest downtime. Consider only cases in which a sequential restart of the storage controllers is required.
The classic solution to this problem is to use software to switch paths on the server (multipath software). In this case, each volume from the storage system must be visible on the server through several independent paths. Paths must go through different storage network switches and different storage controllers.
This scheme is beautiful and with responsible implementation it most often works. The multipath-inga software does its job, but sometimes everything breaks up into harsh reality. Reality tells me that:
1. When I see several hundred servers, each of which runs under its own operating system, is administered by its system administrator with unknown qualifications, I don’t have any desire to check if all of them work switching paths during updating the microcode of storage systems.
2. I have known cases of errors in the operation of multipath-ing software. I do not mean the configuration errors made by the system administrator, but the software errors. Unfortunately, both of these cases have been discovered on productive systems. The manufacturers of this software subsequently released patches, but this is easier for anyone. Therefore, once again to check on a productive system whether this method works or not, there is little desire.
With the next update of 3PAR OS to version 3.1.2, a very useful Persistent Ports function (aka virtual ports) appeared. The meaning of this function is that each host port on the storage system controller is assigned a partner port on another storage system controller. That is, when updating the microcode on one controller of the storage system, the addresses (wwn) of this controller will be transferred to the working controller. At the same time, the host software does not mark any path to the storage system as offline, that is, this operation is transparent for multipath-ing software.
This feature is now activated by default and manifests itself when the storage system controller is restarted and when the microcode is updated online. This feature is implemented using the NPIV functionality (N_Port ID Virtualization). Let me remind you that NPIV allows several nodes of a storage network to use network access through one physical port, but each one has its own unique address (wwn).