Back to basics or a step forward? HPE StoreVirtual VSA
Following current trends in the storage and processing industry, I got to SDS - software-defined storage, namely, an HPE product called HPE StoreVirtual VSA, which I will tell you about in this article.
Software-defined storages (hereinafter referred to as SDS) are considered the next milestone in the development of data storage technologies. This technology is new to the market, so there is no unambiguous and precise definition of this term. A similar situation was present at the dawn of the "clouds", when this term was called everything in a row, not particularly delving into its meaning. By analogy with the mentioned “clouds”, the main idea of SDS is to abstract from the hardware component and come to a business-oriented model for building storage and processing systems.
Now, in most cases, we are talking about software virtualization of data warehouses. This is not about the notorious substitution of one storage system for another due to built-in functions or hardware gateways, but about virtualization of the local drives of your computing nodes.
I have a slight perplexity in this approach, because for many years all the A-class manufacturers of the IT industry have been strenuously inspiring us with the idea that the future lies in storage systems, which are based on the idea of diversity of data storage and processing, which will increase their availability and security.
The beginning of my IT career was at the time of the advent of virtualization, so I personally observed how reluctantly and painfully many companies were moving from the "orthodox" scheme "one application - one server" to virtualized clusters with remote repositories. And now, when the use of storage systems has been repeatedly justified and universally accepted in “combat” infrastructures, the same manufacturers tell us that a new milestone is the return to the use of local drives of our servers. That is why in this article I not only want to express my opinion, but also to hear from you your assessment of both this particular product and the whole idea of SDS in general.
HPE VSA is a software product that installs nodes on a virtual cluster. Currently, VMware vSphere, Microsoft Hyper-V, and KVM products are supported. There are plugins for vSphere and Hyper-V for integration with their management consoles, which allows you to manage the entire infrastructure from a single point.
A typical HPE StoreVirtual VSA deployment requires 3 nodes — two nodes for fault tolerance, and a third for hosting Quorum Witness, which will be responsible for the consistency of data on all nodes in the event of a disconnection. A scenario using only 2 nodes is possible, however, in this case, an independent NFSv3 file ball is needed to host Quorum Witness.
From the perspective of the administrator of a virtual cluster, StoreVirtual VSA is a virtual machine running Enterprise Linux, one for each node in the cluster. Installation can be done either through a special wizard or by deploying an OVF image. In both cases, you will need to configure the disk of the created virtual machine. The size of the virtual disk will depend on the size of the block specified when formatting the datastore.
Like any software, StoreVirtual VSA requires computing resources of the processor and RAM, and HPE developed a number of recommendations to develop a new solution (or to check compatibility with an existing cluster):
This product is licensed in volume, which we present to the hypervisor as a datastore. It is important to understand that this will be a “raw” volume from the point of view of the end user. First of all, the local drives of your servers will be combined into RAID groups at the level of the host RAID controller. The resulting usable volume will be licensed. After starting each separate VSA, they are combined into a cluster and summarize the disk space into one common pool, which is then available to users (servers) via the block protocol (iSCSI).
Nowadays, it’s not enough just to provide some capacity for data storage. The storage system must be capable of protecting data from failures, optimizing storage, management and processing, and StoreVirtual VSA can offer us all the basic functions that are present in hardware storage systems:
• Thin provisioning - “thin” volumes that allow the system to allocate space as actual filling with data;
• Peer Motion - migration of volumes without interrupting access to them;
• Multi-site SAN - a single volume distributed across two or three sites;
• Synchronous and asynchronous replication;
• Creation of consistent snapshots at the application level;
• Network RAID - building fault-tolerant circuits at the cluster node level;
• Split Site - creating geographically dispersed clusters;
• Adaptive Optimization - two-level automatic tearing with granularity of 256 KB.
Management of all functions is available both through the CMC (Centralized Management Console) and through the plug-in for VMWare / Hyper-V management consoles.
In my opinion, the most interesting option will be Network RAID, which protects data from loss in case of failure of an entire cluster node. In essence, this is synchronous replication between nodes, and the RAID level controls the number of copies of data blocks stored in a cluster.
For example, in the case of Network RAID level 10, recommended by the manufacturer as the most optimal, 2 copies of each data block will always be stored in the cluster. Such a backup scheme gives us guaranteed protection against data loss in case of failure of the 1st node of the cluster (and with special luck - up to half of the nodes), however, the overhead will be 1/2. This means that when you purchase a 2TB StoreVirtual VSA license, we get 1TB of virtual storage with usable capacity.
The next level of protection is Network RAID level 10 + 1. In this case, the cluster stores 3 copies of each data block. The permissible level of “losses” is 2 knots. Overhead will be 1/3. It is the Network RAID 10 + 1 backup algorithm that underlies SplitSite technology, allowing you to create a cluster on 3 geographically dispersed sites. The most remarkable thing about this technology is that, unlike the classic replication of hardware storage systems, we lack such concepts as the primary and secondary platforms. The application does not matter on which of the sites the block being processed is currently located, and if the node fails and the redundant block is accessed at another site, the application does not notice the difference and, as a result, there is no downtime. On the other hand, it imposes stringent requirements on the channel, connecting the sites to each other. The delay in data transfer, round trip, should not exceed 5 ms.
Network RAID level 10 + 2 creates copies of blocks on all nodes of the cluster, which allows us to lose n-1 nodes, but the overhead will be 1 / n (where n is the number of nodes in the cluster). At the same time, the minimum supported configuration is from 3 nodes, which in my opinion is excessive and not applicable in real architectures.
By analogy with hardware raids, there are redundancy levels with checksums: Network RAID level 5 and 6. In these architectures, blocks with checksums appear, which are evenly distributed across all nodes of the cluster. This redundancy scheme is the most economical in terms of overhead (n-1 for level 5 and n-2 for level 6), and the fault tolerance level is level 10 and level 10 + 1, respectively. However, the need to read checksums causes a noticeable decrease in the computing performance of the entire cluster, and as a result, they are not recommended for use with highly loaded applications (the most obvious example is databases).
StoreVirtual VSA is part of the hyperconverged systems family of solutions. And, like any other software component of hyperconverged solutions, it is a budget analogue of its hardware original - dedicated storage. In my understanding, the main objective of this product is to create fail-safe secure storage for your virtualized cluster without buying additional equipment. At the same time, do not underestimate the functionality of Network RAID, which allows synchronous replication between cluster nodes, including geographically dispersed ones.
As you can see from the functionality I described, this solution can be called functionally independent, giving its users the possibility of entry-level hardware storage for more modest money.
In conclusion, it is worth noting that at the moment there is a promotion program: when you purchase any HPE Proliant Gen9 server, you get a 1 TB StoreVirtual VSA license for free. This is an excellent opportunity not only to download trial licenses for a detailed study of the product, but also to start introducing it into the productive of new projects without increasing their budget.
Software-defined storages (hereinafter referred to as SDS) are considered the next milestone in the development of data storage technologies. This technology is new to the market, so there is no unambiguous and precise definition of this term. A similar situation was present at the dawn of the "clouds", when this term was called everything in a row, not particularly delving into its meaning. By analogy with the mentioned “clouds”, the main idea of SDS is to abstract from the hardware component and come to a business-oriented model for building storage and processing systems.
Now, in most cases, we are talking about software virtualization of data warehouses. This is not about the notorious substitution of one storage system for another due to built-in functions or hardware gateways, but about virtualization of the local drives of your computing nodes.
I have a slight perplexity in this approach, because for many years all the A-class manufacturers of the IT industry have been strenuously inspiring us with the idea that the future lies in storage systems, which are based on the idea of diversity of data storage and processing, which will increase their availability and security.
The beginning of my IT career was at the time of the advent of virtualization, so I personally observed how reluctantly and painfully many companies were moving from the "orthodox" scheme "one application - one server" to virtualized clusters with remote repositories. And now, when the use of storage systems has been repeatedly justified and universally accepted in “combat” infrastructures, the same manufacturers tell us that a new milestone is the return to the use of local drives of our servers. That is why in this article I not only want to express my opinion, but also to hear from you your assessment of both this particular product and the whole idea of SDS in general.
Architecture
HPE VSA is a software product that installs nodes on a virtual cluster. Currently, VMware vSphere, Microsoft Hyper-V, and KVM products are supported. There are plugins for vSphere and Hyper-V for integration with their management consoles, which allows you to manage the entire infrastructure from a single point.
A typical HPE StoreVirtual VSA deployment requires 3 nodes — two nodes for fault tolerance, and a third for hosting Quorum Witness, which will be responsible for the consistency of data on all nodes in the event of a disconnection. A scenario using only 2 nodes is possible, however, in this case, an independent NFSv3 file ball is needed to host Quorum Witness.
From the perspective of the administrator of a virtual cluster, StoreVirtual VSA is a virtual machine running Enterprise Linux, one for each node in the cluster. Installation can be done either through a special wizard or by deploying an OVF image. In both cases, you will need to configure the disk of the created virtual machine. The size of the virtual disk will depend on the size of the block specified when formatting the datastore.
Like any software, StoreVirtual VSA requires computing resources of the processor and RAM, and HPE developed a number of recommendations to develop a new solution (or to check compatibility with an existing cluster):
| StoreVirtual VSA capacity (total of all storage devices) | Memory required (GB) not using Adaptive Optimization or Space Reclamation | Memory required (GB) when using Adaptive Optimization and / or Space Reclamation |
|---|---|---|
| <= 1 TB | 4 | 4 |
| 1 - <= 4 TB | 5 | 5 |
| 4 - <= 10 TB | 7 | 8 |
| 10 - <= 20 TB | 9 | 12 |
| 20 - <= 30 TB | 12 | 17 |
| 30 - <= 40 TB | fifteen | 21 |
| 40 - <= 50 TB | 18 | 26 |
This product is licensed in volume, which we present to the hypervisor as a datastore. It is important to understand that this will be a “raw” volume from the point of view of the end user. First of all, the local drives of your servers will be combined into RAID groups at the level of the host RAID controller. The resulting usable volume will be licensed. After starting each separate VSA, they are combined into a cluster and summarize the disk space into one common pool, which is then available to users (servers) via the block protocol (iSCSI).
Opportunities
Nowadays, it’s not enough just to provide some capacity for data storage. The storage system must be capable of protecting data from failures, optimizing storage, management and processing, and StoreVirtual VSA can offer us all the basic functions that are present in hardware storage systems:
• Thin provisioning - “thin” volumes that allow the system to allocate space as actual filling with data;
• Peer Motion - migration of volumes without interrupting access to them;
• Multi-site SAN - a single volume distributed across two or three sites;
• Synchronous and asynchronous replication;
• Creation of consistent snapshots at the application level;
• Network RAID - building fault-tolerant circuits at the cluster node level;
• Split Site - creating geographically dispersed clusters;
• Adaptive Optimization - two-level automatic tearing with granularity of 256 KB.
Management of all functions is available both through the CMC (Centralized Management Console) and through the plug-in for VMWare / Hyper-V management consoles.
In my opinion, the most interesting option will be Network RAID, which protects data from loss in case of failure of an entire cluster node. In essence, this is synchronous replication between nodes, and the RAID level controls the number of copies of data blocks stored in a cluster.
For example, in the case of Network RAID level 10, recommended by the manufacturer as the most optimal, 2 copies of each data block will always be stored in the cluster. Such a backup scheme gives us guaranteed protection against data loss in case of failure of the 1st node of the cluster (and with special luck - up to half of the nodes), however, the overhead will be 1/2. This means that when you purchase a 2TB StoreVirtual VSA license, we get 1TB of virtual storage with usable capacity.
The next level of protection is Network RAID level 10 + 1. In this case, the cluster stores 3 copies of each data block. The permissible level of “losses” is 2 knots. Overhead will be 1/3. It is the Network RAID 10 + 1 backup algorithm that underlies SplitSite technology, allowing you to create a cluster on 3 geographically dispersed sites. The most remarkable thing about this technology is that, unlike the classic replication of hardware storage systems, we lack such concepts as the primary and secondary platforms. The application does not matter on which of the sites the block being processed is currently located, and if the node fails and the redundant block is accessed at another site, the application does not notice the difference and, as a result, there is no downtime. On the other hand, it imposes stringent requirements on the channel, connecting the sites to each other. The delay in data transfer, round trip, should not exceed 5 ms.
Network RAID level 10 + 2 creates copies of blocks on all nodes of the cluster, which allows us to lose n-1 nodes, but the overhead will be 1 / n (where n is the number of nodes in the cluster). At the same time, the minimum supported configuration is from 3 nodes, which in my opinion is excessive and not applicable in real architectures.
By analogy with hardware raids, there are redundancy levels with checksums: Network RAID level 5 and 6. In these architectures, blocks with checksums appear, which are evenly distributed across all nodes of the cluster. This redundancy scheme is the most economical in terms of overhead (n-1 for level 5 and n-2 for level 6), and the fault tolerance level is level 10 and level 10 + 1, respectively. However, the need to read checksums causes a noticeable decrease in the computing performance of the entire cluster, and as a result, they are not recommended for use with highly loaded applications (the most obvious example is databases).
Positioning
StoreVirtual VSA is part of the hyperconverged systems family of solutions. And, like any other software component of hyperconverged solutions, it is a budget analogue of its hardware original - dedicated storage. In my understanding, the main objective of this product is to create fail-safe secure storage for your virtualized cluster without buying additional equipment. At the same time, do not underestimate the functionality of Network RAID, which allows synchronous replication between cluster nodes, including geographically dispersed ones.
As you can see from the functionality I described, this solution can be called functionally independent, giving its users the possibility of entry-level hardware storage for more modest money.
In conclusion, it is worth noting that at the moment there is a promotion program: when you purchase any HPE Proliant Gen9 server, you get a 1 TB StoreVirtual VSA license for free. This is an excellent opportunity not only to download trial licenses for a detailed study of the product, but also to start introducing it into the productive of new projects without increasing their budget.
