June 30, 2016 at 12:59NetApp SDS: ONTAP Select
ONTAP Select is a logical development of the Data ONTAP-v line , i.e. Software Defined Storage. ONTAP software (an operating system or firmware in a popular way, if you want) can be used on a specialized FAS hardware platform or as a virtual machine: in public clouds or on comodity equipment.
The last two options are called ONTAP for Cloud and ONTAP Select, respectively.
Like the predecessor of ONTAP Select, this product, which lives in the form of a virtual machine and is completely based on the traditional RAID controller installed in your server. NAS (CIFS, NFS) and IP SAN (iSCSI) protocols are supported and there is no FCP support. In NetApp documents you can find the internal name ONTAP Select - sDOT, this is the same thing.
From the expected innovations:
Along with High Availability and clustering, single-mode configurations are still supported.
To provide High Availability functionality, SyncMirror technology is used (the same as in MetroCluster) which works as RAID1: there are two plexes, main and backup. Reading and writing occurs from the main, and synchronous replica occurs to the backup. If one of the two nodes in the HA configuration with the main plexus dies, the second node uses the backup plex, applications transparently switch and do not notice the loss of the node. In the figure, each node has two data sets (two plexes): its own (primary) and neighbor (backup).
ONTAP Select can act as a source or as a recipient of replicas. Based on SnapMirror, you can build a DR SVM solution . If ONTAP is used only as a file storage, then for replication there is no need to ensure data consistency with third-party software. If IP SAN or virtualization applications, databases and others are used, then backup software is needed to ensure the consistency of information.
The easiest way to take a consistent snapshot is to put out the application. What is data consistency? Consistency refers to backup requirements that provide a guarantee of data recovery at the time of backup. This is if explained in a dry, completely technical language. But in a simple and understandable way, it’s better explained and understood from the opposite: imagine that you have a database living in a virtualization environment or on storage systems that can do snapshots and are configured, so they are removed every day at 12 at night. Will you be able to recover to one of these snapshots when damaged? The answer is “maybe yes, maybe not”, even “rather no than yes”. Because snapshots themselves are molded without any interaction with the application, despite the fact whether the database transactions were completed and whether all the data from the memory was on the disk. In other words, you need to somehow integrate and “make friends” your snaps and your application so that it flushes data from the cache to disk, completes all accumulated transactions, freezes, then the snapshot was removed and only at the end the application “died”. And in order for this to happen unnoticed by users, the snapshot must be removed instantly. And only under this condition you are guaranteed to be able to roll back to such a sepshot. snapshot should be removed instantly. And only under this condition you are guaranteed to be able to roll back to such a sepshot. snapshot should be removed instantly. And only under this condition you are guaranteed to be able to roll back to such a sepshot.
All modern hypervisors, with the help of agents installed inside virtual machines, can achieve consistency and continue to perform consistent snapshots, which is very convenient for small companies that have not very loaded virtual machines. If you do not install a hypervisor agent, there will be no consistency.
For the time being, as long as the organization and load on the vairing machines is not large, consistent snapshots of hypervisors and backup solutions built on this will well be suitable for such companies. But companies and loads can grow and here comes a tipping point:
Snapshots of Hyper-V or the same VMware are arranged, unfortunately, in such a way that the more they exist, the more this snapshot becomes. The larger the snapshot, the longer it takes. The more he retires, the more he loads the disk subsystem. The longer it is removed, the greater the likelihood that the old snap will not have time to retire due to the increased additional load, and another new snap will be created for the new backup process. The more snapshots, the greater the load on the disk subsystem.
Both VMware and Microsoft openly say that hypervisor snapshots are not recommended for highly loaded applications. I personally observed twice in practice that the process of removing a snapshot (consolidation) led to the collapse of the system, in both cases this was due to the fact that the disk subsystem did not manage to cope with the additional load, the block device increased and increased everything due to excessive load response time, until it exceeded the timeout and the virtual machine had no time to consolidate went offline, with its subsequent damage. Once it even turned out to restore it.
In general, at some point it becomes clear that snapshots of hypervisors are not suitable as a backup paradigm.
Snapshots performed on ONTAP have absolutely no effect on storage system performance. This is the internal architecture of ONTAP. But storage snapshots are storage snapshots. And so that they can interact with the hypervisor, you need to use some specialized software solution for interaction. Examples of such solutions include Veeam B&R, vVOL technology and others. It is important to separately note the NetApp SnapCreator solution, which is able to provide this bundle, since ONTAP Select is designed for small organizations, and SnapCreator is a free product.
This is a free web-based framework for managing and integrating with ONTAP and various applications, including Hyper-V, VMware and KVM hypervisors. Due to its modularity, the framework can be integrated with many other software products using plug-ins, and, if necessary, expanded using scripts. Learn more about SnapCreator here and here .
Hypervisors
Equipment:
Licensing
terabyte. The maximum size is 400TB.
ONTAP Select includes licenses: CIFS, NFS, iSCSI, FlexClone, SnapRestore, SnapVault, SnapMirror, Virtual Storage Console.
The SnapCenter license is not included and is purchased for each individual host.
Distinctive features
The following is included in the basic package when purchasing ONTAP Select; in addition, you do not need to buy this functionality:
SDS ONTAP is well suited for medium and small companies, and companies with a branch structure, where there may be a need to replicate data to other ONTAPs. The product allows you to cluster for online migration of your data over cluster nodes, has rich functionality and wide integration with various applications and allows you to build high-availability storage using server comodity. Backup products that integrate ONTAP with virtual environments KVM and ESXi will be very popular in such schemes as they allow to improve the backup process. So Veeam Backup & Replicationand free SnapCreator allow you to perform consistent backups in conjunction with hypervisors and are able to manage SnapMirror replication, which makes these, and other similar products, the optimal tandem in such an infrastructure. SnapMirror for SVM technology will help build a DR solution based on ONTAP. Separately, it is worth highlighting VMware vVOL technology , which natively provides consistent snapshots on ONTAP, which will also be very interesting for such installations.
This may contain links to Habra articles that will be published later .
Please send messages about errors in the text to the LAN .
Comments, additions and questions on the opposite article, please comment .
The last two options are called ONTAP for Cloud and ONTAP Select, respectively.
Like the predecessor of ONTAP Select, this product, which lives in the form of a virtual machine and is completely based on the traditional RAID controller installed in your server. NAS (CIFS, NFS) and IP SAN (iSCSI) protocols are supported and there is no FCP support. In NetApp documents you can find the internal name ONTAP Select - sDOT, this is the same thing.
From the expected innovations:
- High Avalability Support
- Support for clustering up to 4 nodes
- Maximum usable capacity of 400 TB (100 TB per node in a 4-node cluster)
Along with High Availability and clustering, single-mode configurations are still supported.
High availability
To provide High Availability functionality, SyncMirror technology is used (the same as in MetroCluster) which works as RAID1: there are two plexes, main and backup. Reading and writing occurs from the main, and synchronous replica occurs to the backup. If one of the two nodes in the HA configuration with the main plexus dies, the second node uses the backup plex, applications transparently switch and do not notice the loss of the node. In the figure, each node has two data sets (two plexes): its own (primary) and neighbor (backup).
SnapMirror Replication
ONTAP Select can act as a source or as a recipient of replicas. Based on SnapMirror, you can build a DR SVM solution . If ONTAP is used only as a file storage, then for replication there is no need to ensure data consistency with third-party software. If IP SAN or virtualization applications, databases and others are used, then backup software is needed to ensure the consistency of information.
Point-in-time consistency
The easiest way to take a consistent snapshot is to put out the application. What is data consistency? Consistency refers to backup requirements that provide a guarantee of data recovery at the time of backup. This is if explained in a dry, completely technical language. But in a simple and understandable way, it’s better explained and understood from the opposite: imagine that you have a database living in a virtualization environment or on storage systems that can do snapshots and are configured, so they are removed every day at 12 at night. Will you be able to recover to one of these snapshots when damaged? The answer is “maybe yes, maybe not”, even “rather no than yes”. Because snapshots themselves are molded without any interaction with the application, despite the fact whether the database transactions were completed and whether all the data from the memory was on the disk. In other words, you need to somehow integrate and “make friends” your snaps and your application so that it flushes data from the cache to disk, completes all accumulated transactions, freezes, then the snapshot was removed and only at the end the application “died”. And in order for this to happen unnoticed by users, the snapshot must be removed instantly. And only under this condition you are guaranteed to be able to roll back to such a sepshot. snapshot should be removed instantly. And only under this condition you are guaranteed to be able to roll back to such a sepshot. snapshot should be removed instantly. And only under this condition you are guaranteed to be able to roll back to such a sepshot.
Virtualization and Consistency
All modern hypervisors, with the help of agents installed inside virtual machines, can achieve consistency and continue to perform consistent snapshots, which is very convenient for small companies that have not very loaded virtual machines. If you do not install a hypervisor agent, there will be no consistency.
Why are hypervisor snapshots evil?
For the time being, as long as the organization and load on the vairing machines is not large, consistent snapshots of hypervisors and backup solutions built on this will well be suitable for such companies. But companies and loads can grow and here comes a tipping point:
Snapshots of Hyper-V or the same VMware are arranged, unfortunately, in such a way that the more they exist, the more this snapshot becomes. The larger the snapshot, the longer it takes. The more he retires, the more he loads the disk subsystem. The longer it is removed, the greater the likelihood that the old snap will not have time to retire due to the increased additional load, and another new snap will be created for the new backup process. The more snapshots, the greater the load on the disk subsystem.
Both VMware and Microsoft openly say that hypervisor snapshots are not recommended for highly loaded applications. I personally observed twice in practice that the process of removing a snapshot (consolidation) led to the collapse of the system, in both cases this was due to the fact that the disk subsystem did not manage to cope with the additional load, the block device increased and increased everything due to excessive load response time, until it exceeded the timeout and the virtual machine had no time to consolidate went offline, with its subsequent damage. Once it even turned out to restore it.
In general, at some point it becomes clear that snapshots of hypervisors are not suitable as a backup paradigm.
Snapshots ONTAP
Snapshots performed on ONTAP have absolutely no effect on storage system performance. This is the internal architecture of ONTAP. But storage snapshots are storage snapshots. And so that they can interact with the hypervisor, you need to use some specialized software solution for interaction. Examples of such solutions include Veeam B&R, vVOL technology and others. It is important to separately note the NetApp SnapCreator solution, which is able to provide this bundle, since ONTAP Select is designed for small organizations, and SnapCreator is a free product.
Snapcreator
This is a free web-based framework for managing and integrating with ONTAP and various applications, including Hyper-V, VMware and KVM hypervisors. Due to its modularity, the framework can be integrated with many other software products using plug-ins, and, if necessary, expanded using scripts. Learn more about SnapCreator here and here .
Supported ONTAP Select Configurations
Hypervisors
- VMware VSphere 5.X or 6.X, Enterprise or Enterprise Plus
- KVM hypervisor
Equipment:
- Intel Haswell CPU (Xeon E5 v3)
- ONTAP Select virtual machine: 4 virtual
- CPUs / 16GB memory
- Storage:
- SAS, NL-SAS disk support
- Hardware RAID controller with writeback cache
- Network: minimum 2 x 10GbE ports
Licensing
terabyte. The maximum size is 400TB.
ONTAP Select includes licenses: CIFS, NFS, iSCSI, FlexClone, SnapRestore, SnapVault, SnapMirror, Virtual Storage Console.
The SnapCenter license is not included and is purchased for each individual host.
Distinctive features
The following is included in the basic package when purchasing ONTAP Select; in addition, you do not need to buy this functionality:
- Support for SAN (iSCSI) & NAS (CIFS & NFS)
- CIFS (SMB) is supported both in integration with AD, and using Workgroup
- High availability
- Clustering with the ability to migrate SAN / NAS data online by cluster nodes
- Snapshots and clones that do not affect storage performance - FlexClone and SnapRestore licenses are standard
- QoS - file level, moon, volum
- MS ODX & Copy Offload (SAN / NAS), SMB 3.1
- NFS 4.X, pNFS
- NDMP with support for 0x2050 extension
- Thing provisioning, SCSI-3 UNMAP (Space Reclamation)
- resource exhaustion reporting - notification of the host about the end of space in the thin moon
- Compression
- Deduplication
- Storage Virtual Machine (Multi Tenancy) - The ability to create dedicated “virtual” storage systems based on a cluster that can operate independently of each other and managed by various administrators, departments or even companies
- Integration with advanced backup software
- Support for vVol (NFS / iSCSI), VASA, VAAI (NFS / iSCSI)
- Ability to create Disastar Recovery solutions on multiple sites
- Support for thin data replication to remote ONTAP (no matter FAS, ONTAP Select or ONTAP for Cloud) - SnapMirror and SnapVault licenses are standard
- Free SnapCreator - software for integrating snaps with applications (databases, virtualization, etc.)
- Free OnCommand Unified Manager & Performance Manager - software for tracking performance, security and fullness of storage
conclusions
SDS ONTAP is well suited for medium and small companies, and companies with a branch structure, where there may be a need to replicate data to other ONTAPs. The product allows you to cluster for online migration of your data over cluster nodes, has rich functionality and wide integration with various applications and allows you to build high-availability storage using server comodity. Backup products that integrate ONTAP with virtual environments KVM and ESXi will be very popular in such schemes as they allow to improve the backup process. So Veeam Backup & Replicationand free SnapCreator allow you to perform consistent backups in conjunction with hypervisors and are able to manage SnapMirror replication, which makes these, and other similar products, the optimal tandem in such an infrastructure. SnapMirror for SVM technology will help build a DR solution based on ONTAP. Separately, it is worth highlighting VMware vVOL technology , which natively provides consistent snapshots on ONTAP, which will also be very interesting for such installations.
This may contain links to Habra articles that will be published later .
Please send messages about errors in the text to the LAN .
Comments, additions and questions on the opposite article, please comment .