September 27, 2016 at 11:19Using ReFS Features in Veeam Backup & Replication 9.5
One recent post was about Veeam Backup & Replicaion 9.5, which is scheduled for release in the fourth quarter of this year. Today I will dwell in more detail on the declared extended support for ReFS.
The Resilient File System (ReFS) file system is recommended for Windows Server 2016, and our developers did not fail to take advantage of its new features. What was the focus of the new version of Veeam Backup & Replication, and what are the results? Very positive: the use of ReFS in backup repositories significantly increases the reliability of data storage, significantly accelerates the incremental merging of increments on storage systems, and also allows you to store more recovery points - through the use of links to identical data blocks.
For details, welcome to Cat.
First of all, ReFS volumes will be supported on direct attached storage (DAS) internal storage systems and Storage Spaces storage systems, both in their traditional form and deployed using Storage Spaces Direct (S2D). This will make it possible to significantly accelerate the creation of synthetic full backups and the conversion of backups in the chain, as well as increase reliability and reduce disk consumption. Let us dwell on these advantages in more detail.
One of the most “catchy” operations when creating a backup, in particular, a synthetic full backup, is a transformation (conversion at the block level), during which blocks are moved between different files inside the incremental chain. For more information see the article in the Veeam knowledge base , as well as a post on Habré .
In the new version of Veeam Backup & Replication, fast cloning technology (via the ReFS API) will be used to create “synthetics” - this will allow not to move blocks between backup files, but operate on links to blocks stored on the volume. Thus, all operations related to "synthetics" are reduced to updating meta-data and allow you to do without directly reading and writing data from backup.
As you can see from the example below, the creation of a synthetic full backup for a VM backup on ReFS took 58 seconds, while 3 incremental backups were processed. If you do the same for a backup stored on NTFS, it will take more than 36 minutes - as they say, “feel the difference”!
Since the creation of “synthetics” and the transformation operations have now accelerated significantly, and the performance of the backup storage has increased, why not save more full backups to disk? For example, add more GFS backups there. Yes to health! Indeed, thanks to integration with ReFS, it is now possible to share identical blocks for several full backups. This will, of course, reduce the requirements for the capacity of the backup repository and, to a certain extent, approach the efficiency of storage systems with built-in deduplication (happily avoiding, however, performance degradation due to data “rehydration” in such systems). And the best part is that storage can be built on the basis of a typical Windows Server 2016 using internal drives or direct-attached drives. An example of a synthetic full backup in a ReFS repository:
Let's talk about reliability, which, along with performance and optimizing the use of storage space of storage, is a key point in integration with ReFS. Veeam Backup & Replication 9.5 will automatically activate Data Integrity Streams for all backup files that the ReFS data integrity scanner works with. This will allow to detect and signal implicit (silent) violations, which is especially important when using the direct incremental-infinite backup method, and you need to guarantee data integrity with a long storage period. We can say that our engineers “enlarged” the already existing data integrity check functionality at the storage level, and now it has spread not only to the last recovery point, but also to all stored backups.
The following is an example of recording a Windows Event Log when a data integrity violation is detected in a file on a simple ReFS volume:
All these features will work with regular ReFS volumes - but for those who use Storage Spaces, there are even more opportunities. After all, such systems allow you to take measures to repair damaged blocks by using a Mirror or Parity copy of the blocks. Thus, it is possible to ensure data integrity in an infinitely-incremental backup chain.
How to achieve all this wonderful future? Very simple: configure the backup repository on Windows Server 2016. (Many use the same server as the Veeam backup server.) As a result, the ReFS API will be available, and, accordingly, all the above-described functionalities.
" An article on Habré with an overview of new features and products of Veeam
" An article on Veeam backup methods
» Announcement of Veeam Availability Suite 9.5
The Resilient File System (ReFS) file system is recommended for Windows Server 2016, and our developers did not fail to take advantage of its new features. What was the focus of the new version of Veeam Backup & Replication, and what are the results? Very positive: the use of ReFS in backup repositories significantly increases the reliability of data storage, significantly accelerates the incremental merging of increments on storage systems, and also allows you to store more recovery points - through the use of links to identical data blocks.
For details, welcome to Cat.
First of all, ReFS volumes will be supported on direct attached storage (DAS) internal storage systems and Storage Spaces storage systems, both in their traditional form and deployed using Storage Spaces Direct (S2D). This will make it possible to significantly accelerate the creation of synthetic full backups and the conversion of backups in the chain, as well as increase reliability and reduce disk consumption. Let us dwell on these advantages in more detail.
Fast cloning technology
One of the most “catchy” operations when creating a backup, in particular, a synthetic full backup, is a transformation (conversion at the block level), during which blocks are moved between different files inside the incremental chain. For more information see the article in the Veeam knowledge base , as well as a post on Habré .
In the new version of Veeam Backup & Replication, fast cloning technology (via the ReFS API) will be used to create “synthetics” - this will allow not to move blocks between backup files, but operate on links to blocks stored on the volume. Thus, all operations related to "synthetics" are reduced to updating meta-data and allow you to do without directly reading and writing data from backup.
As you can see from the example below, the creation of a synthetic full backup for a VM backup on ReFS took 58 seconds, while 3 incremental backups were processed. If you do the same for a backup stored on NTFS, it will take more than 36 minutes - as they say, “feel the difference”!
Optimization of disk space usage when storing full backups
Since the creation of “synthetics” and the transformation operations have now accelerated significantly, and the performance of the backup storage has increased, why not save more full backups to disk? For example, add more GFS backups there. Yes to health! Indeed, thanks to integration with ReFS, it is now possible to share identical blocks for several full backups. This will, of course, reduce the requirements for the capacity of the backup repository and, to a certain extent, approach the efficiency of storage systems with built-in deduplication (happily avoiding, however, performance degradation due to data “rehydration” in such systems). And the best part is that storage can be built on the basis of a typical Windows Server 2016 using internal drives or direct-attached drives. An example of a synthetic full backup in a ReFS repository:
Write Integrity Streams Method
Let's talk about reliability, which, along with performance and optimizing the use of storage space of storage, is a key point in integration with ReFS. Veeam Backup & Replication 9.5 will automatically activate Data Integrity Streams for all backup files that the ReFS data integrity scanner works with. This will allow to detect and signal implicit (silent) violations, which is especially important when using the direct incremental-infinite backup method, and you need to guarantee data integrity with a long storage period. We can say that our engineers “enlarged” the already existing data integrity check functionality at the storage level, and now it has spread not only to the last recovery point, but also to all stored backups.
The following is an example of recording a Windows Event Log when a data integrity violation is detected in a file on a simple ReFS volume:
All these features will work with regular ReFS volumes - but for those who use Storage Spaces, there are even more opportunities. After all, such systems allow you to take measures to repair damaged blocks by using a Mirror or Parity copy of the blocks. Thus, it is possible to ensure data integrity in an infinitely-incremental backup chain.
How to achieve all this wonderful future? Very simple: configure the backup repository on Windows Server 2016. (Many use the same server as the Veeam backup server.) As a result, the ReFS API will be available, and, accordingly, all the above-described functionalities.
What else to read and see
" An article on Habré with an overview of new features and products of Veeam
" An article on Veeam backup methods
» Announcement of Veeam Availability Suite 9.5