How to set up archiving of Veeam backups in Microsoft Azure Blob Storage using StarWind VTL
- Transfer
I bring to your attention the translation of the article of my colleague Andrew Zhelezko on the use of an integrated backup storage solution based on Veeam, StarWind and Azure products.
Many companies still use tape libraries for their services, however, cloud storages are gaining more and more popularity, which ensure reliable and stable backup systems. And along with support for common virtual tape libraries, Veeam now allows you to work with the StarWind VTL library for Microsoft Azure Blob Storage. This is an excellent opportunity for those who need an inexpensive and reliable cloud storage to safely place backup copies of data in it. Users of such an integrated solution will be able to implement flexible data storage policies: for example, they can hold backups for 1-2 weeks in a regular infrastructure, and then move them to Microsoft Azure Blob Storage long-term cloud storage. In this article I will briefly talk about how to configure the integration.
I note that the efforts that the Veeam user will need to spend on setting up the integration are minimal. However, it should be borne in mind that there are many scenarios for the deployment of components of an integrated solution, so you need to be careful and choose the configuration that is right for you. Scenarios can be very different - from installing Veeam and StarWind solutions on one server in the on-premises infrastructure to installing them on different VMs in the Azure cloud environment.
In the general process of setting up an integrated solution, there are 3 main stages.
To prepare the cloud infrastructure, you must perform the following steps:
You will need these settings when you set up cloud replication in StarWind VTL.
At this stage, we will emulate the installation of a tape library on the selected server. Veeam Backup & Replication will write data to media in this library, then it will be processed and prepared for transfer to the cloud. This is the classic D2D2C (Disk to Disk to Cloud) scheme in action.
You need to complete the following steps:
StarWind VTL emulates the real HPE MSL8096 tape library, with typical operating principles and set of operations.
Note: You may need to install the latest drivers for the server to correctly recognize the specified library. They can be taken here .
You can now connect the library to the server using standard Windows iSCSI tools ( Control Panel > Administrative Tools > iSCSI Initiator ). Go to Discovery > Discovery Portal to initialize and connect to the library ( Targets tab ).
After that, you need to set up archiving to the cloud using replication. To do this, select Microsoft Azure Cloud Storage, then enter the data that you saved in step
1. After that, enter the desired retention period settings.
As for Veeam Backup & Replication, you need to add the server configured in the previous step to the Veeam infrastructure as tape sever. This can be done in the console Veeam Backup & Replication; You will need to specify the IP / DNS and account to access.
When adding a server in this role, the Transport Service and Tape Proxy Service will be automatically installed on it . You can also select the automatic tape inventory option.
After all cassettes are recognized and placed in the Free Media Pool , it is wise to create a multi-cassette target pool. We will need it a little later.
So, our Veeam Backup & Replication is connected to a virtual tape library and can write data there. To do this, you just need to create a backup task to the Backup to Tape tape (or File to Tape ) and specify what to archive (for Backup to Tape , you should already have backed up the machines). Cassettes this task will be taken from a previously prepared target pool (see above).
Depending on the size of the backup \ file, the data will get to the VTL more or less quickly.
Go to the StarWind console and make sure that the backup data is recorded on the tape in slot 1:
Now you can switch to the virtual tape library server and remove the cassette from the slot, if this was not done automatically by exporting the tape upon completion of the task. Since in my case the start time of replication to the cloud was set immediately after recording to the tape, I can personally observe the process of downloading data to the cloud on Azure Blob Storage:
If the download was successful, then the corresponding indicator appears on the Cloud tab . To ensure successful copying, you can go to Azure Blob Storage and check whether the files are loaded in our container.
You can then use the Azure portal to change the access tier for any of these uploaded files.
Similar changes can be made using the StarWind settings, and using PowerShell.
StarWind users can start a restore operation from the Azure cloud storage — to do this, you can open the Veeam Backup & Replication console and follow the standard path with the File or VM Recovery Wizard. But why not perform the restore directly in the Azure infrastructure? Indeed, in the Azure Marketplace there is a virtual machine StarWind appliance - you can deploy it to Azure and, like Veeam Backup & Replication, then restore from the archive storage directly to the new VM in Azure. This approach will save you time and ensure that applications can work in different “bays” of the backup infrastructure.
In addition, you can provide access to these new, restored VMs in Azure via VeeamPN (Veeam-Powered Network) - for example, to provide a secure connection to the main data center of your organization.
Many companies still use tape libraries for their services, however, cloud storages are gaining more and more popularity, which ensure reliable and stable backup systems. And along with support for common virtual tape libraries, Veeam now allows you to work with the StarWind VTL library for Microsoft Azure Blob Storage. This is an excellent opportunity for those who need an inexpensive and reliable cloud storage to safely place backup copies of data in it. Users of such an integrated solution will be able to implement flexible data storage policies: for example, they can hold backups for 1-2 weeks in a regular infrastructure, and then move them to Microsoft Azure Blob Storage long-term cloud storage. In this article I will briefly talk about how to configure the integration.
I note that the efforts that the Veeam user will need to spend on setting up the integration are minimal. However, it should be borne in mind that there are many scenarios for the deployment of components of an integrated solution, so you need to be careful and choose the configuration that is right for you. Scenarios can be very different - from installing Veeam and StarWind solutions on one server in the on-premises infrastructure to installing them on different VMs in the Azure cloud environment.
In the general process of setting up an integrated solution, there are 3 main stages.
Stage 1: Prepare Azure Cloud Infrastructure
To prepare the cloud infrastructure, you must perform the following steps:
- Log in to the Azure portal, select Storage accounts , add a new storage (or an existing Blob storage).
- Enter all the necessary information, from Account kind choose Blob storage .
- Go to the new repository, copy the name and access key (located in Settings > Access keys ).
- Create a container that will then be used to store data ( Blob service > Containers > New ).
You will need these settings when you set up cloud replication in StarWind VTL.
Stage 2: StarWind VTL Setup
At this stage, we will emulate the installation of a tape library on the selected server. Veeam Backup & Replication will write data to media in this library, then it will be processed and prepared for transfer to the cloud. This is the classic D2D2C (Disk to Disk to Cloud) scheme in action.
You need to complete the following steps:
- Install the latest version of StarWind VTL (at the time of writing this article was 8.0.0.12248) on the target server - it could be a physical or virtual machine, a cloud server or even a Veeam server.
- During installation, select the VTL and Cloud Replication option - then the corresponding StarWind components will be installed automatically.
- Specify the path to the storage pool (you can leave the default on drive C ).
- Then go to the StarWind console, connect to the server you need (if you are all on the same machine, specify localhost or 127.0.0.1 for connection) and create a virtual tape device with the required number of cassettes.
StarWind VTL emulates the real HPE MSL8096 tape library, with typical operating principles and set of operations.
Note: You may need to install the latest drivers for the server to correctly recognize the specified library. They can be taken here .
You can now connect the library to the server using standard Windows iSCSI tools ( Control Panel > Administrative Tools > iSCSI Initiator ). Go to Discovery > Discovery Portal to initialize and connect to the library ( Targets tab ).
After that, you need to set up archiving to the cloud using replication. To do this, select Microsoft Azure Cloud Storage, then enter the data that you saved in step
1. After that, enter the desired retention period settings.
Step 3. Configuring on the side of Veeam Backup & Replication
As for Veeam Backup & Replication, you need to add the server configured in the previous step to the Veeam infrastructure as tape sever. This can be done in the console Veeam Backup & Replication; You will need to specify the IP / DNS and account to access.
When adding a server in this role, the Transport Service and Tape Proxy Service will be automatically installed on it . You can also select the automatic tape inventory option.
After all cassettes are recognized and placed in the Free Media Pool , it is wise to create a multi-cassette target pool. We will need it a little later.
So, our Veeam Backup & Replication is connected to a virtual tape library and can write data there. To do this, you just need to create a backup task to the Backup to Tape tape (or File to Tape ) and specify what to archive (for Backup to Tape , you should already have backed up the machines). Cassettes this task will be taken from a previously prepared target pool (see above).
Depending on the size of the backup \ file, the data will get to the VTL more or less quickly.
Go to the StarWind console and make sure that the backup data is recorded on the tape in slot 1:
Now you can switch to the virtual tape library server and remove the cassette from the slot, if this was not done automatically by exporting the tape upon completion of the task. Since in my case the start time of replication to the cloud was set immediately after recording to the tape, I can personally observe the process of downloading data to the cloud on Azure Blob Storage:
If the download was successful, then the corresponding indicator appears on the Cloud tab . To ensure successful copying, you can go to Azure Blob Storage and check whether the files are loaded in our container.
You can then use the Azure portal to change the access tier for any of these uploaded files.
Similar changes can be made using the StarWind settings, and using PowerShell.
It is a little about recovery of virtual computers from the VTL library in Azure
StarWind users can start a restore operation from the Azure cloud storage — to do this, you can open the Veeam Backup & Replication console and follow the standard path with the File or VM Recovery Wizard. But why not perform the restore directly in the Azure infrastructure? Indeed, in the Azure Marketplace there is a virtual machine StarWind appliance - you can deploy it to Azure and, like Veeam Backup & Replication, then restore from the archive storage directly to the new VM in Azure. This approach will save you time and ensure that applications can work in different “bays” of the backup infrastructure.
In addition, you can provide access to these new, restored VMs in Azure via VeeamPN (Veeam-Powered Network) - for example, to provide a secure connection to the main data center of your organization.
Additional links
- Article on Habré about working with magnetic tapes.
- Webinar on integrated solution (in English).