August 9, 2017 at 13:08How to set up backup to S3-compatible cloud from Commvault
Not so long ago, we talked about the launch of our new cloud service - the Technoserv Cloud object storage , compatible with S3. Today we want to move on to specific technical issues in working with our service, and in particular, to the backup system settings for working with S3-compatible storage. Now almost all modern IBSs have built-in support for the S3 API protocol, but not everyone knows about it and, accordingly, do not fully use the functionality of the systems.
A recent study by the Taneja Group identified priorities in the use of object storage among users, according to which the storage of backups takes first place. As an example for today's post we took Commvault SRK. Learn more about research, Commvault, and how to configure cloud connectivity to the system in our new material.
As the volume of data accumulated by companies is growing by more than 50% per year, and over 80% is unstructured data, object storage is of undoubted value. This is the only technology that can keep up with the increasing storage capacity, while remaining within the budget.
According to the results of the study, file backup and archive storage are the most common tasks (57% of respondents noted them). The reason probably lies in the high demand for scalable file backup and document archiving, as well as cost-effective long-term data storage in order to comply with regulatory requirements. Object storage is ideal for large-scale storage of unstructured data, because it can easily be expanded in capacity (up to petabytes and above) by simply adding storage nodes. This approach eliminates a performance bottleneck that limits the single- and dual-controller architectures of traditional file storages. In addition, object storage provides independence from the hardware platform, so companies can increase capacity by adding hardware.
In second place is storage of objects (Storage as a service), 44% of respondents identified it. Facility storage provides service providers with a cost-effective way to manage backups using a robust, scalable, and multi-tenant architecture. And since this is a “service”, companies save on staff, equipment and data center costs. The IT administrator simply rents storage space based on cost per gigabyte and data transfer costs.
The third place in popularity is occupied by big data analytics (35% of respondents). Object storage is designed for large datasets, making it ideal for big data analytics tasks. A recent survey found that about 30% of respondents accumulate 100 TB or more "big data", and the amount of this data increases significantly every month. However, to support great data analytics and artificial intelligence for correlation and interpretation of data, you need to tightly integrate object storage with low latency storage and high performance computing systems.
Finally, another common use case for object storage is secure file sharing (it was also indicated by 35% of respondents).
| Amazon Infrequent Access | Google nearline | Amazon glacier | Google coldline | |
|---|---|---|---|---|
| Durability | 99.999999999% | 99.999999999% | 99.99% | 99.999999999% |
| Availability SLA | 99.9% | 99.9% | N / a | 99% |
| Access time | Milliseconds | Milliseconds | 5 min-12 hrs | Milliseconds |
| Minimum time | thirty | thirty | 90 | 90 |
| Amazon S-3 Standard | Google multiregional | Amazon reduced redundancy | Google regional | |
|---|---|---|---|---|
| Durability | 99.999999999% | 99.999999999% | 99.99% | 99.999999999% |
| Availability SLA | 99.95% | 99.95% | 99% | 99% |
| Access time | Milliseconds | Milliseconds | Milliseconds | Milliseconds |
| Minimum time | None | None | None | None |
Most object stores also offer very high fault tolerance with protection against site, node, and multiple drive failures.
Therefore, today we will focus on such a direction of application of object storage as backup storage, and as an example of a backup system, as already noted, we chose Commvault as one of the leaders in this segment and the most popular solution among corporate clients.
Leaders in data center backup solutions (Gartner, July 2017).
A bit about Commvault
Commvault is a scalable software platform for PK databases, applications, files, virtual machines and OS.
Backup systems that support S3.
Today, almost all modern backup systems have built-in support for the S3 API protocol, but not everyone knows about it and, accordingly, does not fully use the system’s functionality.
Commvault backs up to Amazon S3 and other S3-compatible storage. And the core of the Technoserv Cloud storage is not only built using S3 technologies, but also supports all S3 API commands, which guarantees compatibility with client applications - both now and in the future.
Next, using the example of our cloud, we show how to back up from Commvault to an S3-compatible storage. This instruction applies to any S3-compatible storage.
Configure Technoserv Cloud storage connection to Commvault backup system
To connect the Technoserv Cloud cloud storage to the Commvault backup system, you need to follow three simple steps.
1. Log in to the cloud management console (https://storage.technoserv.cloud ) and create a container for storing data. To do this, go to the "Baskets and objects" tab, then enter the name of the basket in the "Basket" field and select "Create", and leave the remaining fields filled by default.
In this example, we called the basket “bucket”.
2. After this, you need to find out the passkey and password for the cloud storage. To do this, go to the "Password Management" tab.
In the window that opens, the information necessary to connect the cloud storage to the backup system will be indicated, namely:
• information about the service (Service Host);
• access key (Access Key ID);
• and a secret key (Secret Access Key).
3.Now you need to create a cloud library in the Commvault backup system. To do this, right-click on the Libraries tab in the Commcell console and select Add -> Cloud Storage Library.
In the window that opens, in the "General" tab, fill in all the fields:
• Name - indicate the name of the created cloud library;
• Type - select Amazon S3 from the drop-down list;
• MediaAgent - select the media server that will connect to the cloud storage;
• Access Information - specify the server address, access key and password mentioned above. In the Bucket field, specify the name of the previously created basket.
If a proxy server is used to connect to the Internet, then you need to go to the “Advanced” tab and specify the proxy server settings.
That, in fact, is all. This completes the integration of Technoserv Cloud with Commvault.
To send backups to cloud storage, you must specify a storage policy in the backup system console in which to specify the created cloud library. After that, it remains to assign a retention policy to the backup job and run it.
Using cloud storage object Commvault system .
So, as you see, in a simple way we connected our cloud storage to the Commvault backup system when we launched the backup service in the cloud.
In this way, you can quickly realize the benefits of backing up data to the cloud, using it as an extension of your IT infrastructure. Among the advantages of this solution:
• Fast and reliable data backup and recovery.
• No need for expensive gateways and sophisticated intermediate solutions.
• You can replace the tape with cloud storage.
• You can use the cloud for disaster recovery.
According to IDC , the global cloud storage market (Cloud Storage) will grow by 25% per year.
What do you think? We are interested in your opinion: was the material useful? Do you need a similar backup instruction from another system? Which one? We are waiting for your comments!