April 30, 2014 at 13:01Amazon, Google, HP, Microsoft, and Rackspace Cloud Storage Test Results
The company Nasuni, which offers ready-made data warehouse infrastructure solutions in the form of services on the market, has prepared another report on the testing of cloud data warehouses offered by the largest vendors Amazon, Google, HP, Microsoft and Rackspace.
Among others, the following tests were carried out: on the speed of writing, reading and deleting data, responsiveness of services, uptime, scalability and the number of errors when reading / writing data.
For those who do not like to read and understand the numbers, a brief summary of the report:
Testing methodologies can be studied in detail in the presented report , below are the results and comments from the report.
Microsoft showed the best results in tests on the speed of writing data (of any type) to the cloud storage, overtaking competitors in 14 out of 23 separate tests. In second and third place with a large lag behind the cloud from Amazon and HP.
In tests for writing files larger than 1 MB, the results of testing all vendors are close to each other, since the effect of transactions with the storage decreases significantly with increasing file size. In this type of test, the Amazon cloud won, in second place with Microsoft slightly behind, in third place were the clouds from HP and Rackspace with almost the same results.
The test results of reading data (both in general and with files of size> 1MB) from cloud storage also show an advantage in speed for the Microsoft platform.
In the case of testing the reading of any files with a sufficiently large backlog, Amazon comes in second place, followed by other vendors with approximately the same results.
Test results for reading files larger than one megabyte are somewhat different, with HP, Amazon and Rackspace following slightly behind Microsoft. In third place is the Google cloud.
The results of testing file deletion speed again demonstrate a significant advantage of the cloud from Microsoft. In second place with a large lag Amazon, in third - HP. Clouds Google and Rackspace show a significant lag in the speed of data deletion. The report separately notes that the file size does not affect the test results.
Testing the responsiveness of cloud services was measured based on the response time to requests from read / write / delete operations for a period of 30 days.
Microsoft service showed the best results with an average response value of 0.48 seconds. With a slight lag in the results, Amazon comes in second place, then the rest of the cloud vendors are located with a big lag. At the same time, as noted in the report, the cloud from Rackspace showed the most unpredictable behavior during the testing period.
During testing, the level of availability or uptime of cloud services was also measured. Amazon and Google showed 100% availability during the testing period. Microsoft cloud showed uptime - 99.996% and other providers showed uptime below 99.99%.
The scalability of services was tested as follows: 100 million new objects were loaded into the cloud storage, while this operation took no more than 30 days. The numbers of the test results show how the download speed has changed over time, which reflects how much the service’s performance depends on the number of downloaded objects, that is, how much the service scales without losing performance.
Testing showed excellent scalability of Amazon, Microsoft and Google cloud storage, which performance is not affected by the number of downloaded objects. Clouds from HP and Rackspace (which are built on OpenStack) showed the worst scaling results when uploading a large number of objects to the cloud.
In the course of testing the performance of cloud storage, service access errors were separately monitored. When testing records of 100 million objects, Amazon, Google and Microsoft showed a complete absence of errors. Errors in writing data were for vendors HP 0.000017% and Rackspace 0.000001%.
When testing data reading, only the Microsoft cloud showed no errors. Other vendors failed the test one way or another and showed different error levels of read operations.
The following is a literal translation of the final findings of the report:
This year, our tests revealed that Microsoft Azure Blob Storage took a significant step forward, overtaking previous Amazon S3 leader. For all three of the most important tests (performance, scalability, stability), Microsoft showed the best results in each of the categories. Even taking into account the fact that Azure was introduced to the market not so long ago as S3 and does not have such volumes of stored data, it is clear that Microsoft’s investment in the platform is beginning to pay dividends and that the technology they offer to the market is the best.
Testing methodologies can be studied in detail in the presented report , below are the results and comments from the report.
Data writing speed
Microsoft showed the best results in tests on the speed of writing data (of any type) to the cloud storage, overtaking competitors in 14 out of 23 separate tests. In second and third place with a large lag behind the cloud from Amazon and HP.
In tests for writing files larger than 1 MB, the results of testing all vendors are close to each other, since the effect of transactions with the storage decreases significantly with increasing file size. In this type of test, the Amazon cloud won, in second place with Microsoft slightly behind, in third place were the clouds from HP and Rackspace with almost the same results.
Data reading speed
The test results of reading data (both in general and with files of size> 1MB) from cloud storage also show an advantage in speed for the Microsoft platform.
In the case of testing the reading of any files with a sufficiently large backlog, Amazon comes in second place, followed by other vendors with approximately the same results.
Test results for reading files larger than one megabyte are somewhat different, with HP, Amazon and Rackspace following slightly behind Microsoft. In third place is the Google cloud.
Data Deletion Rate
The results of testing file deletion speed again demonstrate a significant advantage of the cloud from Microsoft. In second place with a large lag Amazon, in third - HP. Clouds Google and Rackspace show a significant lag in the speed of data deletion. The report separately notes that the file size does not affect the test results.
Service responsiveness
Testing the responsiveness of cloud services was measured based on the response time to requests from read / write / delete operations for a period of 30 days.
Microsoft service showed the best results with an average response value of 0.48 seconds. With a slight lag in the results, Amazon comes in second place, then the rest of the cloud vendors are located with a big lag. At the same time, as noted in the report, the cloud from Rackspace showed the most unpredictable behavior during the testing period.
Uptime services
During testing, the level of availability or uptime of cloud services was also measured. Amazon and Google showed 100% availability during the testing period. Microsoft cloud showed uptime - 99.996% and other providers showed uptime below 99.99%.
Service Scalability
The scalability of services was tested as follows: 100 million new objects were loaded into the cloud storage, while this operation took no more than 30 days. The numbers of the test results show how the download speed has changed over time, which reflects how much the service’s performance depends on the number of downloaded objects, that is, how much the service scales without losing performance.
Testing showed excellent scalability of Amazon, Microsoft and Google cloud storage, which performance is not affected by the number of downloaded objects. Clouds from HP and Rackspace (which are built on OpenStack) showed the worst scaling results when uploading a large number of objects to the cloud.
Errors of reading, writing data
In the course of testing the performance of cloud storage, service access errors were separately monitored. When testing records of 100 million objects, Amazon, Google and Microsoft showed a complete absence of errors. Errors in writing data were for vendors HP 0.000017% and Rackspace 0.000001%.
When testing data reading, only the Microsoft cloud showed no errors. Other vendors failed the test one way or another and showed different error levels of read operations.
Conclusion
The following is a literal translation of the final findings of the report:
Microsoft confidently overtook other vendors in the test results, offering the best read / write / delete data speed among different data sizes, the highest responsiveness rates and the lowest error rate. Microsoft not only significantly outperformed its competitors in pure performance tests, but it is the only cloud platform with zero error during the test for 100 million read and write operations. In those categories where Microsoft is not in the first place (uptime and scatter of scalability), the vendor is in second place close to the leader.
For these reasons, Microsoft overtook (last year’s leader - approx. Translator) Amazon and won a leadership position in the 2013 report. Cloud storage is a rapidly growing market in which new offers and new suppliers are constantly appearing. The results of this test illustrate that there is no parity among products on the market yet. While offering “cloud storage” is relatively easy, the desire to offer a high-performance, reliable, and scalable solution requires significant focus, advanced technology, and ongoing investment. Despite the fact that Microsoft won first place this year, it is possible that things could change again next year. In the two years that this report has been published, the market leader has been constantly changing. As can be seen from the results of this report,
Additionally, HP, a relative newcomer to cloud storage vendors, has shown strong performance in read and write tests, which, with improvements in stability, can make the company a key player in the market.
At Nasuni, we adhere to our promise to track, test and report on changes in this dynamic market, looking close to those who are the leaders today and who are focused on offering the best full-fledged cloud storage solution for our customers.