Comparison of SATA, SAS, SSD and RAID arrays with them
Everyone knows the performance parameters of disk subsystems in theory. But what in practice? Many ask this question, some build their own hypotheses. I decided to conduct a series of tests and determine “Who is who”. I started testing with all known dd, hdparm utilities, then moved on to fio, sysbench. A series of tests were also performed using UnixBench and several other analogues. A number of graphs were built, but with further testing, it was found that most of this software is unsuitable for adequate comparison of different disks.
Using fio, it was possible to make a comparison table or graph for SAS, SATA, but when testing the SSD it turned out that the results were completely unsuitable. Of course, I respect the developers of this entire software, but at that moment it was decided to create a series of not synthetic tests, but closer to the real situation.
I must say right away that the test parameters and the machines themselves were selected so that the test results were not distorted by the type of processor, its frequency or other parameters.
Over eight cycles, the creation of small files with chaotic content and with a gradual increase in the number of files per cycle was generated. The execution time was measured for each cycle. It can be seen from the graph that KINGSTON SV300S3 SSDs have a high speed of file creation and almost do not depend on their number. It is also worth noting that these disks have a more straightforward scale. According to SAS disks in Hardware RAID, it can be seen that the speed depends on the type of raid, but does not depend on the number of disks. But more time is spent not on creating files, as it turned out, but on overwriting them. Let's move on to the second test.
The operations that were repeated in the first test were repeated, but the files were not created new each time, but the same file was used, in which new information was written each time. The terrible picture on SATA 7,200 rpm MB2000GCVBR disks is immediately evident. Slow recording and 2x 300GB SAS SEAGATE. Therefore, I decided to throw them out of the schedule for clarity on the rest. The fastest subsystem was a single SSD KINGSTON. The second and third places were taken by 8x SEAGATE ST3300657SS and 4x SEAGATE ST3300657SS. We also see that with an increase in the number of SSDs in the array, the speed drops slightly.
An InnoDB table was created with the following structure: Several queries were generated at the same time: - INSERT; - UPDATE with selection by PRIMARY KEY; - UPDATE with FULLTEXT selection (search by 4 characters out of 24): WHERE `status` LIKE '% (string)%'; - DELETE FROM with selection by PRIMARY KEY; - DELETE FROM with selection without using a key: WHERE `time`> (int); - SELECT with selection without using a key: WHERE `time`> (int); - SELECT with selection by PRIMARY KEY; - SELECT with FULLTEXT selection (search by 4 characters out of 24): WHERE `status` LIKE '% (string)%'; - SELECT with keyless selection: WHERE `uid`> (int). And again we observe the same picture as in the second test.
In the following tests, I use the sysbench utility, which generates large files:
128 files with a total size of 10 GB, 30 GB, and 50 GB.
Block size 4 Kb.
I want to note right away that on some graphs, on some servers there is no data on 10 GB. This is due to the fact that these machines have more than 10 GB of RAM and data caching is performed. The lack of some 50 GB results is due to a lack of disk space, in the case of the KINGSTON SV300S3 SSD.
It can be seen that the best indicators are available in all variations with the KINGSTON SV300S3 SSD, as well as in the 8x SEAGATE ST3300657SS in RAID10. Very visible growth rate with an increase in the number of SAS drives.
Here is the very moment where you can clearly see that SSDs are completely different. The difference is 4 times!
The leaders are the same. If you compare 2x SSDs from INTEL and 2x SAS, there is practically no difference.
Here we see a slightly different picture. The leader is the 4x KINGSTON RAID10 SSD, with minimal change in results with increasing file size, and 8x SEAGATE in RAID10, with a gradual decrease in speed, at speeds of 700 Mbps and 600 Mbps.
The lines of 1x SSD KINGSTON and 2x SSD KINGSTON RAID1 coincided. Simply put, for linear reading it is better to take either RAID10 or a single disk. Using RAID1 is not warranted.
It is clearly seen that the 2x SAS RAID1 and 4x SAS RAID10 are very similar. But with an increase in the number of disks twice, a huge increase in speed is visible.
2x SSD Intel RAID1 has a small drop in speed between 10 GB and 30 GB, and then they go at the same speed as SATA RAID1.
The leaders are all SSDs:
- 4x KINGSTON RAID10;
- 2x KINGSTON RAID1, 2x INTEL RAID1;
- 1 KINGSTON.
I copied all the rest to the following chart for clarity. The highest speed among these is naturally 8x SAS RAID10, but the speed drops sharply. But based on the data on 2x SAS and 4x SAS, I will assume that with further growth in volume, the speed will stabilize.
Excellent performance has 2x 120GB SSD INTEL SSDSC2CT12 Hardware RAID1 SAS1068E with a stable speed of 30 Mbps. According to KINGSTON, with an increase in the number of disks, the speed, oddly enough, drops. In fourth place is the 8x SAS SEAGATE.
We all know that on no server there is only read or only write. Both operations are always performed. And in most cases, these are just random operations, not linear ones. And so, let's see what we got. Due to the excellent write speed, there is a large margin of 2x SSD INTEL, followed by the SSD KINGSTON. Third place was shared by 2x SSD KINGSTON and 8x SAS SEAGATE.
After carrying out all these tests, I decided that it would be convenient to deduce the dependence of speed on the ratio of random read and random write operations. Who has a speed increase, who has a drop, and the 8x SAS RAID10 has a straight line.
He also made a comparison of large arrays of SAS disks, which shows that it depends more on disk speed than on their number. It's time to take stock. There were many cars, but not enough. Unfortunately, I was not able to determine whether the indicators for SSD INTEL SSDSC2CT12 are their feature or a feature of the raid controller. But I believe that the same controller.
Unanswered questions:
In addition to the tests and servers listed, there were many more, but they did not get into the results, as they were “calibrated” and many of them were considered incorrect.
We also tested RAMDisk. The performance was pretty good, but not the best. Probably due to the fact that it was a virtual machine.
All tests except the last were performed only on dedicated servers.
Acknowledgments:
Using fio, it was possible to make a comparison table or graph for SAS, SATA, but when testing the SSD it turned out that the results were completely unsuitable. Of course, I respect the developers of this entire software, but at that moment it was decided to create a series of not synthetic tests, but closer to the real situation.
I must say right away that the test parameters and the machines themselves were selected so that the test results were not distorted by the type of processor, its frequency or other parameters.
Test 1
File creation
Over eight cycles, the creation of small files with chaotic content and with a gradual increase in the number of files per cycle was generated. The execution time was measured for each cycle. It can be seen from the graph that KINGSTON SV300S3 SSDs have a high speed of file creation and almost do not depend on their number. It is also worth noting that these disks have a more straightforward scale. According to SAS disks in Hardware RAID, it can be seen that the speed depends on the type of raid, but does not depend on the number of disks. But more time is spent not on creating files, as it turned out, but on overwriting them. Let's move on to the second test.
Test 2
Overwrite files
The operations that were repeated in the first test were repeated, but the files were not created new each time, but the same file was used, in which new information was written each time. The terrible picture on SATA 7,200 rpm MB2000GCVBR disks is immediately evident. Slow recording and 2x 300GB SAS SEAGATE. Therefore, I decided to throw them out of the schedule for clarity on the rest. The fastest subsystem was a single SSD KINGSTON. The second and third places were taken by 8x SEAGATE ST3300657SS and 4x SEAGATE ST3300657SS. We also see that with an increase in the number of SSDs in the array, the speed drops slightly.
Test 3
MySQL Combining sql queries INSERT, SELECT, UPDATE, DELETE
An InnoDB table was created with the following structure: Several queries were generated at the same time: - INSERT; - UPDATE with selection by PRIMARY KEY; - UPDATE with FULLTEXT selection (search by 4 characters out of 24): WHERE `status` LIKE '% (string)%'; - DELETE FROM with selection by PRIMARY KEY; - DELETE FROM with selection without using a key: WHERE `time`> (int); - SELECT with selection without using a key: WHERE `time`> (int); - SELECT with selection by PRIMARY KEY; - SELECT with FULLTEXT selection (search by 4 characters out of 24): WHERE `status` LIKE '% (string)%'; - SELECT with keyless selection: WHERE `uid`> (int). And again we observe the same picture as in the second test.
CREATE TABLE `table` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`time` int(11) NOT NULL,
`uid` int(11) NOT NULL,
`status` varchar(32) NOT NULL,
PRIMARY KEY (`id`),
FULLTEXT KEY `status` (`status`)
) ENGINE=InnoDB DEFAULT CHARSET=cp1251;In the following tests, I use the sysbench utility, which generates large files:
128 files with a total size of 10 GB, 30 GB, and 50 GB.
Block size 4 Kb.
I want to note right away that on some graphs, on some servers there is no data on 10 GB. This is due to the fact that these machines have more than 10 GB of RAM and data caching is performed. The lack of some 50 GB results is due to a lack of disk space, in the case of the KINGSTON SV300S3 SSD.
Test 4
Line recording (file creation)
It can be seen that the best indicators are available in all variations with the KINGSTON SV300S3 SSD, as well as in the 8x SEAGATE ST3300657SS in RAID10. Very visible growth rate with an increase in the number of SAS drives.
Here is the very moment where you can clearly see that SSDs are completely different. The difference is 4 times!
Test 5
Line recording (file overwrite)
The leaders are the same. If you compare 2x SSDs from INTEL and 2x SAS, there is practically no difference.
Test 6
Linear reading
Here we see a slightly different picture. The leader is the 4x KINGSTON RAID10 SSD, with minimal change in results with increasing file size, and 8x SEAGATE in RAID10, with a gradual decrease in speed, at speeds of 700 Mbps and 600 Mbps.
The lines of 1x SSD KINGSTON and 2x SSD KINGSTON RAID1 coincided. Simply put, for linear reading it is better to take either RAID10 or a single disk. Using RAID1 is not warranted.
It is clearly seen that the 2x SAS RAID1 and 4x SAS RAID10 are very similar. But with an increase in the number of disks twice, a huge increase in speed is visible.
2x SSD Intel RAID1 has a small drop in speed between 10 GB and 30 GB, and then they go at the same speed as SATA RAID1.
Test 7
Random reading
The leaders are all SSDs:
- 4x KINGSTON RAID10;
- 2x KINGSTON RAID1, 2x INTEL RAID1;
- 1 KINGSTON.
I copied all the rest to the following chart for clarity. The highest speed among these is naturally 8x SAS RAID10, but the speed drops sharply. But based on the data on 2x SAS and 4x SAS, I will assume that with further growth in volume, the speed will stabilize.
Test 8
Random recording
Excellent performance has 2x 120GB SSD INTEL SSDSC2CT12 Hardware RAID1 SAS1068E with a stable speed of 30 Mbps. According to KINGSTON, with an increase in the number of disks, the speed, oddly enough, drops. In fourth place is the 8x SAS SEAGATE.
Test 9
Combined random read and write operations
We all know that on no server there is only read or only write. Both operations are always performed. And in most cases, these are just random operations, not linear ones. And so, let's see what we got. Due to the excellent write speed, there is a large margin of 2x SSD INTEL, followed by the SSD KINGSTON. Third place was shared by 2x SSD KINGSTON and 8x SAS SEAGATE.
Test 10
After carrying out all these tests, I decided that it would be convenient to deduce the dependence of speed on the ratio of random read and random write operations. Who has a speed increase, who has a drop, and the 8x SAS RAID10 has a straight line.
Test 11
He also made a comparison of large arrays of SAS disks, which shows that it depends more on disk speed than on their number. It's time to take stock. There were many cars, but not enough. Unfortunately, I was not able to determine whether the indicators for SSD INTEL SSDSC2CT12 are their feature or a feature of the raid controller. But I believe that the same controller.
- With the increasing number of SAS disks in the array, all indicators are only improving.
- For MySQL, the slow subsystems are SATA RAID1 and SAS RAID1. For the rest, there are differences, but they are not so significant.
- For linear recording, both large arrays of SAS disks in RAID10 and SSDs are good. There is no point in using arrays from SSD. Cost is growing and productivity is in place.
- Any linear arrays are good for linear reading. But in practice, lin. reading without writing, we hardly meet.
- Random read over single SSDs or in Software RAID.
- For random recording, it is better to use Hardware RAID from the SSD, although single SSDs do not do much.
- Random read / write, that is, one of the most important indicators, has the best results on Hardware RAID from SSD.
- Summarizing all of the above, for most tasks it is better to use large arrays (> = 8) from SAS or Hardware RAID from SSD. But for some tasks it will be more correct to use single SSDs.
- Based on the volumes of SSDs, which are mainly offered in our market, for VDS-nodes it is worth using the maximum performance of processors paired with large SAS arrays or average processors and single SSDs. I believe that using hw raid for two SSDs will be expensive.
- If you need a fast system and there is no need for a large 2x SSD disk space in Hardware RAID is the best choice. If you want to save a little at the expense of performance, then you can take a single SSD or two SSDs in a software raid.
Unanswered questions:
- What happens when you increase the number of SSDs in Hardware RAID?
- What is cheaper for virtual servers: expensive cars and one large array of SAS or several average servers with single SSDs? This issue should also take into account the reliability / longevity of SAS and SSD, as there are various rumors about the latter.
In addition to the tests and servers listed, there were many more, but they did not get into the results, as they were “calibrated” and many of them were considered incorrect.
We also tested RAMDisk. The performance was pretty good, but not the best. Probably due to the fact that it was a virtual machine.
All tests except the last were performed only on dedicated servers.
Acknowledgments:
- vds4you for providing a large number of virtual machines, on the basis of which tests were calibrated;
- PlusServer for providing the first four test machines on SSD / SAS, which have been tested for quite some time. Unfortunately, they fell into these graphs. As well as two SATA from the current tests;
- FastVPS for providing all servers with KINGSTON SSDs;
- ServerClub for providing machines with INTEL Hardware RAID SSDs, as well as all machines with SAS disks;
- everyone who participated in the discussion of the initial tests, in particular on the searchengines forum