June 30, 2019 at 11:37How to evaluate storage performance on Linux: benchmarking using open tools
The last time we talked about tools with open source to evaluate the performance of processors and memory. Today we are talking about benchmarks for file systems and storage systems on Linux - Interbench, Fio, Hdparm, S and Bonnie. Photo - Daniele Levis Pelusi - Unsplash
Fio (stands for Flexible I / O Tester) creates disk I / O streams to evaluate the performance of the Linux file system. The utility can also be run on Windows - you need to install the Cygwin command line interface . The configuration guide is in the fio repository on GitHub .
Fio Author - Jens Aksbo ( by Jens Axboe ), responsible for the IO subsystem in the Linux and developer tools blktrace for routing I / O operations. He created fio because he was tired of writing programs to test a specific load manually.
The utility will calculate the IOPS and system bandwidth, and also allow you to evaluate the depth of the I / O queue. The utility works with special files (extension .fio), in which settings and test conditions are written. There are several test options, for example, there is arbitrary writing, reading and rewriting. Here is an example of the contents of the file for the first case:
Today, fio is used by large companies - they work with the utility in SUSE , Nutanix and IBM .
The utility was written by Canadian developer Mark Lord back in 2005. It is still supported by the author and is part of many popular distributions. The main purpose of hdparm is to configure drive settings. But the tool can be used to conduct simple benchmarks, for example, measuring read speed. To do this, write the command in the console:
The system will generate a similar response:
As for the drive settings, hdparm allows you to change the cache size, modify the sleep mode and power settings, as well as safely erase data on the SSD. But, as ArchLinux experts warn , a careless change in system parameters can make the data on the disk inaccessible and even damage the drive. Before working with hdparm, it is better to familiarize yourself with the manual - just write the man hdparm command in the console.
This is a set of benchmarks for evaluating the performance of I / O systems. The authors of the utility were a development team from the AlgoDev group, which includes employees of the Italian University of Modena and Reggio Emilia .
All benchmarks are bash scripts that evaluate the performance of the storage system - bandwidth, latency, the work of schedulers. For example, the throughput-sync.sh benchmark “bombards” storage systems with read or write requests (in this case, the already mentioned fio utility is used). Here is the code for this script .
Another script - comm_startup_lat.sh - measures the delay in reading data from the disk during the "cold cache" (when it does not have the necessary data). Code alsocan be found in the repository .
Photos - Agê Barros - Unsplash
A utility for evaluating file system performance developed in 1989. Its author was engineer Tim Bray (Tim Bray). With Bonnie's help, he planned to optimize the computing systems involved in the New Oxford English Dictionary project at the University of Waterloo.
Bonnie performs random read and write data to disk. After the utility shows parameters such as the number of processed bytes per CPU second , as well as the percentage of processor utilization. The source code of the benchmark can be found on Google Code .
Based on Bonnie, another set of tools for testing a hard disk is built - Bonnie ++(written in C ++, instead of C). It provides additional benchmark tools. For example, zcav to evaluate the performance of various zones of the HDD. Bonnie ++ is also suitable for testing mail servers and database servers.
The utility was developed by Con Kolivas , an Australian anesthetist who is known for his contributions to the Linux kernel and work on a “ fair processor scheduler. " Interbench helps you configure I / O scheduler and file system settings.
Interbench emulates the behavior of the CPU scheduler when performing interactive tasks. These interactive tasks may include working with sound and video, launching computer games, or simply dragging and dropping a dialog box in the operating system.
The source code, examples and recommendations for setting up the tool can be found in the official repository on GitHub .
Fio
Fio (stands for Flexible I / O Tester) creates disk I / O streams to evaluate the performance of the Linux file system. The utility can also be run on Windows - you need to install the Cygwin command line interface . The configuration guide is in the fio repository on GitHub .
Fio Author - Jens Aksbo ( by Jens Axboe ), responsible for the IO subsystem in the Linux and developer tools blktrace for routing I / O operations. He created fio because he was tired of writing programs to test a specific load manually.
The utility will calculate the IOPS and system bandwidth, and also allow you to evaluate the depth of the I / O queue. The utility works with special files (extension .fio), in which settings and test conditions are written. There are several test options, for example, there is arbitrary writing, reading and rewriting. Here is an example of the contents of the file for the first case:
[global]
name=fio-rand-read
filename=fio-rand-read
rw=randread
bs=4K
direct=0
numjobs=1
time_based=1
runtime=900
Today, fio is used by large companies - they work with the utility in SUSE , Nutanix and IBM .
Hdparm
The utility was written by Canadian developer Mark Lord back in 2005. It is still supported by the author and is part of many popular distributions. The main purpose of hdparm is to configure drive settings. But the tool can be used to conduct simple benchmarks, for example, measuring read speed. To do this, write the command in the console:
$ sudo hdparm -t /dev/sdbThe system will generate a similar response:
Timing buffered disk reads: 242 MB in 3.01 seconds = 80.30 MB/secAs for the drive settings, hdparm allows you to change the cache size, modify the sleep mode and power settings, as well as safely erase data on the SSD. But, as ArchLinux experts warn , a careless change in system parameters can make the data on the disk inaccessible and even damage the drive. Before working with hdparm, it is better to familiarize yourself with the manual - just write the man hdparm command in the console.
S
This is a set of benchmarks for evaluating the performance of I / O systems. The authors of the utility were a development team from the AlgoDev group, which includes employees of the Italian University of Modena and Reggio Emilia .
All benchmarks are bash scripts that evaluate the performance of the storage system - bandwidth, latency, the work of schedulers. For example, the throughput-sync.sh benchmark “bombards” storage systems with read or write requests (in this case, the already mentioned fio utility is used). Here is the code for this script .
Another script - comm_startup_lat.sh - measures the delay in reading data from the disk during the "cold cache" (when it does not have the necessary data). Code alsocan be found in the repository .
Photos - Agê Barros - Unsplash
Bonnie
A utility for evaluating file system performance developed in 1989. Its author was engineer Tim Bray (Tim Bray). With Bonnie's help, he planned to optimize the computing systems involved in the New Oxford English Dictionary project at the University of Waterloo.
Bonnie performs random read and write data to disk. After the utility shows parameters such as the number of processed bytes per CPU second , as well as the percentage of processor utilization. The source code of the benchmark can be found on Google Code .
Based on Bonnie, another set of tools for testing a hard disk is built - Bonnie ++(written in C ++, instead of C). It provides additional benchmark tools. For example, zcav to evaluate the performance of various zones of the HDD. Bonnie ++ is also suitable for testing mail servers and database servers.
Interbench
The utility was developed by Con Kolivas , an Australian anesthetist who is known for his contributions to the Linux kernel and work on a “ fair processor scheduler. " Interbench helps you configure I / O scheduler and file system settings.
Interbench emulates the behavior of the CPU scheduler when performing interactive tasks. These interactive tasks may include working with sound and video, launching computer games, or simply dragging and dropping a dialog box in the operating system.
The source code, examples and recommendations for setting up the tool can be found in the official repository on GitHub .
What we write about in our blogs:Benchmarks for Linux servers: 5 open tools
File backup: how to be safe from data loss
help Gadget examinations at the border: how to act in order not to lose sensitive data?