Work with in-memory database using Intel Optane
Today we publish a guest post written in collaboration with Konstantin Boyandin, a system administrator from Novosibirsk.
Konstantin used a server with an Intel Optane SSD to work with an in-memory database (memcached, Redis in appropriate mode, Apache ignite, and so on). The Intel Optane SSD DC P4800X storage device used in the tests was connected as IMDT (using Intel Memory Drive Technology), accessible to the OS and applications as RAM.
Another area of application may be the use of Memory Drive to host a large farm of virtual machines. It is assumed that a large amount of RAM in this case will significantly simplify and reduce the cost of testing software products.
NVMe alone is not new. NVMe in Memory Drive mode is something new. For one of the projects that Konstantin deals with, the effective processing of a large data array is important. The test results are the most encouraging, DRAM-mode using Intel Memory Drive Technology will help significantly reduce processing resources.
We hope that the experience of Konstantin will be interesting to our readers.
Task description
We investigated the Intel Optane performance issue in IMDT mode (we ran Redis and memcached performance tests, delivered in the Phoronix test package, as well as the RAM, ramspeed performance test from the same package). The same tests worked with Intel Optane in IMDT mode, and, for comparison, with Intel Optane media disconnected, on the same server and the same OS.
The server on which the test was conducted is running CentOS 7.4 (64-bit), and on the following hardware components:
- motherboard: Supermicro X10SRi-F
- RAM: 64Gb (8x8), 2133MHz
- Processor: Intel® Xeon® E5-1650 v4 (3.60GHz)
- drives (SATA): Intel SSD DC S3700 (480Gb)
- NVMe drive: Intel Optane SSD DC P4800X (375Gb)
The first phase of the tests was carried out without a reboot.
Briefly about the indicated DBMS:
- Memcached is used to store and quickly retrieve data in key-value format; Originally created to speed up the LiveJournal blog service, in 2003.
- Redis (short for "remote dictionary server", "remote dictionary server") is a networked journaled data warehouse focused on high-speed data processing. In this case, it is configured to work only in RAM without resetting the WAL (change log in case of emergency shutdown) to the disk.
The purpose of the tests: to compare the performance of the mentioned DBMSs that use storage in the RAM, in the case of using the IMDT mode and without the participation of NVMe in general.
Preparation for work
For the tests, the tests ramspeed (operations in RAM), redis (performance Redis), mcperf (performance memcached), running OS CentOS 7.4 (64-bit) were selected.
In both cases, the OS did not use a swap file so as not to distort the test results.
We used the Phoronix test suite 7.4 package (and its included redis, mcperf), as well as the ramspeed 3.5 package.
In the configuration “Intel Optane in IMDT mode” (hereinafter “IMDT”) OS and applications have 320Gb of RAM available.
In a configuration with disabled Intel Optane (hereinafter referred to as “RAM”) OS and applications, 64Gb of RAM is available.
In each case, at least three test runs were made, averaged values were used (arithmetic mean). All services to be tested were rebooted before each test run.
Measurements, case studies and use cases
For testing, the memtest program was used . The memory was randomly taken from the interval of 50-75% of the unused area of the test.
Sequential write to ramfs
The first test was measuring the performance of RAM connected as ramfs (sequential recording), the following commands were used:
IMDT:
mkdir -p /mnt/ram
mount -t ramfs -o size=300G ramfs /mnt/ram
dd if=/dev/zero of=/mnt/ram/256gb.dat bs=4096 count=67108864
Total:
274877906944 bytes (275 GB) copied, 184.756 s, 1.5 GB / s
RAM:
mkdir -p /mnt/ram
mount -t ramfs -o size=62G ramfs /mnt/ram
dd if=/dev/zero of=/mnt/ram/60gb.dat bs=4096 count=15728640
Summary:
64424509440 bytes (64 GB) copied, 18.0382 s, 3.6 GB / s The
ramfs file systems were unmounted before running the rest of the tests.
Redis test
In testing, it was checked exactly how many database commands (SADD, LPUSH, SET, GET) were executed per second.
The higher the value, the better the performance:
- IMDT - in the Intel Optane configuration, 320Gb of RAM is available for applications;
- RAM - Applications have 64Gb of RAM with Intel Optane disabled.
Redis v3.2.10 | IMDT | RAM |
Sadd | 1,748,516.67 | 1,727,879.00 |
LPUSH | 1 343 129.58 | 1,310,077.82 |
SET | 1,562,944.63 | 1 490 965.52 |
Get | 2,026,167.62 | 2 213 292,12 |
Mcperf test
In testing, it was checked how many commands per second are executed for 1 and 5 simultaneous connections (speed is measured in commands per second for each connection).
Memcached v1.4.15 | IMDT | RAM | ||
1 | 5 | 1 | 5 | |
add | 54 931.2 | 55 197.2 | 59 613.9 | 59 174.8 |
delete | 85 361.3 | 83 259.4 | 89 181.6 | 88 235.4 |
append | 57 834.2 | 57 666.1 | 59 252.8 | 60 652.4 |
prepend | 58,441.9 | 58 168.5 | 59 359.3 | 60 115.8 |
replace | 57 231.2 | 57 046.0 | 59 223.5 | 59 233.0 |
get | 86 657.0 | 85 842.8 | 89 468.9 | 89 107.4 |
set | 54 546.9 | 55 570.1 | 58 311.6 | 57 725.3 |
Ramspeed test
Tests were conducted for 4 simultaneously running threads. Where it matters, the size of the used memory block is 4096 bytes. The higher the value (MB per second), the better the performance.
Ramspeed v3.5 (smp) | IMDT | RAM |
Integer & read | 104 548.28 | 112 535.63 |
Integer & write | 33,238.73 | 34 085.68 |
Integer copy | 30,063.08 | 30 105.97 |
Integer scale | 29 916.03 | 29,757.79 |
Integer add | 33,356.14 | 33 192.47 |
Integer triad | 33,235.32 | 33,184.95 |
Floating point & reading | 102 796.84 | 106 513.01 |
Floating point & writing | 33,826.04 | 34 037.25 |
Floating point copy | 30,061.67 | 30,060.73 |
Floating point scale | 30,094.47 | 29 964.92 |
Floating point add | 33 144.85 | 32 979.58 |
Floating point triad | 33 184.84 | 33,149.31 |
Conclusion
Tests have shown that when using Intel Optane in IMDT mode, despite the noticeable difference in the speed of writing to a file (using the standard dd command), in general, the performance of DBMSs that store data in memory is comparable.
The advantage of NVMe in IMDT mode is that, technically, in most servers it is impossible to install the appropriate number of RAM modules to get the same amount of available RAM.
In terms of cost-performance ratio (for Kingston DDR4 2133MHz RAM 8Gb modules), the Intel Optane volume-equivalent media will cost half as much (note that the Optane used in these tests is not optimal in terms of cost-performance).
For both cases of using Memory Drive (that is, a large amount of RAM), there is either no technical ability to install so much memory (there are no such motherboards), or the price of such a configuration is at least 2 times higher (when calculating the equivalent amount of RAM).
We can assume that NVME with Memory Drive support will become cheaper (now it’s a relative rarity and has a high cost), so the gain in cost-performance can be even more tangible.
The use of NMVe in IMDT mode is promising in cases where a large amount of RAM is required (large databases such as Redis / memcached, a large number of virtual machines on one host, and so on).
In the above tests, rather conditional performance was considered. The construction of a simulation of the environment on which you can run tests close to the design, took too much time.
The author of the article is
Konstantin Boyandin, Provide Labs LLC, Novosibirsk.
He is engaged in software development and system administration, as well as security issues.