The Second Coming of GOST 28147-89: Honest Tests
About ten years ago, symmetric cryptography based on GOST 28147-89 ceased to satisfy the needs of hardware platforms in terms of speed parameters. The cryptographic conversion rates provided by the algorithms implemented on general-purpose registers of processors did not keep up with the exchange rates of information in networks and on disk drives.
On the other hand (American), AES-256 appeared, which showed much better speed parameters with the same degree of cryptographic stability.
In this situation, the FSB Center 8 began work on a new block cipher, which later received the name "Grasshopper" from the initial letters of the authors' names.
Initially, it was a futile undertaking, since the logic of the AES cipher was repeated, but if it was hardware accelerated in Intel and AMD processors, then Grasshopper could not have such hardware acceleration on these processors.
So Grasshopper, this is a classic example of budget money thrown into the wind and not small ...
But there was another option for speeding up cryptographic operations; unfortunately, it did not receive official support for specific reasons. This option involved the development of implementation algorithms for GOST 28147-89 for multithreading and the refinement of the standard itself to the requirements of multithreading.
Multithreading is based on three new methods for implementing GOST 28147-89 in x86-64 architecture processors.
The first is the pipeline operation of the processor in encryption mode. The second is the implementation of the replacement block on the SSE / AVX processor commands. The third is the use of specialized XMM / YMM / ZMM registers with a capacity of 16/32/64 bytes.
In total, these new methods allowed increasing encryption performance in accordance with GOST 28147-89 by at least an order of magnitude and providing high-speed encryption parameters no worse than the American AES-256, which uses a special crypto accelerator in Intel / AMD processors.
But, as they say, God marks the assault, and time puts everything in its rightful place.
At present, the Grasshopper code is perceived as exotic and has no practical use due to the low conversion speed and dubious reputation; instead, manufacturers began to use multi-threaded encryption according to GOST 28147-89. But they try not to talk about this, because they know that they violate copyrights.
Encryption acceleration methods are patented.
Largely because of this, there were no honest tests for the performance of different encryption systems, and all kinds of fabulous and absolutely unsubstantiated numbers that had nothing to do with reality walked in network publications and forums.
Therefore, there was a stable myth about the superiority of AES-256 over GOST 28147-89 in speed parameters almost ten times ...
It is time to conduct honest testing.
Honest tests
. Articles devoted to cryptographic transformations provide various "fantastic" data on the speed of cryptographic functions, there is a lot of cunning in these figures. We will be honest, no "synthetic" tests, all "for real."
Encryption is essentially a service background process. The crypto procedure works in the background of much more important tasks, and the situation with 100% processor utilization with encryption is exotic. Therefore, we will not accelerate cryptofunction by increasing the priority and the number of processor cores used, we will limit the maximum processor load with a cryptofunction to a level of 15%.
In the demo version of the FastSecurityBoxs program ( own craft, for those interested, please write in the comments) the creation of dumps of encrypted disks was carried out on four Skylake nuclear processors with a frequency of 2.6 GHz, hyper-trading was activated (a total of 8 logical cores). The crypto-procedure runs on one logical core (out of 8 available); accordingly, the CPU load it creates does not exceed 12-15 percent, which corresponds to the actual work of the background task. Two SSD disks were used for copying; the speed of reading / writing to them in the file system mode is approximately 450-500 MB / sec. on a cleaned device after running TRIM.
Here is a clean copy without cryptography:
Reading disk sectors ( ProjectFK.exe) occupy 5%, writing to a file (System) takes 2% of the processor time at a speed of 449 MB / s. We will remember these figures, when we turn on cryptography, the costs of cryptoconversion will be added to them, accordingly, it will be possible to estimate the costs of the cryptoconversion of the processor load itself.
Now, by creating a dump, enable cryptography. First, encryption in 8 streams according to GOST 28147-89:
Screenshot of creating a disk backup using crypto conversion strictly according to GOST in 8 streams.
The speed of a really created cryptographic dump is 190 MB / sec. The speed is limited by crypto conversion, since the CPU load created by ProjectFK.exe is 12%, in our case this is the limit on the load of the logical processor core. SSD disks can work much faster, but they are hindered by the restriction on using only half of one physical processor core in the cryptographic procedure.
7% of the processor load is spent on crypto conversion in ProjectFK.
Now encryption in 16 streams according to GOST 28147-89:
Screenshot of creating a backup copy of disks using crypto conversion strictly according to GOST in 16 streams. This mode works effectively only on Intel processors of Skylake generation and higher, but this will be discussed later.
Speed increased to 334 mByte / s. CPU utilization is 10.5%, crypto conversion creates a further limitation here, but the load on the logical processor core is reduced.
5.5% of the processor load is spent on crypto conversion in ProjectFK.
Well, what can American AES encryption provide? Take the most advanced solution using this cipher - Bitlocker. The encryption function is built into his kernel of the OS and is optimized as much as possible, this is not a user application like FastSecurityBoxs, so he has a significant handicap ...
Here's what Bitlocker (AES-128 for 10 rounds) gives when dumping on an encrypted drive is in progress:
Here, ProjectFK.exe only creates a dump, this dump is encrypted with a bitlocker directly in the System process that writes it to disk, so you need to summarize the processor loads created by both of these processes. The load is 12.5 percent of the processor time at an encrypted dump rate of 392 MB / s.
5% of processor load is spent on crypto conversion in the System process.
Not bad at all, just keep in mind that AES-128 is no match for GOST 28147-89, they are from different weight categories of cryptographic strength.
The "dry residue" of tests
Encryption according to GOST 28147-89 in the background (5-7% of the CPU load), at a speed of 200 MB / s, is already fantastic, these parameters are not a single system in real applications, with hardware-accelerated AES-256 cryptography for 14 rounds, not condition.
In AES-128 mode, it is possible to work a little faster on 10 rounds, but the cryptographic strength of this algorithm is much lower than the cryptographic strength of GOST 28147-89 due to half the key size.
We state the obvious. The cryptographic procedure according to strict GOST 28147-89 in a multi-threaded version can be performed in the background at speeds of 200-400 MB / sec., Loading the processor by only 5-7 percent. But nothing prevents you from raising speeds even higher, increasing the processor load.
Certified by the FSB cryptography tool according to strict GOST 28147-89 can in multi-threaded mode provide background encryption of modern SSD and HDD drives on the SATA interface with a processor loading of this operation of no more than 7%. This allows you to use GOST 28147-89 in programs such as Bitlocker even more efficiently than the current American standard AES-128, but with much more "strong" cryptographic protection ...
For disks on the NVMe interface that are already working at speeds of 2-3GB / sec. and for 10G / 10G + networks you need to increase the speed by at least two times.
There are two ways. The first passive path has already been tested when we switched from 8 threads to 16 threads when implementing GOST 28147-89. You can wait for the introduction of the AVX-512 instruction set into the processor. These commands operate with registers of 64 bytes in size and can provide multi-threaded execution according to strict GOST 28147-89 immediately in 32 threads. Which will automatically double the performance of the crypto function.
There is a small “but” here, processors with support for AVX-512 will appear only next year, and the AVX-512 commands will be emulated at first by firmware, which means they will be executed very slowly. So slow that using them in real programs is pointless.
This we have already seen on the example of the introduction of AVX2 commands, 3 years have passed between the appearance of this set of commands and the translation of its execution to the hardware cycle. Only in 2016 did Intel introduce hardware support for AVX2 commands on Skylake generation processors.
It will take about 3 years to wait for Intel to transfer the AVX-512 to the hardware implementation ... The
second way is active, it consists in developing a new encryption algorithm, originally designed for multithreading, because now only a small part of the parallel computing potential is used.
This is what we’ll do next, do not wait for 3 years ... So expect in the near future the third advent of GOST 28147-89.
On the other hand (American), AES-256 appeared, which showed much better speed parameters with the same degree of cryptographic stability.
In this situation, the FSB Center 8 began work on a new block cipher, which later received the name "Grasshopper" from the initial letters of the authors' names.
Initially, it was a futile undertaking, since the logic of the AES cipher was repeated, but if it was hardware accelerated in Intel and AMD processors, then Grasshopper could not have such hardware acceleration on these processors.
So Grasshopper, this is a classic example of budget money thrown into the wind and not small ...
But there was another option for speeding up cryptographic operations; unfortunately, it did not receive official support for specific reasons. This option involved the development of implementation algorithms for GOST 28147-89 for multithreading and the refinement of the standard itself to the requirements of multithreading.
Multithreading is based on three new methods for implementing GOST 28147-89 in x86-64 architecture processors.
The first is the pipeline operation of the processor in encryption mode. The second is the implementation of the replacement block on the SSE / AVX processor commands. The third is the use of specialized XMM / YMM / ZMM registers with a capacity of 16/32/64 bytes.
In total, these new methods allowed increasing encryption performance in accordance with GOST 28147-89 by at least an order of magnitude and providing high-speed encryption parameters no worse than the American AES-256, which uses a special crypto accelerator in Intel / AMD processors.
But, as they say, God marks the assault, and time puts everything in its rightful place.
At present, the Grasshopper code is perceived as exotic and has no practical use due to the low conversion speed and dubious reputation; instead, manufacturers began to use multi-threaded encryption according to GOST 28147-89. But they try not to talk about this, because they know that they violate copyrights.
Encryption acceleration methods are patented.
Largely because of this, there were no honest tests for the performance of different encryption systems, and all kinds of fabulous and absolutely unsubstantiated numbers that had nothing to do with reality walked in network publications and forums.
Therefore, there was a stable myth about the superiority of AES-256 over GOST 28147-89 in speed parameters almost ten times ...
It is time to conduct honest testing.
Honest tests
. Articles devoted to cryptographic transformations provide various "fantastic" data on the speed of cryptographic functions, there is a lot of cunning in these figures. We will be honest, no "synthetic" tests, all "for real."
Encryption is essentially a service background process. The crypto procedure works in the background of much more important tasks, and the situation with 100% processor utilization with encryption is exotic. Therefore, we will not accelerate cryptofunction by increasing the priority and the number of processor cores used, we will limit the maximum processor load with a cryptofunction to a level of 15%.
In the demo version of the FastSecurityBoxs program ( own craft, for those interested, please write in the comments) the creation of dumps of encrypted disks was carried out on four Skylake nuclear processors with a frequency of 2.6 GHz, hyper-trading was activated (a total of 8 logical cores). The crypto-procedure runs on one logical core (out of 8 available); accordingly, the CPU load it creates does not exceed 12-15 percent, which corresponds to the actual work of the background task. Two SSD disks were used for copying; the speed of reading / writing to them in the file system mode is approximately 450-500 MB / sec. on a cleaned device after running TRIM.
Here is a clean copy without cryptography:
Reading disk sectors ( ProjectFK.exe) occupy 5%, writing to a file (System) takes 2% of the processor time at a speed of 449 MB / s. We will remember these figures, when we turn on cryptography, the costs of cryptoconversion will be added to them, accordingly, it will be possible to estimate the costs of the cryptoconversion of the processor load itself.
Now, by creating a dump, enable cryptography. First, encryption in 8 streams according to GOST 28147-89:
Screenshot of creating a disk backup using crypto conversion strictly according to GOST in 8 streams.
The speed of a really created cryptographic dump is 190 MB / sec. The speed is limited by crypto conversion, since the CPU load created by ProjectFK.exe is 12%, in our case this is the limit on the load of the logical processor core. SSD disks can work much faster, but they are hindered by the restriction on using only half of one physical processor core in the cryptographic procedure.
7% of the processor load is spent on crypto conversion in ProjectFK.
Now encryption in 16 streams according to GOST 28147-89:
Screenshot of creating a backup copy of disks using crypto conversion strictly according to GOST in 16 streams. This mode works effectively only on Intel processors of Skylake generation and higher, but this will be discussed later.
Speed increased to 334 mByte / s. CPU utilization is 10.5%, crypto conversion creates a further limitation here, but the load on the logical processor core is reduced.
5.5% of the processor load is spent on crypto conversion in ProjectFK.
Well, what can American AES encryption provide? Take the most advanced solution using this cipher - Bitlocker. The encryption function is built into his kernel of the OS and is optimized as much as possible, this is not a user application like FastSecurityBoxs, so he has a significant handicap ...
Here's what Bitlocker (AES-128 for 10 rounds) gives when dumping on an encrypted drive is in progress:
Here, ProjectFK.exe only creates a dump, this dump is encrypted with a bitlocker directly in the System process that writes it to disk, so you need to summarize the processor loads created by both of these processes. The load is 12.5 percent of the processor time at an encrypted dump rate of 392 MB / s.
5% of processor load is spent on crypto conversion in the System process.
Not bad at all, just keep in mind that AES-128 is no match for GOST 28147-89, they are from different weight categories of cryptographic strength.
The "dry residue" of tests
Encryption according to GOST 28147-89 in the background (5-7% of the CPU load), at a speed of 200 MB / s, is already fantastic, these parameters are not a single system in real applications, with hardware-accelerated AES-256 cryptography for 14 rounds, not condition.
In AES-128 mode, it is possible to work a little faster on 10 rounds, but the cryptographic strength of this algorithm is much lower than the cryptographic strength of GOST 28147-89 due to half the key size.
We state the obvious. The cryptographic procedure according to strict GOST 28147-89 in a multi-threaded version can be performed in the background at speeds of 200-400 MB / sec., Loading the processor by only 5-7 percent. But nothing prevents you from raising speeds even higher, increasing the processor load.
Certified by the FSB cryptography tool according to strict GOST 28147-89 can in multi-threaded mode provide background encryption of modern SSD and HDD drives on the SATA interface with a processor loading of this operation of no more than 7%. This allows you to use GOST 28147-89 in programs such as Bitlocker even more efficiently than the current American standard AES-128, but with much more "strong" cryptographic protection ...
For disks on the NVMe interface that are already working at speeds of 2-3GB / sec. and for 10G / 10G + networks you need to increase the speed by at least two times.
There are two ways. The first passive path has already been tested when we switched from 8 threads to 16 threads when implementing GOST 28147-89. You can wait for the introduction of the AVX-512 instruction set into the processor. These commands operate with registers of 64 bytes in size and can provide multi-threaded execution according to strict GOST 28147-89 immediately in 32 threads. Which will automatically double the performance of the crypto function.
There is a small “but” here, processors with support for AVX-512 will appear only next year, and the AVX-512 commands will be emulated at first by firmware, which means they will be executed very slowly. So slow that using them in real programs is pointless.
This we have already seen on the example of the introduction of AVX2 commands, 3 years have passed between the appearance of this set of commands and the translation of its execution to the hardware cycle. Only in 2016 did Intel introduce hardware support for AVX2 commands on Skylake generation processors.
It will take about 3 years to wait for Intel to transfer the AVX-512 to the hardware implementation ... The
second way is active, it consists in developing a new encryption algorithm, originally designed for multithreading, because now only a small part of the parallel computing potential is used.
This is what we’ll do next, do not wait for 3 years ... So expect in the near future the third advent of GOST 28147-89.
