About open-source implementations of the GOST R 34.11-2012 hash function and their impact on the electronic signature of GOST R 34.10-2012

At one time, the implementation of domestic cryptographic algorithms in the libgcrypt library inspired me very much. It became possible to use these algorithms in Kleopatra and in Kmail and GnuPg in general, to consider the libgcrypt library as an alternative to openssl with the GOST engine. And everything was great until last Friday.

I was asked to verify the electronic signature of GOST R 34.10-2012-256 for a document created in Microsoft Office on MS Windows. And I decided to check it in Kleopatra (I have Linux). And what do you think, the signature turned out to be wrong. Doubts crept in. I decided to check on openssl with GOST-ov engine. The signature has been verified successfully. Urgently re-signed the file in Kleopatra and it did not pass the check on MS Windows. We tried to sign and verify other files, everything was fine. The question was what is the trouble? Since the hash of the document is involved in signing, it was decided to check the calculation of the hash by different programs. First of all, open-source implementations for stribog were involved :

• The famous implementation of Degtyarev
• Libgcrypt implementation
• Libressl
• Openssl engine

And then there was a shock! The hash calculated by the "famous Degtyarev implementation" coincided with the hash calculated in openssl with the GOST endine, but did not match the hash value calculated using libgcrypt and libressl.

How ru_crypt was right when he wrote at the beginning of his article :

I immediately warn you that I did not check the correctness of the implementations.

By the way, the standard on GOST R 34.10-2012 also says that control examples are for reference only. It must be clearly understood that the test cases do not guarantee that different implementations give the same result for all occasions.

To calculate the hash values, the following utilities were used:

1) openssl

$ openssl dgst [–md_gost12_256|-md_gost12_512] 

2) libressl

$libressl dgst [–streebog256|streebog512] 

3) libgcrypt

$gchash [stribog256|stribog512] 

4) The famous implementation of Degtyarev

$gost3411-2012 [-2|-5] 

Here is an interesting thing too: in Latin transcription, stribogs write either stribog or streebog. It would be nice to come to uniformity. And so it seems that these are different functions. Personally, I prefer the first option - stribog.

I needed an arbitrator.

As an arbitrator, it was decided to use the PKCS # 11 token RUTOKEN EDS-2.0, which supports the Russian cryptographic standards GOST R 34.10-2012, GOST R 34.11-2012, VKO GOST R 34.10-2012 (RFC 7836) with a key length of 256 and 512 bits , and is certified by the FSB of Russia as a means of cryptographic information protection (CPSI) and a means of electronic signature.

In addition, the RUTOKEN EDS-2.0 token is widely distributed and many store certificates on it for access to State services and other portals.
To calculate the hash value on the token, we will use the test_digest.tcl script in Tcl :


When does this discrepancy in implementation appear? So far, it has been possible to determine that this discrepancy occurs when calculating the hash of doc files created in MS Office. Moreover, the hash from the first 143 bytes is considered the same, and when calculating the hash from 144 bytes, the values ​​are different.

The first 143 bytes in hexadecimal look like this:

d0cf11e0a1b11ae1000000000000000000000000000000003e000300feff0900060000000000000000000000010000000100000000000000001000002400000001000000feffffff0000000000000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff

Save them in the file Doc1_143_hex.txt.

The first 144 bytes in hexadecimal look like this:

d0cf11e0a1b11ae1000000000000000000000000000000003e000300feff0900060000000000000000000000010000000100000000000000001000002400000001000000feffffff0000000000000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff

Save them in the file Doc1_144_hex.txt.

It is convenient to use the hex2bin.tcl script to translate from hexadecimal to binary form:

#!/usr/bin/tclsh
proc usage {use error} {
    if {$use == 1} {
	puts $error
	puts "Usage:\nhex2bin \n"
    }
}
set countcert [llength $argv]
if { $countcert != 2 } {
    usage 1 "Bad usage!"
    exit
}
set file [lindex $argv 0]
if {![file exists $file]} {
    usage 1 "File $file not exist"
    exit
}
set fd [open $file]
chan configure $fd -translation binary
set cert_user [read $fd]
close $fd
if {$cert_user == "" } {
    usage 1 "Bad file with hex: $file"
    exit
}
set cert_user [binary format H* $cert_user]
set fd [open [lindex $argv 1] w]
chan configure $fd -translation binary
puts -nonewline $fd $cert_user
close $fd

Convert the hexadecimal code to binary:

$./hex2bin Doc1_143_hex.txt Doc1_143.bin
$./hex2bin Doc1_144_hex.txt Doc1_144.bin
$

Now we can check how the hash is calculated by various implementations:
First, we consider the hash for the file Doc1_143, bin:

$ ./openssl  dgst -md_gost12_256 Doc1_143.bin  
md_gost12_256(Doc1_143.bin)= e63bd3edc44f9a03fece4198b690a8ae291b973ae61b2a0f512a9a7479431a63 
$ ./libressl  dgst -streebog256 Doc1_143.bin  
streebog256(Doc1_143.bin)= e63bd3edc44f9a03fece4198b690a8ae291b973ae61b2a0f512a9a7479431a63 
$ ./gchash stribog256 Doc1_143.bin 
e63bd3edc44f9a03fece4198b690a8ae291b973ae61b2a0f512a9a7479431a63  Doc1_143.bin 
$ ./gost3411-2012  -2 Doc1_143.bin  
GOST R 34.11-2012 (Doc1_143.bin) = e63bd3edc44f9a03fece4198b690a8ae291b973ae61b2a0f512a9a7479431a63 
$

The most important moment has come, the moment of verification on a certified cryptographic information protection system:

$ ./test_digest.tcl  stribog256 Doc1_143.bin  
Connect the Token and press Enter 
Loading file for digest: Doc1_143.bin 
digest_hex=
e63bd3edc44f9a03fece4198b690a8ae291b973ae61b2a0f512a9a7479431a63 
$

As you can see, everything ended for good.

Let's see what happens for the Doc1_144.bin file:

$ ./openssl  dgst -md_gost12_256 Doc1_144.bin   
md_gost12_256(Doc1_144.bin)= c766085540caaa8953bfcf7a1ba220619cee50d65dc242f82f23ba4b180b18e0 
$ ./libressl  dgst -streebog256 Doc1_144.bin   
streebog256(Doc1_144.bin)= 3965c99777eb1b64c783496fe950aa6540bc7baa399a3889995145afbdd76250 
$

That's it, the values ​​of the hashes do not match. For the purity of the experiment, we check the remaining implementations:

$ ./gchash_1.7.10 stribog256 Doc1_144.bin 
3965c99777eb1b64c783496fe950aa6540bc7baa399a3889995145afbdd76250  Doc1_144.bin 
$ ./gost3411-2012  -2 Doc1_144.bin   
GOST R 34.11-2012 (Doc1_144.bin) = c766085540caaa8953bfcf7a1ba220619cee50d65dc242f82f23ba4b180b18e0 
$ ./test_digest.tcl  stribog256 Doc1_144.bin  
Connect the Token and press Enter 
Loading file for digest: Doc1_144.bin 
digest_hex= 
c766085540caaa8953bfcf7a1ba220619cee50d65dc242f82f23ba4b180b18e0 
$

The hash calculated by the "famous Degtyarev implementation" matches the hash calculated in openssl with the GOST engine, but does not match the hash value calculated using libgcrypt and libressl.

We get a similar result if we consider the hash stribog512.

There is one conclusion. If you want the GOST R 34.10-2012 electronic signature generated by libressl and libgcrypt (or maybe others) to be compatible with openssl and, most importantly, with cryptographic information protection certified in the FSB certification system of Russia, use verified implementations for computing hashes. I hope this publication avoids many misunderstandings, and the authors of the implementation of stribog in libressl, libgrypt, and possibly others will help to eliminate these discrepancies. Today, I must admit, in the above products, it is actually not GOST R 34.10-2012 that is implemented, but something else. This is a different algorithm. The given test example would probably be nice to include as a test example for GOST R 34.10-2012. And I'm going to edit libgcrypt for Kleopatra and KMail. The legend of Cleopart and Russian cryptography was unfinished.

PS The article was already ready when my colleague said that the discrepancy between the implementations appears when a sufficiently long sequence of 0xFF is encountered. She, this sequence, by the way, is present at the beginning of doc files from MS Office. I checked, the way it is. The file contained 189 bytes.