Does your disassembler work correctly?

Today I want to talk about one interesting complexity of decoding / disassembling IA-32 instructions.

Before reading this article, I recommend contacting the article “Prefixes in the IA-32 Command System” , which describes the general structure of the IA-32 command and the existing prefixes. In this article I will tell you more about the required prefixes ( Eng. Mandatory prefixes) and some of the nuances associated with them.

It all started with reading an article about a language designed to create decoders - GDSL . This article gives some examples already known to me, as well as new features that I have not heard about before. I will tell you about them now.

Some instructions, such as MULSS, MULSDand MULPD(vector multiplication instructions) the same opcode 0x0f 0x59, but different binding prefixes ( 0xf2, 0xf3, and 0x66, respectively). The question arises, what should happen if there are several such prefixes in the instruction code at the same time? It would probably be more logical to determine what the instruction is by the last prefix. But it's not always the case! If the last prefix is 0xf2either0xf3It is considered optional, but 0x66is required only if the prefix 0xf2and 0xf3there is no encoding of this manual. Examples of correct disassembler output for these instructions can be found in the table:

Instruction code	Instruction	Mandatory prefix
66 f3 f2 0f 59 ff	MULSD xmm7, xmm7	f2
66 f2 f3 0f 59 ff	MULSS xmm7, xmm7	f3
66 0f 59 ff	MULPD xmm7, xmm7	66
f2 66 0f 59 ff	MULSD xmm7, xmm7	f2
0f 59 ff	MULPS xmm7, xmm7	-

At some point I questioned the correctness of these statements, but, unfortunately, the documentation gives an ambiguous idea about this issue. After that, it was decided to conduct tests on a real processor, and it turned out that it works that way. This verification was carried out using assembler inserts. An example of one of the tests is shown below:

#include<stdio.h>intmain(){
    double a[2] = {2, 2}, b[2] = {0, 0};
    __asm__ __volatile__ (
         // Copy data from a to xmm7 register"movupd %1, %%xmm7\n"//"mulsd %%xmm7, %%xmm7\n"".byte 0xf2, 0x66, 0x0f, 0x59, 0xff\n"// Copy data from xmm7 register to b"movupd %%xmm7, %0\n"
         :"=m"(*b)
         :"m"(*a)
         :
    ); 
    printf("%lf %lf\n", b[0], b[1]);
    return0;
}

Compiling and running it, you will see the following:

$ gcc -O0 -Wall mulsd.c
$ ./a.out
4.0000002.000000

That is, only the first element of the vector multiplied, while the second remained unchanged, which corresponds to the instructions MULSD, but not MULPD.

In this example, several disassemblers included in the known packages were tested. Few of them coped with their task, which was immediately reported to the developers of these products. A summary of the results is given below:

Product version	Result	The error I reported
Wind River Simics, 4.8	success	-
Xed	success	-
objdump, 2.23	mistake	16083
GNU GDB, 7.5	mistake	16089
nasm, 2.09	mistake	3392269
ODA, 0.2.0	mistake	Email Correspondence
objconv, 2.31	mistake	Email Correspondence
IDA, 6.4 (Evaluation Version)	mistake	Email Correspondence
llvm-objdump, 3.2	mistake	17697

It should be noted that gdb and objdump are part of binutils and use the same library for disassembling. One of the developers of ODA, Anthony DeRosa, said in response to my error message that they use the libopcodes library that comes with binutils. That is, a correction in one place should entail an adjustment of at least three products at once, but, unfortunately, none of the binutils have yet answered me.

Does the disassembler that you use work correctly?