64-bit code in 2015: what's new in diagnosing possible problems?

64-bit errors are hard enough to detect, since they are akin to a time bomb: they may not be felt right away. PVS-Studio static analyzer makes it easy to find and fix such errors. However, several steps were taken in this direction: recently, 64-bit diagnostics have been more carefully revised, as a result of which their distribution by importance levels has changed. This article will discuss these changes, as well as how this affected the work with the tool and the search for errors. Examples of 64-bit errors from real applications are included.

What is the article about?

To begin with, I would like to introduce specifics on the content. The following topics are covered in the article:

1. Changes in the PVS-Studio analyzer affecting the search for 64-bit errors;
2. Review of 64-bit first-level errors found by the PVS-Studio analyzer, and brief comments on them;
3. Comparison of effectiveness in finding the most important errors using PVS-Studio and Microsoft Visual Studio 2013.

The first paragraph speaks for itself: it will consider the main changes of PVS-Studio regarding the analysis of 64-bit errors, as well as how they will affect the work with the tool.

The second, main section is devoted to the found 64-bit errors in real projects. In addition to code snippets from projects, comments will also be given on them, so you may be able to learn something new for yourself.

The third section compares the effectiveness of the search for these errors with the PVS-Studio static analyzer and the tools of the Microsoft Visual Studio 2013 environment. Moreover, in the case of Visual Studio, both the compiler and the static analyzer were used to search for errors.

Do not forget that only some errors are written out here. In a real project, they will surely be much larger and more diverse. At the end of the article, links are offered that will more fully introduce you to the world of 64-bit errors.

Changes in PVS-Studio related to 64-bit errors

Not so long ago, we carefully looked at 64-bit diagnostics and more accurately distributed them according to importance levels.

Now the distribution of 64-bit errors looks like this:

Level 1. Critical errors that cause harm in any application. An example is storing a pointer in a 32-bit int variable. If you are developing a 64-bit application, you should definitely study and fix the first level warnings.

Level 2. Errors that usually manifest themselves only in applications that process large amounts of data. An example is the use of a variable of type 'int' for indexing a huge array.

Level 3All the rest. As a rule, these warnings are not relevant. However, for some applications, a particular diagnostic may be extremely useful.

Thus, by filtering by 64-bit errors of the first level, you will receive a list of messages pointing to sections of code that are more likely to be erroneous. Do not underestimate these warnings, since the consequences of 64-bit errors can be very different, but clearly unpleasant and often unexpected. It is about them that we will speak.

How hard it would be to detect such errors without such a tool as PVS-Studio, I think, you will understand as you read the article.

64-bit error analysis

Care must be taken to ensure that data types are used correctly. Perhaps, we will begin with this.

LRESULT CSaveDlg::OnGraphNotify(WPARAM wParam, LPARAM lParam)
{
  LONG evCode, evParam1, evParam2;
  while (pME && SUCCEEDED(pME->GetEvent(&evCode, 
    (LONG_PTR*)&evParam1, 
    (LONG_PTR*)&evParam2, 0))) 
  {
    ....
  }
  return 0;
}

Analyzer Warnings:

• V114 Dangerous explicit type pointer conversion: (LONG_PTR *) & evParam1 test.cpp 8
• V114 Dangerous explicit type pointer conversion: (LONG_PTR *) & evParam2 test.cpp 8

In order to understand the essence of the error, you need to look at the types of variables 'evParam1', 'evParam2', as well as the declaration of the 'GetEvent' method:

virtual HRESULT STDMETHODCALLTYPE GetEvent( 
            /* [out] */ __RPC__out long *lEventCode,
            /* [out] */ __RPC__out LONG_PTR *lParam1,
            /* [out] */ __RPC__out LONG_PTR *lParam2,
            /* [in] */ long msTimeout) = 0;

As you can see from the analyzer message, a dangerous explicit type conversion is performed. The fact is that the type 'LONG_PTR' is a ' memsize type ', having a size of 32 bits on Win32 ( ILP32 data model ) and 64 bits on Win64 architecture ( LLP64 data model ). At the same time, the 'LONG' type is 32 bits in both architectures. Since the above types have different sizes on 64-bit architecture, incorrect operation with objects referenced by these pointers is possible.

We continue the topic of dangerous type conversions. Take a look at the following code:

BOOL WINAPI TrackPopupMenu(
  _In_      HMENU hMenu,
  _In_      UINT uFlags,
  _In_      int x,
  _In_      int y,
  _In_      int nReserved,
  _In_      HWND hWnd,
  _In_opt_  const RECT *prcRect
);
struct JABBER_LIST_ITEM
{
  ....
};
INT_PTR CJabberDlgGcJoin::DlgProc(....)
{
  ....
  int res = TrackPopupMenu(
    hMenu, TPM_RETURNCMD, rc.left, rc.bottom, 0, m_hwnd, NULL);
  ....
  if (res) {
    JABBER_LIST_ITEM *item = (JABBER_LIST_ITEM *)res;
    ....
  }
  ....
}

Analyzer Warning: V204 Explicit conversion from 32-bit integer type to pointer type: (JABBER_LIST_ITEM *) res test.cpp 57

To begin with, it would be nice to look at the 'TrackPopupMenu' function used in this code. It returns the identifier of the menu item selected by the user, or a null value in case of an error or if there was no choice. The type 'BOOL' for these purposes is clearly chosen unsuccessfully, but what to do.

The result of this function, as can be seen from the code, is entered into the variable 'res'. If some element was nevertheless selected by the user (res! = 0), then this variable is cast to a pointer to a structure. An interesting approach, but since we are talking about 64-bit errors in the article, let's think about how this code will be executed on 32 and 64-bit architectures, and what could be the problem?

The catch is that on a 32-bit architecture such transformations are acceptable and feasible, since the types 'pointer' and 'BOOL' are the same size. But the rake will make itself felt on 64-bit architecture. In Win64 applications, the above types have different sizes (64 and 32 bits, respectively). A potential error is that the most significant bits in the pointer may be lost.

We continue the review. Code snippet:

static int hash_void_ptr(void *ptr)
{
  int hash;
  int i;
  hash = 0;
  for (i = 0; i < (int)sizeof(ptr) * 8 / TABLE_BITS; i++)
  {
    hash ^= (unsigned long)ptr >> i * 8;
    hash += i * 17;
    hash &= TABLE_MASK;
  }
  return hash;
}

Analyzer Warning: V205 Explicit conversion of pointer type to 32-bit integer type: (unsigned long) ptr test.cpp 76

Let's see what the problem of casting a variable of type 'void *' to type 'unsigned long' in this function is. As already mentioned, these types have different sizes in the LLP64 data model, where the type 'void *' occupies 64 bits and 'unsigned long' is 32 bits. As a result of this, the high bits contained in the variable 'ptr' will be truncated (lost). The value of the variable 'i' increases as iterations go through, as a result of this - a bitwise shift to the right as iterations go through will affect an increasing number of bits. Since the size of the 'ptr' variable has been truncated, from some iteration all the bits contained in it will be filled with 0. As a result of all the above, on the Win64-based applications, the 'hash' will not be compiled correctly. By filling 'hash' with zeros,, that is, obtaining the same hashes for different input data (in this case, pointers). As a result, this can lead to inefficient program operation. If casting to the 'memsize type' were performed, truncation would not have occurred, and then the shift (and therefore the compilation of the hash) would be carried out correctly.

Let's look at the following code:

class CValueList : public CListCtrl
{
  ....
  public:
    BOOL SortItems(_In_ PFNLVCOMPARE pfnCompare, 
      _In_ DWORD_PTR dwData);
  ....
}; 
void CLastValuesView::OnListViewColumnClick(....)
{
  ....
  m_wndListCtrl.SortItems(CompareItems, (DWORD)this);
  ....
}

Analyzer Warning: V220 Suspicious sequence of types castings: memsize -> 32-bit integer -> memsize. The value being cast: 'this'. test.cpp 87

Diagnostics V220 signals a double dangerous data conversion. At the beginning, the variable of the 'memsize type' turns into a 32-bit value, and then immediately extends back to the 'memsize type'. In fact, this means that the values ​​of the most significant bits will be “cut off”. This is almost always a mistake.

We continue to expand on the topic of dangerous transformations:

#define YAHOO_LOGINID "yahoo_id"
DWORD_PTR __cdecl CYahooProto::GetCaps(int type, HANDLE /*hContact*/)
{
  int ret = 0;
  switch (type)
  {
    ....
  case PFLAG_UNIQUEIDSETTING:
    ret = (DWORD_PTR)YAHOO_LOGINID;
    break;
    ....
  }
  return ret;
}

Analyzer Warning: V221 Suspicious sequence of types castings: pointer -> memsize -> 32-bit integer. The value being cast: '"yahoo_id"'. test.cpp 99

I noticed a tendency that with each example of transformations it becomes more and more. There are as many as 3. And 2 of them are dangerous, for the same reasons as all of the above. Since 'YAHOO_LOGINID' is a string literal, its type is 'const char *', which in 64-bit architecture is the same size as the type 'DWORD_PTR', so explicit conversion is correct. But then bad things begin. The type 'DWORD_PTR' is implicitly cast to an integer 32-bit. But that's not all. Since the result returned by the function is of type 'DWORD_PTR', another implicit conversion will be performed, this time back to the 'memsize type'. Obviously, in this case, the use of the returned value is at your own risk.

I want to note,

warning C4244: '=': conversion from 'DWORD_PTR' to 'int', possible loss of data

The actual question will be possible here: why is the warning issued by Visual Studio 2013 shown only in this example? The question is fair, but be patient, this will be written below.

In the meantime, we continue to consider errors. Consider the following code containing a class hierarchy:

class CWnd : public CCmdTarget
{
  ....
  virtual void WinHelp(DWORD_PTR dwData, UINT nCmd = HELP_CONTEXT);
  ....
};
class CFrameWnd : public CWnd
{
  ....
};
class CFrameWndEx : public CFrameWnd
{
  ....
  virtual void WinHelp(DWORD dwData, UINT nCmd = HELP_CONTEXT);
  ....
};

Analyzer Warning: V301 Unexpected function overloading behavior. See first argument of function 'WinHelpA' in derived class 'CFrameWndEx' and base class 'CWnd'. test.cpp 122

An example is interesting because it is taken from a report when checking Visual C ++ 2012 libraries. As you can see, even Visual C ++ developers make 64-bit errors.

Enough details about this error are written in the corresponding article.. Here I wanted to explain the essence in brief. On a 32-bit architecture, this code will be correctly processed, since the types 'DWORD' and 'DWORD_PTR' are the same size, in the successor class this function will be overridden, and the code will execute correctly. But the pitfall is still here and it will let you know about yourself on 64-bit architecture. Since in this case the types 'DWORD' and 'DWORD_PTR' will have different sizes, the polymorphism will be destroyed. We will have 2 different functions on hand, which goes against what was meant.

And the last example:

void CSymEngine::GetMemInfo(CMemInfo& rMemInfo)
{
  MEMORYSTATUS ms;
  GlobalMemoryStatus(&ms);
  _ultot_s(ms.dwMemoryLoad, rMemInfo.m_szMemoryLoad,   
    countof(rMemInfo.m_szMemoryLoad), 10);
  ....
}

Analyzer Warning: V303 The function 'GlobalMemoryStatus' is deprecated in the Win64 system. It is safer to use the 'GlobalMemoryStatusEx' function. test.cpp 130

In principle, no special explanation is required, everything is clear from the analyzer message. You must use the 'GlobalMemoryStatusEx' function, because the 'GlobalMemoryStatus' function may not work correctly on a 64-bit architecture. More details on this can be found on the MSDN portal in the description of the corresponding function .

Note.

Please note that all the errors mentioned may occur in the most common application software. For them to arise, the program does not have to work with a large amount of memory. And that is why the diagnostics that detect these errors belong to the first level.

What will Visual Studio 2013 tell us?

Compiler warnings

Before talking about the results of checking the Visual Studio 2013 static analyzer, I would like to dwell on compiler warnings. Attentive readers probably noticed that only 1 such warning was given in the text. What is the matter, you ask? But the fact is that there were simply no more warnings, somehow related to 64-bit errors. And this is at the 3rd level of issuing warnings.

But it’s worth compiling this example with all warnings turned on (EnableAllWarnings), how do we get ...



Moreover, quite unexpectedly, warnings lead to header files (for example, winnt.h). If you are not too lazy and find those related to the project in this heap of warnings, you can still extract something interesting, for example:

warning C4312: 'type cast': conversion from 'int' to 'JABBER_LIST_ITEM *' of greater size

warning C4311: 'type cast': pointer truncation from 'void *' to 'unsigned long'

warning C4311: 'type cast': pointer truncation from 'CLastValuesView * const' to 'DWORD'

warning C4263: 'void CFrameWndEx :: WinHelpA (DWORD , UINT) ': member function does not override any base class virtual member function

In general, the compiler issued 10 warnings in a file with these examples. Only 3 warnings from this list clearly indicate 64-bit errors (compiler warnings C4311 and C4312). Among these warnings are those that indicate a narrowing type conversion (C4244) or that the virtual function will not be overridden (C4263). These warnings also indirectly indicate 64-bit errors.

As a result, having excluded warnings repeating each other in one way or another, we get 5 warnings regarding the 64-bit errors we are considering.

As you can see, the Visual Studio compiler was unable to detect all 64-bit errors. I remind you that the PVS-Studio analyzer found 9 errors of the first level in the same file.

“But what about the static analyzer built into Visual Studio 2013?” - you ask. Maybe he did better and found more mistakes? Let's get a look.

Static Analyzer included with Visual Studio 2013

The result of checking these examples with a static analyzer built into Visual Studio 2013 was 3 warnings:

• C6255 Unprotected use of alloca
  _alloca indicates failure by raising a stack overflow exception. Consider using _malloca instead.
  64BitsErrors - test.cpp (Line 58);
• C6384 Pointer size division
  Dividing sizeof a pointer by another value.
  64BitsErrors - test.cpp (Line 72);
• C28159 Consider using another function instead
  Consider using 'GlobalMemoryStatusEx' instead of 'GlobalMemoryStatus'. Reason: Deprecated. See MSDN for details
  64BitsErrors - test.cpp (Line 128);

But we are looking at 64-bit errors, right? How many errors from this list apply to 64-bit? Only the latter (using a function that can return incorrect results).

It turns out that the Visual Studio 2013 static analyzer found 1 64-bit error against 9 found by the PVS-Studio analyzer. Impressive, isn't it? Imagine what the difference will be in large projects.

And now I want to remind once again that in terms of functionality in terms of error detection, the static code analyzers built into the Visual Studio 2013 and Visual Studio 2015 environments are the same (which is described in more detail in the corresponding note ).

What is the result?

Most clearly will reflect the results of the verification of code examples in the form of a table.



As you can see from the table, 9 64-bit errors were detected using PVS-Studio, and 6 by common means of Microsoft Visual Studio 2013. Perhaps you can say that there is not such a big difference. I do not agree. Let's figure out why:

• We talked only about the most critical 64-bit errors. Even 3 missed errors, that's a lot. And if we take more rare errors, for which the PVS-Studio analyzer generates warnings of the 2nd and 3rd level, then it can find much more than Visual Studio. Some idea of ​​this can give this article . The article is a little outdated, now the gap will be even greater.
• Calling for help the compiler with the 4th warning level turned on is not always possible. But both at the fourth and third levels of warnings we get only 2 warnings (using the analyzer and the compiler) related to 64-bit errors. Not too impressive result.
• If you set the "/ Wall" flag, we get a bunch of warnings that are not related to the project. In practice, using "/ Wall" will be difficult. You can turn on some warnings separately, but there will still be a lot of excess noise.

As can be seen from the above, in order to detect 64-bit errors found by Microsoft Visual Studio 2013, you need to do a certain amount of work. Now imagine how much it will increase, be it a real, really big project.

What about PVS-Studio? Run diagnostics, set filtering by 64-bit errors and the necessary warnings with a few mouse clicks, we get the result.

Conclusion

I hope that I was able to show that porting applications to 64-bit architecture is associated with a number of difficulties. Errors like those described in this article are easy to make, but extremely difficult to find. Add to this the fact that not all such errors are detected by Microsoft Visual Studio 2013, and to find those you need to do a certain amount of work. At the same time, the PVS-Studio static analyzer coped with the task, showing a decent result. Moreover, the process of searching and filtering errors is simpler and more convenient. You have to admit that in really large projects without such a tool you would have to be tight, so in such cases a good static analyzer is simply necessary.

Developing a 64-bit application? Downloadtrial of PVS-Studio, check your project and see how many 64-bit messages of the first level you will have. If a few still show up, please correct them, and make this world a little better.

Additional materials

As promised, I give a list of additional materials on the topic of 64-bit errors:

• Terminology. 64-bit error ;
• Andrey Karpov. A collection of examples of 64-bit errors in real programs .
• Andrey Karpov. C ++ 11 and 64-bit errors .
• Andrey Karpov, Evgeny Ryzhkov. Lessons for developing 64-bit C / C ++ applications .

This article is in English.

If you want to share this article with an English-speaking audience, then please use the link to the translation: Sergey Vasiliev. 64-Bit Code in 2015: New in the Diagnostics of Possible Issues .