As in PVS-Studio we have been solving one engineering problem for several years

At first I wanted to call this note “How PVS-Studio allows CHEAP to introduce static code analysis into the development process”, but I did not dare because of the ambiguous interpretation of the word “cheap”. Therefore, I will talk about one engineering problem that we constantly had to solve in order for people to use our product. Looking ahead, I’ll say that, as it seems to me, we have solved it.



So, having developed the first version of a static code analyzer in early 2007 (which was then called Viva64 and detected 64-bit errors), we were faced with the problem of implementing a tool for clients. Our client is a company with at least several dozen developers, and at least several hundred thousand lines of code. Any static analyzer for such code will generate a bunch of warnings. For example, with our tool we received up to several thousand messages per project.

Yes, of course, there is a problem with false positives of the analyzer. However, any analyzer has false positives and there is no escape from them. The question arose - what to do with a large number of messages that the user receives? That is, the problem looks like this. A potential user downloads the program (trial), launches it and receives ten thousand messages. Naturally, this saddens him, then uninstall and we lost the client.

The first thing we did was immediately remove duplicate messages. The analyzer checks C / C ++ projects and it happens that an error in the .h file may be issued when checking several .cpp files that use it. This is not duplicated here. Then we added the ability to filter analysis results (and constantly improved them): filtering by error code, message text, the ability not to scan files by mask, etc. All this allowed to significantly reduce the number of messages, but only after configuration. For the first time, the user still received a mountain of messages. Thus, message filtering is an important tool, but it did not solve the original problem - the difficulty of introducing the tool into the development process.

Then a new mechanism, Mark as False Alarm, appeared in the analyzer. This is an insertion into the code of comments of a special kind (// - V112) to suppress analyzer messages. Having marked up the code in this way in the future, you can receive messages about problems only in those code fragments where there is no such markup yet. Ideally, this is just new code. Although the problem of introducing the analyzer into the team development process has become a bit simpler, anyway, it is necessary that several people from the team first mark up the code, eliminating obviously garbage messages.

The next step to solving the implementation problem was the ability to check only files modified over the past few days. This is much closer to starting to benefit from static analysis right away. But the problem still remains. After all, the user does not know about this feature, and if you enable it by default, it will not be clear why so few files are checked. But, again, the direction seemed right to us.

So we went ahead and made a new super feature called “Incremental Analysis after Build”. Now the analyzer starts immediately after compilation and checks only those files that were “affected” by user edits. Unlike checking files in a few days (when edits of the development team could be checked), now the user sees ONLY errors in the code that he directly touches.

The programmer will no longer worry about the large amount of code that he does not touch. Perhaps this code is more than 5 years old. It is practically not modified and most of the defects in it have already been fixed. This code does not have to rush to check first and the analyzer does not do this. The programmer will see warnings only in the latest code. And if he has extra time, he will always be able to check the entire project by looking at the most rarely visited places.

Yes, there are still false positives from the analyzer. Yes, all the same, filters have not lost their relevance. But another important thing. We managed to reduce the cost of INTRODUCTION (the cost of human efforts to commission) of a static analyzer to zero. That is, now a person downloads a static analyzer, installs it, and IMMEDIATELY starts to benefit from it without any additional actions.

However, the finishing touch was not enough. Everything would be fine, but the fact that the analyzer found errors was very poorly noticeable. The color change of the PVS-Studio window icon (as we did at first) is not so noticeable, and in Visual Studio 2005 this will not work at all. The solution was a pop-up notification.

Of course, we all don’t like all sorts of annoying pop-up notifications. But in this case, it will be clearly useful to the programmer, and will rarely appear, provided that he does not abuse code errors.

Thus, the engineering task of introducing static analysis into the development process was solved. Therefore, in PVS-Studio 4.34, Incremental Analysis after Build mode will be enabled by default.

The conclusion is simple. Now, developers can not be afraid of the difficulties of implementing static code analysis, just download PVS-Studio, install it and look at the errors that will be detected in the new code that has just been developed.