New service: regular audit of C / C ++ code
Until recently, we were exclusively engaged in the development and sale of the PVS-Studio product. Then we thought and decided to offer a new service: regular code audit. I’ll tell you about her. The article is intended for managers and team leaders. In order not to spoil your mood and not minus, I ask programmers not to read the article.
The trouble of static analysis as a product
Static analysis tools have several drawbacks that cause psychological discomfort and real drawbacks. Many managers understand everything perfectly and are ready to put up with inconveniences, since static analysis as a whole is beneficial. Unfortunately, the introduction of static analysis tools can be met negatively by programmers. Since in some cases it is difficult to sell and implement the regular use of static analysis, we decided to try to sell not a tool, but a service.
Let us examine at the beginning of the reason why the static analysis tool can be negatively perceived by developers.
- Trying the static analysis tool, the programmer sees few real errors and many false positives ( explanations ). The developer does not have time and does not want to go into details of the static analysis methodology. He has a lot of current affairs. Therefore, he seeks to quickly make a decision that the tool is useless and return to previous affairs. Fortunately, this is not so scary. It can be explained to the programmer that the main advantage of static analysis is regular use, not one-time checks. A good analogy is compiler warnings. He does not include them a couple of times a year, but works with them constantly.
- Nevertheless, even during trial runs, PVS-Studio manages to find real errors in the code. Because of this, unfortunately, it can turn into a blood enemy. Some are completely unable to bear criticism. Even if this is the output of the program. Such a conclusion can be made, both by logical conclusions, and by the negative that sometimes falls on us. From the outside it is difficult to understand. If the code contains errors, good for the company. But for the person who made a mistake, this is unpleasant. He will not show, but will unconsciously try to avoid a situation where someone else finds a mistake, and not he. This can be achieved by making static analysis no longer used. Irrational decision from the point of view of teamwork, but the person feels more comfortable. It is precisely because of this sabotage that some heads of development departments take on the role of analyzing diagnostic warnings. In such a situation, the programmer has nowhere to go. Programmers after reading this paragraph will be outraged. That is why I did not recommend this text to them for reading :).
- Static analysis seems to be time-consuming. The time that a person regularly spends on setting up the analyzer and regular work with it is noticeable. But it is completely incomprehensible how much time and effort will be saved by the quickly eliminated typo found during code analysis.
- Let's move from psychological discomfort to practical inconvenience. Often, programmers do not understand what a static code analyzer wants to tell them. Unfortunately, PVS-Studio does not have artificial intelligence to write a story about why there might be a mistake. This is a big real problem. Programmers are not to blame. It takes time and experience to learn to understand the analyzer messages. As a result, the capabilities of PVS-Studio remain underestimated. I see it well from communicating in support. People write that they met one or another false positive and offer to fix it. In the process of clarification, it is often found out that this is a real mistake, and the developers did not notice it or even could not assume that such a situation could be. It is sad. After all, we are talking about those who wrote to us. You can make an assumption that many real mistakes are not perceived and marked as a false positive. By the way, this trouble is not only with static analyzers. Even compiler behavior is interpreted incorrectly (examples ).
- False positives. Unfortunately, they were, are and will be. This is the essence of the static code analysis methodology. False positives can be fought by improving the code, or by suppressing them using the settings. In any case, this is work and it is not interesting to deal with. Plus, it again takes time to gain experience. False positives spoil the impression of the product. But selling PVS-Studio as a product, we cannot do anything with it.
As you can see, there are several objective and subjective reasons that interfere with how to evaluate static analysis and use it regularly.
We decided to offer a workaround for managers, which will allow us to quickly and efficiently begin to regularly use the methodology of static code analysis in our work.
New proposal: audit
I will describe how the audit process and its advantages are built.
- We sign an NDA that will allow us to pass the code.
- We conclude a contract. We focus on annual contracts. However, one always wants to try at the beginning. Therefore, the option of concluding a three-month trial contract is possible. For less than 3 months, making a contract is pointless. We must learn to compile your project, learn how to automatically download updates, learn how to check it, configure the analyzer, make your admins not ban our letters, and so on. While all these issues will be settled, a month will pass. And you need at least 2 more months to evaluate the benefits of cooperation.
- At the first stage, our employee will look through all the general diagnostic messages issued by the analyzer and provide a report on the defects found. After which the daily audit of the new code will already begin.
- A dedicated experienced employee is involved in your project daily. If something breaks, it repairs the project assembly. And most importantly, it scans the results of the analysis. If he finds a suspicious place in the code, then he notifies you. At the same time, he can notify the name of the employee who laid the code leading to the warning.
- In case the error is not obvious, our employee explains it and also provides advice on how best to fix it.
- You do not have to wonder if there is a Linux version of PVS-Studio or not. It is our task to adapt the analyzer for your operating system and compiler. If you do not use something exotic for development, we will be able to check your code.
- Employees cannot consciously or unconsciously sabotage regular code checks. They will have nowhere to go.
- We get money for looking for mistakes. This means that they are interested in carefully treating analyzer warnings. The programmer is inclined to call the error not an error, or just too lazy to change obviously bad code. In the case of an audit, everything will be visible. We get an advantage: the errors that PVS-Studio found are not hushed up, which confirms the value of the tool. You get an advantage: careful analysis of diagnostic messages.
- Your programmers do not work with false messages. In this case, you do not need to make any changes to the code to eliminate false positives. You only get information about real problems in the code or obviously bad code. False positives are our problem that we can handle. A programmer’s dream comes true - you get a tool that will almost never give false positives!
- Information about the defects found comes to the head and specifically to the programmer who wrote the dangerous code. The programmer will be able to easily correct his own code. The manager will see the big picture.
- We well understand how a static analyzer works. Therefore, we will be able to supply the analyzer diagnostic messages with additional comments that will help a person understand the reason for the warning. This is a very important point. A true error will not be inattentively flagged as a false positive.
- If desired, the specialist conducting the audit can give general recommendations for improving the code. It will also help improve the coding standard used by the company.
- A slower response to a defect in the code. The most efficient mode is to check the code immediately after compilation. PVS-Studio has an automatic analysis mode for this . In the case of an audit, you will receive a report once a day. This, of course, is also very good, but still belated.
We estimate 1 month of audit at 100 000 rub. It is economical. Let's count.
In order to audit the code yourself, firstly, you need to purchase a PVS-Studio license. Secondly, a highly paid specialist should appear in the staff who will regularly engage in project analysis, study analyzer reports, report errors to programmers and help them with advice. For simplicity, we leave aside that such a specialist still has to go look. Another task.
Given the purchase of a license, the salary of a new qualified employee, as well as the fact that the employee requires additional overhead costs (payroll tax, office space, a powerful computer to run tests), then you can’t meet 100,000 a month.
Thus, remote auditing can cost your company not only cheaper, but it will be much easier to implement it. They will do everything for you. You only need to provide access to the sources and analyze the information about the errors found in the code.
If you are interested in regular audit of the code, then you can contact us using the " feedback " page or by e-mail: firstname.lastname@example.org .