PVS-Studio ROI: how not to lose millions (draft article)

From time to time, we are asked the question of how much money the company will benefit from using the PVS-Studio analyzer. I decided to implement a ROI calculator on the site and place a detailed description of its principles of operation. But before that, I decided to submit my thoughts and calculations for discussion. I hope to receive interesting and useful comments that will help make the calculator as reliable and convincing as possible.

We offer you a description of a calculator that allows you to calculate the return on investment from using the PVS-Studio code analyzer in the development process.

New version of the article: RU , EN .

The default calculator works in skeptic mode. However, we hope that, after reading the article, you will agree that it will be more correct to switch to the mode: I trust the calculations described in the article “PVS-Studio ROI” :).

Programmer Hours

In order to determine how much money PVS-Studio will return, first you need to calculate what the real value (value) of the programmer’s hour of work is.

The fact is that it is not enough just to take a programmer's monthly salary and divide it by 160 (the average number of hours per month with 40 hour work week).

First, programmers, as well as employees of any other sphere, bring in more money than they receive a salary, otherwise the business will work itself in the negative. Programmers need to provide jobs, pay rent for premises, buy cookies for them, supply them with Internet, and so on. Oh yeah, there are still awards, corporate parties, various bonuses.

In this case, the use of the programmer must be profitable, that is, he must directly or indirectly bring the net profit of the company. In practice, this means that the work of a programmer, depending on the situation, brings 2-10 times more money than is spent on his salary. Once again, the programmers here are no different from any other hired employees. Some features have outsourcing, but this is another story.

For skeptical readers, we take a factor of 2. That is, the programmer brings in 2 times more money than spent on his salary. In fact, a company with such ratios balances to the point of break-even. It is more honest to take a coefficient equal to at least 3.

What does all this mean? If the programmer fell out of the development process for 1 hour, then the company received less than an amount equal to the hour of its work, but 2 or 3 times more.

There is a second factor that influences the price of the present working hour. The fact is that the employee does not program at all 8 hours a day. It is impossible to imagine that a person came in the morning and sat down for 8 hours working on the code without stopping. The programmer works with Trello, participates in meetings, replies in the mail, participates in code-review. In the end, he still needs to go to the toilet and drink tea :). At best, it will work 6 hours directly with the code. And if you are not reading this text in skeptic mode, then you understand that in fact 4 hours is much more likely time.

So it turns out that the cost of an hour must be additionally multiplied by 8/6 = 1.33 or by 2.

Now we multiply the two considered coefficients and obtain the final coefficient by which the programmer’s hour of work needs to be multiplied:

• coefficient for skeptics: 2 * 1.33 = 2.66
• factor closer to reality: 3 * 2 = 6

In practice, the coefficients will be slightly higher, since we do not take into account the vacation in the calculations.

Let's now see what it means for a company to drop a programmer with a salary of 100,000 rubles from the workflow for 1 hour.

Note. To understand, we note that in fact the company spends more than 100,000 rubles on salary payments. It should be noted that the company makes contributions to various funds (“payroll taxes”). And on the hands after the deduction of 13% of the tax a person receives 87,000 rubles. To simplify the calculations, we will not take into account the deductions and assume that the company spends only 100,000. I decided to mention this in order to show that I am rounding off not in favor of PVS-Studio.

With a salary of 100,000 rubles, the rate of 1 hour of work will be 625 rubles. It turns out that if the programmer was distracted for editing an error for 1 hour, the company would not be able to earn money because of this:

• for skepticism: 625 rubles / hour * 2.66 = 1660 rubles / hour
• in reality, more than: 625 rubles / hour * 6 = 3750 rubles / hour

This is the real value (value) of one hour of the programmer when he is busy with useful work.

How many hours does PVS-Studio save?

It’s very hard to say how many hours a year PVS-Studio will save, finding errors early on. Errors are very different. Some of them are immediately noticed by the programmer and immediately corrected. And sometimes a bug can distract a programmer from a useful activity for a few days .

For empirical reasons for the skeptic, we say that the analyzer will save at least 2 hours of a programmer per week, saving him from having to look for bugs found by unit tests or by the testing department. Yes, the bug fix itself usually takes minutes, but attempts to reproduce the problem, correspondence in the bugrecker, test runs, merge and so on will easily eat these 2 hours.

Considering that sometimes the analyzer can prevent the appearance of difficult-to-reproduce heisenbags, the average value is quite possible to specify 3 hours.

About 52 weeks in a year. Per year, the analyzer saves the following hours of this programmer’s work:

• skepticism about static analysis: 2 hours * 52 = 104 hours saved
• positive attitude: 3 hours * 52 = 156 hours saved

It's time to take ROI

Then the use of PVS-Studio by one programmer with a salary of 100,000 rubles will return to the business a year:

• If you are skeptic: 1660 rubles / hour * 104 hours = 172 640 rubles
• Actually: 3750 rubles / hour * 156 hours = 585 000 rubles

Now let's take a typical development team of 10 people. Having implemented PVS-Studio, we can expect that, thanks to the time saved, the team will be able to perform useful work with a cost:

• Skeptic: 1 726 400 rubles
• Reality: 5,850,000 rubles

Final formula

So now let's combine everything into a single formula that is used in the calculator.

Denote the monthly salary of a programmer as S. We denote the

number of programmers in a team as N.

• The formula for the skeptic: N * (S / 160) * 2.66 * 104
• The real formula: N * (S / 160) * 6 * 156

Now we give in the form of tables calculations for teams of other sizes. The table shows the predicted amount of money that the development team for the company can make if, instead of editing the bugs, it is busy creating something new. These numbers should be compared with the cost of the license.

Top line: the number of programmers in a team. Left column: developer salaries.

Table for skeptics:

Table N1. Skeptic. Red: using PVS-Studio may be unjustified. Green: using a static analyzer is justified and useful. Blue: use is definitely beneficial.

Real table:

Table N2. Reality. Red: using PVS-Studio may be unjustified. Green: using a static analyzer is justified and useful. Blue: use is definitely beneficial.

The second table, in my opinion, is reliable, and it is reasonable for her to be guided in assessing the economic feasibility of purchasing a license.

Note

Of course, the above calculations are not always appropriate and not everywhere. For example, if the cost of errors and vulnerabilities for a project is extremely high, then there is no point in associating the value of using PVS-Studio with the programmer's salaries. In such projects, possible monetary and reputational losses should be assessed and already attributed to the reduction of risk when using a code analyzer. This is a separate story, and I still do not know how to approach it from the point of view of calculations.

Also, the calculations may not work for outsourcing companies. This may not sound very nice, but such companies are interested in selling as many hours of development, testing and maintenance as possible. In a sense, the use of the analyzer can only reduce their revenue. This is indirectly confirmed by the fact that there are no outsourcing companies among the clients of PVS-Studio. Plus, in such companies sometimes strange processes can occur at first glance. The company at the time of low load can take a project even at a loss. This is better than dismissing some developers on vacation. Let them be better in business and busy with something.

Conclusion

So, although the calculations may not be suitable for all companies, I hope that I was able to demonstrate how to approach the assessment of the efficiency of using PVS-Studio from the point of view of the business as a whole.