February 28, 2014 at 09:58How much does one error cost in our programs, or what errors happen in PVS-Studio and CppCat
In the list of questions that are always asked to us after reading our articles, there is a question: "Do you check the code of your analyzer by yourself?" And the answer is given in the sense that we always check the code while writing using the incremental analysis mode (checking files immediately after compilation). And also nightly checks for all code are launched. And in this regard, we, unfortunately (irony), will never be able to write an article about errors in our software products. However, today it is an article about one of our mistakes and about how much it cost us. Intrigued?
My today's story will be in person, so first “credits”:
- Michael, user of CppCat;
- Eugene (I), the author of the article, is related to PVS-Studio and CppCat;
- Andrei, my colleague, sometimes writes different articles;
- Pavel, my colleague, knows everything about integration into Visual Studio;
CppCat is what you need! You would have to make a separate post about him. I’ll be begging the management for tea inexpensively :)
Having smiled, my colleague Andrei answered him that already three articles were about CppCat. True, a colleague immediately got “catch up” for the fact that no one knows about our new product. It was at 16:16 Moscow time ...
Already at 16:21 in my CppCat-mail there was a letter from Mikhail with a Wishlist. Michael wanted CppCat error messages to be displayed not in a separate window, as now, but in the standard Visual Studio error window. From my point of view of the “product keeper” of CppCat, this wish did not seem practical, and I tried to explain in the mail that we had a separate window specially designed so as not to confuse our messages and Visual Studio messages. That is very convenient, as it allows us to make the window specialized, etc. But Mikhail “didn’t let up”, citing a simple argument:
“But how can I see errors after recompiling the file?” I wish they were immediately issued out of the window ...
But it is precisely for this purpose that we have a “killer feature” incremental analysis! After recompiling the files, the code analyzer starts automatically. And only if real errors are found, a tray notification is issued and the title bar of the analyzer window changes. A phrase appears in the window stating that new errors have been found. The incremental analysis mode can be turned on / off in the menu.
“Well, yes,” answered Mikhail, “I have a tick“ Incremental analysis ”. But how do I know if there are errors?
After five letters, including screenshots, it became clear that although everything was set up correctly, notifications were not issued in the tray, nor did the text of the window change. And the time was the sixth hour of Friday evening ...
With the thought: “For a long time I didn’t pick up drafts (from the day before yesterday)!”, I asked Michael for a computer via remote access with AMMYY Admin. Having stumbled into my program, I realized that there really are no notifications of new errors, although the program seems to be working. It was close to six in the evening, and I went to my colleague Pavel, who is responsible for integration into Visual Studio. I showed him what I'm doing on the user's computer and that the notification does not work.
Thirty seconds later, Pavel said:
“Eugene ... Well, you forgot that with the“ Rebuild All ”command, an incremental analysis does not start specifically for us. This is done consciously and everything is right. In general, you need to look at your own product more often. And then I already forgot everything. Just to answer letters in the mail, and write articles about the business ...
Blushing, I went away to check ... However, there was no notification either during the reassembly of one file, or during the reassembly of several files. I tried on another project - the result is the same. Does not work. Moreover, a very important function that is a killer feature of our tool does not work. Let me remind you that I tried all this on a remote user machine. I went to Paul again, but it was almost six. And the user wanted to go home, and Pavel ... Perhaps in this connection we together could not understand why it does not work. Having decided on Monday to issue a version with debug logs, we went home.
All weekend I thought about this problem. Is our product to blame? If so, when could this problem appear in the code? And how many people did NOT BUY our product, since a very important thing did not work as it should.
On this sad note came Monday. We gave the user a special version with debug logs. He sent them to us. The logs were not clear, unfortunately. We made more detailed logs, described the playback instructions as detailed as possible ... It didn’t help. According to the logs, it looked as if the code did not work, although it could not work.
For Monday and Tuesday, we gave Mikhail several versions with the most detailed debugging logs. But they could not understand what was happening. There was simply no notification of new errors. Him. And we always had it.
In the middle of Tuesday, Pavel called me:
- Eugene, you won’t believe ... But I don’t get a notification about the errors found in CppCat!
Do you think debugging logs helped us? No, Pavel accidentally clicked on the auto-hide button in the CppCat window inside Visual Studio. And our window began to automatically hide when you take the mouse away from it. Since none of the developers uses the auto-hide button with us, we could not even think that in this mode our notification really does not work!
That is, if auto-hide is turned off (normal mode), then there are notifications of new errors, and if auto-hide is turned on, there are no notifications of new errors.
I asked user Michael to check if the case is really in auto-hide. He confirmed that in it.
After that, it was already a technical matter to correct the mistake. It turned out that the plugin code did not correctly handle a certain window style. For programmers, I asked Pavel to describe the problem and its solution, but I will not provide this text here, since it will be understood only by developers of plug-ins for Visual Studio.
So, we were able to repeat the mistake on Tuesday afternoon. On Wednesday, a revised beta was issued to Mikhail.
While Pavel was doing the patch, I talked with Mikhail about the licensing policy of our CppCat and PVS-Studio products. Although Mikhail got a mistake when he met CppCat, during further communication it turned out that PVS-Studio could be useful for his team.
Let me remind you for those readers who are not familiar with what we are doing. We develop, promote and sell our own CppCat software products for individual developers for $ 250 and PVS-Studio, the minimum license of which for a team of up to 9 people costs a little more than € 5000.
So, discussing licensing issues, Mikhail apparently sent a letter to the management and on Tuesday evening I discussed with his boss PVS-Studio licensing issues. We talked pretty quickly about everything, they came to the conclusion that they did not need CppCat, but PVS-Studio.
But what about that mistake? On Wednesday, we gave Michael a revised beta, he checked that everything works. On Wednesday, I sent my boss Mikhail an invoice to pay PVS-Studio. On Thursday, I saw money for a license on our account. And he decided that it would be nice to describe this story, suddenly it would seem interesting to someone.
So the correction of one error in the product for $ 250 gave us the sale of our other product for completely different money. And all this for one week, because today, on Friday, February 28, you are already reading an article about this story!
If you ask why I wrote all this, then my answer is very simple. I am proud of my team that we can quickly correct our mistakes and that our customers appreciate it.
PS Many thanks to our user Mikhail, who helped us diagnose the error, without this story would not have happened. Our software clients are the best users in the world!
UPDATE1: For some reason, in the comments I write about the fact that it was necessary to give a discount. Of course, a discount was granted, what other options could there be? :-)
UPDATE2: Michael noted in the comments below. Put him a plus in karma for helping developers catch bugs :-).
| Unfortunately, we no longer develop or support the CppCat project. You can read about the reasons here . |