Fakap hit parade: top-5 of our favorite bugs in honor of Tester's Day / Maxifier Development Blog

On the Internet there are already quite a few selections of very interesting bugs - the most amusing ones and those that brought maximum damage (for example, here ).

But alien rakes, of course, are not taught as reliably as their own, adjusted to your height, with a carved handle and sharp teeth, those that are lovingly designed by the crooked hands of developers and carefully laid out by cunning users.

Therefore, I congratulate everyone who is involved in the production process, finding and fixing bugs. By the way, of course, I am proud of our team of testers. These people are able to catch all possible bugs - from primitive-type to unpredictable-magic - and every day they do it very successfully. Therefore, I will not talk about everyday victories. On the contrary, in honor of the holiday I will describe our most epic mistakes and fakapy.

Here, I deliberately do not cite cases where the problem is on the user's side or due to a misunderstanding of the seller, such situations, of course, deserve a separate study, and today - only our own mistakes and mistakes.

To preserve some intrigue, each case will consist of two parts - how it looked from the beginning, and what we found out in the end.
So -

5. Deja vu with unpredictable effect

What it looked like:

Once, on an unkind October day, a local apocalypse occurred. All clients, from different countries, right from the early morning began to complain about the system, and the most diverse complaints came. Someone simply could not enter the program. Someone started, but showed yesterday's data. Someone shamelessly slowed down. Someone gave recommendations for those advertising campaigns that have already ended.

The situation would be clear if, for example, we had just delivered a major release. Or if only one client fell. Or, at worst, all customers are in the same data center. But we did not make any changes on client servers for a long time, and between all cases there was nothing in common, yet ...

What actually:

... until, as usual, we have not figured out the logs. Since our application is very resource-intensive, most of the calculations are carried out at night - so that by morning users will already receive fresh recommendations for improving the network. Plus, the source data for the previous day are ready no earlier than 1–2 a.m.

And so, synchronization and calculations, as usual, waited for hours X, data availability, and started to update. And then, at exactly 3 o’clock in the morning, all over Europe, the servers switched back an hour to winter time.

And our system again on schedule launched a second copy of synchronization and calculations. And two parallel operating systems, simultaneously updating data, are an inexhaustible source of unpredictable situations.

4. Vasya was here!

What it looked like:

One of the client’s operators writes to us: “Oh, cool, you decided to advertise yourself on our sites around the world! Well done, great program, you should know about you. But I don’t understand why you are showing ads to everyone? ”

What actually:

At first we fall into complete bewilderment, because we were not going to advertise at all. And then someone guessed to look at the client’s site. And I saw there, on the main page, the Maxifayer logo. Which, in principle, should never be.

The answer turned out to be banal and offensive. The test advertising campaign, which was required for us to debug the algorithms, was mistakenly launched not on our own test site, but on the entire advertising network of the client. True, to be absolutely accurate, this happened because of the undocumented features of the advertising server with a certain type of targeting, which we just accidentally uncovered during the testing process. As a result of this sweet mistake, we showed our logo throughout the day about 5 million times throughout Europe.

We have to apologize, compensate for losses and so on, so on, so on. Once again you understand that the most serious mistakes are made not by the program, but by the person. Or, as they say, you can provide protection from the fool, but only from the non-inventive.

3. No one can believe in our time. I can!

What it looked like:

We have a demo for a strategic client. The standard demo is not suitable for him, as usual, sellers want to demonstrate everything new, something that existed only in minds yesterday, and today, still uncertainly, in code.

We, in soap and foam, install the version. Half an hour before the show. A letter comes from one of our sellers: “Listen, I know that our system is very, very intelligent and analyzes all the network dependencies in a way that a person can never. And what she predicts more precisely - I also understand. But here is a list of recommendations with a forecast in front of me. And for all recommendations, the forecast is negative. Of course, I understand that our system knows something important and therefore offers them for use. But could you also tell me what to answer to the customer if he asks? ”

What actually:

Well, what can you catch in half an hour if you suddenly realize that a bug has crept into the forecast system, and now it gives incorrect values ​​(read - full bullshit). But there are not only values, there, after all, forecast charts are drawn. The bug is not clear where, for such a period, nothing is fixed. And to increase the chance of selling the product to a new customer, oh how you want.

As a result, we do not change the algorithm, but the output. Random from a small plus value - and now all our recommendations generate quite reasonable numbers. Plus a few static graphs drawn by hand in 5 minutes - and now there is a visualization of the course of advertising campaigns. Next, a short briefing by the seller - and the demo goes with a bang.

I know that it’s not good to deceive customers, I know. But here, almost like in car dealerships, the sparkling and brilliant miracle that you are advertised is not the fact that you can immediately start up and leave the catwalk. But this does not matter to you - but you need to, when you have already bought, sat inside and inserted the key into the ignition, there should not be problems and disappointments at the last moment.

By the way, that's why I love SaaS selling. For he does not allow the developer to sell any idle system, and then dump the money on the Canaries, leaving the deceived customer with a broken trough. For they pay every month - for the result. Well, of course, it’s very problematic for the customer to simply use the results and not pay. Win-win situation.

2. My yours do not understand

What it looked like:

This time, complaints went from our support team. Vida - “The user asks a question about this type of recommendation that I have never heard of” or “The user sends a screenshot of a message that our system cannot generate in principle.”
And at the same time, programmers and developers also do not recognize the system and can not help.

What actually:

To be absolutely honest, in this case it’s not a bug, but a fakap of planning and communication. Initially, the system was developed from Russia, but to the western market. And all the names and messages were written in one way or another in the "rublesh". Of course, the literacy level was sufficient, but probably not ideal. However, customers are used to it, have not complained, and have been working normally with the system for several years.

And then, as part of product improvement, we hired a technical writer in New York. And she enthusiastically got down to business. But while the documentation was being translated, everything was not bad. And then one of the sellers did not understand this or that phrase from the interface, and TsU came to remake all the names and explanations so that it was absolutely correct from the point of view of the native speaker.

And all the names from the short ones became excessively clear. There are only two nuances - firstly, the technical writer understood their meaning on the basis of the existing documentation, which was not always complete and accurate (well, plus to understand and explain the difference between the several hundred types of improvements that we offer the client to improve efficiency, the thing itself it’s nontrivial in itself, and it’s understandable ...). And secondly, the technical writer kept the entire list of correspondences between the old and the new.

And he gave the result at the very last second, when, as usual in the emergency, the next major release was about to happen. And the only programmer responsible for localization had a list of correspondences between the old and the new. And then at the only tester - responsible for the verification.

As a result, a system was sent to customers that was very unusual not only to them, but also to us. And any question put support (also located in Russia) in a stupor - because she simply did not understand what was meant. And the programmers could do nothing to help - and they saw the system for the first time.

It feels very much like the interface suddenly switched to an unfamiliar language. As a result, of course, we found a list of matches, finalized it, printed it out for everyone and after a while got used to it, but the initial shock and spoiled blood were remembered for a long time.

1. How the Christmas tree nearly ruined Christmas

What it looked like:

- Igor, you won’t believe it, but our Christmas tree ruined the whole system.
- Not understood? Are you sure this message is for me at all?
- I remind you. In honor of Christmas, we decided to surprise customers. And we changed the icon of our program to the image of the Christmas tree. Nothing else has changed in the version. But now, not one of the clients has a system that works.
- How??? How is this even possible?

What actually:

As it turned out, it’s easy if you design the system “correctly”. For calculation, we have both a version with visualization for the user and “invisible” versions intended exclusively for computation in cluster mode — forecasting and generating recommendations. those. service launch that does not require a graphical interface.

And so, replacing the plugin, we put it in all versions. As a result, each version is initiated at the time of launch and tries to display the very Christmas tree - it searches for the window title, title bar, and more. And the OS answers, sorry, they say, the graphic display is not assigned to you. And the plugin throws an irreversible exception, which we did not guess to catch, - because initially a similar situation was simply not foreseen.

The conclusions, again, are worthy of Captain Evidence - no need to put the version without testing, no matter how insignificant the changes seem. It is necessary to catch exceptions and handle them, however unlikely they are. Well, of course, you don’t have to do everything at the last moment.

I hope that all these tales have raised your spirits or given you food for thought. If you have curious or instructive stories from your own experience - please share in the comments - it’s interesting.