Do not do this in production.

Original author: Stephen Mann
  • Transfer
Around March 2017, I was asked to make a product review code before launch. That company had problems with memory leaks, spontaneous failures, slow loading, irregular CPU usage, and a release was scheduled in a few weeks. Perhaps you have already heard this story - not from me and not about this company. It is surprisingly typical.

We gathered at the weekend and began to look at the code together. After about half a day, the source of the known problems was discovered, and it took another half a day to write a document on a fix for developers. The launch was a success, but the story made me think: how did the product reach such a state?

When I spoke with the developers, they seemed like smart people. The only obvious problem was the lack of experience. I have come across this before. This is a common and quite normal phenomenon. But in this case, a vile flaw was observed: all the developers lacked experience .

The development department was created recently, and the team was hired in the absence of a technical director. Even a techie is hard to check another programmer - I can't even imagine checking without technical knowledge. They hired the first developer, he checked the second developer - and so on, until a team was formed.

If you are lucky and the first developer has significant experience and a desire to train, then you are in a mess. But if you are unlucky - and the chances of that are very high - then you will get a fast-moving team that creates very fragile software.

“Act fast and break everything,” they say. But this is a pretty bad idea if the business relies on a small number of large customers. Broken products usually scare them away, which in turn hits your business. You can describe an effective strategy in different ways, but the phrase “slowly and steadily move towards the goal” is clearly not impressive.

In fact, there is a balance between fast and slow motion. It is difficult to grope this balance, because each product has its own balance. I assume that intuition is based on experience, and this is a terrible answer for a beginner.

What to do to the new developer?

It seems natural to search for an answer on the Internet. It turns out to be incredibly effective .

But it is also incredibly dangerous .

I collaborated with that company and after the launch of the product. I looked through a significant amount of code, helped in training developers and created new projects for them. Everything went smoothly.

One day my attention caught one piece of code. I could swear I had seen him before. Of course, having run a string, I found an exact copy of the code in one blog entry. Naturally, I read the entire article - right down to the paragraph that said: “Do not do this in production .

But these lines look at me directly from the code base in production.

It did not take long to find a lot of code fragments from similar articles. Almost everywhere there was either a disclaimer, or it was clearly not enough. They all solved one small piece of the problem, but allowed a lot of liberties to make the code easier to read. This can be understood. Most readers appreciate brevity when learning a concept.

The code from these blogs has spread through the code base as an infection, sowing problems here and there without any reasons or patterns. And there was no obvious cure except to read the entire code in a row and manually fix the problems one by one. Without unit tests and automatic deployment, it took almost a year . I am almost sure that the cost of correcting the code exceeded the cost of writing it.

But what choice did these developers have? They needed to release something, and they had never released an application in production before. Therefore, they did what any sane person would try to do - and learned along the way.

Systems in production fail in an incredible number of ways. Without experience or theoretical knowledge of these failures, it is difficult to intuitively understand how to prevent or solve such problems. It is difficult to demand from the new team to achieve an impeccable result, especially without some kind of leadership.

Before proceeding, I want to note that every person who had a hand in this chaos had good intentions. The developers wanted to create a good product and develop. The managers wanted the same thing. Authors of blogs wanted to share useful solutions. Everyone tried to help each other, and it is important to remember this.

The problem is not with people.

I strongly sympathize with the developers who are in this position. They have more information than they ever need, but it is completely disorganized. This is like trying to assemble a puzzle of ten pieces, you just need to find them in a pile of 10,000,000 pieces, where all the pieces are square, and the final result is unknown. Good luck.

If you read it here, hoping for an answer, then I’m sorry: I don’t have a simple answer. This problem is difficult to solve. The solution is too large for one article, it changes every day and differs in nuances for each project.

This problem prompted me to start a blog. I have been fortunate enough to study with incredibly talented mentors, writers and colleagues for almost twenty years. Without the advice of these people, I would still write directives GOTOin QBasic (br-rr). It's time to take the ball and go on the offensive.


This blog covers all aspects of the development of ready-made applications: from infrastructure automation to testing, design, debugging, documentation, development process and security. Each article is independent and suitable for use in the real world -suitable for use in production .

Also popular now: