Hellish project

A few years ago I was invited as a consultant for a single software project for a large French technology company. What I saw is beyond all that I could imagine in development. A simple lack of professional competence was not the worst. Much worse was the extreme contempt for human dignity, which seemed to me comparable to prison in the form in which I imagine it. Here is a list, check it out for yourself.

Scale

• Software development for a government agency.
• The difficulty is low, with a few twists.

The government pays several million euros in advance. A development plan for two to three years. To get started, the company hires several developers and continues to double the size of the team every three months or so, mastering the budget as funds become available.

7 years later, the project has not yet taken shape. They are fined several thousand euros per day. Management decides to cut costs and dismiss experienced employees by hiring people with little or no work experience.

10 years later, given the disastrous state of the project, middle management decides to hire some people with development experience to return to the schedule. Average turnover among beginners: three months. This is the minimum period to be eligible to resign in France.

12 years later, the project is not yet completed. The company again falls into daily fines, billing the government for an ever-growing stream of change requests. 2008 is coming.

Figures

• 6 million lines of code
• Based on C ++
• Over 50,000+ classes
• A specific version of C ++ is deprecated, but is tied to a compiler that is distributed with only one (not supported) operating system
• Based on CORBA
• DBMS from a bankrupt company
• Multiple layers on top of each other for GUI processing, none of which are actually supported by the authors
• Assembly takes 48 hours on 32 parallel machines
• 40 to 50 simultaneous processes to run only the user interface
• Lack of dynamic linking of libraries: the sizes of executable files begin with several hundred megabytes
• Program launch time: about 15 minutes
• Mean time between failures: 30 seconds to 30 minutes

No developer will say that C ++ is a simple language. On the contrary, one of the most difficult. In fact, it is so complex that even its creators admit that they have not yet mastered everything. See Straustrup's confession in his famous “interview” , which was composed as a rally.

Faced with such an incredible labyrinth of bottomless complexity, people react differently. All upstart geeks have heard about C ++ and want to prove that they can also program on it. They fearlessly plunge - and get injured beyond recognition. They spend countless hours trying to figure out why a bunch of gibberish crashes endlessly for no apparent reason. Those who have a mind - they quickly go to other languages ​​and other projects. Life is too short.

Maintaining a large project is difficult regardless of language. Just imagine that employees need to support 6 MILLION LINES of code - and you get an idea of ​​how far crazy development can go. Six million is a big number. If you read one line per second, then you will sit in front of the screen seventy days without a break.

To understand the scale, I will tell you two cases.

One developer was tasked with checking why right-clicking on the interface completely suspends the application. After several days of careful study and an incredible amount of patience, he found that right-clicking works fine. Just the context menu appears after about 45 minutes. All menus are dynamically generated from a heap of (static!) Content each time you right-click in the main window.

At some point, users report that the option “Download data from CD-ROM” does not work at all. It took several weeks to sort it out. But in the end, the bug report was marked as “Solved earlier”, because the data was loaded correctly. Unless downloading 700 MB took seven days. But patience is a virtue.

Version control went awry

Several years passed before one smart guy in the team got the idea to use version control tools. The first attempt did not work out too well, so the team switched to another system. And after a couple of years - on the third, losing the whole story with each transition.

As a result, we chose a disaster tool with a GUI. An abomination coming directly from Sweden. I had to allocate a full-time team of four to complete most of the tasks in this system. It looked something like this:

• For the first checkout, you should make an appointment with the version control team. The meeting was usually scheduled after a week.
• File editing was not permitted without the permission of mid-level managers. You must tell your manager in advance what files you want to edit, and then send an official request for permission, which the version control team can consider within a few days.
• Each code modification triggers a branch. This means that then you need to combine all the modifications received. With so many files in storage, you might think that there are rarely two people working on the same file. But it turned out that most of the work goes on the same 100 files or so.
• Check-in goes through a painful procedure when your code is checked by an automatic program to detect errors, and then by managers. Needless to say, with this approach, errors breed faster than they manage to correct. A careful look at the number of errors recorded showed that for each correction two new errors appeared.
• Version control is generally simple. The old program has the first version, the current one has the second, and the future one has the third. No one can say which version was sent to the customer.

At some point, an official release date was set, completely divorced from the team’s plans. When the day came, the client was actually sent an empty CD with installation instructions, because no one was able to build the software for several weeks. The client found out that they sent him an empty disc, officially complained - and received the old version. He found out about this because the date displayed in the "About" field was the same as last year.

Frames

You pay crumbs - you get idiots

With so many people without any experience in software development, is it any wonder that errors were generated in huge numbers? Some very talented manager found that staff costs are the main costs in a clean software development project. Not at all frightened by this extraordinary discovery, he decided to fire all people with at least some experience, but retain all the managers. It was not uncommon to see C ++ dummies on the tables of many employees.

Meet the team of

55 people in the team: 20 developers, 35 managers. You were not mistaken: there are more managers than real developers.

Managers continue to organize meetings where they show the same PowerPoint AD presentation again and again to the point of disgust, while developers kill time by chatting in a huge open-type office.

Few managers have experience in the software industry. At the time, SCO was suing IBM for Linux. Even if everything was a bluff, but it really worked for all these people who understood that they would soon have to pay for free programs. None of them ever mention Software Libre, but they all know about Software Gratuit. Needless to say, the project is crammed with GNU libraries, and these guys have no idea how the project becomes compatible with GNU. Although all right, given the terrible quality of this code, no one will ever insist that they open source.

Technical knowledge below the plinth. Few people know about the Internet. Those who know think it is made for porn. Mention that you saw something on the Internet - and you will begin to wink and smile.

welcome to Hell

All this would be funny if the top leadership did not behave like the Nazis in a concentration camp. Just a few examples.

• It is forbidden to come to work after 9 in the morning. Once, a senior manager stood in front of the main gate and immediately fired everyone who came after 9:01. The distribution included, among others, a number of managers and salespeople.
• Smokers take more breaks, so they work less. Management tried to force everyone to quit smoking on orders. Did not work out.
• Coffee machines regularly fail for several days. Those who drink coffee work less than those who inseparably sit at their table, typing precious lines of code.
• The same coffee machines are turned off whenever officials come to the office to give the impression that everyone is at work.
• Toilets are the most disgusting I've seen in my life. Probably the idea was to increase productivity: the less time you spend in the toilet, the more (and better) you work.

You are probably wondering why people kept coming to work in such a place. The first and main reason was the deep economic crisis that France was experiencing at that time (and still, to a certain extent, is experiencing). The presence of work and wages is considered a privilege, regardless of conditions.

Another reason was that for many it was the first job. Without some comparison, it is impossible to assess how much your work sucks. Most newcomers considered it perfectly normal that they were forced to come exactly at 9:00 or be fired when absolutely nothing imposed such a restriction, except for the manager’s sick mind.

As for how the government allows this: we all know how it works. The guys in charge of the budget in the ministry are friends with top managers in a number of companies. In a country like France, corruption is not uncommon at this level, it is mostly not disclosed and rarely prosecuted by law. Apparently, this applies not only to France. I heard the same stories from other places in Europe and from the USA.

Next time at the thought that your job sucks - think again.

Published June 24, 2008