3 ways to develop

Engineering Aimed at Creating Garbage

A popular method of the development process in some large firms is Development, Aimed at Creating Garbage, or RNSS for short. RNSM feeds on the belief that the only thing necessary to receive money is a good idea. Obvious nonsense, but it is a support for people with a lack of imagination. This theory says that an Idea is a rare, valuable and unique thing and the whole trick is to catch it.

The main product of the RNSM is meaninglessness, written on such "valuable" ideas: concepts, graphics, descriptions of design and other products created for one purpose only - to be thrown into the basket.

It works like this:

• Creative Guys come with a long list of "features A and B that we can do in our product." An incredibly detailed list of what this future Great Product could do. What fellow they are that they came up with all this! Well, now that the main (creative) part of the work has already been done, there remains only the question of technical implementation, but this is not difficult!
• Managers and their consultants convey brilliant ideas to designers and architects who write tons of specifications. At this stage, a dozen basic ideas turn into several hundred detailed requirements. For a product that will change the world.
• Specifications are passed on to developers who, somewhat ... surprised, carefully ask, but who, ..., it occurred to all of this ... to write? They begin to explain why something is difficult to do, and something is generally impossible, but ... The train has already left. The architecture is approved at the very top, and in fact - who are these developers to argue with the Creative Guys and their highly paid consultants?
• Developers return to their jobs, humiliated and doomed to build a gigantic "creative-and-elegant" pile of garbage. This work turns the brains out, since the theoreticians from above absolutely did not think about the nuances of practical implementation. Elementary and minor changes can take weeks. The project begins to lag, managers click on developers, free evenings and weekends disappear.
• In the end, something that somehow simulates a working product is released. It is awkward and fragile, complex and ugly. Creative Guys call the developers incompetent mediocrity and pay their consultants even more money to attach a bow to this pig, and outwardly it seems to be getting a little better.
• By this time, managers begin to try to sell all this horror, and suddenly (surprise-surprise!) It turns out that no one needs it for three hundred years. In this place, the most important thing is not to fall face down and ... throw a million dollars into an advertising campaign! We have to almost pay extra to convince people to use this product. Ah, this stupid, backward and ungrateful market!
• After a year of intensive marketing, the product is still unprofitable. Moreover, all this year he was haunted by catastrophic failures and negative reviews in the press. The company quietly lowers it into the ballot box, fires consultants, buys a competitive product from a startup company living in a nearby garage and releases it as “Version 2.0”. Hundreds of millions of dollars are thrown into the basket.

In the meantime, another Visionary from top management is drinking an extra glass with the guys from marketing and the Brilliant Idea comes to his mind ...

Development aimed at creating Garbage could be a caricature if it had not happened in life so often. Something like 19 out of 20 products marketed by large firms is a failure (yes, 87% of the statistics come from the ceiling). The remaining 1 out of 20 is successful for some completely random reasons, such as very poor competing products or a new market.

The main conclusion from the use of RNSM is easy to make, but hard to accept: ideas are worth nothing. With no exceptions. There are no "brilliant ideas." Everyone who starts the discussion with an exclamation of “oh, and we can do this!” Must be beaten by those around them with all the passion and energy that they cherished for Jehovah’s witnesses calling at the door. All these Creative Guys with their “brilliant ideas” look like a man sitting in a cafe at the foot of the mountain, sipping his hot chocolate and explaining to others: “Hey, I have a great idea - we could climb this mountain! And build a cottage on top! With two saunas! And the garden! It can be connected to solar panels! Dudes, that's cool! What color will we paint it in? Green! No blue! So, go and build it, and I will stay in this cafe and draw a couple of schedules for our project! ”

The starting point for a good product is to collect real problems that real people have. The second step is to calculate whether it is reasonable to solve these problems at all economically. Good solutions to real problems - this is a successful product. “Good,” among other things, also means “cheap,” which means “simple.”

And now, after defeating the Dragon of Total Inappropriateness, we attack the Demon of Complexity.

Development Aimed at Creating Complexity

Good engineers and small firms are theoretically capable of creating decent products. But the vast majority of products are still too complex and less successful than they could be. This is because teams of specialists are persistently implementing the development process, which I call Development, Direction for Complexity Creation (RNSS).

It works like this:

• Managers correctly find interesting and important problems with the real economic effect of solving them. At this stage, they are one step ahead of companies practicing RNSM from the previous paragraph.
• The development team enthusiastically begins to build a prototype system, and then the basic functionality of the main product. Everything is going according to plan, work is being argued, things are becoming logical and elegant.
• By this time, management comes with even more interesting and important problems. Yes, they are more difficult to solve (something will have to be altered altogether), but if we do this, then we will be two steps ahead of the competition, right?
• Developers do what developers should do - develop. Develop, develop, develop and develop again. Managers come with their interesting and important tasks again and again. And the developers are developing again and again. To get the perfect, solid, massive, HUGE DIFFICULTY as a result.
• The product enters the market, the market scratches turnips and asks “What, was all this really so difficult to do?” People begin to somehow use the product, spending their time and money to figure out its unimaginable complexity, but what if there are no alternatives?
• Management receives positive feedback from major customers - their management believes that if everything is so complicated and so much time needs to be spent on training and use, then this is definitely a good product (?!).

RNSS is characterized by teams of people, frantically and fanatically solving private tasks that are not needed by 99% of users. RNSS products are bulky, ambitious, complex and unpopular. Many open source products were born in an effort to get a simple solution where there was only a complex one. It’s incredibly difficult for some engineers to stop trying to expand the product’s functionality “just in case someone else needs this.” They want to do good to people without thinking whether they need it and whether these people even exist anywhere in the world.

A good example of RNSS is Bluetooth. This is a complex, contrived set of protocols that users hate. It continues to exist thanks to a massive package of patents that are simply impossible not to violate, creating an alternative. Bluetooth is ideally safe, which is close to meaningless for a protocol with such a short range. At the same time, he suffers from the lack of a standard API for developers, forcing everyone to invent an incompatible bike in his application.

#Zeromq Wintre once wrote on the IRC channel how he was furious when he discovered many years ago that the XMMS 2 audio player has a good plug-in system, but cannot play music.

RNSS is a swamp, floating in which developers and architects boil in their own juice, come up with their own problems and heroically solve them, not caring at all about how much this is generally in demand beyond the borders of their imaginary world.

The main conclusions from the RNSS are again easy to draw, but hard to accept:

• Creating a product that you do not need right now is pointless. It doesn’t matter how smart and talented you are if you are doing nonsense to anyone that no one has asked for.
• Problems come in all sizes. Some of them are simple, others are not. Strange, but often the solution of just simple, trifling problems is just what people need. On the other hand, developers are not interested in working on platitudes and trifles; give them a Challenge, a Problem. If development is left to chance, the developers will work on exactly interesting tasks for them, but not the fact that tasks are useful to users.
• Engineers and architects love all sorts of Architecture, Patterns, to generalize something here, and here to optimize, and there to show off a hack - and all this inexorably leads to an increase in complexity. It is critical to have leverage over such things. It can be, for example, a hard deadline, forcing the developer to make a small, simple solution that works "here and now", instead of a brilliant and general one that will never be completed.

Simplicity Design

And finally, we got to a rare, but all the more valuable way of working - Development, aimed at creating simplicity, or RNSP. It starts with a simple thing: you need to realize that we do not know what we want to build before we start doing it. To start with Brilliant Ideas and plans for flexible scaling to thousands of servers is not only pointless, it is rather even a direct obstacle to a neat basic architecture. The real problems are actually hidden in the fog and the only way to dispel it is to start moving, to explore how you explore the hidden map in a computer game. At the first stage, you need to be fast, mobile, move in a word - and so far it is not so important where. Only after exploring a certain area, it will become clear where to go next.

RNSP works like this:

• We collect many interesting problems in a certain area and arrange them in increasing order - from simple to complex, trying to find some common patterns.
• We take and solve the simplest, elementary problem, and we do this in the least costly way - the “patch”. Each patch solves exactly one problem, and exactly one, minimally expensive way.
• We introduce one single metric of the quality of the patch: “Could this be done even easier while still solving this problem?”. Difficulty can be measured in the amount of information that the user will have to learn (or guess) to use the patch. Less is better. An ideal patch solves the problem without requiring any knowledge or skills from the user.
• Our development process begins with a patch that solves the “we need proof of concept” problem and develops into a powerful product consisting of hundreds or thousands of patches that complement and correct each other.
• We do nothing that is not a patch. We build the rules of the development process in such a way that each task is solved with one patch that solves this and only this, conscious and described problem.
• We are building a “supply chain” of tasks, in which users add their problems, and developers solve them. This chain has a good “stop mechanism” that prevents developers from flying on the wings of their imagination into sky-high distances - but try to do it when there is a specific living person waiting for a quick solution to the problem.
• Developers have the right to work on any projects and provide patches for any component. Nobody owns the project alone - there is a “main” repository with its formal criteria for receiving patches, but if someone does not like them, anyone can fork. The same project may have competing forks, encouraging each of them to develop better.
• The project provides a certain set of documented interfaces, allowing other components (clients, plug-ins) to use them. Creating a good API is a way to get many satellite products, build an ecosystem, and conquer the market.
• We withdraw the supply chain of tasks to external users and customers and provide a cycle of frequent releases, so that the user problem can come to us from the outside, be analyzed, evaluated and fixed by the patch in just a few hours.
• At every moment of time, starting with the first patches, our product is ready for delivery. This is important, since a significant part of the patches (10-30%) will be incorrect and only giving users access to the product can we understand what went wrong and needs to be fixed.

RNSP is a reliable way to find optimal solutions to the most significant problems in an unknown subject area. You don’t need to be a genius to apply RNSP, you just need to see the difference between aimless wandering in the fog and the activity of dispelling this fog, as a result of which the outlines of really important problems appear.

Some people point out that such an algorithm has its drawbacks. Walking in small steps, we can get stuck at the “local peak”, missing the opportunity to jump to the top of the ideal. But this, nevertheless, is how everything happens in real life: we take small steps forward here and there for a long time. There is no “rational plan for life.” We reduce the risk of being stuck at a local peak with a wide coverage of the issues under consideration. The theory says that this is how innovations appear, so it is better to use this more or less scientific approach than hoping for luck or solving all problems with some sort of magic.

As soon as you grasp the power of this approach, the power of collective thought and the wonderful endlessness of improving your product, you also realize why some companies and products are stuck in a swamp of disastrous prospects. They simply do not have a variety of thinking, approaches, collective intelligence and imagination for the correct exploration of unknown subject areas and markets. When Nokia killed its open source projects, it cut its throat to itself.

A good architect with a good team can use RNSP to build world-class products. To get the maximum benefit - the RNSP should be used constantly, day after day, developing a scent from the developer for the problems of illogicality. We often overlook all sorts of minor inconsistencies, but a good architect sees them and is able to perceive them as a chance to make the product better. The essence of this type of software construction in non-stop (albeit not large, but constant) approximation to the ideal in using the product.

In the world of opensource, we do all this work in plain sight. There is no “well, let's open the code” moment. Products that decide to open the code only at some stage take away from the user the opportunity to trace the entire exciting history from the very beginning, to create a community around their product from the moment of its birth.