Hey. Recently, I gave a talk to students about what kind of bumps can be stuffed while doing modern web development. How are the various decisions that we make during the development process related to each other, how the choice of technologies influences the size of the team, how the size of the team affects the approaches to testing, how the approaches to testing relate to the structure of the entire company ....

It turned out something like a quest with a distributed choice: from which programming language to choose and to the one who is better to hire in the team that will make the most useful product ever. I suggest you read this post, choose your options, add to the quest and discuss what has become painful.

upd - added some text to kata.

From idea to prototype

Suppose that my friend Valery and I decided to make a startup. Uber for X , or is there anything else like that? Gathered at the bar, discussed this idea, cool topic. Must do. Three months did not sleep, did not eat, did not leave the house. Developed. Launched and realized that nobody needs it.

Sadness Let's try again. This time we studied the market, looked at what needs of users, what problems. We made some kind of prototype completely on the knee, quickly and free of charge in two nights. The prototype took off. Cool, go ahead.

Choosing technology

Now we can take and make from the prototype a large application, develop it. But for this we need to take a more serious approach to the choice of technologies that we will use.

Tongue

In order. What language to write? You can take a trendy functional: Haskell, Erlang, Lisp (very fashionable among grandparents over 70). Or another JS killer, which is very cool, compiles into JS, has all the necessary features. But most likely, we will have nobody to hire in a year, because the next killer of JS will not fly up, and we will have to relearn again or rewrite the project.

Attempt number two. You can take something proven. For example, PHP. This is a good language, it is sometimes fashionable to criticize it, it has its drawbacks, but it is easy to do business logic in it, it is fast enough, it scales well, you can hire people anytime, anywhere. But it is not very productive. Therefore, we need to either have a lot of hardware, or write our own compiler, as Facebook or VK did.

More options? You can take Perl, but then there will be nobody to hire yesterday. Still?
Java. Java - norms. As a language is not very, in my subjective opinion, but the JVM is a great virtual machine, everything is OK, it works fast, but you still need a lot of hardware. And yet while we were writing an abstract builder of the strategy factory instead of creating features, users went to competitors.

Okay, we still have Python. In principle, he has everything ok. But we run the application in Python, it uses one core out of 56, for the rest ... everything is ok. Or you can take something modern: Go, Rust, something else. But they are too low-level, and we just do things for a long time ... We still have to choose some language. Let in the end it will be JS, come down.

Database

Base. JS without a document base - money down the drain. Document bases have their advantages. They allow us to quickly develop prototypes, do not sweat about the scheme, kolbasit data back and forth. It is a lot of pluses, minus one: porridge from data. When we have ten, twenty or forty collections instead of three, when we are trying to glue something good and digestible out of the absence of schemes, it becomes harder and harder to do it. Again we do features for a long time.

Ok, let's take a relational database. MySQL, PostgreSQL, or Oracle, if there is enough money. With relational databases, you can one day come to work and find yourself in hell from transactions and storage. This does not necessarily happen with our project. But if it happens, then these intricacies of logic will be impossible to test. And even if we suddenly reach the vertical limit of our large golden server, on which we host the base, then it will be rather difficult to separate them. Long do the features.

Okay. They took some base, banged up ORM in front of it, so that it was easier to move from one SQL to another. Someday (spoiler: never).

Architecture

What architecture to take? The guys on Habré write that microservices are cool. Oleg Bunin says: “take microservices”.

If we start with microservices, then they will have wrong boundaries with an eighty-percent probability, because they have not fully thought through the domain model and have a poor understanding of where to cut and where not to. Plus, everyone takes microservices, deploit them in packs all over their cluster, and a month later the question arises: “how should we test all this now?”. Services are already in production, and we are not testing them. Using familiar methodologies (pyramid testing, manual integration tests, end-to-end tests), it is difficult to test microservices. Therefore, we make features for a long time.

Ok, then let's bahney the monolith. This is the right idea for a startup. For a very long time, you can live with a great monolith and have no problems. But if we decide to greatly expand the team, then we must be careful. The monolith is normally scaled, while the developers are 20, 30, 50. Next, the feature delivery speed drops exponentially, and we lose users.

Where to run the project?

All this should be started somewhere. 2018, the most logical option to do it in the cloud. You will not run in the cloud - the boys will laugh. But, first, there is a federal law 152, which significantly limits the choice of cloud providers that can be hosted. Secondly, it is very easy to commit a private key to your account on Amazon to Github, and someone will come and spend all your money. And if this does not happen, then at some point cloud tariffs will ruin you.

You can rent a data center. Maybe this is not so resource-effective initially, but in the long run, it will probably be cheaper than hosting in the cloud. But here we need people who will support it. In my experience, those who love it and know how to do, do not like to communicate with everyone else, so they are organized in the department. And the department is separatism. I mean that within the admin team it will be easier to share experiences, but in the future this may not work very well. There will be issues with prioritizing tasks from other colleagues, with synchronization. Other experts will not know what is happening inside the department that supports our data center.
In general, separatism does not suit us. Logically go to the question of recruiting a command.

Team

Development

Suppose we figured out languages, databases and where to host the project. It is time to recruit a team. You can take some very cool guys who will solve all the problems: 100-fold developers, backend ninjas, you understand. Perhaps this is a ride. But in reality, it’s likely that guest stars will be:

• toxic dudes who will not do anything and create a bad atmosphere in the team,
• or idealists building up immaculate architecture bit by bit, placing ORM in front of bases that will never have to be changed ...

In the end ... yes, for a long time we make features. Another option is to take ordinary girls and guys who will just write code, make features fine. But if you take a lot of not very experienced developers with different backgrounds, they can write code in a different style, do things differently, and with a sufficient team size, everyone will be crowded, everyone will have each other's curly braces rearranged in pull-walks. It is not very effective. How can this be solved? The boss can read the entire code. I can read all pullrequests, and my friend and co-founder Valerka will then re-read the second time (just in case, you never know). Clearly, it does not scale and all slowly make features.

A more correct option is to define code style for the company. For many languages, it already exists, and you can simply follow it. Or if someone really wants something, you can take it ready and make a little bit of a bit, and then look at pullrexves and say that the curly brace is not there, it should stand there by the style. You can’t argue with such an argument, but in reality it’s not much better than the previous version, anyway, we are slowly making features. The correct option for all modern languages ​​is to check this automatically.

OK. Recruited developers, figachim code. But we started to release features in production, and we need to somehow make sure that we roll them without bugs, that nothing falls.

Quality Assurance

We can say that we do not need QA-specialists. Many people do it, it sometimes works. But not all developers like to write tests. They can be understood. And it is better to motivate them, so that the tests still write, but the reality is cruel: unit-tests are caught by no means all bugs. And if some developer does not like to write tests and still started writing them, then most likely it will be unit tests.

Plus, there are still approaches when you minimize the mean time between failures instead of mean time to recover. Mean time between failures is when a QA specialist says: “we will not release, I have bad instincts, there will be bugs, let's roll it out in two weeks”. And mean time to recover is when you roll something, you immediately see on the metrics that something has broken, and after two minutes everything is rolled back, fixed and everything is ok. But in order to roll back the project in two minutes, everything must be covered with normal metrics, and this is not always trivial. And if the metrics are in a deplorable state, and we roll out a bad release, we can learn about it after all users leave us for the competition.

Another option: still make the QA department. You remember: the department is not very good, it is separatism, it does not suit us. Separatism can be resolved with the help of cross-functional teams. Yes, they solve the problem of the fact that our admin is sitting separately, testers are separate.

But they create other problems. Since developers, testers, and all other members of cross-functional teams begin to communicate more within their teams and solve previous problems, they less communicate with their colleagues in function: other back-tenders and testers, they begin to reinvent bicycles, do the same things in parallel, isolation between teams. Shilo on soap: there was one separatism, it became another.

How it to settle? Chat with colleagues in hobby groups. Somewhere it is called guilds, somewhere - the community. If we scale the team with cross-functional teams so that they do not become self-contained, we will simply organize a circle of backend lovers, functional languages, security ...

Results

In fact, not everything is so bad. You can find a way out of any situation, find a solution. Maybe not perfect, but the most appropriate in this situation with a minimum of problems. A compromise is always possible.

And yet - all this is interesting. It is interesting to solve problems that someone has already solved, new problems are even more interesting to solve. It is interesting to share knowledge.