From idea to release. Detailed Market Frontend Experience

I always want to come up with something new and necessary in my service. Especially if users like this service. But where do you get the ideas from? How to prioritize? And how to quickly bring an idea to a product without losing anything important along the way?

My name is Alexander, I lead one of the interface development groups in Yandex.Market. Today I will tell the readers of Habr about our experience in solving these problems. We also consider an example of delivering features to production.

Command

Yandex.Market is Russia's largest resource for selecting goods and comparing prices. Every day, 3.5 million people use it - they plan their future purchases here, discuss products and help other people make the right choice.

Now Yandex.Market employs 40 front-end developers, and their number is growing. Yes, 40 people - this is only the frontend of the Market, and there are also fronts of our two other projects - the marketplaces of Beru and Bringly. In 2005, there were only 5 developers in the Market, imagine?

Ideas are born in our circuits. So we call teams consisting of different specialists. Typically, a circuit includes: a project (aka a product technologist), back-end, front-end, analysts, designers and testers. Such a team can independently develop something in the service and solve problems of any complexity. Moreover, each circuit has a clear perimeter of responsibility - a certain “functional circuit” in the product on which it works.

We have, for example:
- conversion circuit - deals with UX, search, filtering and sorting tools;
- communications circuit - works on newsletters and promotions for our users;
- benefit contour - deals with discounts, cashback, promotional codes;
- Community circuit - responsible for reviews, reviews, discussions, achievements, game mechanics and other UGC.

Ideas

Any member of a circuit can offer an idea. We welcome all options, even trivial or absolutely crazy. Usually the guys generate ideas in general meetings - this charges the whole team to work.

In order not to lose anything, we put all our thoughts in a separate turn in the Tracker - our bank of ideas. The collected ideas are evaluated according to the GIST + ICE framework. We evaluate the potential of each idea and the labor costs of implementation to decide what it is worth taking in the first place. More details about this method were told by my colleague.

The ideas that scored the most points turn into projects with the customer and performers. A tremendous set of design tasks appears in the Tracker. Tasks are distributed by the team leader for the sprints and entered the team.

The important point is that the idea is implemented by the circuit that suggested it. That is, the authors of the idea are also its performers. Try to work on your task, and not on a task launched from above - the difference will be easy to feel.

Process

Suppose we decided to develop a carousel of goods for the user, taking into account his interests.

Suppose that the loop backenders have already done their work, the neural network is trained, the API is ready, tested and deployed in a test environment. Layouts from the contour designer are received and look something like this: It



remains to do the work on the front-end and deliver our carousel to the user.

The Frontend of the Market is a web client and stateless NodeJS server, behind which there are dozens of other backends: search, advisory services, analytics, UGC content and more. The frontend lives in the Yandex clouds in several DCs. The NodeJS application is executed in containers, and due to its stateless nature it can be scaled horizontally as we need:



Market front-end developers are developing both the server side on NodeJS and the client on React + Redux. Developing in one language and in one ecosystem is very effective. If you are still thinking about unifying your technology stack between the server and the client, try - that makes sense.

To finalize the carousel, we need to add the code for receiving goods on the server and implement the client part. You can make the carousel lazy - load it just before it appears on the screen. To do this, you will have to additionally implement endpoint in the client API.

When this is done, the developer creates a pool request and sends his code to the code review. Our code review is a must. So that it does not hang, we even have a recommended SLA for its implementation. About how we accelerated the code review, you canread here .

We are developing the front-end at GitHub Enterprise, which is available only on the internal network. There is a bot on our Github that helps organize code reviews. The bot itself finds the reviewers, contacts them in the messenger, adds them to GitHub, controls the ticket status in the Tracker and, if necessary, calls the author of the code.

At the time of creating the pool request on GitHub, a lot is happening: in the test environment, a demo stand with an updated Market is raised, and access links to this stand are delivered to the task ticket. A lot of automated checks take place: linter, formatter, unit-tests. E2E tests are run, client application metrics are automatically analyzed. If any degradation is detected, a detailed report on it is immediately delivered to the task ticket:



If all the checks have passed normally, the ticket is automatically transferred to the “You can check” status, and the task appears on the dashboard of the QA team.

After the QA engineer came to the task ticket and made sure that the new functionality is ready for further travel, the ticket is marked with a release tag. The next release will include all tickets with this tag.

The market is often released, 1-2 times a day, and we are working to increase the frequency of our releases. The person on duty from the QA team is involved in the preparation and release of the release, and the release process is fully automated. Previously, the release package was assembled by hand, it took a lot of time. Now developers can spend this time developing the product. This speeds up the delivery of new functionality to our users.

So, our merry-go-round of recommended products goes to release. How does this happen?

The on-duty QA engineer presses the release button. The release pipeline starts to work. The presentation of the release process report can be viewed here . A release ticket is created in the Tracker, and a release branch on GitHub. All pool requests that have passed checks are automatically poured into it. In a test environment, a stand with a new version of the Market rises. At this stand, all the above automated checks pass, plus a lot of additional ones. For example, load testing with constant load and increasing until the test application is debugged. All audit logs are analyzed on the fly, clear reports are generated, which are immediately delivered to the release ticket:



The QA engineer sees all the stages, with one glance he can evaluate the results of the mass of checks. In the test environment at the stand with the new release, QA also undergoes manual testing. We often conduct A / B testing of new functionality and do not write E2E tests on it.

This is how the release pipeline (clickable) looks like: When the release has passed all the checks in the test environment, it is delivered to the prestable. Prestable is another environment, but with combat backends, closed from real Market visitors. Some companies call it staging. In this environment, a series of specific checks and manual functional testing also take place. If no problems are found, the new version of the Market is sent to our users, it begins to unfold on the production cluster.





At the time of layout for the battle, the work of robots continues. They independently analyze the application logs, look for anomalies and degradation in the work of the service, and monitor client metrics. Log monitoring continues for some time after the release, and the next version of the Market is already undergoing testing in the test environment.

The health of the Market after the release is also monitored by the on-duty QA. He has a large number of graphs at his disposal - we use Graphit and Grafana. The graphs have everything: from the general 500-k background to the errors of the NodeJS application when communicating with various backends broken down by DC. We also have a separate group of people - the Market health group. She monitors Market metrics 24/7.

If a problem is detected, the release automatically rolls back from the combat cluster. The problem is dismantled and the release is returned to battle. If there are no problems, the release is completed, the code merges into master, the tickets that got into the release are closed, and the project branches are deleted. All this also happens automatically.

At this moment, our users already see the products selected for them, and the circuit is working on a new idea.

Here is our carousel on the service:

Instead of a conclusion

Make your teams flexible and independent, let them generate ideas, take care of these ideas, analyze them - there are definitely diamonds among them. Remember that the idea expressed by someone in an uncertain, quiet voice during a meeting can be key to your product and its users.

Let their authors embody ideas as much as possible - so the work goes faster and better. Automate the routine. Take time to translate ideas into reality, let the robots do the rest.

If the reader is interested in talking about something in more detail, write in the comments, I will answer with pleasure, or maybe I will describe this in a new article.