5 stages of microservice development
Dedicated to all those who have heard a lot about microservices, but have not decided to do a single one.
For reasons of creation and description of all the pros and cons, hundreds of articles have already been written - here I will talk about the practical side of creation and use.
So, after much deliberation, you decided to do this miracle, maybe because you had a task on the project that suited well to the microservice approach or everyone around you wrote a few, but you haven’t.
The article does not focus on how to write microservices, each has its own methods, here I will outline in general terms the stages of development and delivery of a service to production and test.
I divided the creation process into stages. And here is the first one:
First, we determine what tasks the microservice will solve. Usually, this is access to some resource on the server (database, queue, or file system) with a little logic on the service itself.
Suppose we need to implement data return for display on a map from slow storage. The rendering itself will naturally be done by the microservice client. The data represents the coordinates of the points on the map with some kind of heading.
There is a problem, now we are determined with the technologies for solving it. The data warehouse is slow and cannot handle a large stream of requests. In such cases, you can use elasticsearch - it copes with heavy loads and can shard and replicate out of the box. We put the data in it in a separate script for subsequent search on them.
Since our company “Wheels Roof Market” decided to write microservices on Go, and I have experience writing on Go, it was decided to write on Go. In your case, it can be anything - Nodejs, Java, PHP, or any other, preferably, compiled language. Since the technology stack also depends on the architecture of the service, at this stage you need to think about it, and also think through the logic of work. It is worth collecting information from all customers who will use our microservice, in particular from the mobile development team, if any.
After all the thought, at the expense of the stack of technology and architecture, the logic of the service, we proceed directly to the coding. Whether we are going to write a codebase from scratch (I do not recommend it) or if we are using some kind of ready-made framework.
In addition to the main service methods, we describe a special stats method that will return service information via http: the microservice version, the number and codes of processed requests, as well as the launch date - will be used for service purposes and, in particular, for healthcheck. In order to be able to run a microservice in different environments (development, testing, production), you need to create configs for each environment.
After writing the main part of the code, we proceed to the documentation of the service, because this is the most important part - there is no documentation, there is no service . We describe absolutely all the features of microservice. For these purposes, it is best to create a Readme file for synchronous updating at the root of the project. Or you can apply a swagger if someone loves prettier.
In principle, we can say that at this stage the development of the microservice is completed, since the code is written, the documentation is written, and you tested it (honestly). But really not. Since the most important, in my humble opinion, issue of service delivery to production has not been resolved. And now we have come to the last stage - delivery.
For these purposes, we will use the CI manager and Docker Swarm technology .
I will talk about the assembly in terms of Bamboo (since we use it at home).
There are 2 related build plans involved in the build cycle. The first is to compile the binary file from the microservice source code pulled from the repository. Compilation takes place in a special container with the version of Go we need. At the forefront, we will have an artifact with a binary ready to run. Immediately after the successful completion of the first plan, the dependent (second) plan starts. In the background - a docker image is being prepared with the service itself, its configs and deployment parameters from another repository.
For convenience, we configure the first build plan so that when changes fall in the master branch, the assembly starts automatically.
Next, you need to create and configure the Deployment Project for the deployment of the microservice. In this project, we describe two environments - Test and Production.
Immediately after a successful deploy of each replica, an attempt will be made to launch it in in case of failure, the deployment stops so as not to put the remaining running replicas.
We configure the trigger for the Test environment, so that when the master branch is successfully assembled, the deployment for the test is carried out automatically.
That's all, now, when changing the master branch, the build of the binary will start, then the image will be built, after which an automatic deployment will take place in the Test environment, and a special Deploy button will be active for the Production environment .
From personal experience I’ll say that the main problem and the complexity of microservices is their delivery to the production server, but if you have people who are friends with the docker, then the problems become many times less. Previously, we used deb packages for the deployment, which forced us to do “manual work” every time the service was updated, and the service was degraded due to the fact that it had to be restarted after installing a new package.
I managed to identify 5 main stages for the development of microservice, I hope this will benefit you and breathe new life into your life that we developers are engaged in.
For reasons of creation and description of all the pros and cons, hundreds of articles have already been written - here I will talk about the practical side of creation and use.
So, after much deliberation, you decided to do this miracle, maybe because you had a task on the project that suited well to the microservice approach or everyone around you wrote a few, but you haven’t.
The article does not focus on how to write microservices, each has its own methods, here I will outline in general terms the stages of development and delivery of a service to production and test.
I divided the creation process into stages. And here is the first one:
1. The challenge
First, we determine what tasks the microservice will solve. Usually, this is access to some resource on the server (database, queue, or file system) with a little logic on the service itself.
Suppose we need to implement data return for display on a map from slow storage. The rendering itself will naturally be done by the microservice client. The data represents the coordinates of the points on the map with some kind of heading.
2. Technology
There is a problem, now we are determined with the technologies for solving it. The data warehouse is slow and cannot handle a large stream of requests. In such cases, you can use elasticsearch - it copes with heavy loads and can shard and replicate out of the box. We put the data in it in a separate script for subsequent search on them.
Since our company “Wheels Roof Market” decided to write microservices on Go, and I have experience writing on Go, it was decided to write on Go. In your case, it can be anything - Nodejs, Java, PHP, or any other, preferably, compiled language. Since the technology stack also depends on the architecture of the service, at this stage you need to think about it, and also think through the logic of work. It is worth collecting information from all customers who will use our microservice, in particular from the mobile development team, if any.
3. Coding
After all the thought, at the expense of the stack of technology and architecture, the logic of the service, we proceed directly to the coding. Whether we are going to write a codebase from scratch (I do not recommend it) or if we are using some kind of ready-made framework.
A lot about Go implementation
It was decided to start github.com/labstack/echo - it seemed to me the most lightweight and lively project. Further, since we have to go to the elastic for data, we need to select the github.com/olivere/elastic to work with the elastic . For logging we take github.com/sirupsen/logrus . As a monitoring, you can take github.com/rcrowley/go-metrics or something simpler github.com/cactus/go-statsd-client .
We include all the dependencies in main.go, and in the same file we describe the processing of all public methods.
We include all the dependencies in main.go, and in the same file we describe the processing of all public methods.
In addition to the main service methods, we describe a special stats method that will return service information via http: the microservice version, the number and codes of processed requests, as well as the launch date - will be used for service purposes and, in particular, for healthcheck. In order to be able to run a microservice in different environments (development, testing, production), you need to create configs for each environment.
4. Documentation
After writing the main part of the code, we proceed to the documentation of the service, because this is the most important part - there is no documentation, there is no service . We describe absolutely all the features of microservice. For these purposes, it is best to create a Readme file for synchronous updating at the root of the project. Or you can apply a swagger if someone loves prettier.
In principle, we can say that at this stage the development of the microservice is completed, since the code is written, the documentation is written, and you tested it (honestly). But really not. Since the most important, in my humble opinion, issue of service delivery to production has not been resolved. And now we have come to the last stage - delivery.
5. Delivery
For these purposes, we will use the CI manager and Docker Swarm technology .
I will talk about the assembly in terms of Bamboo (since we use it at home).
There are 2 related build plans involved in the build cycle. The first is to compile the binary file from the microservice source code pulled from the repository. Compilation takes place in a special container with the version of Go we need. At the forefront, we will have an artifact with a binary ready to run. Immediately after the successful completion of the first plan, the dependent (second) plan starts. In the background - a docker image is being prepared with the service itself, its configs and deployment parameters from another repository.
For convenience, we configure the first build plan so that when changes fall in the master branch, the assembly starts automatically.
Next, you need to create and configure the Deployment Project for the deployment of the microservice. In this project, we describe two environments - Test and Production.
Deployment is carried out through your crutching python-deploiterDeployment parameters, depending on the environment, indicate which cluster to deploy (we currently have 4), the number of replicas, processor, memory limits, etc.
Immediately after a successful deploy of each replica, an attempt will be made to launch it in in case of failure, the deployment stops so as not to put the remaining running replicas.
We configure the trigger for the Test environment, so that when the master branch is successfully assembled, the deployment for the test is carried out automatically.
That's all, now, when changing the master branch, the build of the binary will start, then the image will be built, after which an automatic deployment will take place in the Test environment, and a special Deploy button will be active for the Production environment .
From personal experience I’ll say that the main problem and the complexity of microservices is their delivery to the production server, but if you have people who are friends with the docker, then the problems become many times less. Previously, we used deb packages for the deployment, which forced us to do “manual work” every time the service was updated, and the service was degraded due to the fact that it had to be restarted after installing a new package.
I managed to identify 5 main stages for the development of microservice, I hope this will benefit you and breathe new life into your life that we developers are engaged in.