
Common Code Issues in Microservices
Hello!
Recently, at a PGConf conference in Moscow, one of the speakers demonstrated a “microservice” architecture, mentioning in passing that all microservices inherit from one common base class. Although there were no explanations for the implementation, it seemed that in this company the term “microservices” was not understood in the same way as the classics seemed to teach us. Today we will deal with one of the interesting problems - what can be the common code in microservices and whether it can be at all.
What is a microservice? This is a standalone application. Not a module, not a process, not something that is simply deployed separately, but a full-fledged, real, separate application. It has its own main function, its own repository in the git, its own tests, its own API, its own web server, its own README file, its own database, its own version, its own developers.
Like containers, microservices began to be used when the computing power of HW and the reliability of networks reached such a level that you can afford a function call that lasts 100 times longer than before, you can afford memory consumption 100 times higher, you can afford luxury to settle each "grandmother" not just in a separate "apartment", but in a separate "house". Like any architectural solution, the architecture of microservices once again sacrifices performance, winning in maintainability of the code for developers. But since the person and the speed of his reaction remained the same, the systems continue to satisfy the requirements.
Why split into separate applications? Because we distribute part of the complexity of the system already at the level of system architecture. The programming process is generally speaking a phased “biting off” of the large initial “piece of complexity”, and decomposition (into classes, modules, functions, and in our case, entire applications) is the implementation of part of this complexity in the form of a structure. When we split the system into microservices, we made an architectural decision (successful or not), which the developers no longer need to take in the future when implementing specific parts of the functionality. It is known that this particular microservice is responsible for sending emails, and this one - for authorization, has already been established, so all my new features “fall” on this pattern without discussion.
A key aspect of microservices is poor connectivity. Microservices should be independent of the word "completely." They do not have common data structures, and each microservice may / should have its own architecture, technology, assembly method (and so on). By definition. Because it is an independent application. Changes in the code of one microservice should not affect the others in any way, unless the API is affected. If I have N microservices written in Java, then there should not be any constraining factors not to write the N + 1st microservice in Python, if this is suddenly profitable for some reason. They are loosely coupled, and therefore a developer who starts working with a specific microservice:
a) Very sensitively monitors its API, because it is the only component visible from the outside;
b) Feels completely free in refactoring;
c) Understand the purpose of microservice (here we recall about SRP) and implements a new function accordingly;
d) Selects the persistence method that is most suitable;
etc.
All this is good and sounds logical and harmonious, like many ideologies and theories (and here the ideological theorist puts an end and goes to dinner), but we are practicing. The code does not have to be written on martinfowler.com . And sooner or later we are faced with the fact that all microservices:
and do it identically.
And at some point, the ideological architect comes to work in the morning and discovers that at night a “library” appeared in the system - a new repository with a common code that is used in many microservices. Should an architect be horrified?
It depends.
To correctly assess the situation, we should return to the main idea: microservices are a collection of independent applications that interact with each other through a (network) API. In this we see the main advantage and simplicity of architecture. And we do not want to lose this advantage under any circumstances. Does the general code that was placed in the “library" interfere with this? Let's look at some examples.
1. The user class (or some other business entity) lives in the library.
Unfortunately, any business logic placed in a shared library will have such an effect. General code libraries tend to grow, resulting in the middle of the system forming a logical “tumor” that does not belong to any particular microservice, and the architecture crashes. The “center of logical gravity” of the system begins to move into a repo with a common code, and we get a hellish mixture of monolith and microservices, and we don’t need to go there at all.
2. The parsing code for the message format is placed in the library.
A message parser, or an improved logger, or a wrapped client for sending data to RabbitMQ - it's kind of like helpers, auxiliary components. They are on par with standard libraries from NuGet, Maven or NPM. The microservice developer is always the king; he decides whether to use the standard library, or make his own new code, or use the code from the shared helper library. How it will be more convenient for him, because he writes a SEPARATE AND INDEPENDENT APP. Can a particular helper evolve? Maybe he will probably have versions. Let the developer refer to a specific version in his service, no one forces him to update the service, when updating helpers, this is a question for who supports the service.
3. Java interface, abstract base class, trait.
The team starting to work on a new product lays the foundation for architecture and has the greatest influence on what trends the product will have. If the principles of SRP, successful decomposition, low connectivity, etc. are initially incorporated in the system, then it has a chance to continue to develop correctly. If not, then the centrifugal acceleration of the “time factors” (another team, little time, urgent patches, lack of documentation) will throw this system further to the sidelines faster than it seems.
The question of a common code in microservices remains difficult because it is associated with some sort of trade-off: we weigh what will be more profitable in the future - the degree of independence of microservices, fewer repetitions in the code, the qualifications of engineers, the simplicity of the system, etc. Each time these are reflections and discussions, which can lead to different specific architectural decisions. Nevertheless, let us summarize some of the recommendations:
Recommendation 0: Do not call microservices any thing that is broken into independently existing pieces. Not every table with columns is a matrix, let's use the terms correctly.
Recommendation 1: It is highly desirable that microservices have no common code at all.
Recommendation 2: If there is still a common code, let it be a collection (library) of optional “helpers”. The service developer decides whether to use them or write his own code.
Recommendation 3: Under no circumstances should there be business logic in the common code. All business logic is encapsulated in microservices.
Recommendation 4: Let the common code library be designed as a standard package (NuGet, Maven, NPM, etc), with the option of versioning (or, even better, several separate packages).
Recommendation 5: The “center of logical gravity” of the system should always remain in the microservices themselves, and not in the common code.
Recommendation 6: If you plan to write in the format of microservices, then reconcile in advance that the code between them will sometimes be duplicated. To some extent, our natural “DRY instinct” must be suppressed.
Thank you for your attention and successful microservices.
Recently, at a PGConf conference in Moscow, one of the speakers demonstrated a “microservice” architecture, mentioning in passing that all microservices inherit from one common base class. Although there were no explanations for the implementation, it seemed that in this company the term “microservices” was not understood in the same way as the classics seemed to teach us. Today we will deal with one of the interesting problems - what can be the common code in microservices and whether it can be at all.
What is a microservice? This is a standalone application. Not a module, not a process, not something that is simply deployed separately, but a full-fledged, real, separate application. It has its own main function, its own repository in the git, its own tests, its own API, its own web server, its own README file, its own database, its own version, its own developers.
Like containers, microservices began to be used when the computing power of HW and the reliability of networks reached such a level that you can afford a function call that lasts 100 times longer than before, you can afford memory consumption 100 times higher, you can afford luxury to settle each "grandmother" not just in a separate "apartment", but in a separate "house". Like any architectural solution, the architecture of microservices once again sacrifices performance, winning in maintainability of the code for developers. But since the person and the speed of his reaction remained the same, the systems continue to satisfy the requirements.
Why split into separate applications? Because we distribute part of the complexity of the system already at the level of system architecture. The programming process is generally speaking a phased “biting off” of the large initial “piece of complexity”, and decomposition (into classes, modules, functions, and in our case, entire applications) is the implementation of part of this complexity in the form of a structure. When we split the system into microservices, we made an architectural decision (successful or not), which the developers no longer need to take in the future when implementing specific parts of the functionality. It is known that this particular microservice is responsible for sending emails, and this one - for authorization, has already been established, so all my new features “fall” on this pattern without discussion.
A key aspect of microservices is poor connectivity. Microservices should be independent of the word "completely." They do not have common data structures, and each microservice may / should have its own architecture, technology, assembly method (and so on). By definition. Because it is an independent application. Changes in the code of one microservice should not affect the others in any way, unless the API is affected. If I have N microservices written in Java, then there should not be any constraining factors not to write the N + 1st microservice in Python, if this is suddenly profitable for some reason. They are loosely coupled, and therefore a developer who starts working with a specific microservice:
a) Very sensitively monitors its API, because it is the only component visible from the outside;
b) Feels completely free in refactoring;
c) Understand the purpose of microservice (here we recall about SRP) and implements a new function accordingly;
d) Selects the persistence method that is most suitable;
etc.
All this is good and sounds logical and harmonious, like many ideologies and theories (and here the ideological theorist puts an end and goes to dinner), but we are practicing. The code does not have to be written on martinfowler.com . And sooner or later we are faced with the fact that all microservices:
- log information;
- contain authorization;
- Access message brokers
- return the correct error messages;
- must somehow understand the general entities in the system, if any;
- must work with a common message format (and protocol);
and do it identically.
And at some point, the ideological architect comes to work in the morning and discovers that at night a “library” appeared in the system - a new repository with a common code that is used in many microservices. Should an architect be horrified?
It depends.
To correctly assess the situation, we should return to the main idea: microservices are a collection of independent applications that interact with each other through a (network) API. In this we see the main advantage and simplicity of architecture. And we do not want to lose this advantage under any circumstances. Does the general code that was placed in the “library" interfere with this? Let's look at some examples.
1. The user class (or some other business entity) lives in the library.
- those. a business entity is not encapsulated in one microservice, but spread out differently (otherwise why put it in a shared code library?);
- those. microservices become connected through this business entity; changing the logic of working with an entity will affect several microservices;
- it’s bad, very bad, it’s not microservices at all, although it’s not “big ball of mud”, but very quickly the team’s worldview will lead to “big ball of distributed mud”;
- but microservices in the system work with the same concepts, and concepts are often entities, or just structures with fields, what should I do? read DDD, it’s exactly about how to encapsulate entities inside microservices so that they don’t “fly” through the API.
Unfortunately, any business logic placed in a shared library will have such an effect. General code libraries tend to grow, resulting in the middle of the system forming a logical “tumor” that does not belong to any particular microservice, and the architecture crashes. The “center of logical gravity” of the system begins to move into a repo with a common code, and we get a hellish mixture of monolith and microservices, and we don’t need to go there at all.
2. The parsing code for the message format is placed in the library.
- The code is most likely in Java if all microservices are written in Java;
- If tomorrow I write a service in Python, I won’t be able to use the parser, but it seems like it’s not a problem at all, I’ll write a Python version;
- Key point: if I am writing a new microservice in Java, am I required to use this parser? Yes, probably not. Perhaps I’m not obliged, although it, as a microservice developer, can be very useful. Well, as if I found something useful in the Maven Repository.
A message parser, or an improved logger, or a wrapped client for sending data to RabbitMQ - it's kind of like helpers, auxiliary components. They are on par with standard libraries from NuGet, Maven or NPM. The microservice developer is always the king; he decides whether to use the standard library, or make his own new code, or use the code from the shared helper library. How it will be more convenient for him, because he writes a SEPARATE AND INDEPENDENT APP. Can a particular helper evolve? Maybe he will probably have versions. Let the developer refer to a specific version in his service, no one forces him to update the service, when updating helpers, this is a question for who supports the service.
3. Java interface, abstract base class, trait.
- Or another piece from the category of "torn piece of code";
- Those. I’m here, independent and independent, and a piece of my liver lies somewhere else;
- Here the coherence of microservices at the code level appears, so we will not recommend it;
- At the initial stages, this probably will not bring any tangible problems, but the essence of architectural design is the guarantee of comfort (or discomfort) for years to come.
The team starting to work on a new product lays the foundation for architecture and has the greatest influence on what trends the product will have. If the principles of SRP, successful decomposition, low connectivity, etc. are initially incorporated in the system, then it has a chance to continue to develop correctly. If not, then the centrifugal acceleration of the “time factors” (another team, little time, urgent patches, lack of documentation) will throw this system further to the sidelines faster than it seems.
The question of a common code in microservices remains difficult because it is associated with some sort of trade-off: we weigh what will be more profitable in the future - the degree of independence of microservices, fewer repetitions in the code, the qualifications of engineers, the simplicity of the system, etc. Each time these are reflections and discussions, which can lead to different specific architectural decisions. Nevertheless, let us summarize some of the recommendations:
Recommendation 0: Do not call microservices any thing that is broken into independently existing pieces. Not every table with columns is a matrix, let's use the terms correctly.
Recommendation 1: It is highly desirable that microservices have no common code at all.
Recommendation 2: If there is still a common code, let it be a collection (library) of optional “helpers”. The service developer decides whether to use them or write his own code.
Recommendation 3: Under no circumstances should there be business logic in the common code. All business logic is encapsulated in microservices.
Recommendation 4: Let the common code library be designed as a standard package (NuGet, Maven, NPM, etc), with the option of versioning (or, even better, several separate packages).
Recommendation 5: The “center of logical gravity” of the system should always remain in the microservices themselves, and not in the common code.
Recommendation 6: If you plan to write in the format of microservices, then reconcile in advance that the code between them will sometimes be duplicated. To some extent, our natural “DRY instinct” must be suppressed.
Thank you for your attention and successful microservices.