This issue should be decided by the architect. Or not?

    I have some experience in implementing systems based on microservice architecture and I would like to share the questions (and answers) that arise during the implementation of such projects. Unfortunately, I do not have the right to spread about the projects in which I participated, so I came up with my own spherical project in a vacuum. In this project, we will meet many standard problems.

    I want to immediately note that the implementation will be rudimentary and serves only as a basis for raising questions. In any case, I hope you find a couple of interesting thoughts and links in the article.

    We will see how many interesting moments can arise when writing only three classes and ask ourselves whether in this case the architect should decide or the developer can solve this problem himself.

    image

    The main idea of ​​the project


    So, imagine that a brighter future has come and autopilot cars briskly run through cities and villages. The question arises: do you need a personal car in such conditions?

    I offer you a different approach. Look, you go to my site, register, fill out the form. The questionnaire will be something like this: I want to leave the house every day from Monday to Friday at 8:15 (point A), get into a Mercedes and get to work (point B). Then in the evening at 18:00 to leave the office (point B), sit in the Audi and get to the house (point A). There you can still tick off the point: I want to be able to leave work earlier, I’m ready to wait no more than 8 minutes.

    You can also call the car at any convenient time, the entrance time is not more than five minutes. The only caveat for such cases, the car you will use no more than five hours a week. This is for poorly planned cases or, for example, for a trip to the store.

    For all the pleasure they will ask you 150 cu per month. Sounds good, doesn't it?

    Naturally, under the hood, this service will have a lot of logic, because you have a lot to consider. For example, we must be competitive, i.e. it is necessary to offer people a cheaper option. For this, you can provide for joint trips. We can see that our client’s neighbor, it turns out, is also going in the same direction, but 5 minutes later. Can offer them a ride together, and for this offer a discount?

    Our service will give us a lot of data from which we can get a lot of interesting information:

    • who, where, when and with whom goes.
    • who, where and how much time spends
    • what, when, which car models breaks down
    • who prefers which cars
    • ... ..

    Based on this information, we can draw useful conclusions. For example, since we know where and when our customers go, we can predict traffic jams and adjust the route accordingly so as not to fall into them.

    In order to make this possible, we need to provide for the possibility of collecting and analyzing information.

    In general, we are talking about a large and complex project, which can be called the buzzword “cognitive solution” for a business with elements from the world of Internet of Things.

    Development Methodology and Requirement Analysis


    Naturally, first you need to analyze all the requirements for the project, decide which methodology we will apply (waterfall, rup, scrum, ...). But in this case, we will skip all these steps, because almost all the questions raised in this article will arise in any case, regardless of the chosen methodology.

    Language, framework, architecture


    Initially, I am Java Developer, and therefore the implementation will be in java. Do not blame me.

    By the way, here’s the immediate question: is the choice of a programming language an architect’s task or is it “above this”?

    For those who think that there is no, that the programming language is secondary and in general, only the team is important, I propose to conduct a small thought experiment. Think in what language you would definitely not do such a project, i.e. you think that this would be a clearly wrong decision. Now imagine that the architect decided to make the project in this language. And when you angrily resent, he will tell you: this is a secondary question, the main thing is the team!

    As it was said at the very beginning, we will do it on the basis of microservice architecture. Someone will say that it would be more accurate to start with a monolith and we will agree with him, but we will start right away with microservices.

    And which framework do we take? If you google a little, it will become clear that there aren’t much options, we will do it on the Spring Framework. The reason is simple, Spring Cloud has everything we need.

    We will also have all sorts of API Gateways, Config Services, Message Brokers, Docker, Workflow, Rule Engine and many other trivial words.

    There are two main approaches to microservice design.
    • Domain driven design
    • Functional Driven


    Domain Driven Design means defining domain objects and implementing all the necessary actions that your customer requires. For example, the customer of some pharmacy system says: I need to be able to add new medicine to the system, delete old ones, but editing the already introduced medicine should be prohibited. You make the “Medicine” class with all the necessary fields and implement the named functionality. So you have a MedicineService . Those. with this approach, the starting point is domain objects.

    Functional Driven means that the necessary functionality is taken as the starting point, and what domain objects you have to attract is already a secondary matter.

    Personally, I always start with domain objects, and then check to see if the necessary functionality can be implemented in this way. If not, then I’m looking, maybe I need to add another domain object to the service? In theory, as soon as I had to do this, then it's time to look towards the “functional” approach.

    There are not many services with one domain object, you can immediately start with the functionality, but it’s easier for me.

    Let's decide what services we need. To do this, we will see which domain objects we definitely need, and then we will look at what services we will push them.

    Let's start with the transport. For example with Car . Although just Car will not work. I will explain. For example, suddenly, it will be convenient for a person to get to the station by car, transfer to the train, get to the city, and then ride the bike for the last two kilometers? After all, we want to ask money for this bike from this nice person, too? What if someone wants to drive this very last mile on a monowheel? Do not offend a person, especially if he has money for it? Let's rent him a monowheel! Thus, in the future, we may need many classes describing different vehicles.

    We take as a starting point some abstract class Vehicle

     public abstract class Vehicle {
    ….
       protected String model;
       protected int wheelNumber;
       protected Date manufactureYear;
       protected EngineType engineType; 
       protected Producer producer;
    }

    As we found out we will have different Vehicle , let's make a couple:

    public class Car extends Vehicle {
       public Car() {
           wheelNumber = 4;
       }
    }

    and one more for the poor but athletic:

    public class Bicycle extends Vehicle {
       public Bicycle() {
           wheelNumber = 2;
       }
    }

    Wonderful. Now we need the one for whom we all do this: our client, he is the source of our income. Let's call this entity Customer.

    public class Customer {
       private String firstName;
       private String lastName;
       private Date birthDay;
    }

    We also need a contract with the client, which will indicate which vehicle he will receive from us and how much money we will receive for this.

    public class Contract {
       private long customerId;
       private long vehicleId;
    }

    So we have one class hierarchy with Vehicle at the top, Customer and Contract . I propose to make VehicleService , ContractService and CustomerService out of them .

    What does 'micro' mean in the word microservice?
    Previously, I was tormented by the question, but what does “micro” mean in the word “microservice”? In theory, this means "small." But what does small mean?

    Often there is an opinion that a microservice should fit in the head of one person, or so that a team of three people and all that could make it. This certainly makes sense, but I, as another option, offer a slightly different perspective on this issue.

    When you implement, or better, if you are just thinking of making some kind of microservice, ask yourself: if I am very mistaken with this service, can I afford to throw it out and write a new one from absolute zero?

    If your answer is yes, then this is a microservice. And if not, then no. And what's important, ask yourself this question regularly as it progresses. If you suddenly got the “no” answer, start refactoring / splitting / rethinking this monster. Although usually by this time all polymers are already <.. censored ..>

    You can look at the rudimentary implementation of services here .

    Now let's put the whole thing in the docker. By the way, I was here recently at a meetup on the occasion of Docker's fourth birthday. An interesting link was presented to us there , it’s very nice in places, I recommend to look.

    To put our services in the docker, we need a plug-in for the maven (see pom.xml docker-maven-plugin) and dockerfile.

    We will launch all services through docker-compose, for this docker-compose.yml lies in the root of the project.

    Also pay attention to the .env file and its contents. You will find more about this file in the documentation . Without this file, I couldn’t initialize MySQL on my Windows 7 machine.

    What did I do?


    Let's start with the pros:

    • this thing works.

    On this, the pros unfortunately ended

    Minuses:

    Unfortunately, there are a lot of them, so we will consider only a couple of pieces selectively.

    Before I begin to analyze the shortcomings, I want to mention one small, but extremely important point: the microservice architecture initially assumes that at least two instances of each service work in the system, which share the load. If this rule is not respected, then, in my opinion, we can immediately stop talking about microservice architecture. Yes, I know, this is debatable, but my opinion is just that.

    Unfortunately, redundancy does not guarantee 100% availability of services and therefore, if there is another reasonable way to maintain the system’s performance, then it can and should be used.

    Adding a new type of transport to the system


    Class diagram is not architecture
    Often I hear this statement: a class diagram is not an architecture. This is explained approximately as follows: what and how there in the module did not interest me, it is important for me what the modules do and how they communicate with each other. As a rule, these are people who never worked as a programmer, but somehow somehow immediately became architects. What can I say to that? Maybe they are really right, but my experience suggests otherwise. And now we will consider just such a case.


    So, for starters, we ask ourselves, is the right approach to the decision to drive vehicles we have adopted? I will explain the question. For example, we want to offer our customers scooter rental.

    Now VehicleService looks like this.

    image

    We need to add a new Scooter entity to the system. Everything is based on inheritance, so the end result will look like this:

    image

    We will write a new Scooter class in VehicleService , test, compile, and deploy. And if we have dozens of types of vehicles? Each time we will write a new class, test, compile, etc.? Is there another way?

    You can, for example, do so. Let's make the VehicleType class .

    public class VehicleType {
        private String name;
        private List properties;
        ….
    }

    As you can see, VehicleType has VehicleProperty :

    public class VehicleProperty {
       private String name;
       private T value;
       private String description;
    …..
    }

    Let's make the Vehicle class :

    public class Vehicle {
      private VehicleType vehicleType ;
      private List customProperties;
       …...
    }

    Now, if we want to add a scooter to the system, we will first create the VehicleType “Scooter”:

    VehicleProperty wheelNumberProperty = new VehicleProperty("wheelNumber", 2, "number of wheels");
    …….
    VehicleType scooterType = new VehicleType("Scooter");
     scooterType.addProperty( wheelNumberProperty);
    ……..
    

    And if we need to create an instance of a scooter:

    Vehicle scooter1 = new Vehicle(scooterType);
    ….. 

    Thus, we can add as many new types to the system as possible without creating new classes in VehicleService .

    image

    Do you still remember the last lyrical digression about the class diagram and the question is whether it is a subject of architecture? Here are two class diagrams that describe two fundamentally different approaches to the implementation of the service. Are they part of the architectural solution? In my opinion, very, because in this case, the implementation method even at the class level has far-reaching consequences and can turn the future life of the project into hell.

    Hidden business case
    In general, the issue of changing something in the service requires special attention. I will explain with an example.

    I had such a case. At the rally I present to the client our future architecture. Answered the questions. Everything seems to be fine, everyone likes everything, everyone is happy. And here the main client IT specialist asks such a simple question: how quickly can you add a new field to the “ABC” domain object? A simple question, right? I just answered: add a field - 2 minutes, write tests from a few minutes to a couple of hours, then run all the tests (it may take hours), etc. In general, I could not name any specific figure and I think that no one can, until at least once this has been done. It seems like I answered correctly, but the feeling that the answer was incorrect did not leave me. And then one day I realized how I had to answer.

    To date, my answer is: “How often does this happen?” If this is an exceptional situation, then in principle it doesn’t matter how long the field is added, if only this period is adequate from the point of view of business. If this happens often, then one should ask the question: is this a Business case? And if so, then this functionality needs to be initially put into the system and then the answer should be: 20-30 minutes (this is a lie, of course, but it sounds good), it can take longer if the case is difficult.

    Another question arises: how did it happen that this business case surfaced just now?
    And an even more important question, are there any other similar business cases that we have missed?

    The next moment. See, if our VehicleService crashes, then we can neither create a new instance of the bike in the system (for example, we bought a new batch of bikes and want to add them to the system), or rent a bike. Those. neither our clients nor our office employees can do anything. It would be much better if even in case of problems at the office our customers could place orders and bring us money. How can I do that? It seems like we need to divide our VehicleService into two, one for customers and one for our employees.

    Remember, I was talking about a domain and functional approach to designing services. The problem with the availability of the service for customers and our employees is an excellent example when we started with a domain approach, ran into a problem and moved on to a functional approach. We need two services, their domain objects are basically the same, but the functionality is different.

    Suppose, again, our VehicleService crashes . This means that neither cars nor bicycles can be rented. It would not be bad if the service for cars is not available, then the service for bicycles would work further. How can I do that? Divide VehicleService into several services, one for each type of vehicle?

    In fact, the last two examples of possible problems are not entirely correct, because they can be resolved through redundancy, i.e. several instances of the service should work. But, as I said at the beginning, no redundancy will give 100% availability. That is why we must try to solve the issue of service availability in some other reasonable alternative way.

    A holiday happened on our street, a bunch of customers came running to us who want to rent a car from us. The service does not cope, but this is not a problem, we start another instance and everything is in order again. But now we have two copies of the service, not only for cars, but also for bicycles. Is it bad? I don’t know, but certainly not good. It is very possible that one can live with it, one must look.

    After considering these problems, you can offer another option for the implementation of the service. We will make a separate service for each type of transport. But such a question arises.
    Suppose you are a client, go to the site and want to look at a list of all possible types of transport, i.e. you want to see something like:

    • Car
    • Bicycle
    • Scoter
    • Traktor
    • ... ...

    Where does this list come from? Probably we also need the VehicleTypesService service , which will know what types of vehicles do exist in our system. Where does he get this information from? The first thing that comes to mind is to write to the database with pens. If we provide another type of transport for services, we write a service for it and do not forget to go to VehicleTypesService and add one more line to the database.

    But it’s possible in another way. At start, each service must knock on VehicleTypesServiceinform him of his existence. This solution seems to look good, but does not answer the question of what to do if we need not to add, but to remove the type of vehicle. For example, six months later, we realized that nobody uses monowheels, we want to remove it from the system. How do we do this?

    And here's another interesting question. As you can see, Vehicle has an EngineType field .

    public abstract class Vehicle {
       …...
       protected EngineType engineType;
     

    In my rudimentary implementation, EngineType has enums:

    • Gas
    • Diesel
    • Elektro

    And now the question itself: how do we create a car with a hybrid power plant?

    Instead of EngineType, let's create a list of EngineType 's (and then 99% of the machines will suddenly have a list with one element)?

    public abstract class Vehicle {
       …...
       protected List engineTypes;
     

    Or add a new type to enum, something like Gibrid ?

    Who in this case makes a decision and, accordingly, bears responsibility for it?

    Is it possible to say that we are talking about architecture here or is this a too “small” question?

    Using the example of the EngineType field, I want to ask one more question: do we really need to know what engine the car has? Indeed, it is absolutely unimportant for our client to be very likely, we pour diesel fuel or gasoline into the tank. But, for example, the ability to take a bicycle with you (i.e., is there a large trunk or are there special bike mounts) can be very important.

    In fact, the type of engine is of course important, but not for the client, but for us, the company that provides this service. One of the reasons is statistics (for example, on fuel costs or necessary repairs). It will vary greatly for each type of engine. This begs another question: should domain objects (or their representations) be different for our backend and frontend?

    How to answer questions of this kind? I know only one way: should I talk to people from business more often, ask them what and where they want to see? Unfortunately, they themselves often do not know the answers to these questions.

    Complicated Queries


    Suppose I want to see all the customers with their contracts who live on Orange Street in the glorious city of Berlin. Those. I want to see something like this plate:
    Last Name, First NameContract numberDate of signing the contractThe car
    Pupkin, Vasya1234501/01/2017Audi Q4
    ........................

    As you can see, the table contains data from three microservices: CustomerService , VehicleService and ContractService . How will we put them together? In the case of a monolith, the question is solved with one request to the database, but what to do when we have three bases?

    image

    There are different options for solving this small problem, and next time we will discuss them.

    Thanks, Cap!
    And now a moment of attention. When did you have to ask these questions? Answer: of course, before writing code. And the architect must answer these questions.

    How is the decision-making process and reporting them to a team of programmers
    And now there’s a small “stream of consciousness” on the topic: how does the process of making decisions and conveying them to the team of programmers take place? I know two theoretical possibilities, and everything else is just their mixes in various proportions:

    1. An uncle comes and shows a presentation with all sorts of tails in the form of diagrams flavored with sophisticated words. Often, he makes it clear that he has a lot of experience behind him, understands that he is doing and all that, but because the decision was made completely and irrevocably. Hearing a murmur from a distant row, he quickly declares that he is always open to new ideas. In fact, the architect here is a full-fledged dictator.
    2. An uncle comes, says that he is an “artist” and sees a “picture” like this. At the same time, he directly says that he will not “draw”, but people in the “hall”, and therefore, in their vital interests, the vision of the picture is scolded and, if possible, offers alternatives. In this case, the architect seeks to relieve himself of any responsibility.

    Personally, I’m closer, (I repeat, closer, but not completely satisfied) the second option. There are probably dozens of reasons for this; let us consider some of them.

    I do not want to be responsible. No seriously. I, like any reasonable person, want to get a lot of money, do nothing and do not bear responsibility for this. But objective reality, unfortunately, shows that this is impossible. Well, or at least I have not yet found the right way.

    But for "share with another responsibility" there is another reason and it is no less important. When a programmer participates in the discussion (and thus indirectly in making) a decision, he implements it with a different level of diligence. If a jamb suddenly creeps out, he realizes that this was his fault, and not "this architect idiot invented garbage, but skip it for me."

    Another reason lies in the incomplete knowledge of the system and its environment. Imagine that after the presentation of architecture you get the question: “how does it all fit with the fact that we need to pull data from someone else’s billing system?” And then the architect suddenly finds out that there is a third-party system with which we need to integrate. And everyone knows this, except for the architect. Yes, that happens too.

    It is important to explain why this decision was made, and not any other, what are the advantages and mention the disadvantages. In this case, people understand what is happening, become much more patient.

    True, it should be noted that such a "democracy", when a decision is made by a bunch of people, has its own borders. Moreover, such a “democratic” decision is not always possible. For example, when everyone understands that each of the solutions discussed is bad, and no one knows the good. In the end, if it suddenly turns out that the decision was wrong, the architect is responsible for this. Excuse, they say, I asked everyone, it was a collective decision, unfortunately it will not work. The architect is personally responsible for all decisions, regardless of how they were made.

    In the end, two more thoughts.

    Firstly, if you think that your architecture is good, it means that you have not shown it to anyone yet. By the way, this also applies to the code.

    Secondly, I am a categorical supporter of the opinion that an architect should have practical programming experience so that he knows what it means to realize someone else’s wishes. Even better, if the architect is directly involved in the implementation of the project, i.e. writes code, or at least this code itself is revising. The architect should regularly check that the code is consistent with the architectural decisions made.

    Once I spoiled this moment and it was only at Sprint Review that I accidentally found out what the programmer did not do as he was told. To the question: it is written in black in English, send an “event”, why do you send an http request? the answer was received: well, I thought it would be better. Naturally, it is impossible to control everything, you have to trust your team. But, as the Germans say, trust is good, and control is better.

    Change history


    One day, someone smart will want to find out why customers terminate or do not renew the contract and go to competitors? Maybe the last time the car did not like / was late? Or maybe a man traveled earlier with his colleague, who switched to our competitors and lured his companion to them?

    To answer such questions, we need to know what happened in the system, i.e. we need a change history.

    How to do this we will discuss in the next article.

    Infrastructure issues


    Microservice architecture imposes very specific requirements on the infrastructure of the entire application, namely, we need

    • single point of entry, aka Gateway API
    • search for services, aka Service Discovery
    • one point of configuration, aka Config Service
    • central point for collecting and analyzing logs
    • monitoring
    • ... ...

    These and other questions in the next part.

    Also popular now: