About safety, nuclear physics and love: about the contradictions of the modern IT architecture of front-end solutions
Continuing to make sense of the smallest infinite sets in application architecture through dependency games firsthand.
I have a friend, his profession is related to electrical installation. When he was younger and more cynical, he liked to poison stories about electricians who worked on unpowered networks. The end was always entertaining, but sad for the protagonist. With component architecture, it’s the same: somewhere you don’t isolate one functional from another, it will “shock” you and the one who comes after you. The difference is that isolation in IT is so far more expensive than in electrics.
In the previous article, we already made the first engineering estimates of the functional scope of the new solution for Sberbank - the Unified Frontal System. We focused on quantitative estimates of the volume of functionality, on the methods of its production and development. It's time to talk about its connectedness and ways of isolating functionality.
First of all, we isolated the system, applied, and integration functionality from each other. In fact, these are different “electrical circuits”, with different “voltages” and operating conditions. The result was a metamodel of permissible dependencies, as in the figure below:
Naturally, the applied functional prevails on the scale of Sberbank. Therefore, we will talk about him separately.
An analogue of the “non-linear electric circuit” for the front-end banking system is the front-end customer service scenario. Just like chains, front-end scripts can be branched and include many subprocesses. The natural desire of every architect who has already worked on “de-energized networks” is the desire to granulate processes as much as possible, so that any changes lead to less regression.
The analogue of the "linear electric circuit" for the Unified Frontal System is any state-less logic called from the front-end scenarios. Unlike the scenarios themselves, it has its own life cycle. In Sberbank, he is subordinate to the general program of replacing legacy-functional with solutions based on his own promising automation platform for back- and middle-office banking functionality.
Continuing the analogy, we wrapped individual electrical circuits with electrical tape and introduced the concept of a “module” - a group of applied functionality that is tightly connected inside and loosely connected to another functional. Once implemented in code, this is a regular jar file and pom artifact.
Although we tried to limit ourselves to a small unified number of rules for granulating the applied functionality, we still needed a lot of electrical tape. On the server side, the target number of modules is> = 10 4 . These are the rules:
The following metamodel of the modular concentration of the application functional isolated from each other was obtained:
Granulation limits are determined by the stability of the API module (service API and API call of a separate subprocess) when:
The size of an individual module is 20-50 java classes.
So far, only 5% of the modules have changed less frequently than once a quarter, and this trend will continue in the future. This means that to comply with architectural safety precautions, it is necessary to add electrical tape directly to the modules themselves. It’s good that we have already come up with such patterns as bridges, adapters and facades. All this has become a natural electrical tape inside the modules themselves.
The general typical architecture for the most common application modules is shown in the figure below. Naturally, the emphasis is not on what is happening there, but on how it is isolated from each other.
By the nature of the granulation of the functional, from the point of view of the execution flow, the architecture of the individual modules is most consistent with layered architecture. From the point of view of the final application, which includes, depending on, a whole group of application modules, - process driven architecture.
Granulation and the abundance of isolation, in addition to the above advantages, also has the following:
Among the significant disadvantages:
Among the well-known laboratory work in physics for freshmen, there are experiments with photons that simultaneously behave both as particles (corpuscles) and as a wave. These experiments are used to illustrate the incomprehensible at the household level properties of wave-particle duality. The Sberbank United Front System has very similar behavior.
The ESF has two requirements, each of which is harmless in itself:
In general, both requirements are contradictory and make us think about “interference and diffraction” in architecture. The first requirement is well implemented in microservice architecture, the second - in monolithic.
The solution to the problem is dependency management. We should have dependencies of the same kind, reinforcing concrete guaranteeing minimal regression and transparent, from the point of view of the requirement of introducing products through all channels. And vice versa, to have a group of dependencies “transparent”, in terms of regression volumes, and reinforced concrete ones for changing products.
The first group of dependencies that satisfy these requirements can be created based on the practice of Domain-driven design and a common understanding of the automated banking field. If we draw analogies with nuclear physics, it suffices to single out elementaryparticles are domains whose interactions are forbidden. Each application module must belong to a particular domain.
Below is an illustrative map of the domains we use. None of the modules related to a specific domain should directly depend on the module of another domain, unless located at the top of the dependency chain.
Separate domains have their own hierarchical structure. Separate subdomains can be distinguished for them, between which direct dependencies will also be prohibited.
The second group of “particle-wave” dependencies can be created by dividing the functional into cross-channel and channel-specific. The functional of the first group will be the product functional, and the functional of the second will be auxiliary. Then, isolating the product functionality from the channel-specific way of constructing presentation logic, user authorization, customer identification, etc., you can build the scheme shown in the figure below.
Note:
In our experience, for the last diagram to work, it is important to comply with the following requirements:
Now, dear reader, you have everything for an independent understanding of the Schrödinger equation. ;)
So what do we do with the number of modules? Let me remind you, in the target form of the modules> = 10 4 . Moreover, the customer has the expectation that all the functionality can be replaced as a whole no less than once a quarter.
Despite the fact that the basic design patterns of a typical application functional are less than two dozen, for each automated operation it is necessary to do the following:
Pull on several architectural solutions. Everything is complicated by the fact that the laid down requirements for code reuse require knowledge of the contents of application modules.
Practice has shown that one architect designing the component structure of the applied functional is able to remember the contents of ~ 100 modules. So, it’s enough for us to have ~ 100 architects to manage all the applied functionality. But this is already such a number of specialists in which architects themselves will not know about each other. This means that the problem has no administrative solution.
Fortunately, Gabriel García Márquez had previously proposed a “counter-administrative” solution to the problem, which was made in the section heading. Now we are “letting go” of the direct design of the architecture of the applied functional, since it is impossible to keep it at target scales.
We are going to know nothing about each module and proceed only to control:
It is important here that reducing the IT architecture to large numbers (in particular, to a large number of typical components) is a solution that is good in that it allows for a fairly large number of special cases to develop a common solution.
Chronicle of events:
We are currently working on a number of projects an intermediate solution that allows us to perform internal design of application modules according to the functional configuration, for which we talked about the tool in a previous publication . An architect manually solves only one problem - he “lays out” the functionality for generation by different application modules, positions the modules in the final hierarchy according to dependencies.
Maybe these new methods of forming applied architecture will even be called smart. For example, Statistic driven application architecture. How do you like it? ;)
Technically, we are ready to fully automate the process of applied design. And it is important that, unlike Gabriel García Márquez, we know what reverse engineering is, which will allow us to keep our favorite application architecture on a leash and control the manual completion of automatically generated code. But, it seems, this is already something from Daria Dontsova.
In conclusion, I note the following. There is a common belief that any creator (both García Márquez and Dontsova) will be truly appreciated only by descendants. In IT, and especially in ESF, this is not so. We are ready to consider new ideas and technological solutions. Join our technological competition "Promotion" !
More electrical tape!
I have a friend, his profession is related to electrical installation. When he was younger and more cynical, he liked to poison stories about electricians who worked on unpowered networks. The end was always entertaining, but sad for the protagonist. With component architecture, it’s the same: somewhere you don’t isolate one functional from another, it will “shock” you and the one who comes after you. The difference is that isolation in IT is so far more expensive than in electrics.
In the previous article, we already made the first engineering estimates of the functional scope of the new solution for Sberbank - the Unified Frontal System. We focused on quantitative estimates of the volume of functionality, on the methods of its production and development. It's time to talk about its connectedness and ways of isolating functionality.
First of all, we isolated the system, applied, and integration functionality from each other. In fact, these are different “electrical circuits”, with different “voltages” and operating conditions. The result was a metamodel of permissible dependencies, as in the figure below:
Naturally, the applied functional prevails on the scale of Sberbank. Therefore, we will talk about him separately.
An analogue of the “non-linear electric circuit” for the front-end banking system is the front-end customer service scenario. Just like chains, front-end scripts can be branched and include many subprocesses. The natural desire of every architect who has already worked on “de-energized networks” is the desire to granulate processes as much as possible, so that any changes lead to less regression.
The analogue of the "linear electric circuit" for the Unified Frontal System is any state-less logic called from the front-end scenarios. Unlike the scenarios themselves, it has its own life cycle. In Sberbank, he is subordinate to the general program of replacing legacy-functional with solutions based on his own promising automation platform for back- and middle-office banking functionality.
Continuing the analogy, we wrapped individual electrical circuits with electrical tape and introduced the concept of a “module” - a group of applied functionality that is tightly connected inside and loosely connected to another functional. Once implemented in code, this is a regular jar file and pom artifact.
Although we tried to limit ourselves to a small unified number of rules for granulating the applied functionality, we still needed a lot of electrical tape. On the server side, the target number of modules is> = 10 4 . These are the rules:
- separation of state-less and state-full logic into different modules;
- separation of logic over entities of different domains into different modules;
- removal of any public API into separate modules;
- separation of product-specific and channel-specific logic into different modules;
- allocation of modules for concentration of dependencies.
The following metamodel of the modular concentration of the application functional isolated from each other was obtained:
Granulation limits are determined by the stability of the API module (service API and API call of a separate subprocess) when:
- changes in the requirements of the regulator and the presentation of the customer about the target functionality for automation;
- changing the system or integration functionality to which the ESF delegates state-less logic;
- changing the sequence of state-full processing chains.
The size of an individual module is 20-50 java classes.
So far, only 5% of the modules have changed less frequently than once a quarter, and this trend will continue in the future. This means that to comply with architectural safety precautions, it is necessary to add electrical tape directly to the modules themselves. It’s good that we have already come up with such patterns as bridges, adapters and facades. All this has become a natural electrical tape inside the modules themselves.
The general typical architecture for the most common application modules is shown in the figure below. Naturally, the emphasis is not on what is happening there, but on how it is isolated from each other.
By the nature of the granulation of the functional, from the point of view of the execution flow, the architecture of the individual modules is most consistent with layered architecture. From the point of view of the final application, which includes, depending on, a whole group of application modules, - process driven architecture.
Granulation and the abundance of isolation, in addition to the above advantages, also has the following:
- all forms of interaction are well divided into standard ones, and for their participants you can use either abstractions or marker interfaces;
- several independent development teams can edit the same module for several months, and it somehow miraculously keeps working;
- bridges, adapters and facades are good places for the concentration of "crutches" that appear at one or another stage of the "immaturity" or "overripe" of the code;
- easy top-level control of architectural violations. It is enough to look at the pom-file and compare the actual dependencies with the design ones;
- since there is always a lack of system functionality, its early prototypes are easily isolated at the level of individual application modules, and the resulting excessive connectivity does not “spread”, “poisoning the code”.
Among the significant disadvantages:
- high complexity of maintaining high granularity. Actually, bridges, adapters, facades and a huge amount of code for transferring one data structure to another require implementation;
- you must have developed version control and impact analysis functionality so as not to immerse the entire solution in jar hell;
- end-to-end cross-modular code changes require synchronous switching from branch to branch for a connected group of modules, and pull requests must be accepted taking into account inter-module dependencies.
Microservice-monolithic dualism
Among the well-known laboratory work in physics for freshmen, there are experiments with photons that simultaneously behave both as particles (corpuscles) and as a wave. These experiments are used to illustrate the incomprehensible at the household level properties of wave-particle duality. The Sberbank United Front System has very similar behavior.
The ESF has two requirements, each of which is harmless in itself:
- “Corpuscular requirement”: a change in any application functional should imply minimal regression of the adjacent, including similar functionality in other channels. The entire ESF should never be subject to end-to-end regression. In addition, the “corpuscularity” of the ESF is inevitable due to the large number of teams developing independently applied functionality.
- “Wave requirement”: a change in the logic of customer service should be carried out immediately on all service channels. This is precisely the main idea reflected in the title “The United Frontal System” - to minimize the channel-specific costs for the output of the product to each channel.
In general, both requirements are contradictory and make us think about “interference and diffraction” in architecture. The first requirement is well implemented in microservice architecture, the second - in monolithic.
The solution to the problem is dependency management. We should have dependencies of the same kind, reinforcing concrete guaranteeing minimal regression and transparent, from the point of view of the requirement of introducing products through all channels. And vice versa, to have a group of dependencies “transparent”, in terms of regression volumes, and reinforced concrete ones for changing products.
The first group of dependencies that satisfy these requirements can be created based on the practice of Domain-driven design and a common understanding of the automated banking field. If we draw analogies with nuclear physics, it suffices to single out elementary
Below is an illustrative map of the domains we use. None of the modules related to a specific domain should directly depend on the module of another domain, unless located at the top of the dependency chain.
Separate domains have their own hierarchical structure. Separate subdomains can be distinguished for them, between which direct dependencies will also be prohibited.
The second group of “particle-wave” dependencies can be created by dividing the functional into cross-channel and channel-specific. The functional of the first group will be the product functional, and the functional of the second will be auxiliary. Then, isolating the product functionality from the channel-specific way of constructing presentation logic, user authorization, customer identification, etc., you can build the scheme shown in the figure below.
Note:
- if the requirements of the regulator regarding special procedures for interacting with the client on the channel change, changes in the code will take effect for all channel products;
- if the understanding of the business regarding the product itself changes, all changes become available to all channels.
In our experience, for the last diagram to work, it is important to comply with the following requirements:
- a well-developed cross-channel system functionality is needed to abstract the application scenario of customer service from a specific way of interacting with a client;
- we need a strict business architecture that classifies the ultimate functionality and unifies interaction with the consumer.
Now, dear reader, you have everything for an independent understanding of the Schrödinger equation. ;)
If you love - let go
So what do we do with the number of modules? Let me remind you, in the target form of the modules> = 10 4 . Moreover, the customer has the expectation that all the functionality can be replaced as a whole no less than once a quarter.
Despite the fact that the basic design patterns of a typical application functional are less than two dozen, for each automated operation it is necessary to do the following:
- functional decomposition of modules;
- layout of modules by application;
- designing business entities and APIs;
- fixing inside modular dependencies and isolation.
Pull on several architectural solutions. Everything is complicated by the fact that the laid down requirements for code reuse require knowledge of the contents of application modules.
Practice has shown that one architect designing the component structure of the applied functional is able to remember the contents of ~ 100 modules. So, it’s enough for us to have ~ 100 architects to manage all the applied functionality. But this is already such a number of specialists in which architects themselves will not know about each other. This means that the problem has no administrative solution.
Fortunately, Gabriel García Márquez had previously proposed a “counter-administrative” solution to the problem, which was made in the section heading. Now we are “letting go” of the direct design of the architecture of the applied functional, since it is impossible to keep it at target scales.
We are going to know nothing about each module and proceed only to control:
- the quality of new modules being designed, comparing their quantitative characteristics with the typical values of statistically successful solutions;
- the nature of the changes made to the code through reverse engineering;
- and provide content to automate the search for modules for reuse.
It is important here that reducing the IT architecture to large numbers (in particular, to a large number of typical components) is a solution that is good in that it allows for a fairly large number of special cases to develop a common solution.
Chronicle of events:
- it all started with 20 application modules;
- when there were 50 modules, an estimate of the target number of application modules was obtained and a preliminary dependency metamodel was built;
- 100 modules were enough to stabilize the modular component architecture and raise the issue of automation of the "production" of application modules;
- 300 modules became enough to quantify the "golden section" of ideal and acceptable quality application modules.
We are currently working on a number of projects an intermediate solution that allows us to perform internal design of application modules according to the functional configuration, for which we talked about the tool in a previous publication . An architect manually solves only one problem - he “lays out” the functionality for generation by different application modules, positions the modules in the final hierarchy according to dependencies.
Maybe these new methods of forming applied architecture will even be called smart. For example, Statistic driven application architecture. How do you like it? ;)
Technically, we are ready to fully automate the process of applied design. And it is important that, unlike Gabriel García Márquez, we know what reverse engineering is, which will allow us to keep our favorite application architecture on a leash and control the manual completion of automatically generated code. But, it seems, this is already something from Daria Dontsova.
In conclusion, I note the following. There is a common belief that any creator (both García Márquez and Dontsova) will be truly appreciated only by descendants. In IT, and especially in ESF, this is not so. We are ready to consider new ideas and technological solutions. Join our technological competition "Promotion" !
