June 17, 2019 at 09:00Why do we do Enterprise Service Mesh
Service Mesh is a well-known architectural pattern for integrating microservices and moving to the cloud infrastructure. Today in the cloud-container world it is quite difficult to do without it. Several open-source service mesh implementations are already available on the market, but their functionality, reliability and security are far from always sufficient, especially when it comes to the requirements of large financial companies across the country. Therefore, we at Sbertech decided to customize Service Mesh and want to talk about what is cool in Service Mesh, what is not very and what we are going to do with it.
The popularity of the Service Mesh template is growing with the popularity of cloud technology. It is a dedicated infrastructure layer that simplifies the interaction between different network services. Modern cloud applications consist of hundreds and even thousands of such services, each of which can have thousands of copies.
The interaction between these services and their management is a key task of Service Mesh. In fact, this is a network model from a variety of proxies, managed centrally and performing a set of very useful functions.
At the proxy level (data plane):
At the control plane level:
The circle of users interested in the development of this technology is very wide - from small startups to large Internet corporations, for example, PayPal.
Using Service Mesh brings many obvious benefits. First of all, it’s just convenient for developers: a technology platform appears for writing code that significantly simplifies integration into the cloud infrastructure due to the fact that the transport layer is completely isolated from application logic.
In addition, Service Mesh simplifies the relationship between suppliers and consumers.Today, it is much easier for suppliers and consumers of the API to agree on interfaces and contracts on their own, without involving a special integration intermediary and arbiter for this - a corporate service bus. This approach significantly affects two indicators. The speed of bringing new functionality to the market (time-to-market) is increasing, but at the same time the cost of the solution is increasing, since integration has to be done independently. The use of Service Mesh by business development teams helps maintain balance here. As a result, API providers can focus exclusively on the application component of their service and simply publish it in the Service Mesh - the API will immediately become available to all customers, and the integration quality will be production ready and will not require a single line of additional code.
Another advantage is that the developer, using the Service Mesh, focuses solely on business functionality - on the product, and not on the technological component of his service. For example, you don’t have to think about the fact that in a situation where the service will be called over the network, somewhere the connection may break. In addition, Service Mesh helps balance traffic between copies of the same service: if one of the copies “died”, the system will switch all traffic to the remaining live copies.
Service Mesh is a good foundation for building distributed applications., which hides from the client the details of providing calls to his services both from the inside and the outside. All applications using Service Mesh are isolated on the transport layer both from the network and from each other: there is no connection between them. In this case, the developer gets full control over their services.
It should be noted that updating distributed applications in an environment where Service Mesh is used is becoming easier. For example, blue / green deployment, in which two application environments are available for installation, one of which is not updated and is in standby mode. Rollback to the previous version in case of unsuccessful release is carried out by a special router, the role of which Service Mesh perfectly copes with . To run the new version, you can usecanary release - switch to the new version only 10% of traffic or requests from a pilot group of customers. The main traffic goes to the old version, nothing breaks.
Also, Service Mesh gives us control of SLA in real time . The system of distributed proxies will not allow to flood the service when one of the clients exceeds the quota issued to it. If the API bandwidth is limited, no one will be able to strangle it with a large number of transactions: the Service Mesh is in front of the service and does not allow extra traffic. It will simply beat off in the integration layer, and the services themselves will continue to work without noticing it.
If the company wants to reduce the cost of developing integration solutions, Service Mesh also helps out:its open-source version can be upgraded from commercial products . Our Enterprise Service Mesh is based on the open-source version of Service Mesh.
Another advantage is the availability of a single full-fledged set of integration services . Since all integration is built through this intermediate layer, we can manage all the integration traffic and the connections between the applications that form the business core of the company. It is very comfortable.
Finally, Service Mesh drives the company to transition to a dynamic infrastructure.Now many are looking towards containerization. Sawing a monolith into microservices, it’s beautiful to implement it all - the topic is on the rise. But when you try to transfer to a new track a system that has been in production for many years, you immediately encounter a number of problems: pushing it all into containers and deploying it on the platform is not easy. And the implementation, synchronization and interaction of these distributed components is another difficult topic. How will they communicate with each other? Will there be cascading failures? Service Mesh allows you to solve some of these problems and facilitate the migration from the old architecture to the new due to the fact that you can forget about the logic of network exchange.
At our company hundreds of systems and modules coexist together, and runtime is very busy. So a simple pattern, when one system calls another and receives an answer, is not enough, because in production we want more. What more do you need from a corporate Service Mesh?
Imagine that we need to make a real-time event processing - a system that real-time analyzes the actions of the client and can immediately make him a relevant offer. To implement this functionality, an architectural pattern called event-driven architecture (EDA) is used . None of the relevant Service Mesh natively supports such patterns, but this is very important, especially for the bank!
It is strange that the Remote Procedure Call (RPC) supports all versions of the Service Mesh, and they are not friends with EDA. Because Service Mesh is a semblance of modern distributed integration, and EDA is a very relevant architectural pattern that allows you to do unique things in terms of customer experience.
Our Enterprise Service Mesh should solve this problem. In addition, we want to see in it the implementation of guaranteed delivery, streaming and integrated event processing using a variety of filters and templates.
In addition to EDA, it would be nice to be able to transfer files: on an Enterprise scale, very often only file integration is possible. In particular, the architectural ETL pattern is used (Extract, Transform, Load - “extract, transform, load”). In it, as a rule, everyone exchanges files exclusively: they use big data, which is impractical to shove in separate requests. The ability to natively support file transfers in the Enterprise Service Mesh gives you the flexibility you need for your business.
In large organizations, there are almost always different teams that make different products. For example, in a bank, some teams work with deposits, while others work with credit products, and there are a lot of such cases. These are different people, different teams that make their products, develop their APIs and provide them to others. And very often there is a need for the composition of these services, as well as the implementation of the complex logic of the sequential call of the API set. To solve this problem, a solution is needed in the integration layer, which will simplify all this composite logic (calling several APIs, describing the route of requests, etc.). This is the orchestration service in the Enterprise Service Mesh.
When microservices communicate through a single integration layer, Service Mesh, of course, knows everything about the calls of each service. We collect telemetry: who called whom, when, how long, how many times, and so on. When there are hundreds of thousands of these services and billions of calls, all this accumulates and forms Big Data. These data can be analyzed using AI, machine learning, etc., and then do some useful things based on the results of the analysis. It would be appropriate to at least partially transfer to the artificial intelligence the management of all this network traffic and application calls integrated into the Service Mesh.
As a rule, Service Mesh has proxies and services that access each other inside the trusted perimeter. But there are also external counterparties. The API requirements for this consumer group are much more serious. We divide this task into two main parts.
Currently, most Service Mesh solutions can only natively work with HTTP and HTTP2 traffic or in truncated mode at the TCP / IP level. The Enterprise Service Mesh has many other very specific data transfer protocols. Some systems may use message brokers; others are integrated at the database level. If the company has SAP, then it can also use its own integration system. And all this works and is an important part of the business.
You can’t just say: “Let's give up legacy and create new systems that can use the Service Mesh.” To make friends of all old systems with new ones (on microservice architecture), systems that can use Service Mesh will need some kind of adapter, intermediary, and gateway. Agree, it would be nice if it was delivered in a box along with the service. The AS gateway can support any integration option. Just imagine, you just install the Enterprise Service Mesh, and it is ready to interact with all the protocols that you need. For us, this approach is very important.
This is how we present the corporate version of Service Mesh (Enterprise Service Mesh). The described customization solves most of the problems that arise when trying to use ready-made open-source versions of the integration platform. Having appeared just a couple of years ago, the Service Mesh architecture continues to evolve, and we are glad that we can contribute to its development. We hope that our experience will be useful to you.
The popularity of the Service Mesh template is growing with the popularity of cloud technology. It is a dedicated infrastructure layer that simplifies the interaction between different network services. Modern cloud applications consist of hundreds and even thousands of such services, each of which can have thousands of copies.
The interaction between these services and their management is a key task of Service Mesh. In fact, this is a network model from a variety of proxies, managed centrally and performing a set of very useful functions.
At the proxy level (data plane):
- Assign and distribute routing and traffic balancing policies
- Distribution of keys, certificates, tokens
- Telemetry collection, formation of monitoring metrics
- Integration with security and monitoring infrastructure
At the control plane level:
- Apply routing and traffic balancing policies
- Management of retries and timeouts, definition of "dead" nodes (circuit breaking), management of faulty situations (injecting faults) and ensuring the resilience of services through other mechanisms
- Call Authentication / Authorization
- Discarding metrics (observability)
The circle of users interested in the development of this technology is very wide - from small startups to large Internet corporations, for example, PayPal.
What is Service Mesh for in the corporate sector?
Using Service Mesh brings many obvious benefits. First of all, it’s just convenient for developers: a technology platform appears for writing code that significantly simplifies integration into the cloud infrastructure due to the fact that the transport layer is completely isolated from application logic.
In addition, Service Mesh simplifies the relationship between suppliers and consumers.Today, it is much easier for suppliers and consumers of the API to agree on interfaces and contracts on their own, without involving a special integration intermediary and arbiter for this - a corporate service bus. This approach significantly affects two indicators. The speed of bringing new functionality to the market (time-to-market) is increasing, but at the same time the cost of the solution is increasing, since integration has to be done independently. The use of Service Mesh by business development teams helps maintain balance here. As a result, API providers can focus exclusively on the application component of their service and simply publish it in the Service Mesh - the API will immediately become available to all customers, and the integration quality will be production ready and will not require a single line of additional code.
Another advantage is that the developer, using the Service Mesh, focuses solely on business functionality - on the product, and not on the technological component of his service. For example, you don’t have to think about the fact that in a situation where the service will be called over the network, somewhere the connection may break. In addition, Service Mesh helps balance traffic between copies of the same service: if one of the copies “died”, the system will switch all traffic to the remaining live copies.
Service Mesh is a good foundation for building distributed applications., which hides from the client the details of providing calls to his services both from the inside and the outside. All applications using Service Mesh are isolated on the transport layer both from the network and from each other: there is no connection between them. In this case, the developer gets full control over their services.
It should be noted that updating distributed applications in an environment where Service Mesh is used is becoming easier. For example, blue / green deployment, in which two application environments are available for installation, one of which is not updated and is in standby mode. Rollback to the previous version in case of unsuccessful release is carried out by a special router, the role of which Service Mesh perfectly copes with . To run the new version, you can usecanary release - switch to the new version only 10% of traffic or requests from a pilot group of customers. The main traffic goes to the old version, nothing breaks.
Also, Service Mesh gives us control of SLA in real time . The system of distributed proxies will not allow to flood the service when one of the clients exceeds the quota issued to it. If the API bandwidth is limited, no one will be able to strangle it with a large number of transactions: the Service Mesh is in front of the service and does not allow extra traffic. It will simply beat off in the integration layer, and the services themselves will continue to work without noticing it.
If the company wants to reduce the cost of developing integration solutions, Service Mesh also helps out:its open-source version can be upgraded from commercial products . Our Enterprise Service Mesh is based on the open-source version of Service Mesh.
Another advantage is the availability of a single full-fledged set of integration services . Since all integration is built through this intermediate layer, we can manage all the integration traffic and the connections between the applications that form the business core of the company. It is very comfortable.
Finally, Service Mesh drives the company to transition to a dynamic infrastructure.Now many are looking towards containerization. Sawing a monolith into microservices, it’s beautiful to implement it all - the topic is on the rise. But when you try to transfer to a new track a system that has been in production for many years, you immediately encounter a number of problems: pushing it all into containers and deploying it on the platform is not easy. And the implementation, synchronization and interaction of these distributed components is another difficult topic. How will they communicate with each other? Will there be cascading failures? Service Mesh allows you to solve some of these problems and facilitate the migration from the old architecture to the new due to the fact that you can forget about the logic of network exchange.
Why customize Service Mesh
At our company hundreds of systems and modules coexist together, and runtime is very busy. So a simple pattern, when one system calls another and receives an answer, is not enough, because in production we want more. What more do you need from a corporate Service Mesh?
Event Processing Service
Imagine that we need to make a real-time event processing - a system that real-time analyzes the actions of the client and can immediately make him a relevant offer. To implement this functionality, an architectural pattern called event-driven architecture (EDA) is used . None of the relevant Service Mesh natively supports such patterns, but this is very important, especially for the bank!
It is strange that the Remote Procedure Call (RPC) supports all versions of the Service Mesh, and they are not friends with EDA. Because Service Mesh is a semblance of modern distributed integration, and EDA is a very relevant architectural pattern that allows you to do unique things in terms of customer experience.
Our Enterprise Service Mesh should solve this problem. In addition, we want to see in it the implementation of guaranteed delivery, streaming and integrated event processing using a variety of filters and templates.
File transfer service
In addition to EDA, it would be nice to be able to transfer files: on an Enterprise scale, very often only file integration is possible. In particular, the architectural ETL pattern is used (Extract, Transform, Load - “extract, transform, load”). In it, as a rule, everyone exchanges files exclusively: they use big data, which is impractical to shove in separate requests. The ability to natively support file transfers in the Enterprise Service Mesh gives you the flexibility you need for your business.
Orchestration Service
In large organizations, there are almost always different teams that make different products. For example, in a bank, some teams work with deposits, while others work with credit products, and there are a lot of such cases. These are different people, different teams that make their products, develop their APIs and provide them to others. And very often there is a need for the composition of these services, as well as the implementation of the complex logic of the sequential call of the API set. To solve this problem, a solution is needed in the integration layer, which will simplify all this composite logic (calling several APIs, describing the route of requests, etc.). This is the orchestration service in the Enterprise Service Mesh.
AI and ML
When microservices communicate through a single integration layer, Service Mesh, of course, knows everything about the calls of each service. We collect telemetry: who called whom, when, how long, how many times, and so on. When there are hundreds of thousands of these services and billions of calls, all this accumulates and forms Big Data. These data can be analyzed using AI, machine learning, etc., and then do some useful things based on the results of the analysis. It would be appropriate to at least partially transfer to the artificial intelligence the management of all this network traffic and application calls integrated into the Service Mesh.
API Gateway Service
As a rule, Service Mesh has proxies and services that access each other inside the trusted perimeter. But there are also external counterparties. The API requirements for this consumer group are much more serious. We divide this task into two main parts.
- Security . Issues related to ddos, vulnerability of protocols, applications, operating systems and so on.
- The scale . When the API bill that needs to be given to customers goes to thousands or even hundreds of thousands, there is a need for some means of managing this set of APIs. You need to constantly monitor the API: do they work or not, in what status, what traffic is going on, what statistics, etc. The API gateway must handle this task, making the whole process manageable and secure. Thanks to this component, the Enterprise Service Mesh learns to publish both the internal and external APIs without too much difficulty.
Support service for specific protocols and data formats (AS gateway)
Currently, most Service Mesh solutions can only natively work with HTTP and HTTP2 traffic or in truncated mode at the TCP / IP level. The Enterprise Service Mesh has many other very specific data transfer protocols. Some systems may use message brokers; others are integrated at the database level. If the company has SAP, then it can also use its own integration system. And all this works and is an important part of the business.
You can’t just say: “Let's give up legacy and create new systems that can use the Service Mesh.” To make friends of all old systems with new ones (on microservice architecture), systems that can use Service Mesh will need some kind of adapter, intermediary, and gateway. Agree, it would be nice if it was delivered in a box along with the service. The AS gateway can support any integration option. Just imagine, you just install the Enterprise Service Mesh, and it is ready to interact with all the protocols that you need. For us, this approach is very important.
This is how we present the corporate version of Service Mesh (Enterprise Service Mesh). The described customization solves most of the problems that arise when trying to use ready-made open-source versions of the integration platform. Having appeared just a couple of years ago, the Service Mesh architecture continues to evolve, and we are glad that we can contribute to its development. We hope that our experience will be useful to you.