Microservices, APIs, and Innovations: All the Power of the API
- Transfer
Hello to all!
Today we wanted to offer you a translation of the program article of the notorious Mike Amundsen , the lead architect from API Academy. In this relatively small text, Mike clearly explains why you need to pay special attention to the development of the API, and how the API is done correctly.
During my work as a teacher in API Academy, I happened to travel around the whole world, meeting with amazing people. They are working on stunning projects in all sorts of companies - from young startups to established global enterprises. Incredibly, wherever I was and with whom I spoke, in our conversations a multitude of common ideas, techniques and impressions surfaced. The three most common are microservices, APIs, and a culture of innovation. All three are being introduced to speed up the organization’s digital transformation.
This article is the second in a series of three publications. Here I will talk about what we teach in our API Academy in the context of these three powerful trends, and also describe some techniques that will help your company move from big words to real transformations. In the previous articleI paid special attention to the importance of microservices and focused on three things that you can do today: 1) transition to continuous delivery, 2) provision of prepared deployments and 3) shortening the work queue to reduce the time to product to market. These three most important habits will help your organization to take full advantage of microservices.
APIs provide multi-channel delivery.
Many companies use microservices, trying to encapsulate the key capabilities of their organization precisely to ensure their scalability and reliability. Microservices correspond to important functional elements of the IT component of your company. But this is only half the battle. It is also necessary to figure out how to provide these opportunities so that with their help it would be convenient to solve the actual challenges faced by your business. This is where the APIs come into play.
API - application programming interfaces - play the same role for a machine as a graphical interface - for users of your company's information systems. Often, APIs are used to blend different capabilities into a consistent and affordable solution. For example, in your company you can use services to process transactions related to customers, accounts and sales. Each of these services was carefully designed, programmed, tested, after which it was released and built into the infrastructure of your company; The service provides functionality that is critical for your business. One day you will need to create a new solution to keep users up-to-date - and this solution should work on a wide variety of devices and platforms. So, we start to use API at full capacity.
A single API, tailored for solutions - for example, an API to familiarize users with the system - can be designed so that the most important interactions and task flows that are critical at the stage of such familiarization come to the surface. Here we can use an API that allows you to mix various functional elements relating to customers, accounts and sales into safe, powerful and accessible user interfaces for a wide variety of target audiences within your company. For example, salespeople can log into the system from a browser, office administrators from PC applications, and potential customers from smartphones and tablets.
We can say that the API is a kind of “glue” that holds program elements together, an intermediary layer through which your internal services and external service consumers interact. It is the API that is the means for distributing the key functionality in such a way that consumers of services could use it, whose task is to create important mechanisms for user interaction on a variety of hardware platforms. Such customers may include other teams working in your office, colleagues with whom you communicate remotely, key business partners, or even remote employees involved in client-side development or design.
Design thinking and API
Most companies know how important it is to devote time to developing a user interface for their applications. Because it is known that users like good design, it increases brand loyalty and allows them to promote new business. At the same time, poorly crafted interface design can annoy customers, harm the brand, reduce revenue, and select opportunities. The same is true for designing an API.
If the API is made poorly, it will be difficult for developers to understand it, because of this they can introduce errors into the system, and provoke unnecessary costs to fix bugs and modify the infrastructure. However, if the API is well developed, then it is easier for the employee to work with it effectively. The time it takes to create a stable solution is reduced, the team even gets an incentive to issue fresh, innovative solutions that would help clients and colleagues. API design is such an important area of work that Werner Vogels, Amazon’s chief technical officer, even said:
It is high-quality APIs that help attract partners to your digital ecosystem and simplify internal corporate transformations of your business. The ability to spend time in order to do everything correctly, and, in the long run, is an important skill that we trace just from those companies that have learned to use their API at full capacity.
Essential API design advice
It’s very important to make the API right — and for many reasons. After the release, you can not roll back the API. When customers and key business structures depend on your API, a change in dependency can damage other elements of the system, break down important functionality and reset your investment and time spent. When implementing an API program, you need to keep other important things in mind. I will mention only a few.
There is no canonical API
It is a mistake to try to “pre-select” the canonical design of the API for your entire company. Just trying to implement some objects and data models across the enterprise, that is, trying to create a single API that would take into account all aspects of your organization without exception, now and in the future - most likely, this is a dead end path. It is much better to provide your developers with recommendations and point out restrictions that will help them avoid mistakes, creatively reveal and apply the knowledge of the subject area to create amazing APIs that will appeal to both colleagues and partners.
Implementation process: we cut off all unnecessary
Since an API is just an interface, not a functional as such, you need to be able to design, implement, test, and deploy an API in a matter of days and weeks, but not in months. We know how companies are trying to reduce risks when creating an API, ensuring that the API is convenient to test in advance, automate the release process, make the API itself less costly, and more convenient to assemble them.
Emphasis on interaction, not integration
Another key aspect that successful companies perfectly cope with is the ability to teach development teams to embed interoperability with other elements into the API at the design stage. Such organizations explain how to work with their APIs, so these APIs are not only comprehensible, but also easily bundled with other APIs of this company. Focusing on broad collaboration is more important than designing narrow and tight integration.
Three things you can do today.
Such work, like any important changes, takes time. However, it will not take long to wait for success. I’ll tell you about three measures that I have seen in various companies that you can take today.
Adopt your own practice of creating API
A key component of the long-term success of your API program is to develop an API design practice that is independent of specific technologies. Make sure that such a program will not depend entirely on the latest fashion squeak in programming, using libraries or platforms. For this, it is most convenient to rely on the “first thing API” paradigm.
“First of all - API” means that we first design the API, and only then reflect on the details of its implementation. In principle, the business component is the same regardless of the technology with which you implement the API: be it SOAP, CRUD, REST, gRPC, GraphQL or any other popular thing. In fact, a well-designed program will most likely allow you to formulate recommendations that are relevant to different technological stacks, respectively, helping your team and evaluate possible savings, and preserve the consistency of decisions on future versions of platforms.
We guarantee low risks when creating an API
Having qualitatively designed an API, it’s time to make it real. At the same time, I recommend starting with a draft, then proceed to the prototype and assembly pattern.
The sketches of the API are exactly “thumbnails”. Small approximate "drawings" that help to create an impression about how this API can turn out "to taste and color" from the standpoint of the developer. An API sketch should be made in a few hours, not days. Moreover, on its basis a project should be obtained, which can be shown to colleagues and stakeholders, so that the first round of discussion and first modifications can be done at almost no cost. I like to use the Apiary API template for this. It uses a simple markup language that allows you to simulate a working API server in minutes. The specific tool is not so important, practice is important. Sketches help you do cheap research, and only then proceed to prepare a full-fledged API.
I noticed that when prototyping, tools like Swagger / OpenAPI are popular. Prototypes are much more elaborated models of your final product. They are similar to the scenery for the film. If you do not look closely - you see almost a real scene! The team should be able to prepare a detailed prototype in just a few days. Such a prototype should be free to connect to test tools, virtualization services, and other platform elements, so that you can directly observe how it interacts with your system. Prototypes are needed to test them. They are your last line of defense before you have to spend money on creating a real working API.
This is where the build phase ends. Next, we must formulate a work plan, set deadlines and begin to turn the prototype into a product. Sketch and prototype we needed to work out the details, identify bugs, etc. The build process itself is not so interesting. You just need to show the finished result every day, step by step to implement the API until the work is ready. Many companies seek to ensure that the assembly of the API takes no more than 90 days.
Three API management whales
Finally, if we look at the situation more broadly than at the level of the design and implementation of a single API, the organization must adhere to a viable program for working with the API and apply general API design recommendations that will be known to all teams. Informal regulation allows you to control the development of API across the enterprise, without going into excessive oversight of implementation details.
Eric Wilde, my colleague from API Academy, stresses how important it is to “manage the terrain of your API,” that is, regulate the three key elements of the API program: protocols, formats, terminology.
Protocol regulation is a clear indication of which application-level protocols an API team should support when preparing new releases. Most companies believe that all new APIs should support HTTP. Some also indicate other optional protocols, such as MQTT, Thrift, and other binary protocols. Here it is most important to inform all teams in advance: “If you want to ensure the interoperability of your APIs in the future, you should use these protocols.” To fulfill this rule, it is advisable to use a continuous delivery pipeline.
The regulation of formats means that you need to agree in what formats the data will be sent between services. Almost all browsers expect a response in HTML-format - just at the level of your API you need to decide in what format it will interact with the entire ecosystem. Most companies prefer simple formats, such as JSON, XML or CSV, but their message models lack key metadata, in particular, object names, links, and input forms — and they are necessary for long-term interactions. On the other hand, I know of companies that use more advanced formats, such as HAL, Siren, and Collection + JSON for HTTP-based APIs. For binary interactions between services, many organizations take protobuf and similar formats as the basis. No matter which format you choose,
The third whale of API management is terminology. In this case, we are talking about control over the names of data points and identifiers of actions used when exchanging messages between services. Adhering to the terminology, the organization can have no doubt that new services will be normally accepted by existing ones. Recalling the “common language” proposed by Eric Evans for subject-oriented design (DDD), I note that the terminology you chose is needed in order to talk about the most critical business operation. It should be difficult to release in the production of such a service, which uses "completely new" names of data fields and action identifiers. On the contrary, elements of the assembly line must be tested for compliance with the common terminology adopted throughout the company,
Having mastered these principles: “First of all - API”, “sketch-prototype-assembly” and “three whales of management API”, your team will be able to use their API at full capacity, without risking their stability during execution.
Today we wanted to offer you a translation of the program article of the notorious Mike Amundsen , the lead architect from API Academy. In this relatively small text, Mike clearly explains why you need to pay special attention to the development of the API, and how the API is done correctly.
During my work as a teacher in API Academy, I happened to travel around the whole world, meeting with amazing people. They are working on stunning projects in all sorts of companies - from young startups to established global enterprises. Incredibly, wherever I was and with whom I spoke, in our conversations a multitude of common ideas, techniques and impressions surfaced. The three most common are microservices, APIs, and a culture of innovation. All three are being introduced to speed up the organization’s digital transformation.
This article is the second in a series of three publications. Here I will talk about what we teach in our API Academy in the context of these three powerful trends, and also describe some techniques that will help your company move from big words to real transformations. In the previous articleI paid special attention to the importance of microservices and focused on three things that you can do today: 1) transition to continuous delivery, 2) provision of prepared deployments and 3) shortening the work queue to reduce the time to product to market. These three most important habits will help your organization to take full advantage of microservices.
APIs provide multi-channel delivery.
Many companies use microservices, trying to encapsulate the key capabilities of their organization precisely to ensure their scalability and reliability. Microservices correspond to important functional elements of the IT component of your company. But this is only half the battle. It is also necessary to figure out how to provide these opportunities so that with their help it would be convenient to solve the actual challenges faced by your business. This is where the APIs come into play.
API - application programming interfaces - play the same role for a machine as a graphical interface - for users of your company's information systems. Often, APIs are used to blend different capabilities into a consistent and affordable solution. For example, in your company you can use services to process transactions related to customers, accounts and sales. Each of these services was carefully designed, programmed, tested, after which it was released and built into the infrastructure of your company; The service provides functionality that is critical for your business. One day you will need to create a new solution to keep users up-to-date - and this solution should work on a wide variety of devices and platforms. So, we start to use API at full capacity.
A single API, tailored for solutions - for example, an API to familiarize users with the system - can be designed so that the most important interactions and task flows that are critical at the stage of such familiarization come to the surface. Here we can use an API that allows you to mix various functional elements relating to customers, accounts and sales into safe, powerful and accessible user interfaces for a wide variety of target audiences within your company. For example, salespeople can log into the system from a browser, office administrators from PC applications, and potential customers from smartphones and tablets.
We can say that the API is a kind of “glue” that holds program elements together, an intermediary layer through which your internal services and external service consumers interact. It is the API that is the means for distributing the key functionality in such a way that consumers of services could use it, whose task is to create important mechanisms for user interaction on a variety of hardware platforms. Such customers may include other teams working in your office, colleagues with whom you communicate remotely, key business partners, or even remote employees involved in client-side development or design.
Design thinking and API
Most companies know how important it is to devote time to developing a user interface for their applications. Because it is known that users like good design, it increases brand loyalty and allows them to promote new business. At the same time, poorly crafted interface design can annoy customers, harm the brand, reduce revenue, and select opportunities. The same is true for designing an API.
If the API is made poorly, it will be difficult for developers to understand it, because of this they can introduce errors into the system, and provoke unnecessary costs to fix bugs and modify the infrastructure. However, if the API is well developed, then it is easier for the employee to work with it effectively. The time it takes to create a stable solution is reduced, the team even gets an incentive to issue fresh, innovative solutions that would help clients and colleagues. API design is such an important area of work that Werner Vogels, Amazon’s chief technical officer, even said:
We immediately realized that designing an API is a very important task, since we will have only one attempt to solve it correctly.
It is high-quality APIs that help attract partners to your digital ecosystem and simplify internal corporate transformations of your business. The ability to spend time in order to do everything correctly, and, in the long run, is an important skill that we trace just from those companies that have learned to use their API at full capacity.
Essential API design advice
It’s very important to make the API right — and for many reasons. After the release, you can not roll back the API. When customers and key business structures depend on your API, a change in dependency can damage other elements of the system, break down important functionality and reset your investment and time spent. When implementing an API program, you need to keep other important things in mind. I will mention only a few.
There is no canonical API
It is a mistake to try to “pre-select” the canonical design of the API for your entire company. Just trying to implement some objects and data models across the enterprise, that is, trying to create a single API that would take into account all aspects of your organization without exception, now and in the future - most likely, this is a dead end path. It is much better to provide your developers with recommendations and point out restrictions that will help them avoid mistakes, creatively reveal and apply the knowledge of the subject area to create amazing APIs that will appeal to both colleagues and partners.
Implementation process: we cut off all unnecessary
Since an API is just an interface, not a functional as such, you need to be able to design, implement, test, and deploy an API in a matter of days and weeks, but not in months. We know how companies are trying to reduce risks when creating an API, ensuring that the API is convenient to test in advance, automate the release process, make the API itself less costly, and more convenient to assemble them.
Emphasis on interaction, not integration
Another key aspect that successful companies perfectly cope with is the ability to teach development teams to embed interoperability with other elements into the API at the design stage. Such organizations explain how to work with their APIs, so these APIs are not only comprehensible, but also easily bundled with other APIs of this company. Focusing on broad collaboration is more important than designing narrow and tight integration.
Three things you can do today.
Such work, like any important changes, takes time. However, it will not take long to wait for success. I’ll tell you about three measures that I have seen in various companies that you can take today.
Adopt your own practice of creating API
A key component of the long-term success of your API program is to develop an API design practice that is independent of specific technologies. Make sure that such a program will not depend entirely on the latest fashion squeak in programming, using libraries or platforms. For this, it is most convenient to rely on the “first thing API” paradigm.
“First of all - API” means that we first design the API, and only then reflect on the details of its implementation. In principle, the business component is the same regardless of the technology with which you implement the API: be it SOAP, CRUD, REST, gRPC, GraphQL or any other popular thing. In fact, a well-designed program will most likely allow you to formulate recommendations that are relevant to different technological stacks, respectively, helping your team and evaluate possible savings, and preserve the consistency of decisions on future versions of platforms.
We guarantee low risks when creating an API
Having qualitatively designed an API, it’s time to make it real. At the same time, I recommend starting with a draft, then proceed to the prototype and assembly pattern.
The sketches of the API are exactly “thumbnails”. Small approximate "drawings" that help to create an impression about how this API can turn out "to taste and color" from the standpoint of the developer. An API sketch should be made in a few hours, not days. Moreover, on its basis a project should be obtained, which can be shown to colleagues and stakeholders, so that the first round of discussion and first modifications can be done at almost no cost. I like to use the Apiary API template for this. It uses a simple markup language that allows you to simulate a working API server in minutes. The specific tool is not so important, practice is important. Sketches help you do cheap research, and only then proceed to prepare a full-fledged API.
I noticed that when prototyping, tools like Swagger / OpenAPI are popular. Prototypes are much more elaborated models of your final product. They are similar to the scenery for the film. If you do not look closely - you see almost a real scene! The team should be able to prepare a detailed prototype in just a few days. Such a prototype should be free to connect to test tools, virtualization services, and other platform elements, so that you can directly observe how it interacts with your system. Prototypes are needed to test them. They are your last line of defense before you have to spend money on creating a real working API.
This is where the build phase ends. Next, we must formulate a work plan, set deadlines and begin to turn the prototype into a product. Sketch and prototype we needed to work out the details, identify bugs, etc. The build process itself is not so interesting. You just need to show the finished result every day, step by step to implement the API until the work is ready. Many companies seek to ensure that the assembly of the API takes no more than 90 days.
Three API management whales
Finally, if we look at the situation more broadly than at the level of the design and implementation of a single API, the organization must adhere to a viable program for working with the API and apply general API design recommendations that will be known to all teams. Informal regulation allows you to control the development of API across the enterprise, without going into excessive oversight of implementation details.
Eric Wilde, my colleague from API Academy, stresses how important it is to “manage the terrain of your API,” that is, regulate the three key elements of the API program: protocols, formats, terminology.
Protocol regulation is a clear indication of which application-level protocols an API team should support when preparing new releases. Most companies believe that all new APIs should support HTTP. Some also indicate other optional protocols, such as MQTT, Thrift, and other binary protocols. Here it is most important to inform all teams in advance: “If you want to ensure the interoperability of your APIs in the future, you should use these protocols.” To fulfill this rule, it is advisable to use a continuous delivery pipeline.
The regulation of formats means that you need to agree in what formats the data will be sent between services. Almost all browsers expect a response in HTML-format - just at the level of your API you need to decide in what format it will interact with the entire ecosystem. Most companies prefer simple formats, such as JSON, XML or CSV, but their message models lack key metadata, in particular, object names, links, and input forms — and they are necessary for long-term interactions. On the other hand, I know of companies that use more advanced formats, such as HAL, Siren, and Collection + JSON for HTTP-based APIs. For binary interactions between services, many organizations take protobuf and similar formats as the basis. No matter which format you choose,
The third whale of API management is terminology. In this case, we are talking about control over the names of data points and identifiers of actions used when exchanging messages between services. Adhering to the terminology, the organization can have no doubt that new services will be normally accepted by existing ones. Recalling the “common language” proposed by Eric Evans for subject-oriented design (DDD), I note that the terminology you chose is needed in order to talk about the most critical business operation. It should be difficult to release in the production of such a service, which uses "completely new" names of data fields and action identifiers. On the contrary, elements of the assembly line must be tested for compliance with the common terminology adopted throughout the company,
Having mastered these principles: “First of all - API”, “sketch-prototype-assembly” and “three whales of management API”, your team will be able to use their API at full capacity, without risking their stability during execution.