May 21, 2019 at 11:56Principles of developing modern applications from NGINX. Part 1
- Transfer
Hello friends. In anticipation of the launch of the “Backend PHP Developer” course , we traditionally share with you the translation of useful material.
The software solves more and more everyday tasks, while becoming more and more difficult. As Mark Andressen once said, it swallows the world.
As a result, approaches to application development and delivery have changed dramatically over the past few years. These were shifts of a tectonic scale, which as a result led to the emergence of a set of principles. These principles have proven useful in team building, designing, developing, and delivering your application to end users.
The principles can be summarized as follows:the application should be small, networked and have a developer-oriented architecture . Based on these three principles, you can create a reliable, comprehensive application that can be quickly and safely delivered to the end user, and can be easily scaled and expanded.
Each of the proposed principles has a number of aspects that we will discuss to show how each principle contributes to the achievement of the ultimate goal, which is to quickly deliver reliable applications that are easy to maintain and use. We will consider the principles in comparison with their opposites in order to clarify what it means, say, "Make sure you use the principle of smallness ."
We hope that this article will encourage you to use the proposed principles for building modern applications that will provide a unified approach to design in the context of an ever-growing technology stack.
By applying these principles, you will find yourself taking advantage of the latest software development trends, including the DevOps approach to developing and delivering applications, using containers (like Docker ) and frameworks for container orchestration (like Kubernetes ), using microservices (including NGINX Microservice Architecture) and Network Communications Architecture for Microservice Applications
What is a Modern Application?
Modern applications? Modern stack? What exactly does “modern” mean?
Most developers have only a general idea of what a modern application consists of, so you need to give a clear definition of this concept.
A modern application supports several clients, whether it is a user interface on the JavaScript React library, a mobile application for Android or iOS, or an application that connects to another via the API. A modern application implies the presence of an indefinite number of customers for whom it provides data or services.
A modern application provides an API for accessing the requested data and services. The API should be immutable and constant, and not written specifically for a specific request from any particular client. The API is accessible via HTTP (S) and provides access to all the functionality that is in the GUI or CLI.
Data must be accessible in a generally accepted, compatible format, such as JSON. The API provides objects and services in a clear, organized manner, for example, the RESTful API or GraphQL provide a decent interface.
Modern applications are built on a modern stack, and a modern stack is the stack that supports such applications, respectively. Such a stack allows the developer to easily create an application with an HTTP interface and clear API endpoints. The approach chosen will allow your application to easily receive and send data in JSON format. In other words, the modern stack corresponds to the elements of the Twelve-Factor application for microservices .
Popular versions of this type of stack are based on Java , Python , Node , Ruby , PHP, and Go . Micro Service Architecture NGINXIt represents an example of a modern stack implemented in each of the mentioned languages.
Please note that we do not exclusively advocate a microservice approach. Many of you work with monoliths that need to evolve, while others deal with SOA applications that expand and evolve to become microservice applications. Still others are moving towards serverless applications, and some are introducing combinations of the above. The principles set forth in the article are applicable to each of these systems with only a few minor changes.
Principles
Now that we have a general understanding of what a modern application is and a modern stack, it is time to immerse yourself in the principles of architecture and development that will serve you well in developing, implementing and supporting a modern application.
One of the principles is “create small applications”, let's just call it the principle of smallness . There are incredibly complex applications that consist of a large number of moving components. In turn, building an application from small discrete components simplifies its design, maintenance and work with it as a whole. (Note, we said “simplifies”, not “makes it simple.”)
The second principle is that we can increase the productivity of developers by helping them focus on the functions that they develop, while freeing them from worries about infrastructure and CI / CD during implementation. So, in a nutshell, our approach is focused on developers .
Finally, everything about your application should be connected to the network. Over the past 20 years, we have made great strides towards a networked future, as networks are faster and applications are more complex. As we have already found out, a modern application should be used over the network by many different clients. The use of network thinking in architecture has significant advantages that go well with the principle of smallness and the concept of approach,developer-centric .
If during the development and implementation of the application you will be mindful of these principles, you will have an undeniable advantage in the development and delivery of your product.
Let's look at these three principles in more detail.
The principle of smallness It is
difficult for the human brain to perceive at the same time a large amount of information. In psychology, the term cognitive load means the total amount of mental effort required to store information in memory. Reducing the cognitive load on developers is a priority, because in this case they can focus on solving the problem, instead of keeping in mind the current complex model of the entire application and the developed functions.
Applications are decomposed for the following reasons:
There are several ways to reduce the cognitive burden on developers, and this is where the principle of smallness comes into play.
So, there are three ways to lower cognitive load:
Reducing the development time frame
Let's go back to the times when the methodology
The biggest change in the application development process has been the introduction of the agile methodology. One of the main features of the methodology
Shifting the focus from a massive application to specific small functions that can be completed in a two-week sprint, looking ahead to no more than one function from the next sprint in the head, is a significant change. This allowed us to increase the productivity of the development while reducing the cognitive load, which constantly fluctuated.
The methodology
Small codebases
The next step in reducing cognitive load is to reduce the code base. As a rule, modern applications are massive - a reliable, enterprise application can consist of thousands of files and hundreds of thousands of lines of code. Depending on the organization of the communication files and the dependencies of the code and files, they may be obvious or vice versa. Even debugging the code itself can cause problems, depending on the libraries used and how well the debugging tools distinguish between libraries / packages / modules and user code.
Building a working mental model of the application code can take an impressive amount of time, and again place a large cognitive load on the developer. This is especially true for monolithic code bases, where there is a large amount of code, the interaction between the functional components of which is not clearly defined, and the separation of objects of attention is often blurred, because the functional boundaries are not respected.
One of the effective ways to reduce the cognitive load on engineers is the transition to a microservice architecture. In the microservice approach, each service focuses on one set of functions; the meaning of the service is usually defined and understandable. The boundaries of the service are also clear - remember that communication with the service is carried out using the API, so the data generated by one service can easily be transferred to another.
Interaction with other services is usually limited to several user services and several provider services that use simple and clean API calls, for example, using REST. This means that the cognitive load on the engineer is seriously reduced. The most difficult task is to understand the model of interaction between services and how things like transactions occur in several services. As a result, the use of microservices reduces the cognitive load, reducing the amount of code, indicating clear boundaries of the service and providing an understanding of the relationship between users and providers.
Small incremental changes
The last element of the principle of smallnessIs change management. It’s a special temptation for developers to look at the code base (even, perhaps, on their own, older code) and say: “This is crap, we need to rewrite it all.” Sometimes this is the right decision, and sometimes not. It places the burden of global model change on the development team, which in turn leads to a massive cognitive load. It is better for engineers to focus on the changes that they can make during the sprint, in order to later roll out the necessary functionality in a timely manner, albeit gradually. The final product should be reminiscent of a preplanned one, but with some modifications and testing to meet customer needs.
When rewriting large parts of the code, it is sometimes impossible to quickly deliver changes, since other system dependencies come into play here. In order to control the flow of changes, you can use the feature hiding. In principle, this means the functionality is on production, but it is not available using the settings of environment variables (env-var) or some other configuration mechanism. If the code has passed all the quality control processes, then it may be in production in a hidden state. However, this strategy only works if the feature is ultimately enabled. Otherwise, it will only litter the code and add cognitive load, which the developer will have to cope with for productive work.
Engineers have to overcome many difficulties even with the simple implementation of additional functionality. On the part of the management, it would be reasonable to reduce the extra load on the team so that it can focus on the key elements of the functional. There are three things you can do to help your development team:
If you apply the principle of smallness in your work, your team will become much happier, better focus on the implementation of the necessary functions and more likely to roll out qualitative changes faster. But this does not mean that the work cannot be complicated, sometimes on the contrary, the introduction of new functionality requires modification of several services and this process can be more complicated than the similar one in a monolithic architecture. In any case, the benefits of a small approach are worth it.
The end of the first part.
Soon we will publish the second part of the translation, and now we are waiting for your comments and we invite you to open day , which will be held today at 20.00.
The software solves more and more everyday tasks, while becoming more and more difficult. As Mark Andressen once said, it swallows the world.
As a result, approaches to application development and delivery have changed dramatically over the past few years. These were shifts of a tectonic scale, which as a result led to the emergence of a set of principles. These principles have proven useful in team building, designing, developing, and delivering your application to end users.
The principles can be summarized as follows:the application should be small, networked and have a developer-oriented architecture . Based on these three principles, you can create a reliable, comprehensive application that can be quickly and safely delivered to the end user, and can be easily scaled and expanded.
Each of the proposed principles has a number of aspects that we will discuss to show how each principle contributes to the achievement of the ultimate goal, which is to quickly deliver reliable applications that are easy to maintain and use. We will consider the principles in comparison with their opposites in order to clarify what it means, say, "Make sure you use the principle of smallness ."
We hope that this article will encourage you to use the proposed principles for building modern applications that will provide a unified approach to design in the context of an ever-growing technology stack.
By applying these principles, you will find yourself taking advantage of the latest software development trends, including the DevOps approach to developing and delivering applications, using containers (like Docker ) and frameworks for container orchestration (like Kubernetes ), using microservices (including NGINX Microservice Architecture) and Network Communications Architecture for Microservice Applications
What is a Modern Application?
Modern applications? Modern stack? What exactly does “modern” mean?
Most developers have only a general idea of what a modern application consists of, so you need to give a clear definition of this concept.
A modern application supports several clients, whether it is a user interface on the JavaScript React library, a mobile application for Android or iOS, or an application that connects to another via the API. A modern application implies the presence of an indefinite number of customers for whom it provides data or services.
A modern application provides an API for accessing the requested data and services. The API should be immutable and constant, and not written specifically for a specific request from any particular client. The API is accessible via HTTP (S) and provides access to all the functionality that is in the GUI or CLI.
Data must be accessible in a generally accepted, compatible format, such as JSON. The API provides objects and services in a clear, organized manner, for example, the RESTful API or GraphQL provide a decent interface.
Modern applications are built on a modern stack, and a modern stack is the stack that supports such applications, respectively. Such a stack allows the developer to easily create an application with an HTTP interface and clear API endpoints. The approach chosen will allow your application to easily receive and send data in JSON format. In other words, the modern stack corresponds to the elements of the Twelve-Factor application for microservices .
Popular versions of this type of stack are based on Java , Python , Node , Ruby , PHP, and Go . Micro Service Architecture NGINXIt represents an example of a modern stack implemented in each of the mentioned languages.
Please note that we do not exclusively advocate a microservice approach. Many of you work with monoliths that need to evolve, while others deal with SOA applications that expand and evolve to become microservice applications. Still others are moving towards serverless applications, and some are introducing combinations of the above. The principles set forth in the article are applicable to each of these systems with only a few minor changes.
Principles
Now that we have a general understanding of what a modern application is and a modern stack, it is time to immerse yourself in the principles of architecture and development that will serve you well in developing, implementing and supporting a modern application.
One of the principles is “create small applications”, let's just call it the principle of smallness . There are incredibly complex applications that consist of a large number of moving components. In turn, building an application from small discrete components simplifies its design, maintenance and work with it as a whole. (Note, we said “simplifies”, not “makes it simple.”)
The second principle is that we can increase the productivity of developers by helping them focus on the functions that they develop, while freeing them from worries about infrastructure and CI / CD during implementation. So, in a nutshell, our approach is focused on developers .
Finally, everything about your application should be connected to the network. Over the past 20 years, we have made great strides towards a networked future, as networks are faster and applications are more complex. As we have already found out, a modern application should be used over the network by many different clients. The use of network thinking in architecture has significant advantages that go well with the principle of smallness and the concept of approach,developer-centric .
If during the development and implementation of the application you will be mindful of these principles, you will have an undeniable advantage in the development and delivery of your product.
Let's look at these three principles in more detail.
The principle of smallness It is
difficult for the human brain to perceive at the same time a large amount of information. In psychology, the term cognitive load means the total amount of mental effort required to store information in memory. Reducing the cognitive load on developers is a priority, because in this case they can focus on solving the problem, instead of keeping in mind the current complex model of the entire application and the developed functions.
Applications are decomposed for the following reasons:
- Decrease in cognitive load on developers;
- Speed up and simplify testing;
- Fast delivery of changes in the application.
There are several ways to reduce the cognitive burden on developers, and this is where the principle of smallness comes into play.
So, there are three ways to lower cognitive load:
- Reduce the time frame that they should take into account when developing a new function - the shorter the time frame, the lower the cognitive load.
- Reduce the amount of code on which one-time work is carried out - less code - less load.
- Simplify the process of making incremental changes to the application.
Reducing the development time frame
Let's go back to the times when the methodology
waterfallwas the standard for the development process, and the timeframe from six months to two years for developing or updating the application was common practice. Typically, engineers first read relevant documents, such as product requirements (PRD), system reference document (SRD), architecture plan, and began to combine all these things together into one cognitive model, according to which they wrote code. As requirements and, accordingly, architecture changed, serious efforts had to be made to inform the entire team about updates to the cognitive model. In the worst case, such an approach could simply paralyze work.The biggest change in the application development process has been the introduction of the agile methodology. One of the main features of the methodology
agileis iterative development. In turn, this leads to a decrease in the cognitive load on engineers. Instead of requiring the development team to implement the application in one long cycle, the agileapproach allows you to focus on small amounts of code that can be quickly tested and deployed, while also receiving feedback. The cognitive load of the application has shifted from the timeframe from six months to two years, taking into account the huge number of specifications for a two-week addition or feature changes, aimed at a more vague understanding of a large application.Shifting the focus from a massive application to specific small functions that can be completed in a two-week sprint, looking ahead to no more than one function from the next sprint in the head, is a significant change. This allowed us to increase the productivity of the development while reducing the cognitive load, which constantly fluctuated.
The methodology
agileassumes that the final application will be a slightly modified version of the original concept, so the final development point is necessarily ambiguous. Only the results of each particular sprint can be clear and precise. Small codebases
The next step in reducing cognitive load is to reduce the code base. As a rule, modern applications are massive - a reliable, enterprise application can consist of thousands of files and hundreds of thousands of lines of code. Depending on the organization of the communication files and the dependencies of the code and files, they may be obvious or vice versa. Even debugging the code itself can cause problems, depending on the libraries used and how well the debugging tools distinguish between libraries / packages / modules and user code.
Building a working mental model of the application code can take an impressive amount of time, and again place a large cognitive load on the developer. This is especially true for monolithic code bases, where there is a large amount of code, the interaction between the functional components of which is not clearly defined, and the separation of objects of attention is often blurred, because the functional boundaries are not respected.
One of the effective ways to reduce the cognitive load on engineers is the transition to a microservice architecture. In the microservice approach, each service focuses on one set of functions; the meaning of the service is usually defined and understandable. The boundaries of the service are also clear - remember that communication with the service is carried out using the API, so the data generated by one service can easily be transferred to another.
Interaction with other services is usually limited to several user services and several provider services that use simple and clean API calls, for example, using REST. This means that the cognitive load on the engineer is seriously reduced. The most difficult task is to understand the model of interaction between services and how things like transactions occur in several services. As a result, the use of microservices reduces the cognitive load, reducing the amount of code, indicating clear boundaries of the service and providing an understanding of the relationship between users and providers.
Small incremental changes
The last element of the principle of smallnessIs change management. It’s a special temptation for developers to look at the code base (even, perhaps, on their own, older code) and say: “This is crap, we need to rewrite it all.” Sometimes this is the right decision, and sometimes not. It places the burden of global model change on the development team, which in turn leads to a massive cognitive load. It is better for engineers to focus on the changes that they can make during the sprint, in order to later roll out the necessary functionality in a timely manner, albeit gradually. The final product should be reminiscent of a preplanned one, but with some modifications and testing to meet customer needs.
When rewriting large parts of the code, it is sometimes impossible to quickly deliver changes, since other system dependencies come into play here. In order to control the flow of changes, you can use the feature hiding. In principle, this means the functionality is on production, but it is not available using the settings of environment variables (env-var) or some other configuration mechanism. If the code has passed all the quality control processes, then it may be in production in a hidden state. However, this strategy only works if the feature is ultimately enabled. Otherwise, it will only litter the code and add cognitive load, which the developer will have to cope with for productive work.
Engineers have to overcome many difficulties even with the simple implementation of additional functionality. On the part of the management, it would be reasonable to reduce the extra load on the team so that it can focus on the key elements of the functional. There are three things you can do to help your development team:
- Use a methodology
agileto limit the time frame within which a team should focus on key functions. - Implement your application as multiple microservices. This will limit the number of features being implemented and strengthen the boundaries that hold the cognitive load during work.
- Prefer incremental changes to large and bulky, change small pieces of code. Use feature hiding to implement changes, even if they are not visible immediately after adding.
If you apply the principle of smallness in your work, your team will become much happier, better focus on the implementation of the necessary functions and more likely to roll out qualitative changes faster. But this does not mean that the work cannot be complicated, sometimes on the contrary, the introduction of new functionality requires modification of several services and this process can be more complicated than the similar one in a monolithic architecture. In any case, the benefits of a small approach are worth it.
The end of the first part.
Soon we will publish the second part of the translation, and now we are waiting for your comments and we invite you to open day , which will be held today at 20.00.