June 25, 2015 at 17:41Simple words about the Twelve-Factor App
The well-known heroku provider supports a manifest called the Twelve-Factor App. This is a set of best practices for developing modern web applications on any platform. Practitioners describe applications that are ready:
We can assume that the manifest is partly advertising - 12-factor applications are most conveniently deployed to heroku. But as the manifesto becomes more and more popular, some cloud providers include best practices in their environment, and these practices will be useful both for developers and those who deploy and administer applications.
The manifesto (there is an excellent translation on the hub ) is too detailed and good for a detailed study. In the same article, I will briefly focus on the main advantages.
All application code must be in a version control system. And deploy for development, testing and production servers from one repository. Which, however, does not exclude that in deployments there may be code in various branches that have not yet been added to the release.
If you have several applications using common code, then the common code must be allocated to a separate library and declared as a dependency. If at some stage of development, you have several loosely coupled applications in one repository, then you should divide the application into several, each of which should be a 12-factor application.
Applications should not have implicit dependencies. First, all dependencies, both system and libraries, must be written in the dependency manifest. All modern languages are provided by a package manager with a dependency manifest. In addition, dependencies must be isolated so that the system library does not “leak” into applications.
So in Ruby it uses Gemfile as a manifest format for declaring dependencies, bundle exec for isolating dependencies. In addition to the same operation of the application on different platforms, this will allow new developers to quickly join the project, or quickly connect new dependencies.
Here we should mention Docker, which isolates and explicitly declares even the envy of the OS.
Configuration is all the parameters that vary depending on where the application is running:
The code does not depend on the environment, but the configuration depends. Therefore, the code should be stored in the repository, and the configuration in the environment. If you can publish your code in the public domain without compromising your personal accounts - you are doing everything right.
A fairly popular option is to use pre-configured configuration sets (development, test, and production). This separation cannot be considered a good solution, since adding new environments (development-joe, qa, staging) disproportionately increases the number of parameters that need to be supported.
Third-party services are databases, mail services, cache servers and APIs of various services. A 12-factor application should not distinguish between local and remote services. Each service is a connected resource, the data for connecting to which (address and credentials) must be stored in the configuration.
Replacing the local database with Amazon RDS, or replacing the local mail server with a third-party one, should not affect the application code.
Application deployment consists of three separate steps:
When developing an application of 12 factors, these steps must be strictly separated. The assembly is initiated by the developer when he is ready to post the changes. This may be a longer process, but after this change in the release code can not be made. On the other hand, the execution phase should be as simple as possible so that it can be performed automatically in the event of equipment shutdown, or other errors, without the intervention of the developer.
An application should run as one or more processes that do not retain their internal state. An application can use data in RAM or on disk as temporary storage. For example, when transcoding images. But any user data must be in permanent storage (pluggable resource).
This primarily relates to session data, since the next user connection may occur to another process (due to a hardware failure, or processor restart).
A PHP application can be launched as a module inside Apache HTTPD, or a Java application can be launched inside Tomcat. On the contrary, the 12 factor application is completely self-sufficient, and does not rely on a web server.
This is usually done by using a dependency declaration to add a web server library to an application such as Tornado in Python, Thin in Ruby, Jetty in Java, and ReactPHP in PHP.
Different tasks must be handled by different processes. For example, HTTP requests can be processed by a web process, and lengthy background tasks are processed by a workflow. This will allow in the future to scale only those processes that are necessary.
It is important that processes should not be demonized (should be executed, not run). The process manager must monitor the application output stream and respond to process errors and crashes.
This is a logical consequence of processes that do not retain their state. Maximize reliability with processes that quickly start and correctly complete their work, even if it crashes.
Background processes should return the current task to the queue, and request processing processes should correctly complete requests. Each task of any process should be available for repeated execution (for example, using transactions).
Now there is a significant difference between development and deployment - different people do it, at different times, and with different tools. When introducing 12 factors, you should try to make development and deployment as close as possible:
A log is a stream of events, and you need to access it as a stream of events. The application should not write data to the log file itself, much less manage the files (archive or delete) .12 factor application displays the log of its work in stdout. The manager who starts the process must direct the analysis or archiving system to the logics.
This allows you to aggregate events from different processes, including both application processes and third-party services. The log analysis system allows analysis of the current state of the entire system, as well as analysis of previous work.
In the environment of the developer, the logs are simply viewed in the console.
One-time administration processes (migrations, database fixes) must obey the same rules as other processes:
- to horizontal scaling;
- to continuous deployment;
- to modern cloud hosting.
We can assume that the manifest is partly advertising - 12-factor applications are most conveniently deployed to heroku. But as the manifesto becomes more and more popular, some cloud providers include best practices in their environment, and these practices will be useful both for developers and those who deploy and administer applications.
The manifesto (there is an excellent translation on the hub ) is too detailed and good for a detailed study. In the same article, I will briefly focus on the main advantages.
One application - one repository
All application code must be in a version control system. And deploy for development, testing and production servers from one repository. Which, however, does not exclude that in deployments there may be code in various branches that have not yet been added to the release.
If you have several applications using common code, then the common code must be allocated to a separate library and declared as a dependency. If at some stage of development, you have several loosely coupled applications in one repository, then you should divide the application into several, each of which should be a 12-factor application.
Explicit Dependencies
Applications should not have implicit dependencies. First, all dependencies, both system and libraries, must be written in the dependency manifest. All modern languages are provided by a package manager with a dependency manifest. In addition, dependencies must be isolated so that the system library does not “leak” into applications.
So in Ruby it uses Gemfile as a manifest format for declaring dependencies, bundle exec for isolating dependencies. In addition to the same operation of the application on different platforms, this will allow new developers to quickly join the project, or quickly connect new dependencies.
Here we should mention Docker, which isolates and explicitly declares even the envy of the OS.
Configuration are runtime properties
Configuration is all the parameters that vary depending on where the application is running:
- logins, passwords and database addresses;
- third-party services and API keys.
The code does not depend on the environment, but the configuration depends. Therefore, the code should be stored in the repository, and the configuration in the environment. If you can publish your code in the public domain without compromising your personal accounts - you are doing everything right.
A fairly popular option is to use pre-configured configuration sets (development, test, and production). This separation cannot be considered a good solution, since adding new environments (development-joe, qa, staging) disproportionately increases the number of parameters that need to be supported.
Local and third-party services
Third-party services are databases, mail services, cache servers and APIs of various services. A 12-factor application should not distinguish between local and remote services. Each service is a connected resource, the data for connecting to which (address and credentials) must be stored in the configuration.
Replacing the local database with Amazon RDS, or replacing the local mail server with a third-party one, should not affect the application code.
Separation of assembly, release and execution
Application deployment consists of three separate steps:
- Assembly is the conversion of code into an executable package — loading dependencies, compiling files and resources.
- Release - combining an assembly with a configuration. The release is immediately ready to run in runtime.
- Execution - the launch of a number of processes from the release.
When developing an application of 12 factors, these steps must be strictly separated. The assembly is initiated by the developer when he is ready to post the changes. This may be a longer process, but after this change in the release code can not be made. On the other hand, the execution phase should be as simple as possible so that it can be performed automatically in the event of equipment shutdown, or other errors, without the intervention of the developer.
Application - a set of processes
An application should run as one or more processes that do not retain their internal state. An application can use data in RAM or on disk as temporary storage. For example, when transcoding images. But any user data must be in permanent storage (pluggable resource).
This primarily relates to session data, since the next user connection may occur to another process (due to a hardware failure, or processor restart).
The application is server independent
A PHP application can be launched as a module inside Apache HTTPD, or a Java application can be launched inside Tomcat. On the contrary, the 12 factor application is completely self-sufficient, and does not rely on a web server.
This is usually done by using a dependency declaration to add a web server library to an application such as Tornado in Python, Thin in Ruby, Jetty in Java, and ReactPHP in PHP.
Process Scale
Different tasks must be handled by different processes. For example, HTTP requests can be processed by a web process, and lengthy background tasks are processed by a workflow. This will allow in the future to scale only those processes that are necessary.
It is important that processes should not be demonized (should be executed, not run). The process manager must monitor the application output stream and respond to process errors and crashes.
Quick start and correct termination
This is a logical consequence of processes that do not retain their state. Maximize reliability with processes that quickly start and correctly complete their work, even if it crashes.
Background processes should return the current task to the queue, and request processing processes should correctly complete requests. Each task of any process should be available for repeated execution (for example, using transactions).
Move development to deployment
Now there is a significant difference between development and deployment - different people do it, at different times, and with different tools. When introducing 12 factors, you should try to make development and deployment as close as possible:
- Time - the code should get into the working version several hours after the developer wrote it.
- People - the developer must participate in the deployment of his code and monitor his work in the work.
- Tools - the development environment should maximally correspond to the testing environment (third-party services, OS).
Logs in stdout
A log is a stream of events, and you need to access it as a stream of events. The application should not write data to the log file itself, much less manage the files (archive or delete) .12 factor application displays the log of its work in stdout. The manager who starts the process must direct the analysis or archiving system to the logics.
This allows you to aggregate events from different processes, including both application processes and third-party services. The log analysis system allows analysis of the current state of the entire system, as well as analysis of previous work.
In the environment of the developer, the logs are simply viewed in the console.
Administration Tasks
One-time administration processes (migrations, database fixes) must obey the same rules as other processes:
- The task code must be in the repository in order to match the code of the main application
- Dependencies must be declared in the main manifest of dependencies, so that the process can be completed during normal deployment
- The configuration of the task must be in environment variables so that it can be performed in different environments