Configuration Management in a Software Project
At first everything was simple. Youth, enthusiasm. The project was sawn by several programmers. They all coded, as they were ready, copied the code to a shared virtual machine, occasionally kicked the admin to deliver a package or fix the config. As soon as they understood that everything was going to do a release. First, backup, then the older one gathered all its steepness into a fist, copied the project to the production server and, with the assistance of the administrator, tried to get it working there. The team waited two days, made sure that no queues of grateful users with hatchets formed, and, with a sense of pride in the work done, went to drink beer.
Then they all matured a little. Redmine / jira / etc, git / svn, jenkins, spinx-docs / rubydoc / doxygen / etc, wiki, unit tests appeared and started to be used somehow. Subprojects appeared, the stand grew. Production of Servachoks has become several. The administrator raised salt / puppet / etc, monitoring, sits in his den like a spider, rules configs on salt-master and pulls state.highstate from there.
And this is the right time to sit down and think a little about life (of the project).
There are only seven stages of the life cycle.
Eight. Everyone forgets the last point. But it is also very important (and not only for the nuclear power plant). For a software project, you need to take care of the data. At the stages prior to commissioning, you need to make sure that all the necessary data can be extracted from it, and at the stage of decommissioning that the data has actually been extracted.
This is the basic scheme adopted in system engineering. Depending on the scale, the specifics of the industry and the religious beliefs of PM, the stages can be renamed, glued together or crushed on the contrary, but the sane process can always be correlated with this scheme. If agile is accepted in the command, the diagram describes the life cycle of a single story.
Why is this all. In this context, configuration management is the process of maintaining the product in a holistic state. It begins somewhere near the completion of the first stage and ends only with the death of the project. Moreover, if this process is neglected, death can be sudden.
Library versions. Gathered, sketched a class diagram, agreed to use libcrutch. One team sat for a long time on libcrutch-1.0, the second only found out about it and downloaded libcrutch-2.0 from the Internet. And it turns out only on integration testing. You can catch a bug even on the differences between libcrutch-1.2.14 and libcrutch-1.2.15. And all sorts of LD_PRELOAD or docker only add fuel to the fire. Even if the project is all from itself on microservices, the interfaces can be exchanged with data received from libcrutch and having a different format in different versions.
Component version mismatch. Some saw libbase, others libManagementFacade. In the process, it turned out that in libbase-1.14.3 there is a small but insidious bug. We talked, corrected, forgotten. Tested on libbase-1.14.4, and libbase-1.14.3 went into release.
Changing the environment configuration. One POST request suddenly began to work for a long time. We looked, it is not so important, let it work for ourselves. We increased the timeout for waiting for a backend's response in nginx. The admin on the stand corrected and forgot. They rolled out and again catch the old bugs, but now in combat conditions.
Change of design decisions. We started to do it under Windows, then inspired by RMS’s ideas, decided to switch to Ubuntu, but they didn’t bring the solution to everyone. They started to collect, everyone brought deb packages, and someone who was in the tank exe'shnik.
Loss of user-significant functionality. They brought a new version, talked for a long time about the change of design, about new frameworks, about advanced algorithms. Users listened, nodded their heads, and said: “This is all well and good, but you made a mold for us at our request. She used to be the fifth sub-item in the third menu item, where is she now? ”Lost on some merge request.
Programmers are very lucky to have git. He takes the brunt of the brunt and is required of them quite a bit.
At the stage of completion of conceptual design , when the specialists in the subject say: “We need such a system!”, - technicians unanimously say “Let's do it!”, - managers give the go-ahead: “We will allocate resources - do it!”, You need to make sure that an agreed description of the system has been taken out of the head of experts, cut into requirements and put into documentation. During development, this description will change. You need to make sure that the description is versioned. A good option, if it's text, pick it up in git.
At the stage of architectural designwhen the architect said how he sees it, you need to make sure that this vision is taken out of his head, put into the documentation, a tag with the version is pasted. If this is a notebook sheet with a diagram, you need to scan it, put it in the file system (or wiki) and make a link to it.
At the development stageyou need to make sure the code is documented. It’s good for modules to have separate documents (in git) that describe their requirements and their behavior. Leaving a lot of information in redmine / jira is not worth it. After completing a big feature, before merge in master, you need to make sure that its description from the task tracker is correctly transferred to the documentation. Just because after some time, as part of another task, the behavior may change and collecting documentation for several tasks will be difficult. Task tracker does not provide a complete picture.
User documentation is good at the development stage. Keep (if possible) in git and edit in parallel with the code. If there are no special technical writers, the context will go away, everyone will forget, there will definitely not be documentation.
Upon verificationprogram compliance with advanced requirements is checked. In the end, you need to make sure that all requirements are assigned the status fulfilled / not fulfilled.
At the validation stage , it is checked whether the program can be used. You need to make sure that all changes made to the program behavior are immediately reflected in the documentation.
At the commissioning stage , the correctness of preparation and rolling of the release is checked. You need to make sure that all components of the correct versions are hemmed into it. The main blow here is salt / puppet. It is possible without them, simply by issuing installation instructions, but it is easier with them. You need to cook them correctly and ahead of time.
About the operation phase , everything is clear. You just need to follow the manufacturer's instructions.
At the stage of decommissioning, make sure that all the necessary data is taken out.
About assembly and salt / puppet. This is the second line of defense (immediately after git). The operational scheme of application is approximately the following:
That's all. Manage the configuration correctly, and do not forget that a good process is one that goes around all the rakes and gives an excellent result the first time.
Then they all matured a little. Redmine / jira / etc, git / svn, jenkins, spinx-docs / rubydoc / doxygen / etc, wiki, unit tests appeared and started to be used somehow. Subprojects appeared, the stand grew. Production of Servachoks has become several. The administrator raised salt / puppet / etc, monitoring, sits in his den like a spider, rules configs on salt-master and pulls state.highstate from there.
A life
And this is the right time to sit down and think a little about life (of the project).
There are only seven stages of the life cycle.
- Conceptual design. At this stage, you need to understand what you need to do.
- Architectural design. At this stage, you need to understand how to do it.
- Implementation This is directly coding and unit testing.
- Verification Verification that the program performs all the intended functions.
- Validation Check that the program can still be used. This does not suddenly follow from the previous paragraph.
- Commissioning. It usually includes rolling out a release, data migration, user training.
- Actually the operation itself.
- Removal from service
Eight. Everyone forgets the last point. But it is also very important (and not only for the nuclear power plant). For a software project, you need to take care of the data. At the stages prior to commissioning, you need to make sure that all the necessary data can be extracted from it, and at the stage of decommissioning that the data has actually been extracted.
This is the basic scheme adopted in system engineering. Depending on the scale, the specifics of the industry and the religious beliefs of PM, the stages can be renamed, glued together or crushed on the contrary, but the sane process can always be correlated with this scheme. If agile is accepted in the command, the diagram describes the life cycle of a single story.
Why is this all. In this context, configuration management is the process of maintaining the product in a holistic state. It begins somewhere near the completion of the first stage and ends only with the death of the project. Moreover, if this process is neglected, death can be sudden.
What could break?
Library versions. Gathered, sketched a class diagram, agreed to use libcrutch. One team sat for a long time on libcrutch-1.0, the second only found out about it and downloaded libcrutch-2.0 from the Internet. And it turns out only on integration testing. You can catch a bug even on the differences between libcrutch-1.2.14 and libcrutch-1.2.15. And all sorts of LD_PRELOAD or docker only add fuel to the fire. Even if the project is all from itself on microservices, the interfaces can be exchanged with data received from libcrutch and having a different format in different versions.
Component version mismatch. Some saw libbase, others libManagementFacade. In the process, it turned out that in libbase-1.14.3 there is a small but insidious bug. We talked, corrected, forgotten. Tested on libbase-1.14.4, and libbase-1.14.3 went into release.
Changing the environment configuration. One POST request suddenly began to work for a long time. We looked, it is not so important, let it work for ourselves. We increased the timeout for waiting for a backend's response in nginx. The admin on the stand corrected and forgot. They rolled out and again catch the old bugs, but now in combat conditions.
Change of design decisions. We started to do it under Windows, then inspired by RMS’s ideas, decided to switch to Ubuntu, but they didn’t bring the solution to everyone. They started to collect, everyone brought deb packages, and someone who was in the tank exe'shnik.
Loss of user-significant functionality. They brought a new version, talked for a long time about the change of design, about new frameworks, about advanced algorithms. Users listened, nodded their heads, and said: “This is all well and good, but you made a mold for us at our request. She used to be the fifth sub-item in the third menu item, where is she now? ”Lost on some merge request.
What to do
Programmers are very lucky to have git. He takes the brunt of the brunt and is required of them quite a bit.
- Identify all the components that are necessary for the functioning of the project, make sure that they are correctly versioned. Configuration as a first approximation is a list of components and their versions.
- Understand how configuration is transferred from the stand to production.
- Start requirements management. Generally speaking, requirements management is a separate process. As part of the configuration management, you need to make sure that for each component that falls into the release kit, documentation is attached that accurately describes the requirements for this component and their status: completed, not completed, partially completed, with reservations.
- And indeed, each component should have documentation that describes what, how and why it does.
At the stage of completion of conceptual design , when the specialists in the subject say: “We need such a system!”, - technicians unanimously say “Let's do it!”, - managers give the go-ahead: “We will allocate resources - do it!”, You need to make sure that an agreed description of the system has been taken out of the head of experts, cut into requirements and put into documentation. During development, this description will change. You need to make sure that the description is versioned. A good option, if it's text, pick it up in git.
At the stage of architectural designwhen the architect said how he sees it, you need to make sure that this vision is taken out of his head, put into the documentation, a tag with the version is pasted. If this is a notebook sheet with a diagram, you need to scan it, put it in the file system (or wiki) and make a link to it.
At the development stageyou need to make sure the code is documented. It’s good for modules to have separate documents (in git) that describe their requirements and their behavior. Leaving a lot of information in redmine / jira is not worth it. After completing a big feature, before merge in master, you need to make sure that its description from the task tracker is correctly transferred to the documentation. Just because after some time, as part of another task, the behavior may change and collecting documentation for several tasks will be difficult. Task tracker does not provide a complete picture.
User documentation is good at the development stage. Keep (if possible) in git and edit in parallel with the code. If there are no special technical writers, the context will go away, everyone will forget, there will definitely not be documentation.
Upon verificationprogram compliance with advanced requirements is checked. In the end, you need to make sure that all requirements are assigned the status fulfilled / not fulfilled.
At the validation stage , it is checked whether the program can be used. You need to make sure that all changes made to the program behavior are immediately reflected in the documentation.
At the commissioning stage , the correctness of preparation and rolling of the release is checked. You need to make sure that all components of the correct versions are hemmed into it. The main blow here is salt / puppet. It is possible without them, simply by issuing installation instructions, but it is easier with them. You need to cook them correctly and ahead of time.
About the operation phase , everything is clear. You just need to follow the manufacturer's instructions.
At the stage of decommissioning, make sure that all the necessary data is taken out.
About assembly and salt / puppet. This is the second line of defense (immediately after git). The operational scheme of application is approximately the following:
- Make sure that the situation with each third-party package is clear: where it comes from, which version, which patches were applied. If some kind of radish (a bad person) sticks the same versions on physically different files, you need to convince him that he is wrong, or stick an additional version on all his products.
- If all rpm drop into one repository, you need to make sure that it is clear which version will be rolled. A good option is to have a rebuild script for the entire repository and paste the version on the entire repository. Another is to explicitly indicate the version in the manifest / sls file. By the way, puppet has a bug, the package resource does not know how to lower the version. Why they are not ashamed, I do not know.
- All manifest / sls files are stored in git. In pillar for salt or class parameters for puppet, only what distinguishes the stand from production is displayed. Such things, for example, are web service ulrs, parameters like shared_buffers for postgres, flags that enable debug mode. Everything else is ruthlessly hard. Parameters are set once during the deployment of the stand and are rarely changed in the future. Sls files are perceived as code, it is rolled onto a stand, tested and transferred to production unchanged.
That's all. Manage the configuration correctly, and do not forget that a good process is one that goes around all the rakes and gives an excellent result the first time.