Configuration Management (Part 2, Overview of Tools)
It took a long time before I was ready to write the second part of the article on configuration management. The fact that not so long ago I was lucky enough to speak at the PHPCONF 2009 conference on October 8 ( Web Architect Workshop Day ) with the master class “Method of organizing the source code repository” contributed to the fact that this happened . For the presentation, a presentation was prepared in advance, as well as the text of the report. Despite the excellent organization of the event, the materials of the reports included in the conference program were not posted for public access. In compensation, I decided to publish the material used in my presentation. In addition to this article, (which is a logical continuation of the previous) dedicated to configuration management, presentation slides are available for public viewing .
This article will discuss the tools used in configuration management. Therefore, first of all, I would like to focus on how the tools used in development can affect the process of creating software.
The approaches used in configuration management may well contribute to the construction of such an established process, since this is the main discipline in determining how the working materials of a software project are managed and controlled, changes made to it, as well as information about the status of individual tasks and the entire project as a whole . As mentioned earlier (in the previous article ), they fall into the scope of interests of configuration management (each item will be considered separately below):
Version control is the main configuration management process. The absence of the need to use version control during development is practically equal to zero. Most developers are familiar with version control systems (SLE) such as CVS, Subversion, or VSS. Distributed version control systems are also gaining popularity: git, mercurial, bazaar. The mentioned version control systems have a great influence on the formation of concepts about configuration management among developers. For this reason, usually developers do not think about the thoughtful organization of the repository. The simplest and only solution offers SLE subversion. This solution is used most often: division into trunk, branches and tags. Although you can see this directory hierarchy in most subversion repositories, you cannot say that they are always equally actively used: in the best case, in the branches directory you can see several directories of parallel development branches. The apprehension of developers when creating a branched directory structure of the repository can be understood - this often bodes a lot of problems associated with merging changes between branches. But with team development of applications that have several versions or work on different platforms, it is almost impossible to avoid mergers. But you can minimize their number by regulating situations in which it is possible to carry out mergers, and in which - no. But to determine such a regulation, a thorough understanding of the phenomena associated with maintaining the repository is needed. The apprehension of developers when creating a branched directory structure of the repository can be understood - this often bodes a lot of problems associated with merging changes between branches. But with team development of applications that have several versions or work on different platforms, it is almost impossible to avoid mergers. But you can minimize their number by regulating situations in which it is possible to carry out mergers, and in which - no. But to determine such a regulation, a thorough understanding of the phenomena associated with maintaining the repository is needed. The apprehension of developers when creating a branched directory structure of the repository can be understood - this often bodes a lot of problems associated with merging changes between branches. But with team development of applications that have several versions or work on different platforms, it is almost impossible to avoid mergers. But you can minimize their number by regulating situations in which it is possible to carry out mergers, and in which - no. But to determine such a regulation, a thorough understanding of the phenomena associated with maintaining the repository is needed. But you can minimize their number by regulating situations in which it is possible to carry out mergers, and in which - no. But to determine such a regulation, a thorough understanding of the phenomena associated with maintaining the repository is needed. But you can minimize their number by regulating situations in which it is possible to carry out mergers, and in which - no. But to determine such a regulation, a thorough understanding of the phenomena associated with maintaining the repository is needed.
Another important configuration management process is assembly management. Assembly management is the automation of actions regarding:
Although unit testing cannot be fully attributed to the processes of configuration management, it has quite important properties that determine the special role of unit testing in configuration management. Thus, the use of unit testing in software projects written in interpreted programming languages determines, in addition to the need to manage assemblies, the need for a more thorough approach to configuration management. First, unit testing appears where refactoring is intended. Secondly, when deciding on the use of unit tests, the architectural and logical integrity of the project, which must be maintained at a sufficient level, is usually taken into account. It means,
In addition to supporting architectural and logical integrity in large projects, there is a need to identify syntax errors in the early stages of development. In addition to identifying syntactic and logical errors in the source code, it is often necessary to automatically verify compliance with code conventions. Static code analysis is more relevant for interpreted languages, since in compiled languages most errors are detected at the compilation stage. Sometimes, static analysis can be performed to collect source code metrics and report accordingly. An example of the libraries used for static analysis of the source code are the libraries presented in the table:
The generation of documentation based on source code and comments in the style of javaDoc, phpDoc, etc. is most often relevant for projects with active use of source code: libraries, reusable components, frameworks, etc. The tools most often used in generating documentation are presented in the table:
Continuous Integration (CI) is a software development practice that involves performing frequent automated project assemblies for the earliest possible identification and resolution of integration problems. In a regular project (not using the practice of continuous integration), where developers work independently on each part of the application, the integration stage is final. As rightly noted in a concise Wikipedia article describing CI, continuous integration cannot be applied to any project. To implement continuous integration practice, a project must satisfy a number of requirements:
An important issue that occupies a separate place when considering configuration management tasks is database integration. In most cases, databases are an integral part of applications. Development using agile methodologies involves not only continuous improvement of the program code, but also the structure of the database, as well as its functionality. And usually this happens in parallel - along with a change in the program code, changes are made to field types, field names, functions, triggers, database indexes. The evolution of the database during the life cycle of the project resembles the evolution of program code, having almost the same features when considering the database as an object of configuration management. Although the configuration management of databases and version control of databases have significant differences compared to the configuration management of program code, this is an integral part of the project management and is reflected in the database integration process. An SQL application can often be seen as an application in an application and allocated to a separate project to be versioned. There are two types of identification elements used in versioning databases: DML (Database Manipulation Language) and DDL (Database Definition Language). DML is a subset of SQL used to manipulate data: fetch, insert, delete, updates (CRUD operations). DDL is also a subset of SQL, but used to describe the structure of the database: creating tables, indexes, triggers, integrity constraints, etc.
The tendency to use systems (task setting control systems) indicates more and more frequent attempts by such systems to integrate software project management tools. Often this happens not directly, but indirectly - through the release of various plugins (integration with version control systems, displaying javaDoc, phpDoc or Doxygen documentation, etc.). But even in the basic set of functionality related to CRM (change request management), there are elements that need to be improved. Such an element could be, for example, version naming. But the need to use certain conventions not only in the issue-tracking system, but also in version control, as well as other processes of the Criminal Code, prevents the use of standardized version names.
The approach to software development, which involves the use of a flexible methodology, has a number of distinctive properties that most significantly affect the organization of configuration management. These properties are:
This article discusses the processes that are part of configuration management without going deeper into details. To ensure each process, a separate tool is used, which can be selected from many alternatives. It was noted that due to the heterogeneity of the tools themselves, as well as the difference in the tasks they solve, it is difficult to achieve their maximum integration and effectiveness. The next article will be about this - how, through the introduction of additional formalization (organization of the source code repository), to achieve a more efficient use of the tools used in the Criminal Code.
To be continued
Links:
This article will discuss the tools used in configuration management. Therefore, first of all, I would like to focus on how the tools used in development can affect the process of creating software.
- Firstly, each tool is used to solve a number of specific tasks.
- Secondly, the solution of the problem involves the preceding use of the tool to satisfy a set of requirements, without which the tool will not be effective or will not work
- Third, often the set of requirements put forward by the many tools used is not fully satisfied. In addition, the requirements of different instruments may conflict with each other. All this leads to the fact that the effectiveness of the use of such aids is reduced.
The approaches used in configuration management may well contribute to the construction of such an established process, since this is the main discipline in determining how the working materials of a software project are managed and controlled, changes made to it, as well as information about the status of individual tasks and the entire project as a whole . As mentioned earlier (in the previous article ), they fall into the scope of interests of configuration management (each item will be considered separately below):
- Version control
- Build Management
- Unit testing
- Static code analysis
- Documentation Generation (phpDoc)
- Continuous integration
- Dependency management
- Database integration
- Bug tracking (bug-tracking) and task setting control (issue-tracking)
- Release management
- Adjustment of deliveries (delivery)
- Organization of established development processes
- Coordination of the method of interaction between different parts of a software project
Version control
Version control is the main configuration management process. The absence of the need to use version control during development is practically equal to zero. Most developers are familiar with version control systems (SLE) such as CVS, Subversion, or VSS. Distributed version control systems are also gaining popularity: git, mercurial, bazaar. The mentioned version control systems have a great influence on the formation of concepts about configuration management among developers. For this reason, usually developers do not think about the thoughtful organization of the repository. The simplest and only solution offers SLE subversion. This solution is used most often: division into trunk, branches and tags. Although you can see this directory hierarchy in most subversion repositories, you cannot say that they are always equally actively used: in the best case, in the branches directory you can see several directories of parallel development branches. The apprehension of developers when creating a branched directory structure of the repository can be understood - this often bodes a lot of problems associated with merging changes between branches. But with team development of applications that have several versions or work on different platforms, it is almost impossible to avoid mergers. But you can minimize their number by regulating situations in which it is possible to carry out mergers, and in which - no. But to determine such a regulation, a thorough understanding of the phenomena associated with maintaining the repository is needed. The apprehension of developers when creating a branched directory structure of the repository can be understood - this often bodes a lot of problems associated with merging changes between branches. But with team development of applications that have several versions or work on different platforms, it is almost impossible to avoid mergers. But you can minimize their number by regulating situations in which it is possible to carry out mergers, and in which - no. But to determine such a regulation, a thorough understanding of the phenomena associated with maintaining the repository is needed. The apprehension of developers when creating a branched directory structure of the repository can be understood - this often bodes a lot of problems associated with merging changes between branches. But with team development of applications that have several versions or work on different platforms, it is almost impossible to avoid mergers. But you can minimize their number by regulating situations in which it is possible to carry out mergers, and in which - no. But to determine such a regulation, a thorough understanding of the phenomena associated with maintaining the repository is needed. But you can minimize their number by regulating situations in which it is possible to carry out mergers, and in which - no. But to determine such a regulation, a thorough understanding of the phenomena associated with maintaining the repository is needed. But you can minimize their number by regulating situations in which it is possible to carry out mergers, and in which - no. But to determine such a regulation, a thorough understanding of the phenomena associated with maintaining the repository is needed.
Assembly management
Another important configuration management process is assembly management. Assembly management is the automation of actions regarding:
- Source code compilation
- Application Deployment
- Run unit tests
- Database initialization
- Group file operations
- Compilation
- Deployment
- Interaction with version control systems
Unit testing
Although unit testing cannot be fully attributed to the processes of configuration management, it has quite important properties that determine the special role of unit testing in configuration management. Thus, the use of unit testing in software projects written in interpreted programming languages determines, in addition to the need to manage assemblies, the need for a more thorough approach to configuration management. First, unit testing appears where refactoring is intended. Secondly, when deciding on the use of unit tests, the architectural and logical integrity of the project, which must be maintained at a sufficient level, is usually taken into account. It means,
Static code analysis
In addition to supporting architectural and logical integrity in large projects, there is a need to identify syntax errors in the early stages of development. In addition to identifying syntactic and logical errors in the source code, it is often necessary to automatically verify compliance with code conventions. Static code analysis is more relevant for interpreted languages, since in compiled languages most errors are detected at the compilation stage. Sometimes, static analysis can be performed to collect source code metrics and report accordingly. An example of the libraries used for static analysis of the source code are the libraries presented in the table:
Programming language | Tool |
Java | PMD FindBugs |
C / c ++ | Cppcheck lint |
C # | FxCop StyleCop ReSharper |
Php | PHP_CodeSniffer PHP-sat PHP_Depend Pixy |
Python | PyChecker PyLint PyFlakes |
Ruby | Reek Roodi Rufus Flay Flog |
Programming language independent tools | RATS Yasca |
Documentation Generation
The generation of documentation based on source code and comments in the style of javaDoc, phpDoc, etc. is most often relevant for projects with active use of source code: libraries, reusable components, frameworks, etc. The tools most often used in generating documentation are presented in the table:
Oriented to a specific programming language | Php | phpDocumentor |
Java | Javoc | |
C ++ | Cppdoc | |
Python | pyDoc | |
Ruby | RDoc | |
Deplhi | DelphiCodeToDoc | |
C # | Ndoc | |
Not oriented to use a specific programming language | Doxygen | |
ROBODoc | ||
Twintext |
Continuous integration
Continuous Integration (CI) is a software development practice that involves performing frequent automated project assemblies for the earliest possible identification and resolution of integration problems. In a regular project (not using the practice of continuous integration), where developers work independently on each part of the application, the integration stage is final. As rightly noted in a concise Wikipedia article describing CI, continuous integration cannot be applied to any project. To implement continuous integration practice, a project must satisfy a number of requirements:
- The source code and everything necessary for the assembly is stored either in the source code repository or in an easily accessible place.
- Copy operations from the repository, assembly and testing of the entire project are automated and easily called from an external program.
Database integration
An important issue that occupies a separate place when considering configuration management tasks is database integration. In most cases, databases are an integral part of applications. Development using agile methodologies involves not only continuous improvement of the program code, but also the structure of the database, as well as its functionality. And usually this happens in parallel - along with a change in the program code, changes are made to field types, field names, functions, triggers, database indexes. The evolution of the database during the life cycle of the project resembles the evolution of program code, having almost the same features when considering the database as an object of configuration management. Although the configuration management of databases and version control of databases have significant differences compared to the configuration management of program code, this is an integral part of the project management and is reflected in the database integration process. An SQL application can often be seen as an application in an application and allocated to a separate project to be versioned. There are two types of identification elements used in versioning databases: DML (Database Manipulation Language) and DDL (Database Definition Language). DML is a subset of SQL used to manipulate data: fetch, insert, delete, updates (CRUD operations). DDL is also a subset of SQL, but used to describe the structure of the database: creating tables, indexes, triggers, integrity constraints, etc.
Issue tracking
The tendency to use systems (task setting control systems) indicates more and more frequent attempts by such systems to integrate software project management tools. Often this happens not directly, but indirectly - through the release of various plugins (integration with version control systems, displaying javaDoc, phpDoc or Doxygen documentation, etc.). But even in the basic set of functionality related to CRM (change request management), there are elements that need to be improved. Such an element could be, for example, version naming. But the need to use certain conventions not only in the issue-tracking system, but also in version control, as well as other processes of the Criminal Code, prevents the use of standardized version names.
Agile
The approach to software development, which involves the use of a flexible methodology, has a number of distinctive properties that most significantly affect the organization of configuration management. These properties are:
- changing requirements
- iterativeness
- continuous delivery.
Conclusion
This article discusses the processes that are part of configuration management without going deeper into details. To ensure each process, a separate tool is used, which can be selected from many alternatives. It was noted that due to the heterogeneity of the tools themselves, as well as the difference in the tasks they solve, it is difficult to achieve their maximum integration and effectiveness. The next article will be about this - how, through the introduction of additional formalization (organization of the source code repository), to achieve a more efficient use of the tools used in the Criminal Code.
To be continued
Links:
- Issue tracking system (wiki)
- Continuous integration (wiki)
- Version Control for Multiple Agile Teams
- Continuous deployment in 5 easy steps
- SVN repository metrics build tool
- Code coverage analysis
- Code metrics and their practical implementation in IBM Rational ClearCase
- SLOCCount - a tool for counting the number of lines of code
- Continuous Builds with CruiseControl, Ant and PHPUnit - an example of organizing a simple configuration management platform
- My PHPCONF 2009 Report