Configuration Management (Part 2, Overview of Tools)

    It took a long time before I was ready to write the second part of the article on configuration management. The fact that not so long ago I was lucky enough to speak at the PHPCONF 2009 conference on October 8 ( Web Architect Workshop Day ) with the master class “Method of organizing the source code repository” contributed to the fact that this happened . For the presentation, a presentation was prepared in advance, as well as the text of the report. Despite the excellent organization of the event, the materials of the reports included in the conference program were not posted for public access. In compensation, I decided to publish the material used in my presentation. In addition to this article, (which is a logical continuation of the previous) dedicated to configuration management, presentation slides are available for public viewing .
    This article will discuss the tools used in configuration management. Therefore, first of all, I would like to focus on how the tools used in development can affect the process of creating software.
    • Firstly, each tool is used to solve a number of specific tasks.
    • Secondly, the solution of the problem involves the preceding use of the tool to satisfy a set of requirements, without which the tool will not be effective or will not work
    • Third, often the set of requirements put forward by the many tools used is not fully satisfied. In addition, the requirements of different instruments may conflict with each other. All this leads to the fact that the effectiveness of the use of such aids is reduced.
    Here, any tools used by a programmer during development are used as tools: be it an IDE, a framework, a version control system, or a separate technology. Each developer explicitly or implicitly faces the problem of managing the tools they use. Most often, the solution to this kind of problem either occurs on an intuitive level, or a completely conscious choice of the primary tool is used, on which most of the development process is built. Few people manage to combine most of the tools used in a single platform, while this could serve to build an established development process.

    The approaches used in configuration management may well contribute to the construction of such an established process, since this is the main discipline in determining how the working materials of a software project are managed and controlled, changes made to it, as well as information about the status of individual tasks and the entire project as a whole . As mentioned earlier (in the previous article ), they fall into the scope of interests of configuration management (each item will be considered separately below):
    1. Version control
    2. Build Management
    3. Unit testing
    4. Static code analysis
    5. Documentation Generation (phpDoc)
    6. Continuous integration
    7. Dependency management
    8. Database integration
    9. Bug tracking (bug-tracking) and task setting control (issue-tracking)
    Also, from the previously unmentioned, it can be noted what configuration management itself does:
    • Release management
    • Adjustment of deliveries (delivery)
    • Organization of established development processes
    • Coordination of the method of interaction between different parts of a software project
    It should be noted that we must try to clearly distinguish between the processes of QM (version control, build-management, etc.) and tasks of QM (release management, delivery adjustment, etc.). To identify the essential features of the processes of the UK, you need to consider each such process (as well as its relationship with configuration management) separately.

    Version control


    Version control is the main configuration management process. The absence of the need to use version control during development is practically equal to zero. Most developers are familiar with version control systems (SLE) such as CVS, Subversion, or VSS. Distributed version control systems are also gaining popularity: git, mercurial, bazaar. The mentioned version control systems have a great influence on the formation of concepts about configuration management among developers. For this reason, usually developers do not think about the thoughtful organization of the repository. The simplest and only solution offers SLE subversion. This solution is used most often: division into trunk, branches and tags. Although you can see this directory hierarchy in most subversion repositories, you cannot say that they are always equally actively used: in the best case, in the branches directory you can see several directories of parallel development branches. The apprehension of developers when creating a branched directory structure of the repository can be understood - this often bodes a lot of problems associated with merging changes between branches. But with team development of applications that have several versions or work on different platforms, it is almost impossible to avoid mergers. But you can minimize their number by regulating situations in which it is possible to carry out mergers, and in which - no. But to determine such a regulation, a thorough understanding of the phenomena associated with maintaining the repository is needed. The apprehension of developers when creating a branched directory structure of the repository can be understood - this often bodes a lot of problems associated with merging changes between branches. But with team development of applications that have several versions or work on different platforms, it is almost impossible to avoid mergers. But you can minimize their number by regulating situations in which it is possible to carry out mergers, and in which - no. But to determine such a regulation, a thorough understanding of the phenomena associated with maintaining the repository is needed. The apprehension of developers when creating a branched directory structure of the repository can be understood - this often bodes a lot of problems associated with merging changes between branches. But with team development of applications that have several versions or work on different platforms, it is almost impossible to avoid mergers. But you can minimize their number by regulating situations in which it is possible to carry out mergers, and in which - no. But to determine such a regulation, a thorough understanding of the phenomena associated with maintaining the repository is needed. But you can minimize their number by regulating situations in which it is possible to carry out mergers, and in which - no. But to determine such a regulation, a thorough understanding of the phenomena associated with maintaining the repository is needed. But you can minimize their number by regulating situations in which it is possible to carry out mergers, and in which - no. But to determine such a regulation, a thorough understanding of the phenomena associated with maintaining the repository is needed.

    Assembly management


    Another important configuration management process is assembly management. Assembly management is the automation of actions regarding:
    • Source code compilation
    • Application Deployment
    • Run unit tests
    • Database initialization
    It should be noted that assembly management tools are characterized by the fact that assembly files usually use XML syntax (Ant, Nant, Maven, MSBuild, Phing), although there are exceptions (make, nmake, cmake, rake). Generally, assembly management tools have commands specific to the assembly process, such as:
    • Group file operations
    • Compilation
    • Deployment
    • Interaction with version control systems
    These and other properties of assembly management tools allow for easy integration with other development tools; assembly management tools act as a link for most processes to be automated in configuration management.

    Unit testing


    Although unit testing cannot be fully attributed to the processes of configuration management, it has quite important properties that determine the special role of unit testing in configuration management. Thus, the use of unit testing in software projects written in interpreted programming languages ​​determines, in addition to the need to manage assemblies, the need for a more thorough approach to configuration management. First, unit testing appears where refactoring is intended. Secondly, when deciding on the use of unit tests, the architectural and logical integrity of the project, which must be maintained at a sufficient level, is usually taken into account. It means,

    Static code analysis


    In addition to supporting architectural and logical integrity in large projects, there is a need to identify syntax errors in the early stages of development. In addition to identifying syntactic and logical errors in the source code, it is often necessary to automatically verify compliance with code conventions. Static code analysis is more relevant for interpreted languages, since in compiled languages ​​most errors are detected at the compilation stage. Sometimes, static analysis can be performed to collect source code metrics and report accordingly. An example of the libraries used for static analysis of the source code are the libraries presented in the table:
    Programming languageTool
    JavaPMD
    FindBugs
    C / c ++Cppcheck
    lint
    C #FxCop
    StyleCop
    ReSharper
    PhpPHP_CodeSniffer
    PHP-sat
    PHP_Depend
    Pixy
    PythonPyChecker
    PyLint
    PyFlakes
    RubyReek
    Roodi
    Rufus
    Flay
    Flog
    Programming language independent toolsRATS
    Yasca


    Documentation Generation


    The generation of documentation based on source code and comments in the style of javaDoc, phpDoc, etc. is most often relevant for projects with active use of source code: libraries, reusable components, frameworks, etc. The tools most often used in generating documentation are presented in the table:
    Oriented to a specific programming languagePhpphpDocumentor
    JavaJavoc
    C ++Cppdoc
    PythonpyDoc
    RubyRDoc
    DeplhiDelphiCodeToDoc
    C #Ndoc
    Not oriented to use a specific programming languageDoxygen
    ROBODoc
    Twintext

    Continuous integration


    Continuous Integration (CI) is a software development practice that involves performing frequent automated project assemblies for the earliest possible identification and resolution of integration problems. In a regular project (not using the practice of continuous integration), where developers work independently on each part of the application, the integration stage is final. As rightly noted in a concise Wikipedia article describing CI, continuous integration cannot be applied to any project. To implement continuous integration practice, a project must satisfy a number of requirements:
    1. The source code and everything necessary for the assembly is stored either in the source code repository or in an easily accessible place.
    2. Copy operations from the repository, assembly and testing of the entire project are automated and easily called from an external program.
    Typically, continuous integration tools are configured to trigger a build when a repository update event occurs. With the deployment phase included in the build process, continuous integration can be tailored to provide the testing team with a working and test copy of the application. During development, such continuous integration tools can be used as: CruiseControl, phpUnderControl, Xinc, CruiseControl.rb, TeamCity, Apache Continuum, Hudson, Parabuild, Atlassian Bamboo, etc.

    Database integration


    An important issue that occupies a separate place when considering configuration management tasks is database integration. In most cases, databases are an integral part of applications. Development using agile methodologies involves not only continuous improvement of the program code, but also the structure of the database, as well as its functionality. And usually this happens in parallel - along with a change in the program code, changes are made to field types, field names, functions, triggers, database indexes. The evolution of the database during the life cycle of the project resembles the evolution of program code, having almost the same features when considering the database as an object of configuration management. Although the configuration management of databases and version control of databases have significant differences compared to the configuration management of program code, this is an integral part of the project management and is reflected in the database integration process. An SQL application can often be seen as an application in an application and allocated to a separate project to be versioned. There are two types of identification elements used in versioning databases: DML (Database Manipulation Language) and DDL (Database Definition Language). DML is a subset of SQL used to manipulate data: fetch, insert, delete, updates (CRUD operations). DDL is also a subset of SQL, but used to describe the structure of the database: creating tables, indexes, triggers, integrity constraints, etc.

    Issue tracking


    The tendency to use systems (task setting control systems) indicates more and more frequent attempts by such systems to integrate software project management tools. Often this happens not directly, but indirectly - through the release of various plugins (integration with version control systems, displaying javaDoc, phpDoc or Doxygen documentation, etc.). But even in the basic set of functionality related to CRM (change request management), there are elements that need to be improved. Such an element could be, for example, version naming. But the need to use certain conventions not only in the issue-tracking system, but also in version control, as well as other processes of the Criminal Code, prevents the use of standardized version names.

    Agile


    The approach to software development, which involves the use of a flexible methodology, has a number of distinctive properties that most significantly affect the organization of configuration management. These properties are:
    • changing requirements
    • iterativeness
    • continuous delivery.
    It is assumed that when considering configuration management processes, these properties should be reflected in the individual components of a particular process or the results obtained at one of the stages.

    Conclusion


    This article discusses the processes that are part of configuration management without going deeper into details. To ensure each process, a separate tool is used, which can be selected from many alternatives. It was noted that due to the heterogeneity of the tools themselves, as well as the difference in the tasks they solve, it is difficult to achieve their maximum integration and effectiveness. The next article will be about this - how, through the introduction of additional formalization (organization of the source code repository), to achieve a more efficient use of the tools used in the Criminal Code.

    To be continued

    Links:
    1. Issue tracking system (wiki)
    2. Continuous integration (wiki)
    3. Version Control for Multiple Agile Teams
    4. Continuous deployment in 5 easy steps
    5. SVN repository metrics build tool
    6. Code coverage analysis
    7. Code metrics and their practical implementation in IBM Rational ClearCase
    8. SLOCCount - a tool for counting the number of lines of code
    9. Continuous Builds with CruiseControl, Ant and PHPUnit - an example of organizing a simple configuration management platform
    10. My PHPCONF 2009 Report


    Also popular now: