About how to stop being afraid and fall in love with frequent releases

I must say that, in principle, developers tend to not pay attention to such trifles as tests and deploy. And the situation which in the picture below is not even a joke, but the situation which witnesses, I think there were many.



I'll tell you a little about development at my current place of work.

We use a flexible development methodology ( the Agile ), and at first we practiced the Scrum , and now we are working on the Kanban . Initially, I myself was rather skeptical about such an event, but just a couple of weeks later I was convinced that it works.

A description of such development processes is the topic of a large post, maybe not just one, in addition, there is a large amount of literature on the network, so I will not tell you what it is here, I will limit myself to the fact that these methodologies involve frequent releases, in the case of scrum release occurs after the iteration is completed (the iteration time is chosen by the team itself and usually takes from 1 to 3 weeks), in the case of kanban, the feature is launched into the production immediately after it has been done (well, it’s tested, of course).

Frequent releases are not just good, they are excellent: the streamlined and frequent release process saves the developer from fear of updates, and the customer sees progress.

This development process overlaps with Continuous Integration. About him, too, books and long articles are written, in short, - this is the implementation of frequent automated assemblies of the project for the quickest identification and solution of integration problems ( wikipedia ).

We are developing a distributed system, we have perl as a backend (API), client code is written in javascript, and jenkins ci as a continuous integration server .

For those who are not familiar with it, I will explain on fingers what jenkins is: it is a service with a web interface where tasks for assembly are configured. The task is a step-by-step instruction on what the server must execute to build a package, in our case it is rpm. For example, this usually happens like this:

• make a checkout from VCS (version control system);
• run tests;
• compile and assemble the necessary file structure;
• pack into rpm file;
• put in the repository.

Creating an rpm package requires writing a spec file , which is also a set of instructions for building and installing the package on the destination machine.

So: there is a CI server on which the rpm package is built, the package then gets into the yum repository, and there is a destination server or a cluster of servers on which this package is installed.

In our project, we use modules with CPAN - reusing code is good, but entails some overhead, for which many people don't like perl.

Dependencies

Consider an example: we needed to use a certain hash algorithm, the task is rather trivial, we take a module on CPAN and put it in our development environment:

    # cpanm Some::CPAN::Module

after which we write the code:

   use Some::CPAN::Module;
   sub encrypt {
       my $string = shift;
       return Some::CPAN::Module::mkHash($string);
   }

and cover it with dough:

    ok(encrypt("test") eq Some::CPAN::Module::mkHash("test"), "encrypt function");

the code works, is covered by the test, we are very happy with ourselves. Commit him and try to collect the package. The assembly is configured in our minds: at the stage of assembling the package, unit tests are launched, which is very convenient: if the tests fail, it means bad code, it makes no sense to collect further.

And here's the trouble: the test fails - there is no module required on the build server.

Or another situation: we don’t have any tests, and this code leaves the package in the repository and we get a non-starting service, but already on the production server, where it is much more difficult to catch the error.

To build and pass the tests, add the dependency to the spec-file:

  BuildRequires: perl-Some-CPAN-Module

To deploy to the destination server, add:

  Requires: perl-Some-CPAN-Module

If you are lucky, then the required rpm is already in the yum repository, if not, you need to collect the package and put it in the yum repository.

So, the package in the repository is put on the build / production server, but there is one problem, it has a different version: during the time that we wrote the code and tests, the module author fixed a critical vulnerability in his module and released a new version, which suddenly for some reason became incompatible with the previous one, the test still fails, the assembly fails, the service does not start.

Solution: we find out the version of the module with which our code works, collect rpm with this version and specify the exact version of the module in the spe-file.

   BuildRequires: perl-Some-CPAN-Module = 1.41
   Requires: perl-Some-CPAN-Module = 1.41

Another problem can occur if a module with CPAN has its own dependencies, for example, Some :: CPAN :: Module depends on Other :: CPAN :: Module, as a result, on the development environment and on the build server after assembling-installing version packages again parted and the test fails, the service still does not start. And there can be more than a dozen such modules in a real project. In narrow circles, this is called the jellyfish of addictions.

Solution: you can also track the dependencies of the dependencies and specify all of them in the spec-file, but this solution requires a lot of effort and time for tracking and subsequent assembly. Which is limited in our development methodology.

An alternative solution is possible: raising your own CPAN mirror, for example, using pinto , but this is not the topic of this article.

There is another solution.

The main problem of CPAN is that it was invented for a long time and everyone who at least once laid out their module on CPAN understands what I'm writing about. Fortunately, recent tools for perl have begun to appear.

Carton

I want to talk about a tool called Carton . Its author, Tatsuhiko Miyagawa, is also the author of a bunch of useful utilities, including starman and cpanm.

What is Carton? This is a dependency manager for pearl modules. It was written under the influence of the Bundler for Ruby.

Its essence boils down to the fact that a file is created in the project directory in which the dependencies are declared - cpanfile . In this file, the developer lists the required modules and their versions:

    requires 'Some::CPAN::Module', '1.45'; 

then the command is run:

    carton install 

and the necessary module is put in the local / lib directory and cpanfile.snapshot is also created, in which information about the modules installed with CPAN is stored. This file must also be added to VCS, so that later on the build server when starting carton install dependencies from snapshot arrive, thereby guaranteeing the identity of the environment.

All you need to do now is add to @INCthe local / lib directory.

You can specify the exact version that is needed or a valid range of versions:

    requires 'Some::CPAN::Module', '>= 1.41, < 1.46';

Moreover, you can specify modules specific to different environments, for example:

  osname 'MSWin32' => sub {
      requires 'Win32::File';
  };
  on 'test' => sub {
      requires 'Test::More', '>= 0.96, < 2.0';
      recommends 'Test::TCP', '1.12';
  };
  on 'develop' => sub {
      recommends 'Devel::NYTProf';
  };

Carton works in harness with cpanm, another useful utility of the same author, and we can be grateful to her for the direct installation of modules.

Now that we have all the dependencies in cpanfile, a snapshot of the development environment in cpanfile.snapshot and all this is placed in the VCS in the spec file, you can add the carton install command before running the tests.

The same command can be added to the installation stage when the package is deployed to the server for which the rpm package is intended.

But we went further.

Carton’s author has a vision of his utility working as follows: Carton is installed on all necessary servers, then a capistrano- type utility runs a command immediately on all necessary remote machines:

  # cap carton install

It’s a little strange that you need to download a whole bunch of dependencies from the Internet or from a local CPAN mirror, and this can take quite a while, but what if at this point there were any network problems?

There is a more consistent way, we can install the modules at the rpm package build stage. And we decided to create a über-rpm package in which we placed all the dependencies at once. This package is declared as a package dependency with the code and it can be installed both on the build server for passing tests during assembly and on the production.

This approach is popular for projects, for example, on scala - a über-jar is created in which all the project dependencies + code, go go even further - they create a über-binar in which all the dependencies are already compiled.

Now that we have set up and automated the build process, we can not be afraid of the releases and jellyfish of dependencies, but focus on the algorithms and the product that we produce :)