Second life with Maven

I think that for most habrazhitel it will not be a revelation that among large software developments, technologies and languages ​​are often found far behind the advanced ones. Unfortunately, this is inevitable, because it is impossible at some point to take and rewrite several million lines of code into another language or using a more modern framework.

But at some point the lag becomes so significant that the product ceases to be competitive and gradually disappears from the market. It was such a situation of gradual attenuation that I happened to observe.

Here I want to talk about how in a particular case I tried to delay the inevitable and revitalize the development using the Maven project collector for this.

Perhaps, the main advantage of the system I developed can be called the fact that with a small amount of labor, you can use Maven to teach a project implemented in any language, to assemble from separate Packages that respect their version and are automatically updated when the project is built. All that is needed is a command line utility that can compile the source, and in the case of scripting languages, a validation utility would be useful.

So given:


- A system, the "core" of which consists of a set of dlls and is given to application developers without sources.
- An applied programming language that is similar to Pascal, but apart from the standard features of Pascal (of which not all are available) allows you to use objects implemented in the dll "core". This application language is compiled into a kind of code byte and executed by a separate “core” dll.

Thus, the main development is carried out in the magic applied language described above , all the charms of which, perhaps, could horrify the respected community, and the “core” is periodically (new version once every couple of months) delivered from the head office and cannot be modified.

It is very likely that developing and maintaining your own programming language similar to Pascal and made on the basis of Delphi was probably a bad idea, but no one thought about this 15-17 years ago when it all started. Other goals were pursued, which, in general, were successfully achieved.

The system itself is a product sold to a customer with a server, client, and thin client. Naturally, most customers are not satisfied with the standard functionality of the system, so there are several departments of applied development that “finish” the system to their needs. Each project is engaged in a separate group of programmers. The interaction and exchange of experience between groups is disastrously small. Thus, each group gets its chance to reinvent the wheel and step on the same rake for the tenth time.

After analyzing, it became clear that such a lack of interaction between departments is due primarily to the lack of mechanisms for using common code. In other words, if, nevertheless, there was an exchange of experience and source codes between groups, then the library (source) was simply copied to another project (product for another client) and began to live an independent life, preserving all the glitches and shortcomings that were present at the time copying.

To remedy the situation, it was decided to organize a repository (repository) and a project collection system using a set of shared code libraries ( Packages ).

Maven was chosen as a tool for this, because by default it performs similar functions, has a flexible life cycle and great flexibility of the plug-in system.

But first things first


The main problem, which I did not suspect at the beginning of my research, turned out to be that the Packages should contain files that were not compiled into byte code, namely the source, as compilation should take place exactly under that version of the "kernel" for which they will be used. For simplicity, we call our application source bpas. This is slightly contrary to the standard LC in Maven.

After such revelations, it was decided to form a package from the sources, but during formation it is necessary to check their atomic compilation. In other words, use the almost standard Maven LC when .class is first compiled, and then, together with resources, it is packed into a jar. Only in my case, instead of compiled files, the package was formed from the source bpas packed in zip.

A little bit about how Maven works


The Maven life cycle is quite complete, and the set of phases takes into account almost all the stages of the project assembly that may be required.

And while trying to start some phase of the LC, for example, “mvn compile”, I actually run the entire phase chain from validate (project validation) to compile, without missing a single one. For some phases, there are so-called default plugins that will be called despite the fact that in pom.xml (the main file of the Maven project) there is only a header and not a single indication of the launch of plugins.

It is worth mentioning separately that Maven is a fully plug-in system. In other words, he can do almost nothing but launch plugins, but they already know how to do amazingly much. It turns out that when we want to teach Maven some peculiarities of the project assembly, we must add in pom.xml an indication to launch the right plug-in in the right phase and with the right parameters.

Here is such an absolutely valid empty pom.xml, despite its emptiness, upon receipt of the mvn deploy command, it will launch the Initialization, Compilation, Packaging and Deploy Java Plug-ins from the src / main folder.

4.0.0com.mycompany.projectgroupprojectname01.03.03-00


The main usage policy in Maven is that for any action there are default parameters and additional settings are required only when these defaults are not enough or they are grossly violated. In my case, I had to abandon so many defaults, so pom.xml no longer looks so modest.

Building a new lifecycle in pom.xml


To implement the packages, the following life cycle was selected.

initialize (initialization) - Read the settings from the config (property or key = value) file and add them to the properties tag. We’ll talk about the properties tag a bit later.
generate-sources (source code generation) - Download and unpack from zip all Packages that are dependencies of this package / project into a separate directory for subsequent compilation with the source code of the current package / project.
compile - Run our compilation plugin for our bpas, which determines the correct compilation order and starts the command line compiler for our language. I’ll briefly talk about my own plugin below, but I suggest moving the guide for writing it outside the scope of this article.
assembly - we pack the package consisting of bpas sources in zip while preserving the structure of the subdirectories of the source files.
deploy (in our case, unloading to the repository) - The assembly zip collected in the assembly phase is sent to the local Maven repository with pom.xml and other package information added to it. This procedure is almost identical to the normal deploy jar file, but with special parameters.

clean - This phase is not included in the general LC of the assembly phases, but is somewhat different, but since it has also been upgraded, it is also worth mentioning. In addition to the standard removal of the directory in which the compiled files are located. (targetDirectory), it was required to remove the garbage that is generated during the downloading and unpacking of dependency packages.

General structure of pom.xml


I conditionally divide pom.xml into two parts: the header and the assembly.

The header is the package identification (groupId, artifactId, version), properties (properties that act as internal constants), the local repository (distributionManagement), the local plugin repository (pluginRepositories), the local package repository (repositories) and the dependencies this package (dependencies). At the same time, all three repositories can point to the same repository, but fundamentally these are three different entities, each of which must be specified separately. So, for example, we can decide to store plugins separately from the rest of the code, and use http access to access packages in the repository, while we will “deploy” there as in a file repository.

The assembly (build tag) is the second part of pom.xml, in which the processing features of a particular phase of the life cycle by various plugins with non-default settings are configured. In addition, directories and parameters that will be involved in the assembly of the project are
configured there : sourceDirectory - the directory in which the sources for compilation are located.
finalName - the final file name after packing into the archive.
directory - the working directory in which the compiled files will be placed.
In addition, I want to remind once again that for all these parameters, there are default values, and their separate indication in our case is required only because they must differ from these defaults.

Lifecycle implementation in build tag


Now, let's return to the LC we have defined and see how each phase of the life cycle is implemented by calling the desired plug-in with the configuration that we need.

initialize

Here again, let's digress a little and separately mention the properties tag. We explain why it is needed and how it is used.

Speaking very roughly, this tag is similar to declaring constants that will be used in our program (pom.xml). But values ​​can get there in three different ways. And each method has a priority that determines what the result will be at the moment when it is really needed.

The lowest priority is to write properties directly to the properties tag, for example like this:
Hello my friend
  


Direct priority is given a higher priority when Maven starts, something like “mvn –DhelloText = Hi initialize”
When Maven starts with this parameter, the initial value of the helloText tag will be replaced by the value passed to the start line for the current session, ie in xml it will not be saved. If such a constant did not exist at all, then for this session it will exist and can be used. The values ​​of all non-existent constants are an empty string.

The highest priority is given to adding values ​​to the proprties tag by plugins in the current session. They are also not saved in pom.xml. It is this mechanism that we will use to make individual assembly settings to a properties file containing “name = value”.
For this, the properties-maven-plugin is used
org.codehaus.mojoproperties-maven-plugin1.0-alpha-11initializeread-project-properties..\build.conf2pre-cleanread-project-properties..\build.conf


In addition to an interesting solution for putting context-sensitive settings into a separate file, there is a good demonstration of how the same plug-in runs in completely different phases of the life cycle. This is ensured by the tag., where for each desired phase a separate tag is created . TagIt must include a unique id (if execution is more than one), and it may also contain a separate configuration tag whose priority is higher than all others.

generate-sources

At the source generation stage, we recursively download from the repository and unpack all the packages we need, which are indicated in the dependencies. Again, practically nothing needs to be done by ourselves. The plugin will do everything for us after indicating the correct settings for it.
org.apache.maven.pluginsmaven-assembly-plugin2.2.1unpackSourcesgenerate-sourcesunpack${packagesPath}

The construction of $ {packagesPath} means that you need to take the value from the tag “/ project / properties / packagesPath”.

I would also like to note that the use of the maven-assembly-plugin plug -in for unpacking is considered deprecated and is not recommended for use in Maven 3. Instead, it uses the maven-dependency-plugin with settings similar to those indicated above. I’m using an older version of the plugin to once again demonstrate how the same plugin is configured to perform several tasks from its range.

compile

I had to tinker a lot with the compilation stage, but the main difficulties arose with writing my own compilation plugin. Step-by-step instructions for writing your own plugin for Maven is a topic for a separate article, so now we will not focus on this. In the end, the material presented here can be used for scripting languages, compilation of which is not required at all.
One thing is certain, no matter how hard you try, you won’t be able to disable the launch of the native maven-compile-plugin , the calling of which is to compile Java sources (Without considering the editing possibilities of superPom.xml). So the settings of my compilation plugin are as follows:
com.companyname.utilsBPASCompilerPlugin0.1.0.2compilebpascompile${protectionServerName}${protectionServerAlias}${pathToBPASCC}${env.BINARY_VERSION}

parameters used:
protectionServer - protection server, without which it is impossible to start the command line compiler.
protectionAlias - section of the used license of the protection server.
bpasccPath - The full or relative path to the command line compiler.
binaryVersion - The version that will be "mounted" into the compiled library.

This is not all the settings of my plugin, but as I said, this is a topic for a separate large article. In principle, the configuration section could be omitted altogether, and then all the parameters that the plug-in needed would be initialized by the plug-in with default values, which corresponds to the basic concept of Maven.

assembly

When passing the assembly phase, Maven is configured to run maven-assembly-plugin by default, explicitly indicating its launch in the assembly phase in the build tag, we can override its settings and make it work for us. We used the same plugin for unpacking packages before compilation, so now I will give a full version of the settings for this plugin, including both packing and unpacking.

org.apache.maven.pluginsmaven-assembly-plugin2.2.1unpackSourcesgenerate-sourcesunpackpackSourcesassemblyassembly..\src.xml${packagesPath}


Here we see the second execution section with id = packSources and the necessary settings for this phase .. \ src.xml. Src.xml contains settings for how to package the sources. In fact, all of these settings can be placed directly in the descriptor tag, but this can be very cluttered with pom.xml.
Here is an example of src.xml to complete the picture.
srczippom.xmlSOURCE.svn**/*${packagesDirName}*/**

packagesDirName is a constant from / project / properties of the pom.xml file.
I also want to note that putting the packing settings into a separate file allowed me to create one packing config for all Packages, which is extremely convenient.

deploy

The deploy phase is also launched by Maven, regardless of whether we specified the settings for this plugin or not. Overriding them, we made this plugin work for ourselves.
org.apache.maven.pluginsmaven-deploy-plugin2.5${project.build.directory}\${project.artifactId}-src.zip${project.distributionManagement.repository.url}${project.distributionManagement.repository.id}${project.groupId}${project.artifactId}${project.version}zippom.xml


With such manual settings, maven-deploy-plugin allows any file (or even a group of files) to be uploaded to the Maven repository as a valid library (Package). Now let's analyze the settings in order.
file - the file that will be sent to the Maven repository as a Package.
url - path to the Maven
repository repositoryId - identifier of the repository to which the deploy will be performed.
groupId , artifactId , version- standard package identification in the Maven repository. It is with these three parameters that a library can be uniquely identified. packaging - the deployment functionality also includes packing the files that will be sent to the repository, so you must again tell it zip so that it does not do anything with the package, otherwise it will unpack and pack like jar :-).
pomFile - if this parameter is specified, then the file that we specify as pom.xml will be added to the package; the original name may be different. Maintaining the original pom.xml is beneficial for many reasons, so we will not disdain this feature.

clean

The clean phase, as I said, is not part of the standard LC. Its initial task is that after the mvn clean command is executed, no traces of vital activity remain in the project. In our case, in addition to the standard targetSource folder (indicated in the build tag), you also need to delete all Packages that were “merged” as dependencies for successful compilation of the package / project.

So, the settings:

org.apache.maven.pluginsmaven-clean-plugin2.4.1${packagesPath}


directory - the actual directory to be deleted. It should be noted that here the plug-in creators departed from the generally accepted concept, and explicitly specifying the plug-in settings does not cancel its default actions. But in this case, this is very good, because it saves us from unnecessary settings.

Bearing in mind how difficult it is to deal with individual settings branches at first, I will give the full text pom.xml of one of the packages below.

4.0.0com.companyname.packagesString0.1.0.1Stringhttp://maven.apache.orgwindows-1251${basedir}/SOURCE/${packagesDirName}spb-maven-repospb-maven-repofile://\\SPB-FS\maven-repospb-maven-repospb-maven-repotruealwaysfailtruealwaysfailhttp://spb-maven-repodefaultspb-maven-repohttp://spb-maven-repocom.companyname.packagesArithm0.1.0.1zipcom.companyname.packagesbUnit0.9.0.0zipUSER${basedir}/SOURCE${project.artifactId}org.codehaus.mojoproperties-maven-plugin1.0-alpha-11initializeread-project-properties..\build.conf2pre-cleanread-project-properties..\build.conforg.apache.maven.pluginsmaven-assembly-plugin2.2.1unpackSourcesgenerate-sourcesunpackpackSourcesassemblyassembly..\src.xml${packagesPath}com.companyname.utilsCompilerPlugin0.1.0.2compilecompile${protectionServerName}${protectionServerAlias}${pathToBPASCC}${env.BINARY_VERSIONorg.apache.maven.pluginsmaven-deploy-plugin2.5${project.build.directory}\${project.artifactId}-src.zip${project.distributionManagement.repository.url}${project.distributionManagement.repository.id}${project.groupId}${project.artifactId}${project.version}zippom.xmlorg.apache.maven.pluginsmaven-clean-plugin2.4.1${packagesPath}


Summary


I tried to explain everything as detailed as possible, but all the same, behind the scenes there were still a huge number of questions that I plan to cover in subsequent articles.
  • phase and goal are their relationship between themselves and the plugins.
  • creating archetypes - a project template for quickly deploying your pom.xml configuration and a set of default files.
  • Maven repositories and their management.
  • how to write and debug your Maven plugin.
  • how NOT to write your own plugin for every "sneeze".


Despite the fact that I myself was damn pleased with my work, I am ready to admit that in this form the system of batch assembly of the project is still far from perfect, however, the description given illustrates the implementation of the full life cycle of the batch / project. In fact, this is a workable framework, on top of which you can almost unlimitedly expand the capabilities.

The main advantages of such a batch assembly are:
  • ease of adding generic code
  • ease of adding dependencies and building a project
  • transparency of the project (it can be seen what is used and what is the real volume of non-standard improvements)
  • identification of cyclic dependencies between Packages, thanks to Maven mechanisms.
  • automatic distribution of corrections in the general code.
  • ease of use by an ordinary developer / tester (enough to know 3-4 teams).

useful links


Versioning of the dependencies used
The most basic set of plugins for Maven and the documentation for them
An additional set of plugins included in the main Maven repository

Also popular now: