February 12, 2017 at 22:16Versioning build artifacts in Gradle using git tag, brunch, and commit names
With the move from SVN to GIT and gitlab (plus the move from Jenkins to Gitlab-CI, but we will also use it), the question arose of versioning the resulting artifacts of the application assembly.
In SVN there was a familiar revision number for everyone, monotonously increasing with each commit. It was convenient to add to the version number, and this solved most of the problems. But git, of course, provides a lot of goodies, and it was worth convincing the management and the whole team to transfer the project to it ...
But I had to rebuild the process of versioning the resulting assembly artifacts.
As a result, we stopped at a very good Gradle plugin github.com/nemerosa/versioning , and I'm going to tell about its use.
We have been using Gradle for a long time in our application, and for SVN we just used the inherited function inherited from the family, written directly in the build.gradle file. Fortunately, among the other advantages of Gradle, you can mention that it is a wonderful Groovy language, and it does not limit you in writing the logic of the build - you have connected the necessary libraries from the Java world, and go ahead, at least rewrite the entire application in one file!
However, do you understand the harmfulness of this approach? If the logic of getting the version number takes more than 5-10 lines, and also if we crush our crutch for any reason, then it will just be impossible to support it very soon ...
You can see similar "solutions" to manual parsing, for example, in the Jenkins article for Android on a clean system and without UI orThrough the thorns to the assembly where it is proposed to manually call git describe and parse the output with regular expressions ...
I wanted something simpler, more reliable and initially working.
In our application, a couple of jar files, 3 war artifacts, 3 RPMs including them are collected, and at the end of Docker, an image of the application with RPMs installed, which after automatic testing is immediately sent to gitlab-ci to a private repository.
In general, with the transition to git / gitlab, we adhere to the logic inherited from the standard github flow with minor changes, which means for versioning:
Having looked around at the available plugins for gradle, I found such a wonderful option: github.com/nemerosa/versioning
Its documentation immediately captivates - everything is simple, logical and understandable for what it was done.
Plus, the semantic separation of release, feature
So connecting to the project is very simple, follow the instructions:
That's it, in most cases it is already possible to use the version in your build scripts further, where it is needed:
Well, or closer to the point, say in the name of the war artifact:
After assembling from the feature-1 brunch, we get a file with approximately the following naming: portal-api ## feature-1.3e46dc.war (the example uses Tomcat-style naming ). The options for setting and parsing values for more interesting situations will be discussed later.
2 tasks are immediately available:
versionDisplay - showing information and versions and displays on the console. It is very convenient in debugging and versionFile - creating a build / version.properties file with ready-made variables for import into bash scripts outside:
just fine.
I would like to note right away that there are many options for parsing names, handling prefixes, suffixes, interpreting versions. There is also support for SVN by the way. In general, you are in the customization section .
However, there is not a fly in the ointment . At the time I started using it, the documentation looked different .
Yes, you can set your own closure on how to interpret the brunch name (for example, consider 'release / 1' as release, and 'qa / 0.1' otherwise):
This is all great, but we want a tag instead of brunch, if any !?
I did not want to give up this idea. Of course, I cut a temporary workaround, but the author created a request to make the parsing logic more general: github.com/nemerosa/versioning/issues/32
Damien Coraboeuf , who is the author of this plugin, turned out to be very responsive, and quickly corrected a couple of small things quickly.
In general, as often happens, he proposed to realize for me what I am offering.
I followed his advice - quickly banged pull request .
Now, after its adoption, we get the SCM commit information object (SVN or GIT) and are free to choose the way how to build the version for us. For example, the same code as above can be implemented like this:
Same thing in a full closure.
What does this give us? Well, for example, as described in the requirements, we use this in order to take the brunch name in one case and the tag name in the other, and are not limited only to the string representation of the brunch name. For us, it now looks something like this:
Lowercase is used for use in Docker image tags.
As I mentioned, the options are not limited to this, we also control the dirty suffix, and also add the build time to the same object using Groovy meta magic ...
Well, since I started talking about convenient integration, I should immediately pay attention to one pitfall, which I also stumbled upon. And this plugin has already been taken care of!
The specified code worked fine, was tested, and commited. But the first push and build on CI brought a strange result - the name of the brunch was something like HEAD.
In fact, the reason is simple, if we look at what the builder does, he collects not a branch, but a specific commit. At the time of assembly, there may already be others in the same branch. Therefore, it always makes a checkout named commit hash. Thus, we get the git repository in the detached head state .
As I said, looking ahead, this situation is normal and most work like that, but in this plugin you just need to write one line, indicating the name of the external variable or variables from which you need to take the real brunch name, for gitlab-ci I just needed add:
In Jenkins, such variables were also added a long time ago at the request of JENKINS-30252 . So, if you want to support both systems at once, you can simply write:
I hope you find it more convenient to work with versions in gradle. Yes, and in every way I recommend to put bugs or write requests to the author - he answers them very quickly. Good coding!
In SVN there was a familiar revision number for everyone, monotonously increasing with each commit. It was convenient to add to the version number, and this solved most of the problems. But git, of course, provides a lot of goodies, and it was worth convincing the management and the whole team to transfer the project to it ...
But I had to rebuild the process of versioning the resulting assembly artifacts.
As a result, we stopped at a very good Gradle plugin github.com/nemerosa/versioning , and I'm going to tell about its use.
Problem
We have been using Gradle for a long time in our application, and for SVN we just used the inherited function inherited from the family, written directly in the build.gradle file. Fortunately, among the other advantages of Gradle, you can mention that it is a wonderful Groovy language, and it does not limit you in writing the logic of the build - you have connected the necessary libraries from the Java world, and go ahead, at least rewrite the entire application in one file!
However, do you understand the harmfulness of this approach? If the logic of getting the version number takes more than 5-10 lines, and also if we crush our crutch for any reason, then it will just be impossible to support it very soon ...
You can see similar "solutions" to manual parsing, for example, in the Jenkins article for Android on a clean system and without UI orThrough the thorns to the assembly where it is proposed to manually call git describe and parse the output with regular expressions ...
I wanted something simpler, more reliable and initially working.
Our workflow and Wishlist
In our application, a couple of jar files, 3 war artifacts, 3 RPMs including them are collected, and at the end of Docker, an image of the application with RPMs installed, which after automatic testing is immediately sent to gitlab-ci to a private repository.
In general, with the transition to git / gitlab, we adhere to the logic inherited from the standard github flow with minor changes, which means for versioning:
- We want to distinguish between local builds, from builds on CI
- We want to distinguish the build from the fork (or feature branch) from the release one. In this case, you need to add a piece of the hash of the commit to know exactly where it was collected
- For releases, we decided to use tagging directly through the gitlab WEB interface - this is convenient. But at the same time, the hash is no longer needed because it does not look very nice for users, and the tag in the git is normal, Read-only (and does not require any hooks like in svn) and uniquely identifies the commit with the convenient name that we gave. That is, it should already be used in the version
- Plus, we need a structured version object to register in some parts of the system (js, html) for some shuffles
- We also need to export this information somehow, for gitlab-ci so that it knows which containers, which versions need to be raised in the next steps for testing
Proposed Solution: Gradle plugin net.nemerosa: versioning
Having looked around at the available plugins for gradle, I found such a wonderful option: github.com/nemerosa/versioning
Its documentation immediately captivates - everything is simple, logical and understandable for what it was done.
Plus, the semantic separation of release, feature
Let's try in business
So connecting to the project is very simple, follow the instructions:
plugins {
id 'net.nemerosa.versioning' version '2.4.0'
}That's it, in most cases it is already possible to use the version in your build scripts further, where it is needed:
version = versioning.info.fullWell, or closer to the point, say in the name of the war artifact:
war {
archiveName = "portal-api##${versioning.info.full}.war"
}After assembling from the feature-1 brunch, we get a file with approximately the following naming: portal-api ## feature-1.3e46dc.war (the example uses Tomcat-style naming ). The options for setting and parsing values for more interesting situations will be discussed later.
2 tasks are immediately available:
versionDisplay - showing information and versions and displays on the console. It is very convenient in debugging and versionFile - creating a build / version.properties file with ready-made variables for import into bash scripts outside:
> ./gradlew versionDisplay
:versionDisplay
[version] scm = git
[version] branch = release/0.3
[version] branchType = release
[version] branchId = release-0.3
[version] commit = da50c50567073d3d3a7756829926a9590f2644c6
[version] full = release-0.3-da50c50
[version] base = 0.3
[version] build = da50c50
[version] display = 0.3.0> ./gradlew versionFile
> cat build/version.properties
VERSION_BUILD=da50c50
VERSION_BRANCH=release/0.3
VERSION_BASE=0.3
VERSION_BRANCHID=release-0.3
VERSION_BRANCHTYPE=release
VERSION_COMMIT=da50c50567073d3d3a7756829926a9590f2644c6
VERSION_DISPLAY=0.3.0
VERSION_FULL=release-0.3-da50c50
VERSION_SCM=gitjust fine.
Custom logic parsing versions
I would like to note right away that there are many options for parsing names, handling prefixes, suffixes, interpreting versions. There is also support for SVN by the way. In general, you are in the customization section .
However, there is not a fly in the ointment . At the time I started using it, the documentation looked different .
Yes, you can set your own closure on how to interpret the brunch name (for example, consider 'release / 1' as release, and 'qa / 0.1' otherwise):
versioning {
branchParser = { String branch, String separator = '/' ->
int pos = branch.indexOf(separator)
if (pos > 0) {
new BranchInfo(
type: branch.substring(0, pos),
base: branch.substring(pos + 1))
} else {
new BranchInfo(type: branch, base: '')
}
}
}This is all great, but we want a tag instead of brunch, if any !?
I did not want to give up this idea. Of course, I cut a temporary workaround, but the author created a request to make the parsing logic more general: github.com/nemerosa/versioning/issues/32
Damien Coraboeuf , who is the author of this plugin, turned out to be very responsive, and quickly corrected a couple of small things quickly.
In general, as often happens, he proposed to realize for me what I am offering.
I followed his advice - quickly banged pull request .
Now, after its adoption, we get the SCM commit information object (SVN or GIT) and are free to choose the way how to build the version for us. For example, the same code as above can be implemented like this:
versioning {
releaseParser = { scmInfo, separator = '/' ->
List part = scmInfo.tag.split(separator) + ''
new net.nemerosa.versioning.ReleaseInfo(type: part[0], base: part[1])
}
} Same thing in a full closure.
What does this give us? Well, for example, as described in the requirements, we use this in order to take the brunch name in one case and the tag name in the other, and are not limited only to the string representation of the brunch name. For us, it now looks something like this:
versioning{
full = { scmInfo ->
// Tag name, or '_branch_name_' if it is not 'master'
(scmInfo.tag ?: ( 'master' == scmInfo.branch ? '' : "_${scmInfo.branch}_." ) + scmInfo.abbreviated).toLowerCase().replaceAll(/[^a-z0-9-_\.]/, '_')
}
}Lowercase is used for use in Docker image tags.
As I mentioned, the options are not limited to this, we also control the dirty suffix, and also add the build time to the same object using Groovy meta magic ...
Integration with CI
Well, since I started talking about convenient integration, I should immediately pay attention to one pitfall, which I also stumbled upon. And this plugin has already been taken care of!
The specified code worked fine, was tested, and commited. But the first push and build on CI brought a strange result - the name of the brunch was something like HEAD.
In fact, the reason is simple, if we look at what the builder does, he collects not a branch, but a specific commit. At the time of assembly, there may already be others in the same branch. Therefore, it always makes a checkout named commit hash. Thus, we get the git repository in the detached head state .
As I said, looking ahead, this situation is normal and most work like that, but in this plugin you just need to write one line, indicating the name of the external variable or variables from which you need to take the real brunch name, for gitlab-ci I just needed add:
branchEnv = ['CI_BUILD_REF_NAME']In Jenkins, such variables were also added a long time ago at the request of JENKINS-30252 . So, if you want to support both systems at once, you can simply write:
branchEnv = ['CI_BUILD_REF_NAME', 'GIT_LOCAL_BRANCH']I hope you find it more convenient to work with versions in gradle. Yes, and in every way I recommend to put bugs or write requests to the author - he answers them very quickly. Good coding!