October 11, 2011 at 00:49Assembly Environment for Midnight Commander - Continued
Beginning
in the first part , I introduced a new binary package assembly service Midnight Commander file manager for Debian / Ubuntu. The comments rightly pointed out the lack of a technical description of the environment itself and I promised that I would post the details as soon as the code was brought into a divine form. Two weeks passed, a little time appeared to stabilize the functionality and comb the code (it is still terrible, but it seems to work as intended), so it's time to fulfill the promise.
I specifically do not include source codes in the topic, there are many of them, and in my opinion the article is overloaded. Anyone interested can download them from googlecode .
The assembly environment consists of several shell scripts:
As I mentioned earlier, this is the most important part of the build environment. It performs two types of operations: assembling / reassembling releases and assembling night slices from GIT. If the release build was selected, then the script:
The assembly of the night slice differs only in that the sources are formed from the GIT repository and the version increment code is different. If the script detects that there were no changes in the master branch of the repository, then the assembly is interrupted.
This script calls build-mc-from-git.sh and, if successful, does an additional binary package check in piuparts, and if there are no errors, it starts the target assembly process (squeeze-i386, squeeze-amd64, natty-i386, etc. .). After each successful build, the temporary structure of the repository is updated. If all the goals were collected successfully and buildbot.sh was called by one of the CRON tasks (mc-release-builds.sh or mc-nightly-builds.sh), then the script to update my repository is run. At this point, APT overhead is updated, Release files are signed, and the server copy is updated.
Now the full assembly cycle of 8 targets takes a little more than an hour. In the future, it will be possible to expand the targets for rpm-based distributions (as far as I understand, the upstream has not restored the normal operation of the previous buildbot). Additional info or feasible support can be obtained on the project page .
in the first part , I introduced a new binary package assembly service Midnight Commander file manager for Debian / Ubuntu. The comments rightly pointed out the lack of a technical description of the environment itself and I promised that I would post the details as soon as the code was brought into a divine form. Two weeks passed, a little time appeared to stabilize the functionality and comb the code (it is still terrible, but it seems to work as intended), so it's time to fulfill the promise.
I specifically do not include source codes in the topic, there are many of them, and in my opinion the article is overloaded. Anyone interested can download them from googlecode .
Assembly environment
The assembly environment consists of several shell scripts:
- contrib / mc-release-builds.sh - a script that runs as a CRON job and controls the assembly of new releases, or the rebuilding of already released ones.
- contrib / mc-nightly-builds.sh - a script that is also a CRON job and builds night slices.
- contrib / prepare-environment.sh - a script that initializes environment variables specific to my server (in particular, environment variables are exported for operation of SSH and GPG agents)
- contrib / local-repo-update - the script with which I update my repository, now I’m thinking about giving up apt-ftparchive and trying reprepro.
- contrib / pbuilder - configs of assembly targets, one of them (contrib / pbuilder / buildbot) is used to prepare the package from source or GIT.
- contrib / apt-ftparchive - my settings for apt-ftparchive to work (maybe it will be useful to someone)
Next are the scripts that make up the assembly environment itself:
- build-mc-from-git.sh - build a package from source or VCS (system core)
- buildbot.sh - a wrapper over the previous script, which is an intermediate link between the tasks of the crown, the user and the kernel of the assembly environment.
- initial-build.sh - an anachronism, which remained thanks to my natural laziness. Adds a single buildbot user to the pbuilder (buildbot) assembly target.
build-mc-from-git.sh
As I mentioned earlier, this is the most important part of the build environment. It performs two types of operations: assembling / reassembling releases and assembling night slices from GIT. If the release build was selected, then the script:
- updates from the repository a local copy of the contrib / debian / directory, which is necessary to build the package (dh_make is not used when building the package)
- unpacks the source archive, which is passed to the script as one of the arguments by higher scripts or by the user.
- checks if the source code is "out of the box". To do this, in a directory with unpacked sources, a familiar sequence of actions is performed:
[ -x ./autogen.sh ] && ./autogen.sh ./configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib/mc make make install DESTDIR=/tmp/install
If this step goes without errors, then further problems with the package assembly indicate the need to revise the debian-specific part of the package (contrib / debian / directory in VCS) - copies the debian / directory from the local copy of VCS to the sources, checks if there is a local changelog for the version of the sources (if a new revision of the package is being assembled, and not a new release), and if it exists, then the changelog from VCS is replaced by the local one.
- increments the version and / or revision number and starts building the package by calling dpkg-buildpackage -rfakeroot -us -uc
- checks the package at lintian for compliance with the Debian policy. The output of lintian is saved in a separate file. (recently it helped to catch an error in the man pages of nightly builds and send the patch to upstream)
The assembly of the night slice differs only in that the sources are formed from the GIT repository and the version increment code is different. If the script detects that there were no changes in the master branch of the repository, then the assembly is interrupted.
buildbot.sh
This script calls build-mc-from-git.sh and, if successful, does an additional binary package check in piuparts, and if there are no errors, it starts the target assembly process (squeeze-i386, squeeze-amd64, natty-i386, etc. .). After each successful build, the temporary structure of the repository is updated. If all the goals were collected successfully and buildbot.sh was called by one of the CRON tasks (mc-release-builds.sh or mc-nightly-builds.sh), then the script to update my repository is run. At this point, APT overhead is updated, Release files are signed, and the server copy is updated.
Now the full assembly cycle of 8 targets takes a little more than an hour. In the future, it will be possible to expand the targets for rpm-based distributions (as far as I understand, the upstream has not restored the normal operation of the previous buildbot). Additional info or feasible support can be obtained on the project page .