Automated NuGet Package Creation


Kohl you wanted to convey the assembly
And with them fiery hello
Nugetom do not forget to pack
In a package!


Immediately make a reservation that in this article we will focus on the Microsoft .NET technology stack.

It often happens that a subset of projects begins to be used in different solutions.

As a rule, programmers, having spotted something useful in a neighboring project, do not bother at first - they create the lib folder (dll, assemblies, etc.) and put the compiled assemblies from the original solution there. Over time, it becomes clear that this is not the most convenient option, and here's why:

  • the original solution begins to develop in its own direction, without taking into account “consumers”: new dependencies are added, .net versions are updated, etc. "Jokes";
  • even if they think about “consumers”, they forget to update assemblies when they receive a critical update or just a new version, and then everything gets worse when assemblies become more than one and there are some dependencies between them - updating one assembly, we get problems in execution time, as another assembly may be the wrong version;
  • The original solution is no longer being developed.

The answer to all these troubles can be bringing projects to a separate solution and creating a NuGet package that includes common assemblies, and changing the development paradigm of these assemblies. By and large, all this can be done without NuGet, but it’s much less fun. How to make sure that the NuGet package is automatically built together with the project compilation on the build server and includes all the necessary whistles and buzzes - this will be ours story.

Making NuGet Packages


The process of making NuGet packages is quite simple. The entire general theoretical part is accessible and, in general, understandable. Different content can be packaged in packages, not only compiled assemblies, but also debugging symbols, pictures, etc. resources, and even source code.

In this description, we confine ourselves to the most pressing issue of packaging compiled assemblies.

Preparation of the first NuGet package


In order to set up automated creation of NuGet packages on the build server, you need to "cook up" the first version of the package. The easiest and most understandable way to create a package is to use a NuSpec file that describes what the package will be. You can get this NuSpec file in different ways:


In principle, you can completely create the entire NuSpec file in the GUI, but understanding how NuSpec works is still useful.

For example, one of our NuSpec files with abbreviations looks something like this:

NuSpec File Contents
NewPlatform.Flexberry.ORM2.1.0-alpha1Flexberry ORMNew Platform LtdFlexberry ORM package.
      ...
    Copyright New Platform Ltd 2015Flexberry ORM


Here are some explanations regarding some sections:

  • Id must be unique within the common namespace of all packages in order to avoid collisions. Someone indicates the name of the company in the package name , then the name of the project and a specific product, while someone does not bother .
  • As for the versions: the use of the principles of semantic versioning is considered good practice . A small rule that we have developed in our team - all pre-release versions (which apart from 3 numbers have something else at the end, for example, alpha1) we publish with assemblies assembled in the Debug configuration, and releases, respectively, in Release.
  • Release notes (releaseNotes) - a very useful thing, be sure to write there that has changed from the previous version. Users must understand what they get with each update.
  • Dependencies When describing dependencies, you need to think about how your package will be installed: if the user only needs your package and nothing more, then there are no dependencies. If your assemblies will work only if there is another package, for example, SharpZipLib, then you definitely need to register this dependency. It is important to understand that SharpZipLib, in turn, may have its own dependencies, and they will also “fly” to the user during installation, even if you do not specify them at home.
    Installation occurs recursively, so that a user in one of the hypothetical situations can start installing one package, and more than a hundred will be installed for him - just through dependencies. During package installation, choosing the version of the dependent package is very tricky.. If you do not specify the version number, the latest release version will be installed, otherwise the one that is explicitly specified in the dependency. By the way, if you use several unrelated packages from time to time, then you can create an empty package with dependencies on the packages you need and install this package of your own - the rest will be installed after it themselves.
  • File descriptions may include specific names or masks. We highly recommend that you observe the correct package structure when the content type, version of .net framework and other things are written in target, in accordance with the agreement . It is important to understand that in the src attribute when specifying the path to the file, you need to start from the current directory, in the context of which the package packaging command will be executed.

After the NuSpec file is ready, you can start trial creation of the package. To do this, a simple NuGet.exe utility command is run: nuget pack MyAssembly.nuspec.

Thus, we should get the coveted “first package”, or “prototype package,” that is, a nupkg file that can be used for installation in projects through the NuGet Package Manager or through NuGet.exe .

Exhibition of ready-made packages


So, we have a package that needs to be delivered to users somehow through some kind of “package distribution channel”. We believe that most users will install packages through Visual Studio. The NuGet Package Manager built into it understands two options for placing packages:

  • Package Gallery, accessible over the network;
  • Windows folder (local or network).

You can add your own package sources in the settings, they will be searched in turn when installing or restoring packages until the desired id is found. The option when the same (!) Package lies in several sources is quite acceptable.

The easiest option for distributing packages is to create a network folder and put the packages there.

It is worth noting that NuGet allows you to work not only with the general gallery of packages https://nuget.org , but also to create your own galleries, for this you can deploy the same engine somewhere that is used at https://nuget.org. Our team prefers this option, because in this case it becomes possible to track download statistics, manage permissions through the site, in the end, it's just beautiful.



Installing a gallery may require small dances with a tambourine, at least in the issue of authorization, but there is nothing complicated about it. Packages are published in the same way as on NuGet.org, it’s important when updating the gallery site not to lose the archive with already downloaded packages - they are stored in the site directory. Setting up NuGet Package Manager for users in this case will look something like this:



If the local source of packages is somewhere near users, for example, on the same local network, it is recommended to download all packages with dependencies into it - this will reduce the time of downloading packages for new users. Finding nupkg files from dependent packages is very easy - they are always in the packages folder in which these packages are installed (usually in the directory with the sln file). Also, the order is important in the package source settings window - the studio will iterate over the sources in case of package recovery in the order specified in the settings. Therefore, if your package is available only locally, then first place your source so that there are no unnecessary requests to nuget.org.

NuGet Packet Factory


After the “prototype package” has been made and the “distribution channel for packages” has been established, you can proceed to automate the assembly of packages so that with the very first click of the mouse we can get the hot and freshest NuGet package.

Let's see how this is done in the case of Team Foundation Server 2013/2015. For other similar CI systems, the process will be similar.
In the Build Definition (XAML) properties, you can specify the PowerShell script that will be executed if the build is successful. It is in this script that we will call our “packer”, passing the path to the NuSpec file as a parameter.

There are several points that should be clarified for yourself: where will NuGet.exe itself and all the files it needs (at least the configuration file) lie, where will the NuSpec file be located? On the one hand, you can rely on the fact that NuGet.exe will be located in a certain place on the build server, but if there are several build servers and there is no desire to administer them, it is easiest to put NuGet.exe in Source Control and add a directory with its location in the Workspace with which the build will be performed. As for NuSpec, it is convenient to keep it next to the sln-file and even include it in Solution Items for quick access to it through the Solution Explorer.

If there are several solutions and you plan to create several packages, it is recommended to implement one common PowerShell script, which will receive the path to the NuSpec file as a parameter.

Below are excerpts from such a script:

Excerpts from the PowerShell script
# Create NuGet Package after successfully server build.
# Enable -Verbose option for this script call.
[CmdletBinding()]
Param(
    # Disable parameter.
    # Convenience option so you can debug this script or disable it in 
    # your build definition without having to remove it from
    # the 'Post-build script path' build process parameter.
    [switch] $Disable,
    # This script used NuGet.exe from current directory by default.
    # You can change this path to meet your needs.
    [String] $NuGetExecutablePath = (Get-Item -Path ".\" -Verbose).FullName + "\NuGet.exe",
    $BinariesDirectoryPostfixes = @("\Debug", "\Release"),
    # Path to the nuspec file. Path relative TFS project root directory.
    [Parameter(Mandatory=$True)]
    [String] $NuspecFilePath,
    # Disable Doxygen.
    [switch] $NoDoxygen
    # ...
    # Go, go, go!
    $nugetOutputLines = & $NuGetExecutablePath pack $realNuspecFilePath -BasePath $basePath
        -OutputDirectory $outputDirectory -NonInteractive;
    ForEach ($outputLine in $nugetOutputLines) {
        Write-Verbose $outputLine;
    }
    # ...


In the script, operations are performed to convert relative paths to absolute ones (you can easily find a description of the available variables that are indicated by the CI system when the script is run). In some cases, modification of the NuSpec file in this script is required. For example, this way you can handle the creation of packages for various configurations (Any CPU, x86).

This, in fact, sets up the automatic mechanism for creating NuGet packages. We start the assembly on the build server, check that everything worked. To get debugging information, if something went wrong, do not forget to write –Verbose in the script parameters in the build definition settings. Pour the finished packages into a shared resource or gallery and invite the first users.

The subtleties of the process


As the saying goes, "the main task of the programmer is to kill the perfectionist in himself." If the internal perfectionist has not yet given up, then the following points should come in handy.

In addition to the ability to create NuGet packages, a script for the build server for each package can run a utility for generating auto-documentation based on XML comments in the code. This feature is convenient in the sense that for each version of the package we have our own version of auto-documentation, it is convenient if users use different versions of NuGet packages. We use Doxygen to generate auto- documentation . Here is the script section on auto-documentation:

Excerpts from a PowerShell script to generate auto-documentation
if($NoDoxygen)
{
    Write-Verbose "Doxygen option is disabled. Skip generation of the project documentation.";
}
else
{
    Write-Verbose "Doxygen option is enabled. Start documentation generation.";
    # Copy doxygen config file.
    $doxyConfigSourcePath = Join-Path -Path $toolsFolderPath -ChildPath "DoxyConfig" -Resolve;
    $doxyConfigDestinationPath = Join-Path -Path $Env:TF_BUILD_BINARIESDIRECTORY -ChildPath "DoxyConfig";
    # Modify doxigen config file according with given nuspec.
    $nuspecXml = [xml](Get-Content $NuspecFilePath);
    $doxyConfig = Get-Content -Path $doxyConfigSourcePath;
    $projectName = $nuspecXml.GetElementsByTagName("title").Item(0).InnerText + " " +
    $nuspecXml.GetElementsByTagName("version").Item(0).InnerText;
    $doxyConfig = $doxyConfig -replace "FlexberryProjectName", $projectName;
    $projectLogoPath = Join-Path -Path $toolsFolderPath -ChildPath "logo.png" -Resolve;
    $doxyConfig = $doxyConfig -replace "FlexberryProjectLogo", $projectLogoPath -replace "\\", "/";
    $doxyConfig = $doxyConfig -replace "FlexberryOutputDirectory", $Env:TF_BUILD_BINARIESDIRECTORY -replace "\\", "/";
    $doxyConfig = $doxyConfig -replace "FlexberryInputDirectory", $Env:TF_BUILD_SOURCESDIRECTORY -replace "\\", "/";
    $doxyWarnLogFilePath = Join-Path -Path $Env:TF_BUILD_BINARIESDIRECTORY -ChildPath "doxygen_log.txt";
    $doxyConfig = $doxyConfig -replace "FlexberryWarnLogFile", $doxyWarnLogFilePath -replace "\\", "/";
    $doxyConfig | Out-File $doxyConfigDestinationPath default;
    # Run doxygen.
    $doxygenExecutablePath = Join-Path -Path $toolsFolderPath -ChildPath "doxygen.exe" -Resolve;
    $doxygenOutputLines = & $doxygenExecutablePath $doxyConfigDestinationPath
    ForEach ($outputLine in $doxygenOutputLines) {
        Write-Verbose $outputLine;
    }
    Write-Verbose "Documentation generation done. Packing to the archive.";
    # Do archive.
    $archiveSourceFolder = Join-Path -Path $Env:TF_BUILD_BINARIESDIRECTORY -ChildPath "html" -Resolve;
    $archiveFileName = $nuspecXml.GetElementsByTagName("id").Item(0).InnerText + "." +
    $nuspecXml.GetElementsByTagName("version").Item(0).InnerText;
    $archiveDestinationFolder = Join-Path -Path $Env:TF_BUILD_BINARIESDIRECTORY -ChildPath ($archiveFileName + ".zip");
    Add-Type -assembly "system.io.compression.filesystem";
    [io.compression.zipfile]::CreateFromDirectory($archiveSourceFolder, $archiveDestinationFolder);
    # Remove html documentation files.
    Remove-Item $archiveSourceFolder -recurse;
    Write-Verbose "Done.";
}


The second item will concern the assembly of the project if different versions of assemblies for different versions of the .net framework are packaged in one package.

The tricks begin with forcing the build server to build assemblies for different versions of the .net framework. Consider the projects to be assembled in csproj format, rather than the new json project file format (ASP.NET5). Visual Studio supports assembly configuration mechanism. Usually 2 configurations are used - Debug and Release, but the same mechanism allows you to configure .net version switching.

You can create your own configurations, which we do. Unfortunately, in order to fine-tune all the necessary parameters, you will have to open the csproj file and, at a minimum, set TargetFrameworkVersion there in each of the configuration sections.
Excerpts from the .csproj file
truefullfalsebin\Debug-Net35\DEBUG;TRACEv3.5AnyCPUprompt4AllRules.rulesetbin\Debug-Net35\LogService.XMLpdbonlytruebin\Release-Net35\TRACEv3.5AnyCPUprompt4bin\Release-Net35\LogService.XMLAllRules.rulesettruebin\Debug-Net40\DEBUG;TRACEbin\Debug-Net40\LogService.XMLfullv4.0AnyCPUpromptAllRules.rulesettruebin\Debug-Net45\DEBUG;TRACEbin\Debug-Net45\LogService.XMLfullv4.5AnyCPUpromptAllRules.rulesetbin\Release-Net40\TRACEbin\Release-Net40\LogService.XMLtruepdbonlyv4.0AnyCPUpromptAllRules.rulesetbin\Release-Net45\TRACEbin\Release-Net45\LogService.XMLtruepdbonlyv4.5AnyCPUpromptAllRules.ruleset


The configurations in Visual Studio are switched mainly in the toolbar; in the assembly definition on the server, you can select several configurations at once, which will be compiled sequentially.

It is worth noting that if your code for different versions of the .net framework starts to differ, then this can be processed using the directives:

        #if NETFX_35
            for (int i = 0; i < resValueLength; i++) 
        #else
            System.Threading.Tasks.Parallel.For(0, resValueLength, i =>
        #endif

In this case, the constants must be defined in the corresponding section of the csproj file:

DEBUG;TRACE;NETFX_35

When we have ready-made compiled assemblies, let's figure out how to properly configure nuspec. In nuspec, special directories are specified for specific versions of the .net framework.

An example of the files section in a NuSpec file:


Another problem that you can often encounter when using (not even when creating) NuGet packages is the problem of connecting one project to several solutions. The fact is that in the csproj-file assembly links are put down to specific dlls, which are restored by default to Visual Studio in the packages folder next to the sln-file. Hence the problem arises when the same project is included in several solutions located in different folders. To solve this problem, you can use the NuGet package, which includes a special Target, which rewrites the links before the build: https://www.nuget.org/packages/NuGetReferenceHintPathRewrite .

Another feature of using NuGet packages is the topic of package recovery during assembly. The fact is that for some time Visual Studio did not have built-in package recovery tools, so a special Target was added to csproj, which was responsible for recovery. In modern Visual Studio (2013+) this is no longer relevant, keep your csproj files clean, no Target is needed to restore NuGet packages anymore.

And finally, we can talk about the fact that when using TFS, the packages folder by default crawls into Source Control and someone can periodically blink and still check all the assemblies in TFS. To prevent this from happening (we are sure that for those who cheat assemblies in TFS there must be a separate boiler in hell), you can use the .tfignore filewho must save from this scourge.

Result


So, having completed everything that is described in the instruction we have proposed, you can get a ready-made package packaging mechanism that works without human intervention. Our packages are built that way. Unless, the publication itself requires some attention.

Useful links:




Also popular now: