Earlier, when we did not have our corporate blog, I wrote about how we use Microsoft TFS (Visual Studio Team Servives on Premises) to manage the software development life cycle and to automate testing. In particular, we have put together a large set of autotests for different systems in one package, which we run every day. I talked about this in more detail at the DevOpsDaysMoscow conference ( presentation , video presentation ) During the implementation, we encountered several problems:

• Running autotests in a single thread in succession takes too much time
• Some autotest crashes are not recorded

• Jenkins builds do not publish test results in VSTS

  
  

  All of these problems have been successfully resolved using our own VSTS extensions:

  
  
• Parallel Assemblies
• Auto Defects

• Refinement of the standard Jenkins job launch task

  
  

  More about the problems and how we looked for solutions - in my last article . Today I want to talk about how to make these extensions and how we can help extension developers.

  
  

  How ?

  
  

  Almost from the very beginning, we realized that the easiest way to solve our problems is with your own tasks or build steps in TFS. Tasks can be written either in powershell or in typescript. We chose typescript, and that's why - in fact it is javascript, but with typing support. There are many articles about the advantages of typescript and its use, including on the Habr. For us, the main advantages were the following:

  
  
• IntelliSense and static checks
• Powered by linux agents
• the ability to use any npm modules. This means that you do not need to reinvent the wheel - you need UUID generation - it is there, the generation of test results in Visual Studio format (trx files) - it is, there is also work with XML.
• Language level asynchronous support - async / await. Allows you to get rid of callback hell. Asynchronous methods and calls look like synchronous.

• To work with TFS, Microsoft created and published 2 npm modules, which again eliminated the need to reinvent the wheel.

  
  

  It also turned out to be of great help that many tasks from Microsoft are written and typescript and published under an open license. In addition to the fact that it allows you to use them as examples, it made it possible to make your own fork of this repository and make your own bootstrap kit for quickly creating tasks and automating the assembly and packaging of extensions.

  
  

  What ?

  
  

  What is the challenge for VSTS? This is a set of required components:

  
  
• task definition
• main task script
• package.json - dependency definition

• task icon

  
  

  Task definition

  
  

  A task definition contains several blocks.

  
  

  identification block:

  
  

  {
  "id": "b5525419-bae1-44de-a479-d6de6a3ccb2f",
  "name": "TestTask",
  "friendlyName": "TestTask",
  "description": "TestTask",
  "helpMarkDown": "",
  "category": "Build",
  "author": "authorName",
  "version": {
      "Major": 1,
      "Minor": 0,
      "Patch": 0
  },
  "instanceNameFormat": "TestTask $(testparam)"
  }

  
  

This block describes the unique id of the task, its name, category and version. When creating a task, you must specify all of these fields. The instanceNameFormat field determines what the task name will look like in the default VSTS assembly. It can contain parameters from the parameter block in the form $ (parameter name)

parameter block

The parameters block indicates the input parameters of the task, their names, descriptions and types. Parameters can be grouped for ease of presentation on the task settings page. The following block is the AutoDefects task parameter:

{
 "groups": [
        {
            "name": "authentication",
            "displayName": "Authentication",
            "isExpanded": false
        }
    ],
    "inputs": [
        {
            "name": "Assignees",
            "type": "filePath",
            "label": "Assignees list file",
            "defaultValue": "assignees.json",
            "required": false,
            "helpMarkDown": "Bug assignees list in json format. Format: {\"testrunname\":\"username\"}"
        },
        {
            "name": "authtype",
            "type": "pickList",
            "label": "Authentication type",
            "defaultValue": "oauth",
            "required": false,
            "helpMarkDown": "Authentication type to access the tfs rest api",
            "options": {
                "oauth": "OAuth",
                "NTLM": "NTLM",
                "Basic": "Basic"
            },
            "groupName" : "authentication"
        },
        {
            "name": "Username",
            "type": "string",
            "label": "Username",
            "defaultValue": "",
            "required": false,
            "helpMarkDown": "Username to access tfs rest api (NTLM and Basic types)",
            "groupName" : "authentication",
            "visibilityRule" : "authtype != OAuth"
        },
        {
            "name": "Password",
            "type": "string",
            "label": "Password",
            "defaultValue": "",
            "required": false,
            "helpMarkDown": "Password to access tfs rest api (NTLM and Basic types)",
            "groupName" : "authentication",
            "visibilityRule" : "authtype != OAuth"
        }
    ]
}

The parameters that determine the authentication scheme are placed in a separate group, which is collapsed by default.

The most commonly used types of parameters are:

• string is a regular string.
• pickList - select box with a limited list of values
• filePath - select a file inside the assembly repository.

execution unit

contains a link to the main executable file of the task

{
 "execution": {
        "Node": {
            "target": "testtask.ts"
        }
    }
}

localization unit

{
    "messages": {
        "taskSucceeded": "All done",
        "taskFailed": "Task Failed"
    }
}

contains a set of localized lines for logging the task in the build log file. Used less often than the blocks above. Messages for the current local settings can be received by calling task.loc ("messagename");

Main executable file

The main executable file is a script that executes VSTS when the task starts. At a minimum, it should contain code for importing the necessary modules for the task to work and error handling. For instance:

import tl = require('vsts-task-lib/task');
import trm = require('vsts-task-lib/toolrunner');
import path = require('path');
import fs = require('fs');
import Q = require("q");
import * as vm from 'vso-node-api';
import * as bi from 'vso-node-api/interfaces/BuildInterfaces';
import * as ci from 'vso-node-api/interfaces/CoreInterfaces';
import * as ti from 'vso-node-api/interfaces/TestInterfaces';
import * as wi from 'vso-node-api/interfaces/WorkItemTrackingInterfaces';
async function run() {
    tl.setResourcePath(path.join(__dirname, 'task.json'));
    let projId = tl.getVariable("System.TeamProjectId");
    try {
    } catch(err) {
        console.log(err);
        console.log(err.stack);
        throw err;
    }
}
run()
.then(r => tl.setResult(tl.TaskResult.Succeeded,tl.loc("taskSucceeded")))
.catch(r => tl.setResult(tl.TaskResult.Failed,tl.loc("taskFailed")))

As you can see, the task is a set of standard components that change little from task to task. Therefore, when I created the third task, the idea came up to automate the creation of tasks. This is how our bootstrap appeared, which greatly facilitates the development of extensions for VSTS.

How faster?

What needs to be done when creating a task for VSTS, except writing the code of the task itself? The steps are usually the same:

1. Create task skeleton
2. Assemble a task into an isolated component
3. Pack task on vsix for publishing to VSTS

All of these steps can be automated to speed up development and eliminate unnecessary manual labor. To automate these steps, you can use our bootstrap. The workflow of our collector is similar to the task collector from Microsoft - the tasks for the assembly are listed in the make-options.json file in the root of the project:

{
    "tasks": [
        "AutoDefects",
        "ChainBuildsAwaiter",
        "ChainBuildsStarter",
        "TestTask"
    ],
...
}

Prerequisite

To create extensions, you will need the following software:

• Local JavaScript machine - nodejs
• Typescript interpreter - npm install -g typescript
• Gulp collector - npm install -g gulp
• Console utility for working with VSTS tfx-cli - npm install -g tfx-cli

Task creation

TaskName task is created by the command:
gulp generate –-name TaskName

As a result of the execution of the command, the following occurs:

• The task is added to the project list of tasks for the assembly.
• Создание каталога задачи и «скелетных» файлов – taskname.ts, task.json, package.json, typings.json, icon.png

Скелетные файлы содержат минимально необходимый набор данных и кода.

Сборка задач проекта

Сборка задач проекта осуществляется комадной gulp

При этом для всех задач, перечисленных в make-options.json происходит следующее:

• Трансляция .ts в .js
• Установка node_modules в каталог задачи
• Генерация языковых файлов

Упаковка задач

Упаковка задач осуществляется командой gulp mkext [--all] [--exts ext1,ext2]

По умолчанию каждая задача упаковывается в отдельный vsix файл, если указан параметр --all, то все задачи собираются в один большой vsix файл.

By default, all tasks listed in make-options.json are packed, if the --exts option is specified, only those listed in the extension parameter are packed.

We are open

Bootstrap published on GitHub - forks, feature requests, pull requests are welcome.

I really hope that this article will arouse interest in Miscosoft VSTS, which in my opinion is an excellent group work tool not only for large companies, but also for small flexible teams.

Konstantin Neradovsky,
Head of Testing Automation,
Otkritie Bank