Introducing CLI Builder

Original author: Hans Larsen
  • Transfer
Introducing CLI Builder

In this article, we will look at the new Angular CLI API, which will allow you to extend existing CLI features and add new ones. We will discuss how to work with this API, and what points of its extension exist, which allow adding new functionality to the CLI.

History


About a year ago, we introduced the workspace file ( angular.json ) in the Angular CLI and rethought many of the basic principles for implementing its commands. We came to the fact that we placed the teams in the “boxes”:

  1. Schematic commands - "Schematic commands . " By now, you've probably already heard about Schematics, the library used by the CLI to generate and modify your code. It appeared in version 5 and is currently used in most commands that concern your code, such as new , generate , add and update .
  2. Miscellaneous commands - "Other teams" . These are commands that are not directly related to your project: help , version , config , doc . Recently, analytics has also appeared , as well as our Easter eggs (Shhh! Not a word to anyone!).
  3. Task commands - "Task commands . " This category, by and large, "launches processes executed on other people's code." - As an example, build is a project build, lint is debugging and test is testing.

We started designing angular.json a long time ago. Initially, it was conceived as a replacement for the Webpack configuration. In addition, it was supposed to allow developers to independently choose the implementation of the project assembly. As a result, we got a basic task launch system, which remained simple and convenient for our experiments. We called this API "Architect."

Despite the fact that Architect was not officially supported, it was popular among developers who wanted to customize the assembly of projects, as well as among third-party libraries that needed to control their workflow. Nx used it to execute Bazel commands, Ionic used it to run unit tests on Jest, and users could expand their Webpack configurations with tools such asngx-build-plus . And that was just the beginning.

An officially supported, stable, and improved version of this API is used in Angular CLI version 8.

Concept


The Architect API offers tools for scheduling and coordinating tasks that the Angular CLI uses to implement its commands. It uses functions called
“builders” - “collectors”, which can act as tasks or as schedulers of other collectors. In addition, it uses angular.json as a set of instructions for the collectors themselves.

This is a very general system designed to be flexible and extensible. It contains an API for reporting, logging and testing. If necessary, the system can be expanded for new tasks.

Pickers


Assemblers are functions that implement logic and behavior for a task that can replace the CLI command. - For example, start the linter.

The collector function takes two arguments: the input value (or options) and the context that provides the relationship between the CLI and the collector itself. The division of responsibility here is the same as in Schematics - the CLI user sets the options, the API is responsible for the context, and you (the developer) set the necessary behavior. The behavior can be implemented synchronously, asynchronously, or simply display a certain number of values. The output must be of type BuilderOutput , which contains the logical field success and the optional field error , which contains the error message.

Workspace file and tasks


The Architect API relies on angular.json , a workspace file for storing tasks and their settings.

angular.json divides the workspace into projects, and they, in turn, into tasks. For example, your application created with the ng new command is one such project. One of the tasks in this project will be the build task , which can be launched using the ng build command . By default, this task has three keys:

  1. builder - the name of the collector to use to complete the task, in the format PACKAGE_NAME: ASSEMBLY_NAME .
  2. options - settings used when starting a task by default.
  3. configurations - settings that will be applied when starting a task with the specified configuration.

The settings are applied as follows: when the task starts, the settings are taken from the options block, then, if a configuration has been specified, its settings are written on top of the existing ones. After that, if additional settings were passed to scheduleTarget () - the overrides block , they will be written last. When using the Angular CLI, command line arguments are passed to overrides . After all the settings are transferred to the collector, he checks them according to his scheme, and only if the settings correspond to it, the context will be created and the collector will start working.

More information on the workspace here .

Create your own collector


As an example, let's create a collector that will run a command on the command line. To create a collector, use the createBuilder factory and return the BuilderOutput object :

import { BuilderOutput, createBuilder } from '@angular-devkit/architect';
export default createBuilder((options, context) => {
  return new Promise(resolve => {
    resolve({ success: true });
  });
});

Now, let's add some logic to our collector: we want to control the collector through the settings, create new processes, wait for the process to complete, and if the process completed successfully (that is, return code 0), signal this to Architect:

import { BuilderOutput, createBuilder } from '@angular-devkit/architect';
import * as childProcess from 'child_process';
export default createBuilder((options, context) => {
  const child = childProcess.spawn(options.command, options.args);
  return new Promise(resolve => {
    child.on('close', code => {
      resolve({ success: code === 0 });
    });
  });
});

Output processing


Now, the spawn method passes all the data to the standard output of the process. We may want to transfer them to the logger - logger. In this case, firstly, debugging during testing will be facilitated, and secondly, Architect itself can run our collector in a separate process or disable standard output of processes (for example, in the Electron application).

To do this, we can use Logger , available in the context object , which will allow us to redirect the output of the process:

import { BuilderOutput, createBuilder } from '@angular-devkit/architect';
import * as childProcess from 'child_process';
export default createBuilder((options, context) => {
  const child = childProcess.spawn(options.command, options.args, { stdio: 'pipe' });
  child.stdout.on('data', (data) => {
    context.logger.info(data.toString());
  });
  child.stderr.on('data', (data) => {
    context.logger.error(data.toString());
  });
  return new Promise(resolve => {
    child.on('close', code => {
      resolve({ success: code === 0 });
    });
  });
});

Performance and Status Reports


The final part of the API related to the implementation of your own collector is the progress and current status reports.

In our case, the command is either completed or executed, so it makes no sense to add a progress report. However, we can communicate our status to the parent collector so that he understands what is happening.

import { BuilderOutput, createBuilder } from '@angular-devkit/architect';
import * as childProcess from 'child_process';
export default createBuilder((options, context) => {
  context.reportStatus(`Executing "${options.command}"...`);
  const child = childProcess.spawn(options.command, options.args, { stdio: 'pipe' });
  child.stdout.on('data', (data) => {
    context.logger.info(data.toString());
  });
  child.stderr.on('data', (data) => {
    context.logger.error(data.toString());
  });
  return new Promise(resolve => {
    context.reportStatus(`Done.`);
    child.on('close', code => {
      resolve({ success: code === 0 });
    });
  });
});

To pass a progress report, use the reportProgress method with current and (optionally) summary values ​​as arguments. total can be any number. For example, if you know how many files you need to process, you can transfer their number to total , then to current you can transfer the number of already processed files. This is how the tslint collector reports on its progress.

Input Validation


The options object passed to the collector is checked using JSON Schema. This is similar to Schematics if you know what it is.

In our collector example, we expect that our parameters will be an object that receives two keys: command - command (string) and args - arguments (array of strings). Our verification scheme will look like this:

{
  "$schema": "http://json-schema.org/schema",
  "type": "object",
  "properties": {
    "command": {
      "type": "string"
    },
    "args": {
      "type": "array",
      "items": {
        "type": "string"
      }
    }
  }

Schemes are really powerful tools that can carry out a large number of checks. For more information about JSON schemes, you can refer to the official JSON Schema website .

Create a build package


There is one key file that we need to create for our own collector in order to make it compatible with the Angular CLI - builders.json , which is responsible for the relationship between our implementation of the collector, its name and the verification scheme. The file itself looks like this:

{
  "builders": {
    "command": {
      "implementation": "./command",
      "schema": "./command/schema.json",
      "description": "Runs any command line in the operating system."
    }
  }
}

Then, in the package.json file, we add the builders key , pointing to the builders.json file :

{
  "name": "@example/command-runner",
  "version": "1.0.0",
  "description": "Builder for Architect",
  "builders": "builders.json",
  "devDependencies": {
    "@angular-devkit/architect": "^1.0.0"
  }
}

This will tell Architect where to look for the collector definition file.

Thus, the name of our collector is "@ example / command-runner: command" . The first part of the name, before the colon (:) is the name of the package, defined using package.json . The second part is the name of the collector, defined using the builders.json file .

Testing your own builders


The recommended way to test assemblers is through integration testing. This is because creating a context is not easy, so you should use the scheduler from Architect.

To simplify the patterns, we thought of a simple way to create an Architect instance: first you create a JsonSchemaRegistry (to test the schema), then TestingArchitectHost and, finally, an Architect instance . Now you can compile the builders.json configuration file .

Here is an example of running the collector, which executes the ls command and verifies that the command completed successfully. Please note that we will use the standard output of processes in logger .

import { Architect, ArchitectHost } from '@angular-devkit/architect';
import { TestingArchitectHost } from '@angular-devkit/architect/testing';
import { logging, schema } from '@angular-devkit/core';
describe('Command Runner Builder', () => {
  let architect: Architect;
  let architectHost: ArchitectHost;
  beforeEach(async () => {
    const registry = new schema.CoreSchemaRegistry();
    registry.addPostTransform(schema.transforms.addUndefinedDefaults);
    // Аргументы TestingArchitectHost – рабочая и текущая директории.
    // Сейчас мы их не используем, поэтому они одинаковые.
    architectHost = new TestingArchitectHost(__dirname, __dirname);
    architect = new Architect(architectHost, registry);
    // Тут мы передаем либо имя NPM-пакета,
    // либо путь до package.json файла пакета.
    await architectHost.addBuilderFromPackage('..');
  });
  // Это может не работать в Windows
  it('can run ls', async () => {
    // Создаем регистратор, хранящий массив всех зарегистрированных сообщений.
    const logger = new logging.Logger('');
    const logs = [];
    logger.subscribe(ev => logs.push(ev.message));
    // "run" может содержать множество выводов, а также информацию о ходе работы сборщика.
    const run = await architect.scheduleBuilder('@example/command-runner:command', {
      command: 'ls',
      args: [__dirname],
    }, { logger });
    // "result" – следующий вывод выполняемого процесса.
    // Он имеет тип "BuilderOutput".
    const output = await run.result;
    // Останавливаем сборщик. Architect действительно прекращает сохранение состояний 
    // сборщика в памяти, так как сборщики ждут, чтобы снова быть запущенными.
    await run.stop();
    // Ожидаем успешное завершение.
    expect(output.success).toBe(true);
    // Ожидаем, что этот файл будет выведен.
    // `ls $__dirname`.
    expect(logs).toContain('index_spec.ts');
  });
});

To run the example above, you need the ts-node package . If you intend to use Node, rename index_spec.ts to index_spec.js .

Using the collector in a project


Let's create a simple angular.json that demonstrates everything we learned about assemblers. Assuming we packed our collector in example / command-runner and then created a new application using ng new builder-test , the angular.json file might look like this (some of the content has been removed for brevity):

{
  // ... удалено для краткости.
  "projects": {
    // ...
    "builder-test": {
      // ...
      "architect": {
        // ...
        "build": {
          "builder": "@angular-devkit/build-angular:browser",
          "options": {
            // ... разные опции
            "outputPath": "dist/builder-test",
            "index": "src/index.html",
            "main": "src/main.ts",
            "polyfills": "src/polyfills.ts",
            "tsConfig": "src/tsconfig.app.json"
          },
          "configurations": {
            "production": {
              // ... разные опции
              "optimization": true,
              "aot": true,
              "buildOptimizer": true
            }
          }
        }
      }

If we decided to add a new task to apply (for example) the touch command to the file (updates the file modification date) using our collector, we would run npm install example / command-runner , and then make changes to angular.json :


{
  "projects": {
    "builder-test": {
      "architect": {
        "touch": {
          "builder": "@example/command-runner:command",
          "options": {
            "command": "touch",
            "args": [
              "src/main.ts"
            ]
          }
        },
        "build": {
          "builder": "@angular-devkit/build-angular:browser",
          "options": {
            "outputPath": "dist/builder-test",
            "index": "src/index.html",
            "main": "src/main.ts",
            "polyfills": "src/polyfills.ts",
            "tsConfig": "src/tsconfig.app.json"
          },
          "configurations": {
            "production": {
              "fileReplacements": [
                {
                  "replace": "src/environments/environment.ts",
                  "with": "src/environments/environment.prod.ts"
                }
              ],
              "optimization": true,
              "aot": true,
              "buildOptimizer": true
            }
          }
        }
      }
    }
  }
}

The Angular CLI has a run command , which is the main command to run collectors. As the first argument, it takes a string of the format PROJECT: TASK [: CONFIGURATION] . To run our task, we can use the ng run builder-test: touch command .

Now we may want to redefine some arguments. Unfortunately, we cannot redefine arrays from the command line so far, however we can change the command itself for demonstration: ng run builder-test: touch --command = ls . - This will output the src / main.ts file .

Watch Mode


By default, it is assumed that collectors will be called once and terminated, however, they can return Observable to implement their own observing mode (as the Webpack collector does ). Architect will subscribe to the Observable until it finishes or stops and can subscribe to the collector again if the collector is called with the same parameters (albeit not guaranteed).

  1. The collector should return a BuilderOutput object after each execution. After completion, it can enter observing mode caused by an external event and, if it starts again, it will have to call the context.reportRunning () function to notify Architect that the collector is working again. This will protect the collector from stopping it by the Architect on a new call.
  2. Architect himself unsubscribes from Observable when the collector stops (using run.stop (), for example), using the Teardown logic - the destruction algorithm. This will allow you to stop and clear the assembly if this process is already running.

Summarizing the above, if your collector watches external events, it works in three stages:

  1. Performance. For example, compilation of Webpack. This step ends when Webpack finishes building and your collector sends BuilderOutput to the Observable .
  2. Observation. - Between two launches, external events are monitored. For example, Webpack monitors the file system for any changes. This step ends when Webpack resumes the build and context.reportRunning () is called. After this step, step 1 begins again.
  3. Completion. - The task is fully completed (for example, it was expected that Webpack would start a certain number of times) or the start of the collector was stopped (using run.stop () ). In this case, the Observable destruction algorithm is executed , and it is cleared.

Conclusion


Here is a summary of what we learned in this publication:

  1. We provide a new API that will allow developers to change the behavior of Angular CLI commands and add new ones using assemblers that implement the necessary logic.
  2. Collectors can be synchronous, asynchronous, and responsive to external events. They can be called multiple times, as well as by other collectors.
  3. The parameters that the collector receives when the task starts are first read from the angular.json file , then they are overwritten by the parameters from the configuration, if any, and then overwritten by the command line flags if they were added.
  4. The recommended way to test collectors is through integration tests, however you can perform unit testing separately from the collector logic.
  5. If the collector returns an Observable, it should be cleared after passing through the destruction algorithm.

In the near future, the frequency of use of these APIs will increase. For example, the Bazel implementation is strongly associated with them.

We already see how the community creates new CLI collectors for use, for example, jest and cypress for testing.

Also popular now: