Node.js Part 4: npm guide, package.json and package-lock.json files

Today we publish the fourth part of the translation guide for Node.js. In this article we will start talking about npm and also consider the features of files package.jsonand package-lock.json.

Basics npm

Npm (node ​​package manager) is the Node.js package manager. In the first part of this material, we already mentioned that now there are more than half a million packages in npm, which makes it the world's largest repository of code written in one language. This suggests that in npm you can find packages designed to solve almost any problem.

Initially, npm was created as a package management system for Node.js, but today it is also used in the development of JavaScript front-end projects. To interact with the npm registry, the same-name command is used, which gives the developer a huge number of possibilities.

▍Download packages

Using the command, npmyou can download packages from the registry. Below we look at examples of its use.

▍Install all project dependencies

If the project has a file package.json, then you can install all the dependencies of this project with the following command:

npm install

This command will load everything the project needs and put these materials in a folder node_modules, creating it if it does not exist in the project directory.

▍ Installing a separate package

Separate can be set with the following command:

npm install <package-name>

You can often see how this command is used not in this simple form, but with some flags. Consider them:

• The flag --saveallows you to install a package and add an entry about it to the section of the dependenciesfile package.jsonthat describes the dependencies of the project. These dependencies are used by the project to implement its main functionality, they are installed during its deployment on the server (after npm 5 is released, records about installed packages in the dependency section are made automatically without using this flag).
• The flag --save-devallows you to install a package and add an entry about it to a section containing a list of development dependencies (that is, packages that are needed during project development, such as libraries for testing, but not required for its operation) of a file package.jsoncalled devDependencies.

▍ Pack Updates

To update packages, use the following command:

npm update

After receiving this command, npm will check all packages for the presence of their new versions, and if it finds new versions that meet the restrictions on the package versions specified in package.json, install them.

You can also upgrade a separate package:

npm update <package-name>

▍Download packages of certain versions

In addition to the standard package downloads, npm also supports downloading of certain versions. In particular, it can be noted that some libraries are compatible only with certain major releases of other libraries, that is, if the dependencies of such libraries were installed without taking into account versions, this could disrupt their work. The ability to install a certain version of a certain package is also useful in situations where, for example, the most recent release of this package suits you perfectly, but it turns out that there is an error in it. Waiting for the release of the revised version of the package, you can use its older but stable release.

The ability to specify specific versions of the required libraries for a project is useful in team development, when all team members use exactly the same libraries. The transition to their new versions is also carried out centrally, by making changes to the project file package.json.

In all these cases, the ability to specify the package versions required by the project is extremely useful. Npm follows the standard of semantic versioning (semver).

▍ Run scripts

The file package.jsonsupports the ability to describe commands (scripts), which can be run using this construct:

npm <task-name>

For example, here’s what the list of scripts looks like in the corresponding section of the file:

{
  "scripts": {
    "start-dev": "node lib/server-development",
    "start": "node lib/server-production"
  }
}

It is quite common to use this feature to launch a webpack:

{
  "scripts": {
    "watch": "webpack --watch --progress --colors --config webpack.conf.js",
    "dev": "webpack --progress --colors --config webpack.conf.js",
    "prod": "NODE_ENV=production webpack -p --config webpack.conf.js",
  }
}

This approach makes it possible to replace the input of long commands, fraught with errors, with the following simple constructions:

$ npm watch
$ npm dev
$ npm prod

▍ Where does npm install packages?

When installing packages using npm (or yarn ), two installation options are available: local and global.

By default, when using a command like to install a package npm install lodash, the package is in a folder node_moduleslocated in the project folder. In addition, if the above command was executed, npm will also add a library entry lodashto the dependenciesfile section package.jsonthat is in the current directory.

Global installation of packages is performed using the flag -g:

npm install -g lodash

By executing such a command, npm does not install the package in the local project folder. Instead, it copies package files to a global location. Where exactly do these files go?

To find out, use the following command:

npm root -g

On macOS or Linux, package files may be in a directory /usr/local/lib/node_modules. On Windows, this may be something like C:\Users\YOU\AppData\Roaming\npm\node_modules.

However, if you are using the version management of Node.js nvm, the path to the global packages folder may change.

For example, I use nvm, and the above command tells me that global packages are installed at that address: /Users/flavio/.nvm/versions/node/v8.9.0/lib/node_modules.

▍Using and executing packages installed using npm

How to use modules installed using npm, locally or globally, that fall into folders node_modules? Suppose you have installed a popular library lodashthat contains many support functions used in JavaScript development:

npm install lodash

This command will install the library in the local project folder node_modules.

In order to use it in your code, it is enough to import it using the command require:

const _ = require('lodash')

What if the package is an executable file?

In this case, the executable file will fall into the folder node_modules/.bin/ folder.

You can look at how the work of this mechanism looks like by installing the cowsay package . It is a comic program written for the command line. If you transfer to this package any text, in the console, in the style of ASCII art, an image of a cow will be displayed that “pronounces” the corresponding text. Other creatures can “voice” the text.

So, after installing the package using the command npm install cowsay, it, along with its dependencies, will fall into node_modules. And in the hidden folder .binwill be written symbolic links to binary files cowsay.

How to do them?

Of course, you can, to call the program, enter something like in the terminal ./node_modules/.bin/cowsay, this is a working approach, but it is much better to use npx , a tool for running executable files of npm-packages, included in npm starting with version 5.2. Namely, in our case we need the following command:

npx cowsay

The path to the npx package will find automatically.

Package.json file

The file package.jsonis the most important element of a multitude of projects based on the Node.js ecosystem. If you programmed in JavaScript, whether it was server-side or client-side development, then you probably already encountered this file. Why is it needed? What should you know about him and what opportunities does he give you?

Package.jsonis something like a manifest file for a project. He gives the developer a lot of diverse opportunities. For example, it is the central repository of settings for the tools used in the project. In addition, it is the place where npm and yarn record information about the names and versions of installed packages.

ФайлаFile structure

Here is an example of the simplest file package.json:

{
}

As you can see, it is empty. There are no strict requirements regarding what should be present in such a file for some application. The only requirement for the file structure is that it must follow the rules of the JSON format. Otherwise, this file cannot be read by programs that attempt to access its contents.

If you create a Node.js package that you are going to distribute through npm, then everything changes radically, and you package.jsonshould have a set of properties that will help other people use the package. We will talk more about this later.

Here is another example package.json:

{
  "name": "test-project"
}

It contains a property namewhose value is the name of the application or package whose materials are contained in the same folder as this file.

Here is a more complicated example, which I took from an example application written using Vue.js:

{
  "name": "test-project",
  "version": "1.0.0",
  "description": "A Vue.js project",
  "main": "src/main.js",
  "private": true,
  "scripts": {
    "dev": "webpack-dev-server --inline --progress --config build/webpack.dev.conf.js",
    "start": "npm run dev",
    "unit": "jest --config test/unit/jest.conf.js --coverage",
    "test": "npm run unit",
    "lint": "eslint --ext .js,.vue src test/unit",
    "build": "node build/build.js"
  },
  "dependencies": {
    "vue": "^2.5.2"
  },
  "devDependencies": {
    "autoprefixer": "^7.1.2",
    "babel-core": "^6.22.1",
    "babel-eslint": "^8.2.1",
    "babel-helper-vue-jsx-merge-props": "^2.0.3",
    "babel-jest": "^21.0.2",
    "babel-loader": "^7.1.1",
    "babel-plugin-dynamic-import-node": "^1.2.0",
    "babel-plugin-syntax-jsx": "^6.18.0",
    "babel-plugin-transform-es2015-modules-commonjs": "^6.26.0",
    "babel-plugin-transform-runtime": "^6.22.0",
    "babel-plugin-transform-vue-jsx": "^3.5.0",
    "babel-preset-env": "^1.3.2",
    "babel-preset-stage-2": "^6.22.0",
    "chalk": "^2.0.1",
    "copy-webpack-plugin": "^4.0.1",
    "css-loader": "^0.28.0",
    "eslint": "^4.15.0",
    "eslint-config-airbnb-base": "^11.3.0",
    "eslint-friendly-formatter": "^3.0.0",
    "eslint-import-resolver-webpack": "^0.8.3",
    "eslint-loader": "^1.7.1",
    "eslint-plugin-import": "^2.7.0",
    "eslint-plugin-vue": "^4.0.0",
    "extract-text-webpack-plugin": "^3.0.0",
    "file-loader": "^1.1.4",
    "friendly-errors-webpack-plugin": "^1.6.1",
    "html-webpack-plugin": "^2.30.1",
    "jest": "^22.0.4",
    "jest-serializer-vue": "^0.3.0",
    "node-notifier": "^5.1.2",
    "optimize-css-assets-webpack-plugin": "^3.2.0",
    "ora": "^1.2.0",
    "portfinder": "^1.0.13",
    "postcss-import": "^11.0.0",
    "postcss-loader": "^2.0.8",
    "postcss-url": "^7.2.1",
    "rimraf": "^2.6.0",
    "semver": "^5.3.0",
    "shelljs": "^0.7.6",
    "uglifyjs-webpack-plugin": "^1.1.1",
    "url-loader": "^0.5.8",
    "vue-jest": "^1.0.2",
    "vue-loader": "^13.3.0",
    "vue-style-loader": "^3.0.1",
    "vue-template-compiler": "^2.5.2",
    "webpack": "^3.6.0",
    "webpack-bundle-analyzer": "^2.9.0",
    "webpack-dev-server": "^2.9.1",
    "webpack-merge": "^4.1.0"
  },
  "engines": {
    "node": ">= 6.0.0",
    "npm": ">= 3.0.0"
  },
  "browserslist": [
    "> 1%",
    "last 2 versions",
    "not ie <= 8"
  ]
}

As you can see, here is really nemeryanno everything interesting. Namely, the following properties can be distinguished here:

• name - sets the name of the application (package).
• version - contains information about the current version of the application.
• description - a brief description of the application.
• main - sets the entry point to the application.
• private- if this property is set to true, this prevents the package from being accidentally published to npm.
• scripts - sets a set of Node.js scripts that can be run.
• dependencies - contains a list of npm-packages on which the application depends.
• devDependencies - contains a list of npm-packages used in the development of the project, but not in its actual work.
• engines - sets the list of Node.js versions on which the application runs.
• browserlist - used to store the list of browsers (and their versions) that the application should support.

All of these properties are used either by npm or by other tools used during the application life cycle.

▍Properties used in package.json

Talk about properties that can be used in package.json. Here we will use the term “package”, but everything that is said about packages is also true for local applications that are not planned to be used as packages.

Most of the properties that we describe are used only for the needs of the npm repository , some are used by programs that interact with code like the same npm.

Name property

The property namesets the package name:

"name": "test-project"

The name must be shorter than 214 characters, must not include spaces, must consist only of capital letters, hyphens ( -) and underscores ( _).

Such restrictions exist because when a package is published to npm, its name is used to form the URL of the package page.

If you publish the package code on GitHub, in a public domain, then the name of the corresponding GitHub repository is a good option for the package name.

Property author

The property authorcontains information about the package author:

{
  "author": "Flavio Copes <flavio@flaviocopes.com> (https://flaviocopes.com)"
}

It can be presented in this format:

{
  "author": {
    "name": "Flavio Copes",
    "email": "flavio@flaviocopes.com",
    "url": "https://flaviocopes.com"
  }
}

Property contributors

The property contributorscontains an array with information about the people who contributed to the project:

{
  "contributors": [
    "Flavio Copes <flavio@flaviocopes.com> (https://flaviocopes.com)"
  ]
}

This property may look like this:

{
  "contributors": [
    {
      "name": "Flavio Copes",
      "email": "flavio@flaviocopes.com",
      "url": "https://flaviocopes.com"
    }
  ]
}

Bugs property

The property bugscontains a link to the project's bug tracker. It is very likely that such a link will lead to the GitHub bug tracking system page:

{
  "bugs": "https://github.com/flaviocopes/package/issues"
}

Property homepage

This property homepageallows you to set the package home page:

{
  "homepage": "https://flaviocopes.com/package"
}

Version property

This property versioncontains information about the current version of the package:

"version": "1.0.0"

When forming the value of this property, one should follow the rules of semantic versioning . This means, in particular, that the version number is always represented by three digits: xxx

The first number is the major version of the package, the second is the minor version, and the third is the patch version.

Changing these numbers carries a certain meaning. Thus, the release of the package, which only fixes errors, leads to an increase in the value of the patch version. If the release of the package comes out, the changes made in which are distinguished by backward compatibility with the previous release - then the minor version changes. Major versions of packages may contain changes that make these packages incompatible with packages of previous major versions.

License property

The property licensecontains information about the license package:

"license": "MIT"

Property keywords

The property keywordscontains an array of keywords related to the functionality of the package:

"keywords": [
  "email",
  "machine learning",
  "ai"
]

Proper selection of keywords helps people find what they need when searching for packages to solve certain problems, allows them to group packages and quickly evaluate their possible functionality when browsing the npm site.

Property description

The property descriptioncontains a brief description of the package:

"description": "A package to work with strings"

This property is especially important if you plan to publish the package to npm, as it allows users of the npm site to understand the purpose of the package.

Repository property

The property repositoryindicates where the package repository is located:

"repository": "github:flaviocopes/testing",

Note that the value of this property has a prefix github. Npm supports prefixes for some other popular services of this kind:

"repository": "gitlab:flaviocopes/testing",
"repository": "bitbucket:flaviocopes/testing",

The version control system used when developing a package can also be specified in the explicit form:

"repository": {
  "type": "git",
  "url": "https://github.com/flaviocopes/testing.git"
}

The same package can use different version control systems:

"repository": {
  "type": "svn",
  "url": "..."
}

Main property

The property mainsets the entry point to the package:

"main": "src/main.js"

When a package is imported into an application, this is where the search will be made for what the corresponding module is exporting.

Private property

The property privateset trueto prevent the package from being accidentally published to npm:

"private": true

Scripts property

This property scriptsspecifies a list of scripts or utilities that can be run using npm tools:

"scripts": {
  "dev": "webpack-dev-server --inline --progress --config build/webpack.dev.conf.js",
  "start": "npm run dev",
  "unit": "jest --config test/unit/jest.conf.js --coverage",
  "test": "npm run unit",
  "lint": "eslint --ext .js,.vue src test/unit",
  "build": "node build/build.js"
}

These scripts are command line applications. They can be run using npm or yarn, executing, respectively, commands of the form npm run XXXXor yarn XXXX, where XXXXis the name of the script. For example, it may look like this:

npm run dev

Scripts can be called as you want, they can do almost everything that a developer can wish for.

Dependencies property

The property dependenciescontains the list of npm-packages installed as package dependencies:

"dependencies": {
  "vue": "^2.5.2"
}

When installing a package using npm or yarn, the following commands are used:

npm install <PACKAGENAME>
yarn add <PACKAGENAME>

These packages are automatically added to the list of dependencies of the developed package.

DevDependencies property

The property devDependenciescontains a list of npm packages installed as development dependencies:

"devDependencies": {
  "autoprefixer": "^7.1.2",
  "babel-core": "^6.22.1"
}

This list is different from the one that is stored in the property dependencies, since the packages it contains are installed only in the package developer’s system; they are not used in practical use of the package.

Packages fall into this list when they are installed using npm or yarn, performed as follows:

npm install --dev <PACKAGENAME>
yarn add --dev <PACKAGENAME>

Property engines

This property enginesindicates which versions of Node.js and other software products are used to make the package work:

"engines": {
  "node": ">= 6.0.0",
  "npm": ">= 3.0.0",
  "yarn": "^0.13.0"
}

Browserlist property

This property browserlistallows you to report which browsers (and their versions) the developer of the package is going to support:

"browserslist": [
  "> 1%",
  "last 2 versions",
  "not ie <= 8"
]

Babel, Autoprefixer and other tools use this feature. The analysis of this list allows them to add to the package only those polyfills and auxiliary mechanisms that are needed for the listed browsers.

The property value shown here as an example browserlistmeans that you want to support at least 2 major versions of all browsers with at least 1% of use (this data is taken from CanIUse.com ), except for IE 8 and older versions of this browser (more about this can be found on the browserlists package page ).

▍Saving in package.json settings for various software tools

You package.jsoncan store settings for various auxiliary tools like Babel or ESLint.

Each of these tools corresponds to a particular property, like eslintConfigor babel. Details on the use of such properties can be found in the documentation of relevant projects.

▍About package versions and semantic versioning

In the examples above, you could see that the version numbers of the packages are not only defined as ordinary numbers separated by dots, but also using certain special characters. For example, in the form ~3.0.0or ^0.13.0. Here we use the so-called version specifiers, which determine the range of package versions that are suitable for use in our package.

Taking into account the fact that when using semantic versioning, all version numbers of packages consist of sequences representing three numbers, the meaning of which we spoke above, we will describe the following rules for using version specifiers:

• ~: if you specify a version in the form, ~0.13.0this means that you are only interested in the patch releases of the package. That is, the package 0.13.1will suit you, but 0.14.0not.
• ^: If the version number is specified in the form ^0.13.0, this means that new patch versions and minor versions of the package are suitable for you. That is, you will arrange a version of the package 0.13.1, 0.14.0and so on.
• *: using this symbol, you inform the system that you will be satisfied with any fresh versions of the package, including its new major releases.
• >: Any versions of the package that are larger than the specified one are suitable.
• >=: Any package versions that are equal to or greater than the specified one are suitable.
• <=: You will be satisfied with packages whose versions are equal to the specified one or less.
• <: you are interested in packages whose versions are less than the specified.
• =: you only need the specified version of the package.
• -: used to specify a range of suitable versions, for example - 2.1.0 - 2.6.2.
• ||: allows you to combine sets of package related conditions. For example, it might look like < 2.1 || > 2.6.

There are some more rules:

• no extra characters: if the version number of the package without additional characters is used, it means that your package only needs the specified version of the dependency package and no other.
• latest: indicates that you need the most recent version of a certain package.

Most of the above qualifiers can be combined, for example, by specifying ranges of suitable versions of dependency packages. For example, a type construct 1.0.0 || >=1.1.0 <1.2.0indicates that it is planned to use either a package 1.0.0version or a version whose number is greater than or equal to 1.1.0, but smaller 1.2.0.

File package-lock.json

The file has been package-lock.jsonused since the advent of npm version 5. It is created automatically when you install Node.js packages. What is this file? Perhaps you are not familiar with him even if you knew about him package.json, which exists much longer than him.

The purpose of this file is to track the exact versions of the installed packages, which allows the developed product to be 100% reproducible in its original form, even if those who support the packages have updated them.

This file solves a very specific problem that cannot be solved by means package.json. You package.jsoncan specify which updates of a certain package are suitable for you (patch versions or minor versions) using the above version specifiers.

Git doesn't commit foldernode_modules, as it usually has huge dimensions. When you try to recreate a project on another computer, using the command npm installwill result in that if using the specifier ~as applied to a version of a package, its patch release was released, the package that was used during the development will be installed, but this patch release.

The same goes for the specifier ^. If, while specifying the package version, the specifiers were not used, then the specified version will be installed and the problem in question will be irrelevant in this situation.

So, someone is trying to initialize the project using the commandnpm install. When new versions of packages are released, it will turn out that this project is different from the original one. Even if following the rules of semantic versioning, minor releases and patch releases should not contain changes that prevent backward compatibility, we all know that bugs are able to penetrate (and penetrate) anywhere.

The file package-lock.jsonkeeps unchanged the version information of each installed package, and npm will use these versions of the packages when executing the command npm install.

This concept is not new, package managers used in other programming languages ​​(like the Composer manager in PHP) have been using a similar system for many years.

Filepackage-lock.jsonYou need to send it to a Git repository, which will allow other people to download it if the project is publicly available, or when a team of programmers is developing it, or if you use Git to deploy the project.

Versions of dependencies will be updated in package-lock.jsonafter the command is executed npm update.

▍Sample package-lock.json file

This example shows the structure of the file package-lock.jsonthat is included with the cowsay package installed in an empty folder with the npm command install cowsay:

{
  "requires": true,
  "lockfileVersion": 1,
  "dependencies": {
    "ansi-regex": {
      "version": "3.0.0",
      "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-3.
0.0.tgz",
      "integrity": "sha1-7QMXwyIGT3lGbAKWa922Bas32Zg="
    },
    "cowsay": {
      "version": "1.3.1",
      "resolved": "https://registry.npmjs.org/cowsay/-/cowsay-1.3.1.tgz"
,
      "integrity": "sha512-3PVFe6FePVtPj1HTeLin9v8WyLl+VmM1l1H/5P+BTTDkM
Ajufp+0F9eLjzRnOHzVAYeIYFF5po5NjRrgefnRMQ==",
      "requires": {
        "get-stdin": "^5.0.1",
        "optimist": "~0.6.1",
        "string-width": "~2.1.1",
        "strip-eof": "^1.0.0"
      }
    },
    "get-stdin": {
      "version": "5.0.1",
      "resolved": "https://registry.npmjs.org/get-stdin/-/get-stdin-5.0.
1.tgz",
      "integrity": "sha1-Ei4WFZHiH/TFJTAwVpPyDmOTo5g="
    },
    "is-fullwidth-code-point": {
      "version": "2.0.0",
      "resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/
is-fullwidth-code-point-2.0.0.tgz",
      "integrity": "sha1-o7MKXE8ZkYMWeqq5O+764937ZU8="
    },
    "minimist": {
      "version": "0.0.10",
      "resolved": "https://registry.npmjs.org/minimist/-/minimist-0.0.10
.tgz",
      "integrity": "sha1-3j+YVD2/lggr5IrRoMfNqDYwHc8="
    },
    "optimist": {
      "version": "0.6.1",
      "resolved": "https://registry.npmjs.org/optimist/-/optimist-0.6.1.tgz",
      "integrity": "sha1-2j6nRob6IaGaERwybpDrFaAZZoY=",
      "requires": {
        "minimist": "~0.0.1",
        "wordwrap": "~0.0.2"
      }
    },
    "string-width": {
      "version": "2.1.1",
      "resolved": "https://registry.npmjs.org/string-width/-/string-width-2.1.1.tgz",
      "integrity": "sha512-nOqH59deCq9SRHlxq1Aw85Jnt4w6KvLKqWVik6oA9ZklXLNIOlqg4F2yrT1MVaTjAqvVwdfeZ7w7aCvJD7ugkw==",
      "requires": {
        "is-fullwidth-code-point": "^2.0.0",
        "strip-ansi": "^4.0.0"
      }
    },
    "strip-ansi": {
      "version": "4.0.0",
      "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-4.0.0.tgz",
      "integrity": "sha1-qEeQIusaw2iocTibY1JixQXuNo8=",
      "requires": {
        "ansi-regex": "^3.0.0"
      }
    },
    "strip-eof": {
      "version": "1.0.0",
      "resolved": "https://registry.npmjs.org/strip-eof/-/strip-eof-1.0.0.tgz",
      "integrity": "sha1-u0P/VZim6wXYm1n80SnJgzE2Br8="
    },
    "wordwrap": {
      "version": "0.0.3",
      "resolved": "https://registry.npmjs.org/wordwrap/-/wordwrap-0.0.3.tgz",
      "integrity": "sha1-o9XabNXAvAAI03I0u68b7WMFkQc="
    }
  }
}

Let's sort this file. We install the cowsay package, which depends on the following packages:

• get-stdin
• optimist
• string-width
• strip-eof

These packages, in turn, depend on other packages, information about which we can learn from the properties requiresthat some of them have:

• ansi-regex
• is-fullwidth-code-point
• minimist
• wordwrap
• strip-eof

They are added to the file in alphabetical order, each has a field version, there is a field resolvedindicating the location of the package, and a string property integritythat can be used to check the integrity of the package.

Results

Today we started talking about npm and figured out the structure and purpose of the files package.jsonand package-lock.json. Next time, we’ll continue to analyze the capabilities of npm and touch on using npx.

Dear readers! Which package manager do you prefer - npm or yarn?