Another tool to check your npm dependencies - wtfwith

    Have you ever wondered how many versions of the same library your client or server assembly is dragging on? At one point, it became interesting to me. Offhand finding a ready-made tool for this did not work, and looking through the eyes of package-lock is too tiring. As we know, in any incomprehensible situation you need to write your npm package, so I did just that ... Further in the post I will consider the result of the analysis of a live project and draw a couple of controversial conclusions.

    Well, you can’t do without this classic picture:


    How does it work


    In general, no rocket science - we take package.json and package-lock.json and go through all the dependencies that are there, collecting identical modules or fork modules. Forks, of course, have to be written with your hands, so now only lodash and undescore are listed in them. But you can always help and expand this list.



    How to use


    The module is published on npm, it is easiest to run using npx , although no one bothers you to install it locally or even globally. Next I will consider the option of running through npx. For example, I’ll take one of my old projects, which is essentially a monolith on a node (this was before the microservice boom, and then sawing it was lazy and impractical).

    1. npx wtfwith moduleName(for example, npm wtfwith lodash) - we check the number of occurrences of a certain module, displays the results if there are more than two. The result can be, for example, such:

    example
    npx wtfwith lodash
    npx: installed 3 in 2.047s
    Searching for lodash
    Checking path /web/xxx/package-lock.json
    Huston, we have a problem:

    30 versions of lodash:
    - 2.4.2 from root, cheerio@0.18.0
    - 3.10.1 from xmlbuilder@2.6.4
    - 4.0.0 from twilio@3.15.0
    - 4.17.4 from graphlib@2.1.1, sway@1.0.0
    - 4.17.5 from aws-sdk@2.211.0, xmlbuilder@4.2.1, request-promise-core@1.1.1, requestretry@1.13.0
    - lodash.assign:4.2.0 from ioredis@3.2.2
    - lodash.bind:4.2.1 from ioredis@3.2.2
    - lodash.clone:4.5.0 from ioredis@3.2.2
    - lodash.clonedeep:4.5.0 from ioredis@3.2.2
    - lodash.defaults:4.2.0 from ioredis@3.2.2
    - lodash.difference:4.5.0 from ioredis@3.2.2
    - lodash.flatten:4.4.0 from ioredis@3.2.2
    - lodash.foreach:4.5.0 from ioredis@3.2.2
    - lodash.get:4.4.2 from z-schema@3.18.4
    - lodash.includes:4.3.0 from jsonwebtoken@8.2.1
    - lodash.isboolean:3.0.3 from jsonwebtoken@8.2.1
    - lodash.isempty:4.4.0 from ioredis@3.2.2
    - lodash.isequal:4.5.0 from z-schema@3.18.4
    - lodash.isinteger:4.0.4 from jsonwebtoken@8.2.1
    - lodash.isnumber:3.0.3 from jsonwebtoken@8.2.1
    - lodash.isplainobject:4.0.6 from jsonwebtoken@8.2.1
    - lodash.isstring:4.0.1 from jsonwebtoken@8.2.1
    - lodash.keys:4.2.0 from ioredis@3.2.2
    - lodash.noop:3.0.1 from ioredis@3.2.2
    - lodash.once:4.1.1 from jsonwebtoken@8.2.1
    - lodash.partial:4.2.1 from ioredis@3.2.2
    - lodash.pick:4.4.0 from ioredis@3.2.2
    - lodash.sample:4.2.1 from ioredis@3.2.2
    - lodash.shuffle:4.2.0 from ioredis@3.2.2
    - lodash.values:4.3.0 from ioredis@3.2.2

    Advice: Sometimes it is good to make a fresh start: rm ./ -rf && git commit -am 'nevermore' && git push origin master


    2. npx wtfwith everything(you can simply omit the argument and write “npx wtfwith”, but this is depressing) - it checks generally all the dependencies, which are more than two versions. Maybe, for example, this result:

    Example
    npx wtfwith lodash
    npx: installed 3 in 1.813s
    Searching all strange things...
    Checking path /web/xxx/package-lock.json
    Huston, we have a problem:

    30 versions of lodash:
    // ну, это вы уже видели

    4 versions of punycode:
    - 1.2.4 from dkim-signer@0.1.2
    - 1.3.2 from url@0.10.3
    - 1.4.1 from tough-cookie@2.3.3
    - 2.1.0 from uri-js@3.0.2

    4 versions of xmlbuilder:
    - 0.4.2 from nodemailer@0.7.1, aws-sdk@2.0.5
    - 2.6.4 from root, xml2js@0.4.8
    - 4.2.1 from aws-sdk@2.211.0, xml2js@0.4.17
    - 9.0.1 from twilio@3.15.0

    3 versions of xmldom:
    - 0.1.19 from xml-crypto@0.10.1
    - 0.1.27 from pdf2json@0.6.2
    - 0.1.7 from ws.js@2.0.22

    3 versions of mime:
    - 1.2.11 from mailcomposer@0.2.12
    - 1.4.1 from superagent@3.8.0
    - 2.3.1 from root

    3 versions of sax:
    - 0.4.2 from xml2js@0.2.6
    - 0.6.1 from xml2js@0.4.8
    - 1.2.1 from aws-sdk@2.211.0, xml2js@0.4.17

    3 versions of uuid:
    - 1.4.2 from root
    - 3.1.0 from aws-sdk@2.211.0
    - 3.2.1 from request@2.83.0, request@2.85.0, requestretry@1.13.0, twilio@3.15.0

    3 versions of xml2js:
    - 0.2.6 from nodemailer@0.7.1, aws-sdk@2.0.5
    - 0.4.17 from aws-sdk@2.211.0
    - 0.4.8 from root

    3 versions of moment:
    - 2.19.3 from twilio@3.15.0
    - 2.20.1 from moment-timezone@0.5.14
    - 2.22.1 from root, joi@4.9.0, moment-range@1.0.9, moment-timezone@0.4.0, moment-timezone@0.5.15

    3 versions of readable-stream:
    - 1.0.34 from match-stream@0.0.2, pullstream@0.4.1, slice-stream@1.0.0, unzip@0.1.11
    - 1.1.14 from dicer@0.2.5, ftp@0.3.10, htmlparser2@3.8.3, nodemailer@0.7.1
    - 2.3.3 from mysql@2.14.1, superagent@3.8.0

    Advice: What about lunch?



    3. You can also specify an option --dev- in case, for example, if you are building a bundle of dependency maids. Although it’s better to use front-end tools for analyzing bundles. I will not show an example of use, the command looks like npx wtfwith everything --dev, and you can imagine about what the hell goes there.

    Real examples


    Of course, it would be interesting to analyze several popular packages, which I did.

    1. express - surprisingly, nothing interesting was found.

    2. gulp - all sorts of little things were found:

    looking?
    4 versions of kind-of:
    - 3.2.2 from is-accessor-descriptor@0.1.6, is-data-descriptor@0.1.4, is-number@3.0.0, make-iterator@1.0.0, object-copy@0.1.0, snapdragon-util@3.0.1, to-object-path@0.3.0
    - 4.0.0 from has-values@1.0.0
    - 5.1.0 from array-sort@1.0.0, class-utils@0.3.6, is-descriptor@0.1.6, default-compare@1.0.0, expand-brackets@2.1.4, snapdragon@0.8.2, static-extend@0.1.2
    - 6.0.2 from braces@2.3.1, is-accessor-descriptor@1.0.0, is-data-descriptor@1.0.0, is-descriptor@1.0.2, micromatch@3.1.10, nanomatch@1.2.9, use@3.1.0

    3 versions of define-property:
    - 0.2.5 from class-utils@0.3.6, expand-brackets@2.1.4, object-copy@0.1.0, snapdragon@0.8.2, static-extend@0.1.2
    - 1.0.0 from base@0.11.2, braces@2.3.1, extglob@2.0.4, snapdragon-node@2.1.1
    - 2.0.2 from micromatch@3.1.10, nanomatch@1.2.9, to-regex@3.0.2



    3. npm is a pretty funny result:
    look!
    14 versions of lodash:
    - 3.10.1 from cli-table2@0.2.0
    - lodash._baseindexof:3.1.0 from root
    - lodash._baseuniq:4.6.0 from root
    - lodash._bindcallback:3.0.1 from root
    - lodash._cacheindexof:3.0.2 from root
    - lodash._createcache:3.1.2 from root
    - lodash._createset:4.0.3 from lodash._baseuniq@4.6.0
    - lodash._getnative:3.9.1 from root, lodash._createcache@3.1.2
    - lodash._root:3.0.1 from lodash._baseuniq@4.6.0
    - lodash.clonedeep:4.5.0 from root
    - lodash.restparam:3.6.1 from root
    - lodash.union:4.6.0 from root
    - lodash.uniq:4.5.0 from root
    - lodash.without:4.4.0 from root

    3 versions of mississippi:
    - 1.3.1 from make-fetch-happen@2.6.0
    - 2.0.0 from cacache@10.0.4
    - 3.0.0 from root, pacote@7.6.1

    3 versions of pump:
    - 1.0.3 from mississippi@1.3.1
    - 2.0.1 from mississippi@2.0.0, pumpify@1.4.0
    - 3.0.0 from mississippi@3.0.0


    At least it’s funny that npm is clearly trying to minimize the use of lodas, but one of its dependencies is completely overwhelming it ...

    Todo


    The project was written overnight on the knee, so there are a number of things that are not implemented there:

    1. There is no work with the shrinkwrap file - but this is not very necessary, you can just generate package-lock for verification;
    2. Only node 6+ is supported. Also not very critical, because there is nvm;
    3. The larn file from yarn is not supported. Again not too critical;
    4. Surely there are a number of bugs and inaccuracies.

    In general, if the tool is relevant to you, then pull requests are welcome.

    conclusions


    The results do not look very nice, but to be honest, I expected more hell. Apparently, many people still use tools for analyzing updates and dependency protection.

    From the controversial recommendations:

    1. I would not recommend for deliberately back-end modules to tighten dependencies like lodash in pieces - anyway, someone will surely drag out the full version of the library. Well, do not forget about semantic versioning - of course, without exact version restrictions everything may break at some point, but this should be solved by fixing dependencies in package-lock and interaction between developers, and not by exact restrictions. That is, if you find breaking change in some kind of dependence, it would be nice to go to the author of the package and say that it is wrong - this will save others a lot of effort and terabytes of disk space.

    Although here the author respected by me ioredis does exactly the opposite .

    2. It is worth periodically running through your dependencies and looking at what is suspicious in them, and then unsubscribing issues or doing pull requests to problem modules.

    If the problem of updating dependencies is close to you, then in the end I can advise several useful tools:

    1. npm-check-updates - we quickly look at package version updates, we quickly install them. Just be careful!
    2. snyk - checking the security of packages, I already wrote about it a little earlier .
    3. node security project - another tool for checking dependency security, which can be considered native to the ecosystem, because it was recently acquired by npm.
    4. In case you are working with the front, you are probably using something like webpack-bundle-analyzer, which will tell you what exactly you got as a result. But I don’t work with the front, so I can’t give any clear recommendations.
    5. It is also useful to have all sorts of badges and CI integrations that will clearly show and notify you if something is out of date or has become unsafe. I will not give examples of badges, there are a lot of them, and you can look at a part on shields.io . There are many integrations too, and if you have an open source project, then travis-ci can run almost any kind of test.

    Such posts appear periodically, but time goes by - so tell us what useful checks and tools you use in your projects now.

    Also popular now: