NPM 5, NPM 6, Yarn: real world installation times comparisons

Tests do not show the expected NPM@6 performance enhancements

6 min readMay 9, 2018

Context

Versions
Node: v9.11.1 | NPM: 5.6.0 | NPM: 6.0.0 | Yarn: 1.6.0

Measures
I used time command to measure the running time of each installation. More information about the outputs of this command here.
Edit: to get a formatted output, I used gnu-time.

Process

The first installation is done with no cache, no lock files (package-lock.json and yarn.lock deleted) and node_modules directory is deleted
The second installation is done with no cache but keeping lock files package-lock.json or yarn.lock and node_modules directory is deleted
The last installation is done keeping the cache and the lock files, just removing node_modules directory.

Script
I made a bash script available here to run these steps and get all the outputs in a file. It requires NMV, and gnu-time to be installed.

The output for each project is as follows (here: a professional project with
#babel #enzyme #webpack #jest #sass # lodash #eslint #stylelint #react #react-redux #hammerjs #counterpart etc):

unlogged-front
2018-05-07 14:41:13
...
NVM 0.33.8
************************************************


================================================
INSTALLATION WITH NPM 5
===============================================
Node version: v9.11.1
NPM version: 5.6.0

REAL		USER		SYS	
------------------------------------------------
1:52.18		81.31		33.01

REAL		USER		SYS	
------------------------------------------------
1:45.59		64.44		27.51

REAL		USER		SYS	
------------------------------------------------
1:10.27		44.60		20.45

================================================
INSTALLATION WITH NPM 6
================================================
Node version: v9.11.1
NPM version: 6.0.0

REAL		USER		SYS	
------------------------------------------------
1:44.79		87.17		31.38

REAL		USER		SYS	
------------------------------------------------
1:13.30		65.84		25.30

REAL		USER		SYS	
------------------------------------------------
1:06.67		51.53		19.91

================================================
INSTALLATION WITH YARN 
================================================
Node version: v9.11.1
Yarn version: 1.6.0

REAL		USER		SYS	
------------------------------------------------
5:03.54		54.32		44.33

REAL		USER		SYS	
------------------------------------------------
4:14.89		45.39		41.78

REAL		USER		SYS	
------------------------------------------------
0:29.48		20.14		23.28


>>> ALL DONE!

All detailed outputs for each project are available here.

Real time results

I used a script to get all results compiled in a file with only the real time for each installation on each projects. Here are the results:

=================================
1. fbrgsmn.mobile.app
=================================
NPM5: 0:46.24 - 0:30.96 - 0:29.19
NPM6: 0:45.08 - 0:32.31 - 0:30.07
YARN: 1:23.84 - 0:42.34 - 0:25.14=================================
2. nktest
=================================
NPM5: 0:02.21 - 0:01.81 - 0:01.62
NPM6: 0:02.40 - 0:02.09 - 0:01.78
YARN: 0:01.87 - 0:01.70 - 0:00.84=================================
3. expensify-app
=================================
NPM5: 1:37.74 - 1:30.33 - 1:28.81
NPM6: 1:38.97 - 1:38.31 - 1:38.17
YARN: 1:47.59 - 1:49.31 - 1:30.16=================================
4. indecision-app
=================================
NPM5: 1:44.38 - 1:36.41 - 1:34.80
NPM6: 1:46.98 - 1:36.99 - 1:34.42
YARN: 1:38.43 - 1:36.90 - 1:30.39=================================
5. d3-ionic-examples
=================================
NPM5: 0:25.13 - 0:14.93 - 0:13.11
NPM6: 0:24.45 - 0:17.23 - 0:13.95
YARN: 0:55.55 - 0:33.53 - 0:16.47=================================
6. demo-rellax
=================================
NPM5: 0:42.31 - 0:24.09 - 0:22.13
NPM6: 0:33.49 - 0:26.13 - 0:23.84
YARN: 0:40.21 - 0:35.91 - 0:24.67=================================
7. ion-prism
=================================
NPM5: 0:08.87 - 0:06.83 - 0:06.29
NPM6: 0:09.28 - 0:07.23 - 0:06.51
YARN: 0:20.28 - 0:12.69 - 0:06.56=================================
8. ion-prism-app-example
=================================
NPM5: 1:47.16 - 1:34.38 - 1:34.91
NPM6: 1:44.31 - 1:37.60 - 1:37.44
YARN: 2:02.53 - 1:47.53 -=================================
9. occurences
=================================
NPM5: 0:05.12 - 0:03.73 - 0:03.13
NPM6: 0:05.67 - 0:04.16 - 0:03.47
YARN: 0:05.35 - 0:04.71 - 0:02.21=================================
10. react-js-chatbox
=================================
NPM5: 0:34.43 - 0:23.24 - 0:18.84
NPM6: 0:37.47 - 0:25.50 - 0:22.67
YARN: 0:39.28 - 0:27.79 - 0:14.14=================================
11. react-js-markdown-editor
=================================
NPM5: 0:33.02 - 0:18.61 - 0:17.92
NPM6: 0:33.88 - 0:19.90 - 0:18.94
YARN: 0:33.26 - 0:31.28 - 0:18.81=================================
12. react-js-recipe-box
=================================
NPM5: 0:37.18 - 0:21.64 - 0:20.43
NPM6: 0:37.07 - 0:22.90 - 0:21.68
YARN: 0:42.21 - 0:35.52 - 0:18.30=================================
13. statsify
=================================
NPM5: 0:27.98 - 0:17.29 - 0:14.92
NPM6: 0:27.88 - 0:18.80 - 0:16.59
YARN: 0:56.34 - 0:31.39 - 0:15.22=================================
14. svg-loader-es6
=================================
NPM5: 0:16.08 - 0:10.23 - 0:08.92
NPM6: 0:16.61 - 0:11.58 - 0:10.05
YARN: 0:18.54 - 0:12.96 - 0:06.28=================================
15. svg-loader-es6-example
=================================
NPM5: 0:22.67 - 0:15.47 - 0:12.58
NPM6: 0:23.41 - 0:15.65 - 0:14.59
YARN: 0:25.04 - 0:18.74 - 0:10.44=================================
16. tictactoe-simple
=================================
NPM5: 0:27.56 - 0:18.55 - 0:16.69
NPM6: 0:28.43 - 0:20.29 - 0:18.04
YARN: 0:30.48 - 0:21.35 - 0:11.54=================================
17. to-go-zone
=================================
NPM5: 3:17.85 - 3:34.37 - 3:18.23
NPM6: 4:23.58 - 2:56.03 - 3:04.76
YARN: 3:29.28 - 3:19.27 - 1:35.30=================================
18. xbk
=================================
NPM5: 0:39.10 - 0:22.56 - 0:21.16
NPM6: 0:35.93 - 0:25.78 - 0:23.81
YARN: 0:41.91 - 0:30.99 - 0:16.26=================================
19. unlogged-api
=================================
NPM5: 0:50.86 - 0:34.52 - 0:31.22
NPM6: 0:47.87 - 0:34.08 - 0:32.52
YARN: 3:15.65 - 3:25.36 - 0:18.56=================================
20. unlogged-front
=================================
NPM5: 1:52.18 - 1:45.59 - 1:10.27
NPM6: 1:44.79 - 1:13.30 - 1:06.67
YARN: 5:03.54 - 4:14.89 - 0:29.48

Why these results are surprising?

This is not what I expected but yep ! This is what you see: NPM 5 is globally faster than NPM 6 (in each of the 3 installations).
And surprise ! Yarn is slower than NPM 5 except when there is npm cache.
However we can read that npm@6 should include benefits like performance enhancements (up to 17x). The evidence is that it’s not the case in real world and in real conditions of work!

It also seems that NPM@5 is faster than NPM@6 when there’s a package-lock.json file. But more important: NPM@6 losts 30 points when there is this package-lock.json !

I’ll continue to use Yarn

I’m a little bit disappointed because when I used Yarn the first time, it was pleasant to install packages very fastly. But now I see it’s not the fastest tool of all.

When NPM@6 arrived, I thought I could let it a chance. But day to day, the cache is more likely here (I don’t delete it every day!).
So Yarn still remains my favorite packages manager as it is faster 70% of the time, while NPM@6 is never the fastest in my tests!

These graphs show the installation times in milliseconds for each project.

Help me to improve the tests

I don’t understand these results! I checked my tools a lot of times to be sure my tries were running in the right environments. If people wanna use the script to get detailed results.

I’ll be happy to add it to the compiled results.