Technology13 minute read

A Guide to npm: The Node.js Package Manager

In this article, Toptal Software Engineer Martin Gouws explains advanced features of the Node Package Manager that should be in every JavaScript developer’s knowledge base.


Toptalauthors are vetted experts in their fields and write on topics in which they have demonstrated experience. All of our content is peer reviewed and validated by Toptal experts in the same field.

In this article, Toptal Software Engineer Martin Gouws explains advanced features of the Node Package Manager that should be in every JavaScript developer’s knowledge base.


Toptalauthors are vetted experts in their fields and write on topics in which they have demonstrated experience. All of our content is peer reviewed and validated by Toptal experts in the same field.
Martin Gouws
Verified Expert in Engineering

Martin is a young and talented JavaScript developer and software engineer who has his roots in enterprise Java.

Read More

Expertise

PREVIOUSLY AT

Discovery Limited, South Africa
Share

JavaScript is easily the most used language when it comes to the development of websites and web applications. The multitude of resources are astounding, and even more so, the number of libraries available.

At first, these libraries are few and easy to maintain; however, soon enough dependency hell sets in and a more mature solution is required.

npm is probably the most popular package manager for JavaScript.

Enter the Node Package Manager (npm) – a JavaScript package manager most notably used in conjunction with Node.js, although it can be used independently as well. It gives you exceptional control over your project’s dependencies and provides a great way to contribute to the open-source world.

You can get started by simply running npm install <package name> and injecting it into your JavaScript file.

Want to install a specific version? No problem. Run npm install <package name>@1.2.3.

Want to install a package globally (like Mocha, or Angular-CLI)? Just add a -g like so: npm install -g angular-cli mocha.

Admittedly, most use cases stop at an npm install, and there isn’t a need for anything else. However, npm has loads of additional features, which I’ll walk you through, highlighting those I consider essential, really useful, or just plain awesome.

CLI commands

The CLI is where users spend most of their time interacting with npm, and its help interface is actually helpful.

Querying help (npm help) spews out an entire array of options, and running npm help-search <searchText> gives you a list of search results direct from the npm markdown.

Here are the core commands that stand out.

  • install: It’s mentioned here because of its sheer necessity when working with npm. Used to either install a new package locally or globally (when adding -g) or to install dependencies listed in the package.json file (more on that later).

  • uninstall: This is also an essential. It’s used to purge a specific package from the node_modules directory either locally or globally (when adding -g).

  • access: This is the playground of npm user permission administrators within the context npm-organizations and scoped (private) packages. Seriously powerful stuff. Used in conjunction with adduser, owner, team, etc, it gives fine grained control over who has access to what.

  • bin: Where on earth are packages installed? Run this command to see the absolute file path.

  • cache: If you start installing packages from npm left, right, and center, this command is quite useful. Either call it with the ls subcommand to see a list of locally cached packages or with the clean subcommand to clear all packages that are in the cache. Back when the npm registry was still a bit unstable, this was essential to get back to a stable environment or to reset things when you didn’t properly set up npm permissions.

  • config: We will get into the different configuration options later, but this command deals primarily with persisting configuration properties in the local or global configuration file by using the set, get or delete subcommands.

  • dedupe or ddp: When working on a project over an extensive period of time and installing packages straight from npm, this command will walk the local package tree and attempt to simplify dependencies.

  • link: When you are developing your own npm package, this allows you to create a symlink to the global context so it can be tested as if it was installed globally from the npm registry. For example, if you are writing an assembly tool in node that has a CLI installed globally, you can run this command and test your CLI’s behavior without needing to deploy it first.

  • ls: It’s used to visualize package dependencies and their dependencies, in a tree structure. It’s cool to see and is also useful for comparisons with other projects.

  • outdated: This is used to evaluate the current state of installed dependencies and whether or not they are outdated. In projects where the root dependency list is hundreds of lines long, a manual check on packages is close to impossible. Adding -g --depth=0 to this command allows you to also check your globally installed packages.

  • publish: This command is essential when developing your own package for npm. It does exactly as the name suggests; it publishes your package to the npm registry.

  • search: Use this to search the registry for all packages containing the text provided in the third argument.

  • shrinkwrap: In short, this commands allows you to lock down specific dependency versions in a package in order, so that a relaxed semver (semantic versioning) number doesn’t break production code.

  • star: Do you really like a package you are using? Use this command to show your appreciation directly from the terminal, which then reflects on the package’s page on the npm registry.

  • update: This usually follows the outdated command to update any outdated packages.

  • version: This gives you a shorthand to bump the package.json version property, and do a git tag all in one.

Note that most of these commands can take subcommands and/or configurations, and this list is by no means an end-all discussion on the CLI.

npm-config

Configuration is a major part of npm, and there are multiple ways configuration variables can be set.

Configuration via CLI and Environmental Variables

First, configuration can be set via the CLI from the terminal.

Typically it would look something like this: npm <command> --<configuration option> [<optional value>].

If the value is not specified, the option will be set to true by default.

For example, let’s say that you are working on a scoped (private) npm package, and you decide to publish it as a public package.

This is easily done by appending --access=public to your publish command. If we did not specify the property to be public, the default would be restricted (private).

Configuration appended to other commands like this does not persist everywhere, so it can become tiresome to set an array of configurations via the CLI.

In those cases, it may be better to set configuration using environmental variables.

Any environmental variable set with the npm_config_ prefix will be used to configure npm.

For example: export npm_config_registry=localhost:4321 would set the registry configuration option globally, and when npm is executed it will use the npm registry located at localhost on port 4321.

Configuration via the npmrc file

You can also set configuration options using the special .npmrc file, which can be set at different levels depending on your requirements:

  • Project level: In the root of a project’s code along with its package.json file, typically path/to/project/.npmrc
  • User level: The directory that configures a specific user’s account, typically ~/.npmrc
  • Global level: The directory where npm looks for global configurations, typically $PREFIX/etc/npmrc
  • Built-in level: Be cautious. This configuration is not only global but it is also part of the npm source code, and best practice recommends (actually demands) that we do not change code we are not responsible for maintaining. It can typically be found at /path/to/npm/npmrc.

Configuration settings in the .npmrc file can be modified and persisted using the CLI by running a command in this format: npm config set <key> <value>.

For example, you can run npm config set access public to make the publishing configuration of a scoped (private) package persistently public.

By default, this command would persist the configuration locally (the user level configuration as described above), but you can add -g to persist it globally.

When persisted configuration needs to happen on the project level, or on the built-in level, the .npmrc file must be modified using a text editor.

Configuration via package.json

Finally, configuration can be set from the package.json file. However, this is seldom used (and should only be used if explicitly required), because a project level .npmrc file is the conventionally prefered place to set package configuration.

Notable Configuration Settings

  • access: As discussed above, it’s used to set permissions.

  • always-auth: It’s important to note that this setting defaults to false. When it is set to true, npm will always require authentication when contacting the registry.

  • ca: Defaults to the npm certificate authority (CA). It can be changed to null to allow access to only known registrars, or to a specific CA certificate to only grant access to that specific one. This setting, along with cafile, cert and strict-ssl, are seldom used, but speak toward the security and reliability aspect of npm, allowing peace of mind in knowing that the package you are installing is coming from the source you expect.

  • color: This defaults to true, giving you a break from the standard bleakness of the terminal by coloring the stdout that is allowed by tty file descriptors. If set to false the terminal remains dull. When it is set to always, it always outputs in color.

  • depth: This setting allows for granular control over what you see with recursive commands, such as lsand outdated, by assigning how deeply they’re executed. A value of 0 will only evaluate the first level of dependencies whereas infinity (the default) will cause all levels of dependencies to be evaluated. The exception to this rule is when using it with outdated; in that case, infinity is interpreted as 0 to ensure more relevant output.

  • dev: This is set to false by default, but when it is set to true (when doing an npm install) all development dependencies in the package.json file will be installed along with the normal dependencies.

  • dry-run: When this setting is set to true, npm will not make any changes to your package but will instead tell you what it would have done. This can be very useful when running certain commands, such as dedupe or update.

  • git-tag-version: It is set to true by default. This setting tags a version in git when running the npm version command. If you are using npm as the package manager for a large project that has tagged versions in git, it can save you time, and remember to update the package.json file for you.

  • loglevel: By default, this is set to warn, which gives an error and warning output when running npm commands. Other settings include silent, which provides no output; error, which only logs errors to the output; http, which only announces http request errors; info, for want informative output); verbose, which logs almost everything; and silly, which, like the name suggests, gives a silly amount of output and then some.

  • production: When this is set to true, npm acts accordingly and runs all commands in production mode. This means that development or optional dependencies will not be installed, nor will any development related tasks be executed.

  • rollback: When set to true, all failed installs are removed. This comes in handy when an installation of dependencies fail. Depending on your level of logging, you should be able to see what installations failed, make a note of these, and run the npm install command with the rollback option set to true. Then with your notes and a dry-run installation (as described above), you can then debug the problem.

  • save: When installing a package directly from the registry, you can append –save to the command which will add the installed package to the dependencies option in the package.json file. For example, npm install lodash` will add lodash to your dependencies.

  • save-dev: Similar to the save configuration option, add --save-dev when installing a package, and it will then be added to the devDependencies option in the package.json file.

  • save-optional: Similar to the save configuration option, add --save-optional when installing a package, and it will then be added to the optionalDependencies option in the package.json file.

  • save-exact: When installing packages, the save, save-dev and save-optional options modify the package.json file by inserting the installed package into its respective property with a semver range operator. When invoking the ‘save-exact’ configuration setting with a value of true, in conjunction with one of the mentioned settings above, a specific version number is used, ignoring the semver range.

  • save-prefix: This sets the semver range operator when using save, save-dev or save-optional. The default is ^, allowing for minor upgrades on packages to happen on install. This can be set to any valid prefixed semver range operator.

  • tag-version-prefix: The conventional default is v, specifying what is prepended to the git tag version when running npm version.

Updating npm Using npm

You can also use npm to update itself.

Simply run npm install -g npm@latest, and npm will be updated to the latest stable release. It is important to note that each version of Node.js ships with a specific version of npm, and in my experience, you shouldn’t mess with that pairing too much.

At the end of the day, my recommendation is to stick with the pairing as they are intended.

When using npm as a standalone tool, make sure you understand the implications of using whatever version you choose. There is a great tool for managing different Node.js versions (and in turn npm versions) on the same system called nvm.

The package.json File

The package.json file is the crucial element that links everything together.

It’s a requirement for publishing a package to the npm registry, and it’s where the management part of dependencies comes to life.

It has two required fields, namely “name” and “version,” and together these properties should be a unique identifier.

The name field should adhere to certain rules, as defined by the npm documentation on naming, and the version field is subject to the semver specificiations.

npm reads the package.json file for dependency management.

Over and above that, you can have a list of dependencies a mile long, and define a specific version to be used for each of them, using the semver versions and range operators. Here’s a list of other notable properties.

“main”

“main” defines the entry point to your application, which defaults to index.js. Depending on convention or your framework, it might be app.js or main.js. You can, of course, make it anything you want.

“scripts”

This is an underrated property.

First, it can be used to do things on prepublish.

Second, it provides a place where you can alias an array of frequently used commands, ranging from build tasks (defined in gulp or grunt), triggering the installation of other dependencies (with something like bower), starting a development server with webpack, or running a set of bash commands.

“dependencies”

This property is a list of packages needed by your application, along with the compatible semver number. It is a notable property because it can be modified from the terminal when you install local packages.

This is done by adding --save (or the shorthand -S) at the end of the npm install command.

When you do this, the newly install package(s) gets added to the list of dependencies in your package.json file.

Similarly a dependency can also be removed by adding --save when running the npm uninstall command.

It is important to be aware of the semver versioning patterns of each of the dependencies and what they mean.

If the semver rule is too strict, you lose out on new features and improvements, whereas if the semver rule is too relaxed, a breaking version of a package can be installed along the line.

A broken package install can prove to be quite difficult to resolve, especially when the minified version of the package is used.

“devDependencies”

Separate from the dependencies property, the “devDependencies” property allows you to define dependencies that are only used during the development phase and are not required for the production build (like ESLint, grunt-contrib packages, and Protractor). Just as with dependencies, this property can be modified from the terminal by adding --save-dev (or the shorthand -D) to the end of the npm install command or the npm uninstall command. The same caution applies to versioning as mentioned under dependencies.

“bin”

This is where you can specify your package’s executable file(s), like the path to a CLI utility. This property tells npm to create local or global symlinks to your executables when your package is installed.

“config”

As discussed earlier, this is where you define configuration settings though your package.json file.

“private”

When set to true, npm will refuse to publish the package.

This should not be confused with the access configuration setting.

This is a handy setting when you have a project that utilizes npm along with its package.json but it is not intended to be published to the npm registry, either scoped or public.

If your intention changes, simply change the setting to false, and you will be able to publish your package.

Custom Properties

The package.json file also accepts custom properties, as long as the name is not already defined or reserved.

Developing Your Own npm Package

The npm ecosystem is filled with packages, written by thousands of different developers around the world. Each solves some sort of problem, providing an abstraction, or presenting an implementation of something.

Chances are that, at some point, you too will want to develop your own package to share.

First, you need to author a package.json file with the minimum required properties of “name” and “version”, and then the “main” property to specify the entry point, for example index.js.

Write your code in that index.js file, login with your npm user account, or create a new user from the terminal, and you are ready to publish it to the npm registry.

Packages can be public or private.

Public packages are free to publish and are available for everyone to utilize.

Private packages, called scoped packages, can only be published if you paid private modules user, and they can be identified by the distinct @username/ that is prepended to the package name.

Scoped packages can also be published publically by calling the publish command with --access=public.

Furthermore, if you spend some more time expanding and improving your package’s code base, and it is time for a new version to be published, you simply alter the version (as per the semver rules and convention) of the package in the package.json file and type npm publish.

You can also use the command line interface and call npm version <update_type>, where update_type is either patch, minor, or major, as described by semver, and this then automatically increments the version number in the package.json file.

npm Organizations

Again, the npm documentation for this is excellent, and it would be futile to just repeat their words.

What can be said about organizations in the npm context is that it is extremely fine grained, and when managed correctly, large teams and individuals, working on scoped or public packages under one name, can be very well managed and restricted.

While it is complex to master, it’s very rewarding.

The Power of npm

Ultimately the documentation that npm provides is extensive and should be consulted for specifics, but this article provides a useful overview of both basic and more advanced, involved functionality, conveying the awesomeness of npm.

As with all things, strong opinions exist and many faults can be found. But if you have never tried npm (or node, for that matter), dive in, and explore it for yourself. Chances are you’ll enjoy it more than you think.

For more interesting articles on npm, consider reading Using Scala.js with npm and Browserify.

Hire a Toptal expert on this topic.
Hire Now
Martin Gouws

Martin Gouws

Verified Expert in Engineering

Pretoria, Gauteng, South Africa

Member since November 7, 2016

About the author

Martin is a young and talented JavaScript developer and software engineer who has his roots in enterprise Java.

Read More
authors are vetted experts in their fields and write on topics in which they have demonstrated experience. All of our content is peer reviewed and validated by Toptal experts in the same field.

Expertise

PREVIOUSLY AT

Discovery Limited, South Africa

World-class articles, delivered weekly.

Subscription implies consent to our privacy policy

World-class articles, delivered weekly.

Subscription implies consent to our privacy policy

Join the Toptal® community.