npm

npm 1.0: Global vs Local installation

Thu, 24 Mar 2011 06:07:13 UTC - Isaac Schlueter - npm

npm 1.0 is in release candidate mode. Go get it!

More than anything else, the driving force behind the npm 1.0 rearchitecture was the desire to simplify what a package installation directory structure looks like.

In npm 0.x, there was a command called bundle that a lot of people liked. bundle let you install your dependencies locally in your project, but even still, it was basically a hack that never really worked very reliably.

Also, there was that activation/deactivation thing. That’s confusing.

Two paths

In npm 1.0, there are two ways to install things:

globally —- This drops modules in {prefix}/lib/node_modules, and puts executable files in {prefix}/bin, where {prefix} is usually something like /usr/local. It also installs man pages in {prefix}/share/man, if they’re supplied.
locally —- This installs your package in the current working directory. Node modules go in ./node_modules, executables go in ./node_modules/.bin/, and man pages aren’t installed at all.

Which to choose

Whether to install a package globally or locally depends on the global config, which is aliased to the -g command line switch.

Just like how global variables are kind of gross, but also necessary in some cases, global packages are important, but best avoided if not needed.

In general, the rule of thumb is:

If you’re installing something that you want to use in your program, using require('whatever'), then install it locally, at the root of your project.
If you’re installing something that you want to use in your shell, on the command line or something, install it globally, so that its binaries end up in your PATH environment variable.

When you can't choose

Of course, there are some cases where you want to do both. Coffee-script and Express both are good examples of apps that have a command line interface, as well as a library. In those cases, you can do one of the following:

Install it in both places. Seriously, are you that short on disk space? It’s fine, really. They’re tiny JavaScript programs.
Install it globally, and then npm link coffee-script or npm link express (if you’re on a platform that supports symbolic links.) Then you only need to update the global copy to update all the symlinks as well.

The first option is the best in my opinion. Simple, clear, explicit. The second is really handy if you are going to re-use the same library in a bunch of different projects. (More on npm link in a future installment.)

You can probably think of other ways to do it by messing with environment variables. But I don’t recommend those ways. Go with the grain.

Slight exception: It’s not always the cwd.

Let’s say you do something like this:

cd ~/projects/foo     # go into my project
npm install express   # ./node_modules/express
cd lib/utils          # move around in there
vim some-thing.js     # edit some stuff, work work work
npm install redis     # ./lib/utils/node_modules/redis!? ew.

In this case, npm will install redis into ~/projects/foo/node_modules/redis. Sort of like how git will work anywhere within a git repository, npm will work anywhere within a package, defined by having a node_modules folder.

Test runners and stuff

If your package's scripts.test command uses a command-line program installed by one of your dependencies, not to worry. npm makes ./node_modules/.bin the first entry in the PATH environment variable when running any lifecycle scripts, so this will work fine, even if your program is not globally installed:

{ "name" : "my-program"
, "version" : "1.2.3"
, "dependencies": { "express": "*", "coffee-script": "*" }
, "devDependencies": { "vows": "*" }
, "scripts":
  { "test": "vows test/*.js"
  , "preinstall": "cake build" } }

npm 1.0: The New "ls"

Fri, 18 Mar 2011 06:22:17 UTC - Isaac Schlueter - npm

This is the first in a series of hopefully more than 1 posts, each detailing some aspect of npm 1.0.

In npm 0.x, the ls command was a combination of both searching the registry as well as reporting on what you have installed.

As the registry has grown in size, this has gotten unwieldy. Also, since npm 1.0 manages dependencies differently, nesting them in node_modules folder and installing locally by default, there are different things that you want to view.

The functionality of the ls command was split into two different parts. search is now the way to find things on the registry (and it only reports one line per package, instead of one line per version), and ls shows a tree view of the packages that are installed locally.

Here’s an example of the output:

$ npm ls
npm@1.0.0 /Users/isaacs/dev-src/js/npm
├── semver@1.0.1 
├─┬ ronn@0.3.5 
│ └── opts@1.2.1 
└─┬ express@2.0.0rc3 extraneous 
  ├─┬ connect@1.1.0 
  │ ├── qs@0.0.7 
  │ └── mime@1.2.1 
  ├── mime@1.2.1 
  └── qs@0.0.7

This is after I’ve done npm install semver ronn express in the npm source directory. Since express isn’t actually a dependency of npm, it shows up with that “extraneous” marker.

Let’s see what happens when we create a broken situation:

$ rm -rf ./node_modules/express/node_modules/connect
$ npm ls
npm@1.0.0 /Users/isaacs/dev-src/js/npm
├── semver@1.0.1 
├─┬ ronn@0.3.5 
│ └── opts@1.2.1 
└─┬ express@2.0.0rc3 extraneous 
  ├── UNMET DEPENDENCY connect >= 1.1.0 < 2.0.0
  ├── mime@1.2.1 
  └── qs@0.0.7

Tree views are great for human readability, but some times you want to pipe that stuff to another program. For that output, I took the same datastructure, but instead of building up a treeview string for each line, it spits out just the folders like this:

$ npm ls -p
/Users/isaacs/dev-src/js/npm
/Users/isaacs/dev-src/js/npm/node_modules/semver
/Users/isaacs/dev-src/js/npm/node_modules/ronn
/Users/isaacs/dev-src/js/npm/node_modules/ronn/node_modules/opts
/Users/isaacs/dev-src/js/npm/node_modules/express
/Users/isaacs/dev-src/js/npm/node_modules/express/node_modules/connect
/Users/isaacs/dev-src/js/npm/node_modules/express/node_modules/connect/node_modules/qs
/Users/isaacs/dev-src/js/npm/node_modules/express/node_modules/connect/node_modules/mime
/Users/isaacs/dev-src/js/npm/node_modules/express/node_modules/mime
/Users/isaacs/dev-src/js/npm/node_modules/express/node_modules/qs

Since you sometimes want a bigger view, I added the --long option to (shorthand: -l) to spit out more info:

$ npm ls -l
npm@1.0.0 
│ /Users/isaacs/dev-src/js/npm
│ A package manager for node
│ git://github.com/isaacs/npm.git
│ http://npmjs.org/
├── semver@1.0.1 
│   ./node_modules/semver
│   The semantic version parser used by npm.
│   git://github.com/isaacs/node-semver.git
├─┬ ronn@0.3.5 
│ │ ./node_modules/ronn
│ │ markdown to roff and html converter
│ └── opts@1.2.1 
│     ./node_modules/ronn/node_modules/opts
│     Command line argument parser written in the style of commonjs. To be used with node.js
└─┬ express@2.0.0rc3 extraneous 
  │ ./node_modules/express
  │ Sinatra inspired web development framework
  ├─┬ connect@1.1.0 
  │ │ ./node_modules/express/node_modules/connect
  │ │ High performance middleware framework
  │ │ git://github.com/senchalabs/connect.git
  │ ├── qs@0.0.7 
  │ │   ./node_modules/express/node_modules/connect/node_modules/qs
  │ │   querystring parser
  │ └── mime@1.2.1 
  │     ./node_modules/express/node_modules/connect/node_modules/mime
  │     A comprehensive library for mime-type mapping
  ├── mime@1.2.1 
  │   ./node_modules/express/node_modules/mime
  │   A comprehensive library for mime-type mapping
  └── qs@0.0.7 
      ./node_modules/express/node_modules/qs
      querystring parser

$ npm ls -lp
/Users/isaacs/dev-src/js/npm:npm@1.0.0::::
/Users/isaacs/dev-src/js/npm/node_modules/semver:semver@1.0.1::::
/Users/isaacs/dev-src/js/npm/node_modules/ronn:ronn@0.3.5::::
/Users/isaacs/dev-src/js/npm/node_modules/ronn/node_modules/opts:opts@1.2.1::::
/Users/isaacs/dev-src/js/npm/node_modules/express:express@2.0.0rc3:EXTRANEOUS:::
/Users/isaacs/dev-src/js/npm/node_modules/express/node_modules/connect:connect@1.1.0::::
/Users/isaacs/dev-src/js/npm/node_modules/express/node_modules/connect/node_modules/qs:qs@0.0.7::::
/Users/isaacs/dev-src/js/npm/node_modules/express/node_modules/connect/node_modules/mime:mime@1.2.1::::
/Users/isaacs/dev-src/js/npm/node_modules/express/node_modules/mime:mime@1.2.1::::
/Users/isaacs/dev-src/js/npm/node_modules/express/node_modules/qs:qs@0.0.7::::

And, if you want to get at the globally-installed modules, you can use ls with the global flag:

$ npm ls -g
/usr/local
├─┬ A@1.2.3 -> /Users/isaacs/dev-src/js/A
│ ├── B@1.2.3 -> /Users/isaacs/dev-src/js/B
│ └─┬ npm@0.3.15 
│   └── semver@1.0.1 
├─┬ B@1.2.3 -> /Users/isaacs/dev-src/js/B
│ └── A@1.2.3 -> /Users/isaacs/dev-src/js/A
├── glob@2.0.5 
├─┬ npm@1.0.0 -> /Users/isaacs/dev-src/js/npm
│ ├── semver@1.0.1 
│ └─┬ ronn@0.3.5 
│   └── opts@1.2.1 
└── supervisor@0.1.2 -> /Users/isaacs/dev-src/js/node-supervisor

$ npm ls -gpl
/usr/local:::::
/usr/local/lib/node_modules/A:A@1.2.3::::/Users/isaacs/dev-src/js/A
/usr/local/lib/node_modules/A/node_modules/npm:npm@0.3.15::::/Users/isaacs/dev-src/js/A/node_modules/npm
/usr/local/lib/node_modules/A/node_modules/npm/node_modules/semver:semver@1.0.1::::/Users/isaacs/dev-src/js/A/node_modules/npm/node_modules/semver
/usr/local/lib/node_modules/B:B@1.2.3::::/Users/isaacs/dev-src/js/B
/usr/local/lib/node_modules/glob:glob@2.0.5::::
/usr/local/lib/node_modules/npm:npm@1.0.0::::/Users/isaacs/dev-src/js/npm
/usr/local/lib/node_modules/npm/node_modules/semver:semver@1.0.1::::/Users/isaacs/dev-src/js/npm/node_modules/semver
/usr/local/lib/node_modules/npm/node_modules/ronn:ronn@0.3.5::::/Users/isaacs/dev-src/js/npm/node_modules/ronn
/usr/local/lib/node_modules/npm/node_modules/ronn/node_modules/opts:opts@1.2.1::::/Users/isaacs/dev-src/js/npm/node_modules/ronn/node_modules/opts
/usr/local/lib/node_modules/supervisor:supervisor@0.1.2::::/Users/isaacs/dev-src/js/node-supervisor

Those -> flags are indications that the package is link-installed, which will be covered in the next installment.

← Page 1