Wed, 11 Jun 2014 16:00:00 UTC - tjfontaine - Uncategorized

Notes from the Road

As Project Lead for Node.js, I was excited to have the opportunity to go on the road and bring production stories to all of our users. We've had amazing speakers and turn out in San Francisco, Seattle, Portland, Boston, and New York. But I wanted to make sure we reached more than just our coasts, so soon we'll be in Minneapolis and I'll be returning to my home state of Ohio and doing an event in Cincinnati. The Node.js community is all over the world, and hopefully Node on the Road can reach as many of you as it can. Nominate your city to be a future stop on the Node.js on the Road series here.

These Node on the Road events are successful because of the incredible support from the community and the existing meetup organizations in their respective cities. But the biggest advantage is that the project gets to solicit feedback directly from our users about what is and isn't working for them in Node.js, what modules they're using, and where they need Node to do better.

Release schedules

Some of the feedback we've received has been about the upgrade process for Node. Veteran Node.js alums will occasionally sit around campfires and tell the stories of when things would break every release, or how long they stayed on 0.4 before upgrading to 0.6. Some production companies are still out there running on 0.8 afraid to make the jump to 0.10. While other companies advise people to avoid upgrading to a new release of a Node version until the patch number hits double digits. It's those sorts of stories that make it important for us to get the release for 0.12 right, from the get go.

Node is in a fantastic place right now, it's maturing quickly and finding its footing in new environments with new users and new use cases. The expectation for Node is getting higher each day with every release. There are multiple interests at stake, keeping Node lean, keeping it up to date with languages and standards, keeping it fast, and balanced with keeping it stable such that we don't upset the adoption rate. That means Node needs to make the right choices that balance the needs of all of our users without closing the doors to others.

All of these conversations are helping to shape the release process going forward, and helping to scope just what does go into a release and how fast people want to see those happen. In fact something we've been considering is eliminating the confusion around our Stable/Unstable branches, and instead moving to releases that are always stable. But it's important that the features and changes that go into a release are shaped by user feedback, which is why events like Node on the Road are vital.

Better Documentation

Another key piece of feedback has consistently been around our documentation. Users need us to clean up our API reference documentation, there are lots of undocumented and under-documented methods and properties that are being used or should be used. Node needs to include what errors may be delivered as part of the operation of your application, as well as what methods will throw and under what circumstances.

But mostly users are looking for more general purpose documentation that can help both new and veteran Node.js users be more productive with Node. And the people who are most equipped to provide that documentation are the users themselves who've already been successful.

Easier Contribution

Aside from soliciting feedback from users of Node.js and bringing production stories to our users, Node on the Road has also been about highlighting the various ways you as a member of the community can contribute. There are many ways you can contribute from meetups and conferences, to publishing modules, to finding issues in modules or core, to fixing issues in modules or core, or even adding features to modules or core. Where ever you are passionate about Node.js there are ways you can contribute back to Node.

Node.js has inherited many things from our largest dependency V8, we've adopted their build system GYP, we use their test runner (which is unfortunately in python), and when we were structuring the project we brought along the Contributor License Agreement (CLA) that Google uses to manage contributions for Chromium and V8. The CLA is there as a way for a project to audit itself and to give itself the opportunity to relicense itself in the future if necessary. Node.js though is distributed under the venerable MIT license, and that's not going to change. The MIT license is one of the most permissible open source licenses out there, and has fostered a ton of development with Node.js and we want that to continue.

In an effort to make it easier for users to contribute to Node.js the project has decided to lift the requirement of signing the CLA before contributions are eligible for integration. Having to sign the CLA could at times be a stumbling block for a contribution. It could involve a long conversation with your legal department to ultimately contribute typo corrections.

I'm excited to see what contributions will be coming from the community in the future, excited to see where our users take Node.js, and excited to be participating with all of you on this project.

Wed, 25 Apr 2012 20:48:58 UTC - Dave Pacheco - Uncategorized

It's incredibly easy to visualize where your Node program spends its time using DTrace and node-stackvis (a Node port of Brendan Gregg's FlameGraph tool):

  1. Run your Node.js program as usual.
  2. In another terminal, run:
    $ dtrace -n 'profile-97/execname == "node" && arg1/{
        @[jstack(150, 8000)] = count(); } tick-60s { exit(0); }' > stacks.out
    This will sample about 100 times per second for 60 seconds and emit results to stacks.out. Note that this will sample all running programs called "node". If you want a specific process, replace execname == "node" with pid == 12345 (the process id).
  3. Use the "stackvis" tool to transform this directly into a flame graph. First, install it:
    $ npm install -g stackvis
    then use stackvis to convert the DTrace output to a flamegraph:
    $ stackvis dtrace flamegraph-svg < stacks.out > stacks.svg
  4. Open stacks.svg in your favorite browser.

You'll be looking at something like this:

This is a visualization of all of the profiled call stacks. This example is from the "hello world" HTTP server on the Node.js home page under load. Start at the bottom, where you have "main", which is present in most Node stacks because Node spends most on-CPU time in the main thread. Above each row, you have the functions called by the frame beneath it. As you move up, you'll see actual JavaScript function names. The boxes in each row are not in chronological order, but their width indicates how much time was spent there. When you hover over each box, you can see exactly what percentage of time is spent in each function. This lets you see at a glance where your program spends its time.

That's the summary. There are a few prerequisites:

  • You must gather data on a system that supports DTrace with the Node.js ustack helper. For now, this pretty much means illumos-based systems like SmartOS, including the Joyent Cloud. MacOS users: OS X supports DTrace, but not ustack helpers. The way to get this changed is to contact your Apple developer liaison (if you're lucky enough to have one) or file a bug report at bugreport.apple.com. I'd suggest referencing existing bugs 5273057 and 11206497. More bugs filed (even if closed as dups) show more interest and make it more likely Apple will choose to fix this.
  • You must be on 32-bit Node.js 0.6.7 or later, built --with-dtrace. The helper doesn't work with 64-bit Node yet. On illumos (including SmartOS), development releases (the 0.7.x train) include DTrace support by default.

There are a few other notes:

  • You can absolutely profile apps in production, not just development, since compiling with DTrace support has very minimal overhead. You can start and stop profiling without restarting your program.
  • You may want to run the stacks.out output through c++filt to demangle C++ symbols. Be sure to use the c++filt that came with the compiler you used to build Node. For example:
    c++filt < stacks.out > demangled.out
    then you can use demangled.out to create the flamegraph.
  • If you want, you can filter stacks containing a particular function. The best way to do this is to first collapse the original DTrace output, then grep out what you want:
    $ stackvis dtrace collapsed < stacks.out | grep SomeFunction > collapsed.out
    $ stackvis collapsed flamegraph-svg < collapsed.out > stacks.svg
  • If you've used Brendan's FlameGraph tools, you'll notice the coloring is a little different in the above flamegraph. I ported his tools to Node first so I could incorporate it more easily into other Node programs, but I've also been playing with different coloring options. The current default uses hue to denote stack depth and saturation to indicate time spent. (These are also indicated by position and size.) Other ideas include coloring by module (so V8, JavaScript, libc, etc. show up as different colors.)

For more on the underlying pieces, see my previous post on Node.js profiling and Brendan's post on Flame Graphs.


Dave Pacheco blogs at dtrace.org

Thu, 15 Dec 2011 19:59:15 UTC - ryandahl - Uncategorized

This week Microsoft announced support for Node in Windows Azure, their cloud computing platform. For the Node core team and the community, this is an important milestone. We've worked hard over the past six months reworking Node's machinery to support IO completion ports and Visual Studio to provide a good native port to Windows. The overarching goal of the port was to expand our user base to the largest number of developers. Happily, this has paid off in the form of being a first class citizen on Azure. Many users who would have never used Node as a pure unix tool are now up and running on the Windows platform. More users translates into a deeper and better ecosystem of modules, which makes for a better experience for everyone.

We also redesigned our website - something that we've put off for a long time because we felt that Node was too nascent to dedicate marketing to it. But now that we have binary distributions for Macintosh and Windows, have bundled npm, and are serving millions of users at various companies, we felt ready to indulge in a new website and share of a few of our success stories on the home page.

Work is on-going. We continue to improve the software, making performance improvements and adding isolate support, but Node is growing up.

Tue, 25 Oct 2011 22:26:23 UTC - ryandahl - Uncategorized

Version 0.6.0 will be released next week. Please spend some time this week upgrading your code to v0.5.10. Report any API differences at a href="https://github.com/joyent/node/wiki/API-changes-between-v0.4-and-v0.6"https://github.com/joyent/node/wiki/API-changes-between-v0.4-and-v0.6 or report a bug to us at a href="http://github.com/joyent/node/issues"http://github.com/joyent/node/issues if you hit problems.

The API changes between v0.4.12 and v0.5.10 are 99% cosmetic, minor, and easy to fix. Most people are able to migrate their code in 10 minutes. Don't fear.

Once you've ported your code to v0.5.10 please help out by testing third party modules. Make bug reports. Encourage authors to publish new versions of their modules. Go through the list of modules at a href="http://npmjs.org/"http://npmjs.org/ and try out random ones. This is especially encouraged of Windows users!

Page 2 →