Tue, 29 Jul 2014 21:00:00 UTC - tjfontaine - Community

Node.js is reaching more people than ever, it's attracting new and interesting use cases, at the same time as seeing heavy adoption from traditional engineering departments. Managing the project to make sure it continues to satisfy the needs of its end users requires a higher level of precision and diligence. It requires taking the time to communicate and reach out to new and old parties alike. It means seeking out new and dedicated resources. It means properly scoping a change in concert with end users, and documenting and regularly check pointing your progress. These are just some of the ways we're working to improve our process, and work to deliver higher quality software that meets our goals.

Documentation

One of the big things we've wanted to do is to change the way the website works, which is something I've mentioned before. It should be a living breathing website whose content is created by our end users and team. The website should be the canonical location for documentation on how to use Node.js, how Node.js works, and how to find out what's going on in the Node community. We have seeded the initial documentation with how to contribute, who the core team is, and some basic documentation of the project itself. From there we're looking to enable the community to come in and build out the rest of the framework for documentation.

One of the key changes here is that we're extending the tools that generate API documentation to work for the website in general. That means the website is now written in markdown. Contributions work with the same pull-request way as contributions to Node itself. The intent here is to be able to quickly generate new documentation and improve it with feedback from the community.

The website should also be where we host information about where the project is going and the features we're currently working on (more about that later). But it's crucial we communicate to our end users what improvements will be coming, and the reasons we've made those decisions. That way it's clear what is coming in what release, and also can inspire you to collaborate on the design of that API. This is not a replacement for our issue tracking, but an enhancement that can allow us to reach more people.

Features

Which brings us to the conversation about features. During the Q & A portions of the Node.js on the Road events there are often questions about what does and doesn't go into core. How the team identifies what those features are and when you decide to integrate them. I've spent a lot of time talking about that but I've also added it to the new documentation on the site.

It's pretty straight forward, but in short if Node.js needs an interface to provide an abstraction, or if everyone in the community is using the same interface, then those interfaces are candidates for being exposed as public interfaces for Node. But what's important is that the addition of an API should not be taken lightly. It is important for us to consider just how much of an interface we can commit to, because once we add the API it's incredibly hard for us to change or remove it. At least in a way that allows people to write software that will continue to work.

So new features and APIs need to come with known use cases and consumers, and with working test suites. That information is clearly and concisely present on the website to reach as wide of an audience as possible. Once an implementation meets those requirements it can be integrated into the project. Then and only then, when we have an implementation that meets the design specification and satisfies the test suite, will we be able to integrate it. That's how we'll scope our releases going forward, that's how we'll know when we're ready to release a new version of Node. This will be a great change for Node, as it's a step forward on moving to an always production ready master branch.

Quality Software

And it's because Node.js is focused on quality software and a commitment to backwards compatibility that it's important for us to seek ways to get more information from the community about when and where we might be breaking them. Having downstream users test their code bases with recent versions of Node.js (even from our master branch) is an important way we derive feedback for our changes. The sooner we can get that information, the more test coverage we can add, the better the software we deliver is.

Recently I had the opportunity to speak with Dav Glass from Yahoo!, and we're going to be finding ways to get automated test results back from some larger test suites. The more automation we can get for downstream integration testing the better the project can be at delivering quality software.

If you're interested in participating in the conversation about how Node.js can be proactively testing your software/modules when we've changed things, please join the conversation.

Current release

Before we can release v0.12, we need to ensure we're providing a high quality release that addresses the needs of the users as well as what we've previously committed to as going into this release. Sometimes what can seem like an innocuous change that solves an immediate symptom, doesn't actually treat the disease, but instead results in other symptoms that need to be treated. Specifically in our streams API, it can be easy to subtly break people while trying to fix another bug with good intent.

This serves as a reminder that we need to properly scope our releases. We need to know who the consumers are for new APIs and features. We need to make sure those features' test cases are met. We need to make sure we're adopting APIs that have broad appeal. And while we're able to work around some of these things through external modules and experimenting with JavaScript APIs, that's not a replacement for quality engineering.

Those are the things that we could have done better before embarking on 0.12, and now to release it we need to fix some of the underlying issues. Moving forward I'm working with consumers of the tracing APIs to work on getting a maintainable interface for Node that will satisfy their needs. We'll publicly document those things, we'll reach out to other stakeholders, and we'll make sure that as we implement that we can deliver discretely on what they need.

That's why it's important for us to get our releases right, and diagnose and fix root causes. We want to make sure that your first experience with 0.12 results in your software still working. This is why we're working with large production environments to get their feedback, and we're looking for those environments and you to file bugs that you find.

The Team

The great part about Node's contribution process and our fantastic community is that we have a lot of very enthusiastic members who want to work as much as possible on Node. Maybe they want to contribute because they have free time, maybe they want to contribute to make their job easier, or perhaps they want to contribute because their company wants them to spend their time on open source. Whatever the reason, we welcome contributions of every stripe!

We have our core team that manages the day to day of Node, and that works mostly by people wanting to maintain subsystems. They alone are not solely responsible for the entirety of that subsystem, but they guide its progress by communicating with end users, reviewing bugs and pull requests, and identifying test cases and consumers of new features. People come and go from the core team, and recently we've added some documentation that describes how you find your way onto that team. It's based largely around our contribution process. It's not about who you work for, or about who you know, it's about your ability to provide technical improvement to the project itself.

For instance, Chris Dickinson was recently hired to work full time on Node.js, and has expressed interest in working on the current and future state of streams. But it's not who employs Chris that makes him an ideal candidate, but it will be the quality of his contributions, and his understanding of the ethos of Node.js. That's how we find members of the team. And Chris gets that, in his blog about working full time on Node.js he says (and I couldn't have said it better myself):

I will not automatically get commit access to these repositories — like any other community member, I will have to continually submit work of consistent quality and put in the time to earn the commit bit. The existing core team will have final say on whether or not I get the commit bit — which is as it should be!

Exactly. And not only does he understand how mechanism works, but he's already started getting feedback from consumers of streams and documenting some of his plans.

In addition to Chris being hired to work full time on Node.js, Joyent has recently hired Julien Gilli to work full time with me on Node. I'm really excited for all of the team to be seeking out new contributors, and getting to know Chris and Julien. They're both fantastic and highly motivated, and I want to do my best to enable them to be successful and join the team. But that's not all, I've been talking to other companies who are excited to participate in this model, and in fact Modulus.io themselves are looking to find someone this year to work full time on Node.js.

Node.js is bigger than the core team, it's bigger than our community, and we are excited to continue to get new contributors, and to enable everyone. So while we're working on the project we can't just focus on one area, but instead consider the connected system as a whole. How we scale Node, how we scale the team, how we scale your contributions, and how we integrate your feedback -- this is what we have to consider while taking this project forward, together.

Mon, 16 Jun 2014 15:46:10 UTC - vulnerability

Today we are releasing new versions of Node:

First and foremost these releases address the current OpenSSL vulnerability CVE-2014-0224, for both 0.8 and 0.10 we've upgraded the version of the bundled OpenSSL to their fixed versions v1.0.0m and v1.0.1h respectively.

Additionally these releases address the fact that V8 UTF-8 encoding would allow unmatched surrogate pairs. That is to say, previously you could construct a valid JavaScript string (which are stored internally as UCS-2), pass it to a Buffer as UTF-8, send and consume that string in another process and it would fail to interpret because the UTF-8 string was invalid.

Note, the results encoded by V8 in this case are exactly what was passed into the encoding routine. There is no overflow, underflow, or the inclusion of other arbitrary memory, merely an unmatched UTF-8 surrogate resulting in invalid UTF-8.

As of these releases, if you try and pass a string with an unmatched surrogate pair, Node will replace that character with the unknown unicode character (U+FFFD). To preserve the old behavior set the environment variable NODE_INVALID_UTF8 to anything (even nothing). If the environment variable is present at all it will revert to the old behavior.

This breaks backward compatibility for the specific reason that unsanitized strings sent as a text payload for an RFC compliant WebSocket implementation should result in the disconnection of the client. If the client attempts to reconnect and receives another invalid payload it must disconnect again. If there is no logic to handle the reconnection attempts, this may lead to a denial of service attack. For instance socket.io attempts to reconnect by default.

// Prior to these releases:
new Buffer('ab\ud800cd', 'utf8');
// <Buffer 61 62 ed a0 80 63 64>

// After this release:
new Buffer('ab\ud800cd', 'utf8');
// <Buffer 61 62 ef bf bd 63 64>

// This is an explicit conversion to a Buffer, but the implicit
// .write('ab\ud800cd') also results in the same pattern
websocket.write(new Buffer('ab\ud800cd', 'utf8'));
// This would result in the client disconnecting.

Node's default encoding for strings is UTF-8, so even if you're not explicitly creating Buffers out of strings, Node may be doing so under the hood. If what you're passing is not actually UTF-8 then when you call .write(str) you could be specific and say .write(str, 'binary') which signals Node to pass the string through without interpreting it.

You can also mitigate this in pure JavaScript by sanitizing your strings, as an example see node-unicode-sanitize which will similarly replace unmatched surrogate pairs with the unknown unicode character.

Thanks to Node.js alum Felix Geisendörfer for finding, getting the fixes upstreamed, and helping with the testing and mitigation. Also for helping to inform and improve the process for Node.js security issues.

To float these fixes in your own builds you can apply the following patch with git am

Mon, 16 Jun 2014 15:33:45 UTC - release

2014.06.09, Version 0.8.27 (maintenance)

  • openssl: update to 1.0.0m (CVE-2014-0224)

  • utf8: Prevent Node from sending invalid UTF-8 (Felix Geisendörfer)

    • NOTE this introduces a breaking change, previously you could construct invalid UTF-8 and invoke an error in a client that was expecting valid UTF-8, now unmatched surrogate pairs are replaced with the unknown UTF-8 character. To restore the old functionality simply have NODE_INVALID_UTF8 environment variable set.
  • tls: fix pool usage race (Fedor Indutny)

  • fs: close file if fstat() fails in readFile() (cjihrig)

Source Code: http://nodejs.org/dist/v0.8.27/node-v0.8.27.tar.gz

Macintosh Installer (Universal): http://nodejs.org/dist/v0.8.27/node-v0.8.27.pkg

Windows Installer: http://nodejs.org/dist/v0.8.27/node-v0.8.27-x86.msi

Windows x64 Installer: http://nodejs.org/dist/v0.8.27/x64/node-v0.8.27-x64.msi

Windows x64 Files: http://nodejs.org/dist/v0.8.27/x64/

Linux 32-bit Binary: http://nodejs.org/dist/v0.8.27/node-v0.8.27-linux-x86.tar.gz

Linux 64-bit Binary: http://nodejs.org/dist/v0.8.27/node-v0.8.27-linux-x64.tar.gz

Solaris 32-bit Binary: http://nodejs.org/dist/v0.8.27/node-v0.8.27-sunos-x86.tar.gz

Solaris 64-bit Binary: http://nodejs.org/dist/v0.8.27/node-v0.8.27-sunos-x64.tar.gz

Other release files: http://nodejs.org/dist/v0.8.27/

Website: http://nodejs.org/docs/v0.8.27/

Documentation: http://nodejs.org/docs/v0.8.27/api/

Shasums:

089e075cd556b6124f27d41045f6c5a4c119f9da  node-v0.8.27-darwin-x64.tar.gz
88014cb30b27f410b89c99a7ccbf32dd8aa19bfd  node-v0.8.27-darwin-x86.tar.gz
fe2471f055af273c222c4d982e92ffe32369d41f  node-v0.8.27-linux-x64.tar.gz
6096cba0eb9f0cb8a3b618aa7dcf7f4c29cda82b  node-v0.8.27-linux-x86.tar.gz
b775af237c0e6c62969ab7886bab3b0cf1563f11  node-v0.8.27-sunos-x64.tar.gz
8d396f1e546b104616a0e9f3fd20bc7eaa9a370f  node-v0.8.27-sunos-x86.tar.gz
9e1acfc8970ef80011257080576321dc136f7236  node-v0.8.27-x86.msi
04a1961f5a4fa9e953aecd589fe5aea5f17e3a56  node-v0.8.27.pkg
13ebdc277c76732089982b29fb25941dbee5af3f  node-v0.8.27.tar.gz
d1a41a081663ecf036390dd2f511fa7b452cc1ad  x64/node-v0.8.27-x64.msi

Mon, 16 Jun 2014 15:06:10 UTC - release

2014.06.05, Version 0.10.29 (Stable)

  • openssl: to 1.0.1h (CVE-2014-0224)

  • npm: upgrade to 1.4.14

  • utf8: Prevent Node from sending invalid UTF-8 (Felix Geisendörfer)

    • NOTE this introduces a breaking change, previously you could construct invalid UTF-8 and invoke an error in a client that was expecting valid UTF-8, now unmatched surrogate pairs are replaced with the unknown UTF-8 character. To restore the old functionality simply have NODE_INVALID_UTF8 environment variable set.
  • child_process: do not set args before throwing (Greg Sabia Tucker)

  • child_process: spawn() does not throw TypeError (Greg Sabia Tucker)

  • constants: export O_NONBLOCK (Fedor Indutny)

  • crypto: improve memory usage (Alexis Campailla)

  • fs: close file if fstat() fails in readFile() (cjihrig)

  • lib: name EventEmitter prototype methods (Ben Noordhuis)

  • tls: fix performance issue (Alexis Campailla)

Source Code: http://nodejs.org/dist/v0.10.29/node-v0.10.29.tar.gz

Macintosh Installer (Universal): http://nodejs.org/dist/v0.10.29/node-v0.10.29.pkg

Windows Installer: http://nodejs.org/dist/v0.10.29/node-v0.10.29-x86.msi

Windows x64 Installer: http://nodejs.org/dist/v0.10.29/x64/node-v0.10.29-x64.msi

Windows x64 Files: http://nodejs.org/dist/v0.10.29/x64/

Linux 32-bit Binary: http://nodejs.org/dist/v0.10.29/node-v0.10.29-linux-x86.tar.gz

Linux 64-bit Binary: http://nodejs.org/dist/v0.10.29/node-v0.10.29-linux-x64.tar.gz

Solaris 32-bit Binary: http://nodejs.org/dist/v0.10.29/node-v0.10.29-sunos-x86.tar.gz

Solaris 64-bit Binary: http://nodejs.org/dist/v0.10.29/node-v0.10.29-sunos-x64.tar.gz

Other release files: http://nodejs.org/dist/v0.10.29/

Website: http://nodejs.org/docs/v0.10.29/

Documentation: http://nodejs.org/docs/v0.10.29/api/

Shasums:

45a26f4c01086275dba4657e3c347eb8648beda5  node-v0.10.29-darwin-x64.tar.gz
cdfd3cbb5b4a5c99fed4b5f29bb0d9f39af0ad15  node-v0.10.29-darwin-x86.tar.gz
570c45653fec04d29d2208bb2967bc88b2821537  node-v0.10.29-linux-x64.tar.gz
81a0f08f9b4485d0a7ed0c049c79110b93ae03bc  node-v0.10.29-linux-x86.tar.gz
44cf39e6eba77b23abf31f0780b85d83f6e8d93a  node-v0.10.29-sunos-x64.tar.gz
c91a5b34dec50f91102a1cc846b79cb50d6460af  node-v0.10.29-sunos-x86.tar.gz
dd512e5ceb74988ede95517554888f2b3d716ff4  node-v0.10.29-x86.msi
7401f3047373813cf788fa4de487d0766cef76f7  node-v0.10.29.pkg
0d5dc62090404f7c903f29779295758935529242  node-v0.10.29.tar.gz
7e1fc5162e177aa32d626635e941aa2791092035  node.exe
9f83dbfc6db23f1fa701259359cbb8593e6edb58  node.exp
b9090fd83fcd66a91ecde139acb14c741c4b4af7  node.lib
2599a5723ea8aac115f72e48f053e0d7ac1dd5c7  node.pdb
9387d64902a3c7ff97ce3bc07f519b238ab9df4f  pkgsrc/nodejs-ia32-0.10.29.tgz
cc4c90afe93456176c9ea984a08aefd065d85b34  pkgsrc/nodejs-x64-0.10.29.tgz
c6ae603f73ccb2e33fed2d34fbcf55fd8d952775  x64/node-v0.10.29-x64.msi
1bf313dfb5259e8712ed1f0783739070f5c1453d  x64/node.exe
14c902ddaa40a1c37826691d691e594d0d657fe8  x64/node.exp
0bf14903526c93fcfe4bab57844b4046ce922b31  x64/node.lib
af759c0b56d829439b08b46877641000bbff81d6  x64/node.pdb

← Page 1

Page 3 →