Hot Loading, Redux.

[Blaine Cook]

(see http://groups.google.com/group/nodejs/browse_thread/thread/c9b82171d3c7aac2
for an earlier discussion).

The ability to re-require modules (aka, hot load new code) is *huge*;
I've posted on it briefly here:
http://romeda.org/blog/2010/01/hot-code-loading-in-nodejs.html

For web servers in particular, change-driven (i.e., event-driven
stat()) hot code loading means that new code can be loaded without
impacting in-progress requests and with no performance impact. For
anyone who's managed large deployments of web servers, this is an
immensely important feature. Along with process' uncaughtException, it
means that you can realistically build a server with 99.9999999%
software uptime. Huge.

The implementation I've done[1] basically moves the module cache into
a per-module cache that inherits from the parent's cache, but allows
individual modules to remove specific modules from the cache so that
they can be re-required. This guarantees that a single module and all
its descendants will see a persistent snapshot of code, while also
guaranteeing that re-required code will see a current and contemporary
snapshot of code.

The approach could do with some cleaning up, but I wanted to build
something simple to demonstrate the approach before building a pretty
API.

Basically, it works like this:

var requestHandler = require('./myRequestHandler');

process.watchFile('./myRequestHandler', function () {
module.unCacheModule('./myRequestHandler');
requestHandler = require('./myRequestHandler');
}

http.createServer(function () { requestHandler.handle(req, res) }).listen(8000);

Obviously, the watchFile chunk could be moved to a helper method off
of module, for example.

There is one caveat: code changes must be atomic between require()s to
guarantee consistency. If a module tree has some deferred require()s,
it's possible that a running handler will require() new code, if it's
not already in the cache. There are a couple of ways to fix this, but
I don't know that it's necessary, since this is really meant to be a
tool to enable frameworks to make it easier for developers to
dynamically upgrade code. I just don't see a use-case that justifies
building a (relatively) complex versioned/time-based (and therefore
flaky) dependency hierarchy.

thoughts? Any chance this could make it into core, or should I just
ship it as an external module?

b.

[1] http://github.com/blaine/node/tree/hotload

[Felix Geisendörfer]

THANK YOU!

Thank you for working on this again. I had a very similar patch
(giving each module its own cache) ready to go previously, but there
were issues with it under heavy I/O load. So basically the hot code
reloading would work perfectly, but as soon as you were trying to do
it during something like a file upload (network traffic that gets
written to disk), the require callbacks would simply never fired.

I'm not exactly sure what changed, I retried the patch each time ryan
fixed some libeio issues, but the problem is gone now. (Sad but likely
reason: My build cache was not properly purged?)

So, I just took my old patch and rebased it to the current HEAD:

http://github.com/felixge/node/commit/326847b5e47339786274572c09ae1c3e3e7fae5a

Looking at it again, it could probably be done much simpler.

Anyway, I have a few thoughts on what should be accomplished here:

A) Each module should get its own cache (by default the parent module
cache is inherited). I think most people will agree with that.
B) It should be possible to either purge a single child module, or all
modules from a modules cache.
C) There should be a convenience function called require.hot() that
purges all of the current modules cache, and then asynchronously loads
the desired module. This allows an entire "module tree" to be
reloaded.

Let me know what you think! The patched linked above implements all of
those requirements, and I just tested it for hot reloading modules
with ~70 child modules in my app. I don't really like the patch
itself, but I would like it if we could agree on the scope for hot
code reloading.

-- fg

PS: If you're wondering why I'm so excited about this: I probably
spend ~30h total with this, trying to hunt down why the reloading
would previously fail under heavy load ... ; )

On Jan 31, 2:29 pm, Blaine Cook <rom...@gmail.com> wrote:
> (seehttp://groups.google.com/group/nodejs/browse_thread/thread/c9b82171d3...

[r...@tinyclouds.org]

On Sun, Jan 31, 2010 at 5:29 AM, Blaine Cook <rom...@gmail.com> wrote:
> thoughts? Any chance this could make it into core, or should I just
> ship it as an external module?
>

> [1] http://github.com/blaine/node/tree/hotload

The patch looks good - thanks! I'd like to include this in the core.
One complaint - would it be possible to modify your tests to not use
wait()? I see that it's pretty dependent on it- it's something I want
to possibly remove in the future - so I don't want so much exposure in
the tests.

[Blaine Cook]

Awesome!

Sure, I was just using wait() to keep things relatively linear in the
tests, but it definitely doesn't depend on that. I'll fix it up and
sync up with your branch tomorrow when I wake up. 3 AM is no time to
be fixing tests. ;-)

b.

[Felix Geisendörfer]

Blaine: Please have a look at:

http://github.com/felixge/node/commits/hotload

I ported my patch to use yours as a base (I think yours is a lot
cleaner). Please let me know what you think.

Ryan: Please don't merge Blaine's patch without considering my
additions. A very common scenario I foresee for this are web
frameworks that want to reload models/controllers for each request
during development. Without the additional hand holding I'm
suggesting, people will very likely run into caching race conditions
and co-routine limitations (I ran into them myself when trying with
transload.it).

require.async() should probably be queued in the same way as
require.hot(). Let me know if you'd like that to be implemented as
well.

(We wouldn't have any of this problems if module loading was entirely
sync, but this would either mean blocking networking support for
dropping remote module loading).

-- fg

[Blaine Cook]

Hi Felix,

I'll definitely take another look - sorry I didn't reply to your
earlier message, my emails, they blew up! I liked a lot of your ideas,
and actually went looking for your hot loading patch before I started,
but I think it'd been removed at some point.

b.

2010/2/2 Felix Geisendörfer <fe...@debuggable.com>:

> Ryan: Please don't merge Blaine's patch without considering my
> additions. A very common scenario I foresee for this are web
> frameworks that want to reload models/controllers for each request
> during development. Without the additional hand holding I'm
> suggesting, people will very likely run into caching race conditions
> and co-routine limitations (I ran into them myself when trying with
> transload.it).

Could you describe this scenario a bit more? I'm not sure I understand
the motivation for reloading modules on each request, versus when they
change on disk?

I've considered having a helper, but because of the closures, it would
depend on an async callback rather than something that could happen
automatically. Though, perhaps something like this:

var server = http.createServer(function (req, res) {
require('myHandler').handle(req, res) });

would work, since a background helper could clear and re-prime the cache?

b.

[Felix Geisendörfer]

> and actually went looking for your hot loading patch before I started,
> but I think it'd been removed at some point.

I think the commit got untangled and GitHub garbage collected it.
Luckily I never clean my local repositories so I was able to retrieve
it ; ).

> Could you describe this scenario a bit more? I'm not sure I understand
> the motivation for reloading modules on each request, versus when they
> change on disk?

Personally, I started reloading my modules by listening to SIGUSR1
signals to my process.

Watching for file changes is actually tricky. You'd have to
recursively attach watchers to all modules referenced by the module
you want to reload (scenario: let's say a utility module gets a bug
fix).

Either way, if you are issuing multiple reload requests in a short
time, Promise.wait() will be called too often which may not blow
anything up, but is certainly not desirable as it will hog memory.
That's why I added a queue to avoid running into this problem all-
together.

> I've considered having a helper, but because of the closures, it would
> depend on an async callback rather than something that could happen
> automatically. Though, perhaps something like this:
>
> var server = http.createServer(function (req, res) {
> require('myHandler').handle(req, res) });
>
> would work, since a background helper could clear and re-prime the cache?

Essentially this is what I think people will do inside of web
frameworks. I just hope they are going to do it async like this:

var server = http.createServer(function (req, res) {

require.async('myHandler').addCallback(function(myHandler) {
myHandler.handle(req, res) });
});

--fg

PS: I'm hanging out as 'felixge' in IRC if you want to hop in and
chat.