When require() is slow: ideas on improving node.js require() performance on startup

[Benjamin Pasero]

Hi,

in my environment it is very important to startup a node.js server instance in a very short time. Now, my server has many dependencies (around 10.000 files in node_modules). My measurements show that the pure fact that node.js resolves all require() calls from the entry point is making the startup very slow. In this case, around 500 files are being loaded through require(). The problem is even worse for me when I host node.js from a UNC share where loading files one by one is a real bottleneck.

I would like to ask the community if there is any idea for a solution that would speed up require() in node.js. So far I had two ideas:

1. parallelize fs operations in require()
2. allow to resolve require() calls from a zip file
3. implement true async module loading through requireJS or similar

Option 1 is the obvious: Instead of fs.readFileSync each required file one by one, do them all in one batch and collect the results. This should increase the performance by an order of magnitude, especially if the disk is slow (as in my UNC case). The only drawback from this solution I could see is that it would no longer be possible to overwrite the behavior of the require() function from within a dependency because file loading would happen in parallel. But this seems like not a typical use case.

Option 2 would allow to specify an archive file to resolve require() against. This seems like actually supporting commonjs packages in zip archive form (http://wiki.commonjs.org/wiki/Packages/1.0). The idea is that my entire node_modules folder could be one zip file that gets loaded into memory on startup (at least the file/folder structure) and all require() calls resolve against the archive.

Option 3 is something I was hoping to get already using the requireJS node module. However they make clear that module loading is sync to the execution context. So this is not an option currently, but maybe there is a plugin that does this?

Any other ideas are greatly appreciated. Maybe there is even a solution out there I could benefit from ?

Thanks,

Ben

[Matt]

Did you think about a possibility of 4) redesign your application so it's not loading 10k modules? That's an INSANE amount of dependencies. Even making them parallel in some way is still going to require thousands of syscalls. In general I don't think it's going to be a priority for node to optimise for startup time, but I could be wrong - I'm not a core team member.

--
--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
 
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Angel Java Lopez]

No experience.... but I would measure the file system time in serving your application.

You could store the files in a SSD disk (a "flash" disk, sorry, I'm not a hardware  guy). Comparing times will give you more context about the issue.

[Azer Koçulu]

How about bundling it in parts ?

[Benjamin Pasero]

I should clarify that those 10.000 files are not loaded on startup, it is really just around 300 files. I could defer loading to a later time but since require is always sync this would just mean to block the V8 thread at a later time which is not what I want to do.

Moving the files off the UNC to a different place is unfortunately not an option in my scenario.

[Ben Noordhuis]

On Mon, Jul 29, 2013 at 3:48 PM, Matt <hel...@gmail.com> wrote:
> Did you think about a possibility of 4) redesign your application so it's
> not loading 10k modules? That's an INSANE amount of dependencies. Even
> making them parallel in some way is still going to require thousands of
> syscalls. In general I don't think it's going to be a priority for node to
> optimise for startup time, but I could be wrong - I'm not a core team
> member.

Well, we try to keep node.js snappy. But loading 10k files at
start-up is pretty out of the ordinary, I'm not sure it's worth the
time trying to optimize for edge cases like that.

[Benjamin Pasero]

Is there a technical reason why node.js does not execute the require in parallel?

[Matt]

Because it returns the module.exports object so it can't.

--

[greelgorke]

that, and the load-order of modules is more predictable with sync loading. sometimes that matters.

[phidelta]

It sounds like you don't habe to load all files on startup, but do so in order to not block later. So maybe your solution is to only require what is needes at start and then switch to async require.

I wrote an async require last week as a test. Since you can in fact require('module') and interact with the stuff in there it's pretty easy to go async with it.

of course you do loose some benefits such as automatic load ordering, but if performance in loading a lot of modules is your concern, then that might be a way to go.

have a look at https://github.com/joyent/node/blob/master/lib/module.js and you'll see that the module loader is actually quite simple.

If you need help, give me a shout out, as I said I played with something similar last week but killes it as overkill for my scenario.

[phidelta]

[phidelta]

[phidelta]

[Benjamin Pasero]

ok, so far I have not heard a technical reason. the fact that require() returns module.exports obviously would have to change to AMD require with callback. And the order of modules could probably be ensured by using a similar approach as requireJS load order (http://requirejs.org/docs/1.0/docs/api.html#order). 

[Ben Noordhuis]

There are no doubt modules on npm that implement AMD-style module
loading. If there aren't, you should be able to implement it quite
easily.

That said, I don't think I/O will be your only bottleneck when loading
10k JS files (unless none of those files are in the operating system's
file cache, then I/O will probably dominate). I expect that V8 will
spend a non-negligible amount of CPU time parsing and compiling all
that source code.

On a final note, asynchronous file I/O is relatively slow in node.js.
Each call to fs.open(), fs.read() and fs.close() translates into a
round-trip to a thread pool of limited size (defaults to 4 threads,
configurable with the UV_THREADPOOL_SIZE environment variable which is
clamped to the range 1-128.)

What that means is that you can fire off 10k concurrent file I/O
requests but the _effective_ concurrency is the size of the thread
pool.

Effective thread pool management is a topic of ongoing research but
it's hard. With many kinds of workloads, having more threads actually
_decreases_ the rate of throughput rather than increase it. I have
alternative thread pool implementations sitting in branches that speed
up some workloads 10x while slowing down others by that same amount or
more. :-(

[Jorge Chamorro]

On 30/07/2013, at 11:52, Ben Noordhuis wrote:
>
>
> Effective thread pool management is a topic of ongoing research but
> it's hard. With many kinds of workloads, having more threads actually
> _decreases_ the rate of throughput rather than increase it. I have
> alternative thread pool implementations sitting in branches that speed
> up some workloads 10x while slowing down others by that same amount or
> more. :-(

In the end, it can't go any faster than the disk, but the disk is much faster doing large sequential read/writes than random access ones (even on SSDs random access is much slower than sequential r/w), which (random access) is what ends up happening when dozens of threads fight for i/o in parallel: i/o troughput gone haywire.

IOW, reading/writing a hundred files sequentially is *always* faster than reading/writing them in parallel (to the same disk, of course).
--
( Jorge )();

[Ben Noordhuis]

That's a big part of it.

Another aspect is that concurrent reads from the same file are
effectively unordered, destroying the operating system's ability to do
proper read-ahead. Serializing reads would go some way towards
addressing that but kills performance with yet other workloads. You
just can't win. :-/

[Benjamin Pasero]

Why is node.js not using the native capabilities of loading files async on platforms where it is supported like windows?

Btw one positive aspect of AMD for modules is that you can truly do lazy loading of dependencies on a node.js server that is composed from many plugins that you dont want to load all on startup.

I realize that AMD is not possible in node.js because while you can write your own code in AMD, all dependencies are not. One would have to rewrite all dependencies to AMD style to have true async dependency loading.

[Jorge Chamorro]

A queue+thread pool per I/O *device* would speedup things under certain circumstances, istm, but perhaps it's not worth the effort because that circumstances don't happen so often.
--
( Jorge )();

[Ben Noordhuis]

On Tue, Jul 30, 2013 at 8:36 PM, Benjamin Pasero
<benjami...@gmail.com> wrote:
> Why is node.js not using the native capabilities of loading files async on
> platforms where it is supported like windows?

Because there are restrictions on native asynchronous file I/O that
make it impractical for anything that isn't a database.

That's not a knock on Windows, by the way - Linux and FreeBSD have
similar limitations.

[Ryan Dahl]

Benjamin, out of curiosity how slow is slow? What are the absolute
startup time numbers?

On Tue, Jul 30, 2013 at 5:52 AM, Ben Noordhuis <in...@bnoordhuis.nl> wrote:
> On Tue, Jul 30, 2013 at 8:46 AM, Benjamin Pasero
> <benjami...@gmail.com> wrote:
>>> ok, so far I have not heard a technical reason. the fact that require()
>>> returns module.exports obviously would have to change to AMD require with
>>> callback. And the order of modules could probably be ensured by using a
>>> similar approach as requireJS load order
>>> (http://requirejs.org/docs/1.0/docs/api.html#order).
>
> There are no doubt modules on npm that implement AMD-style module
> loading. If there aren't, you should be able to implement it quite
> easily.
>
> That said, I don't think I/O will be your only bottleneck when loading
> 10k JS files (unless none of those files are in the operating system's
> file cache, then I/O will probably dominate). I expect that V8 will
> spend a non-negligible amount of CPU time parsing and compiling all
> that source code.

Seems like a case where precompiled JS files would help. We did some
work on this at some point investigating whether V8's snapshot
facility could be extended to Node's library or even user code. My
vague memory of the conclusion was: it would be very difficult but
perhaps doable.

[Ben Noordhuis]

Hi Ryan, long time no chat.

Pre-compiling has some limitations. IIRC, the generated code must be
< 1 MB and it's not subject to the same optimizations as regular JS
code.

[Benjamin Pasero]

Around 15 seconds to load 300 files when UNC hasn't cached any files. We will change this situation by avoiding UNC drive to host node.js and instead use a local disk. On my local SSD user code gets hit around 780ms after server startup. So, 780ms is spent in require() calls and compile calls. This makes about 50% of the entire startup time of my server (the rest is spent doing stuff on startup).