Using "require" in browser and standalone

[John English]

I have a bunch of JS which I want to run as a standalone program using node.js and also within a browser. The code is divided into several file which are loaded as needed using "require", This works well with node.js, but not so well in a browser. In the browser I've tried using Smoothie, which provides a compatible version of "require", but I get a warning:

"Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help http://xhr.spec.whatwg.org/"

When I look at http://xhr.spec.whatwg.org/ I see this:

Synchronous XMLHttpRequest outside of workers is in the process of being removed from the web platform as it has detrimental effects to the end user's experience.

This presumably means that I should find a better implementation, otherwise one day I will suddenly find that my code no longer works.

So, can anyone suggest an implementation of "require" that I can use in a browser that is
(a) compatible with node.js, and
(b) doesn't break any rules that will lead to trouble in the future?

Many thanks for any advice...

[Ryan Schmidt]

I like browserify... It's not an implementation of require but another solution to the problem.

[Aria Stewart]

Indeed that is difficult: the way  most people solve it now is with webpack or browserify, where the requires are done statically, at build time, not at runtime. There's a few modules where that won't work, but for the most part it's a workable way.

You really don't want to be blocking the browser interaction thread, so synchronous is in fact not good there (though you're likely only doing it at startup time), and requests have a real cost in latency, especially unparallelized like a synchronous interface will allow you to do.

It's a set of trade-offs: node went with synchronous IO because it simplified the interface suitably for the use-case, and browser module systems went for pre-building, because of the constraints and latency involved in IO there.

[John English]

On Tuesday, November 17, 2015 at 7:13:49 PM UTC+2, ryandesign wrote:

I like browserify... It's not an implementation of require but another solution to the problem.

Ah, I've just realised what browserify actually does. Yes, that may be the best solution, but it means making some bloated bundles... I suppose there are worse things than that, though. Thanks for the tip.

[John English]

On Tuesday, November 17, 2015 at 7:17:00 PM UTC+2, Aria Stewart wrote:

Indeed that is difficult: the way  most people solve it now is with webpack or browserify, where the requires are done statically, at build time, not at runtime. There's a few modules where that won't work, but for the most part it's a workable way.

OK. I was trying to do it on-the-fly, but I suppose I could look at just bundling everything up, or maybe have different bundles for different situations, or... (sigh).
 

You really don't want to be blocking the browser interaction thread, so synchronous is in fact not good there (though you're likely only doing it at startup time), and requests have a real cost in latency, especially unparallelized like a synchronous interface will allow you to do.

 The only other ideas I had were
(a) to fork it off into a separate thread (which I could do easily in Java, but in JS... maybe use setTimeout? I don't know enough about the JS threading model)
(b) write my own version of "require" which injects a <script> node into the DOM tree and let the browser do the loading rather than using XHR...

Either way it seems I would have to figure out a way to notify the main thread when things were ready to roll. (More sighs,)

Does any of this sound remotely practical?

[John English]

On Tuesday, November 17, 2015 at 7:13:49 PM UTC+2, ryandesign wrote:

I like browserify... It's not an implementation of require but another solution to the problem.

I want to be able to have a common code base; modules that I require can require other modules, and I don't want them to have to figure out which environment they're running in first.

[Kevin "Ingwie Phoenix" Ingwersen]

Excuse my shameless advertistment… But WebPack might do that, actually. Even in a browser, it can emulate/pollyfill common nodejs things, so you actually dont have to think about the environment, you just code. o.o

https://github.com/webpack/webpack 

[Aria Stewart]

That in fact works -- modules can provide alternate implementations when the runtimes can't provide the features, and browserify provides shims when they can mostly. 

[Aria Stewart]

On Wednesday, November 18, 2015 at 4:43:09 PM UTC-5, John English wrote:

On Tuesday, November 17, 2015 at 7:17:00 PM UTC+2, Aria Stewart wrote:

Indeed that is difficult: the way  most people solve it now is with webpack or browserify, where the requires are done statically, at build time, not at runtime. There's a few modules where that won't work, but for the most part it's a workable way.

OK. I was trying to do it on-the-fly, but I suppose I could look at just bundling everything up, or maybe have different bundles for different situations, or... (sigh).
 

You really don't want to be blocking the browser interaction thread, so synchronous is in fact not good there (though you're likely only doing it at startup time), and requests have a real cost in latency, especially unparallelized like a synchronous interface will allow you to do.

 The only other ideas I had were
(a) to fork it off into a separate thread (which I could do easily in Java, but in JS... maybe use setTimeout? I don't know enough about the JS threading model)
(b) write my own version of "require" which injects a <script> node into the DOM tree and let the browser do the loading rather than using XHR...

Yeah, there are no threads available.

There may be clever options available with Service Workers in some browsers, but that's bleeding edge stuff. And in any case, _any_ asynchronous interface in node is going to have a different interface than a sync one -- the limitation is in the synchronous interface.

require.js is similar concepts to node's require, but fully asynchronous. I find it complex and that browserify is a better trade-off for me.

[Jimb Esser]

If you want it to at least "feel" like it's "on-the-fly" (and, most importantly, require no external build steps), check out browserify-middleware - you just include the middleware and don't think about it anymore, just any JS in a script tag and it handles bundling everything which that file require()s.  For a production environment, it has just slightly more start-up time (as it will bundle and minify at startup instead of at build time), but I find it's a good trade-off.  Not sure if WebPack also has an on-the-fly middleware kind of thing as well.

[John English]

OK, thanks for all the advice. Browserify seems OK but requires an external build, which may be a limitation for me; I'm going to have a play with require.js (thanks, Aria), webpack (thanks, Ingwie) and browserify-middleware (thanks, Jimb) and see how they pan out as alternatives.