nexe compatibility (packaging node-red as a standalone exe)

[Simon H]

Hi all,

I've successfully packaged node-red as a standalone exe using nexe.  

There were a few minor changes to node-red to do so; these are all related to making dynamic 'require' statements work (like how all 'node' files are loaded!!).

Nexe bundles all required .js files, and compiles them into node.exe; but it does not know what to bundle if require statements contain variables.  However, it can be fooled by having static require statements in the right places).

The (current) resulting executable works for everything except 'manage pallette' is missing (not checked why yet...).

Is there any appetite for incorporating the minor mods to enable this into node-red?  

If so, I'll clean it up best i can and give a pull request.

best regards,

Simon

[Dave C-J]

do you have an example of the specific mods required ?

How does this approach compare with something like Electron ?

as attempted here... 

https://github.com/dceejay/electron-node-red 

Palette Management requires npm to be available... so that needs to be an executable on the path somewhere.

[Simon H]

Dear Dave,

ohh... Electron; did not find this in my research, but seems much more mature than nexe....  I'll do some serious research on that based on your repo.... see if i can build in my environment (windows or linux... ).

The mods consist of changing the dynamic 'require' calls to call a function 'noderequire', which by default is 'require', unless a js called 'modulerequire' is in the folder below node-red.

This mod is done in registry/installer, registry/loader, registry/localfilesystem, and storage/index.

The file 'modulerequire', if it exists, contains static requires for all the dynamic .js files, with 'if (false)' around them, so they never actually get required; but browserify sees these require statements, and includes the files into the combined source chunk, and notes where they are by some module id.

The function 'noderequire' just does a require, but from inside the 'modulerequire.js' (which must be the way, as browserify only maps from where the require is done....).  This then means that the dynamic requires are fulfilled.

In order to wrap the whole thing, 'modulerequire.js' is created dynamically at build by searching certain paths, and at the same time, a file containing all the other resources is created (all the node html and json files, for example).

The resources are later accessed by node-red by the initial inclusion of a js which replaces certain fs. functions (readFile, stat, plus others); so node-red sees the resources as being there, even though they are actually built in.

In this way, flows, settings, etc. can also be 'wrapped in'.  It's not the most efficient wrapping of resources as all the resource files are bin64 encoded :(.  But I'm thinking they could almost as easily be read from a zip or other archive.

I just pushed the basic node-red mods to here https://github.com/btsimonh/node-red

I've not got a repo of the actual wrapping process yet.

There was also a dynamic require issue with oauth2orize, which I patch, and a wrapper for the lot which loads the fs modification before node-red.

I will try Electron and report back on differences.  I guess the immediate difference (from literally 1 minute looking at Electron) is that nexe only wraps node + node-red, and has no browser integration.  Basically, it patches the node sources to include the (compiled) single browserified js, and builds a version of node with the startup js you specify; so fairly raw and bare, but workable result.  I am guessing that Electron has options to NOT include the browser if required, and so can produce a very similar result.

When you say 'attempted' when referring to your repository, what issues did you come across, and what remains to be resolved?  From a quick look, there's no node-red modification required?

best regards,

Simon

[Dave C-J]

Hi,

correct - for electron no Node-RED modification is required. - I actually like the browser being included as we need that for editing things anyway - and by default I include the dashboard project also so "that" is the app - with the editor available behind it...

I said attempted - as it is still fairly basic - but it does produce working dmg, deb and .exe files... but the docs are woefully incomplete - I am building on Mac so it "works for me" - but I know I did some step I forgot to document - just can't recall what it was... was around prereqs of course. 

[Simon H]

Hi again Dave,

taking a look at electron.

For me, the packaging as asar archive is a little 'open'.  I can see advantages to the fact the JS will be compiled on the target (JIT), as optimisation can be better (nexe's use of pre-compiled js is nice; smaller; but some optimisation steps are skipped).  I'm guessing that the asar archive access could be adapted to read from an encrypted version, or even from a zipped version (there is enough code out there which does this - physicsfs is one of my favourites).

I don't think either can automagically build in any native components, so likely not going to end up with a true single exe if doing anything a bit specialised.  (I would love it if MS would make it easier to load and run DLLs from memory; but the chances of anything getting past AV is low).

So for my main app, nexe is still a first choice; still needs adaptation to achieve exactly what I want (like modifying specific native modules to be static libraries, and working out how to integrate these into the node build), but it's smaller and a little obscure.

However, Electron has huge advantages where GUI is concerned; I was considering a 'web' app and restricting users to chrome; but if built with electron I would never need to be concerned about browser compatibility.... one whopping plus :).

I think my proposed mods to the 'require' part where nodes are dynamically loaded may have merit in other areas; dynamic require is not something which is 'common' practice in node (i.e. I only had to patch in 3 places in red, an one other dependency), and so providing a hook to override the dynamic require functionality may be of use to others.

Could you take a couple of minutes to review and assess if the mods have other merit?

best regards,

Simon