Embedding nodejs and addon in the same binary (C++).

[Piotr Szczepanski]

Hello,

I've started doing a simple project, but I've hit a brick-wall on how to achieve certain "effects"

First things first. I have a Win32 application that embeds nodejs enviroment, for simplicity, let's say I have this simple code responsible for embedding node:

int main(int argc, char* argv[])
{
node::Start(argc, argv);
return 0;
}

When run, it works as expected.

Now, I wanted to add some addons for it. Let's say I have written a simple addon:

#include <nan.h>
NAN_METHOD(hello)
{
info.GetReturnValue().Set(Nan::New("hello world!").ToLocalChecked());
}
NAN_MODULE_INIT(Init)
{
NAN_EXPORT(target, hello);
}
NODE_MODULE(addon, Init) 

If I compile it as external module and execute var addon = require('./addon.node') everything works great, but I want to have this already embedded when nodejs starts.

I know this is possible as nodejs itself does this, but I haven't yet figured out how to do this. Does anyone have an idea how do I combine the two c++ files into one binary that will work without the need to import the plugin?

With best regards,

Piotr

[Piotr Szczepanski]

I have managed to achieve my goal. In the hopes it might help someone, somewhere, sometime, below you will find the process I used to do this on Windows 10 using Visual Studio 2015 (should also work on Visual Studio 2017, but I haven't tested it).

The idea is thus: we want to build nodejs as a static library and then link it with our own program that contains an addon and launches node through node::Start(argc,argv). To do this, let's start with our own project first, and then link it with node.

In my project I will be using Nan (Native Abstraction for Node.js - https://github.com/nodejs/nan) to create my own C++ addon, which then will be bound to nodejs internally and will act defacto as internal module. 

1. Addon code

I have created a file addon.cpp and inside I have put a simple skeleton for addon. This is the file that we will be developing in later on to include more functionality and eventually more addons. The structure of the file is almost identical to how you would write normal addon with one exception.

#include <nan.h>
using namespace Nan;

class MyAddon
{
public:
  static NAN_METHOD(Hello)
  {
    info.GetReturnValue().Set(New("hello world").ToLocalChecked());
  }

  static NAN_MODULE_INIT(Init)
  {
    SetMethod(target, "hello", Hello);
  }
};

NODE_MODULE_CONTEXT_AWARE_BUILTIN(myaddon, MyAddon::Init)

Notice that you initialize your addon using NODE_MODULE_CONTEXT_AWARE_BUILTIN instead of usual NODE_MODULE. We do this, so that the addon is treated as built-in by node and we can bind it in bootstrap.

2. Addon registration

Unfortunately, when building nodejs as a static library internal addon registration functions are stripped and never executed. If they are not executed, node will fail to launch with the following error:

bootstrap_node.js:465
  const ContextifyScript = process.binding('contextify').ContextifyScript;
                                   ^
Error: No such module: contextify
    at bootstrap_node.js:465:36

To circumvent this, we need to manually execute all of them when our program launches. To achieve this, I have created a file called external.inl and excluded it from build. But you might as well put it in your main cpp file.

The file itself consists of two parts, first is registration function declaration (as they are not exposed in any header file) and a function that will register all of them. 

The declarations are in C, so we must use extern to declare them properly:

extern "C" {
  // builtin modules
  void _register_contextify();
  void _register_config();
  void _register_async_wrap();
  void _register_cares_wrap();
  void _register_fs_event_wrap();
  void _register_js_stream();
  void _register_buffer();
  void _register_contextify();
  void _register_crypto();
  void _register_fs();
  void _register_http_parser();
  void _register_os();
  void _register_util();
  void _register_v8();
  void _register_zlib();
  void _register_pipe_wrap();
  void _register_process_wrap();
  void _register_signal_wrap();
  void _register_spawn_sync();
  void _register_stream_wrap();
  void _register_tcp_wrap();
  void _register_timer_wrap();
  void _register_tls_wrap();
  void _register_tty_wrap();
  void _register_udp_wrap();
  void _register_uv();
  void _register_icu();
  void _register_url();
  void _register_inspector();

  // my addons
  void _register_myaddon();
}

After that we can create definition of registration function:

void _register_modules()
{
  _register_async_wrap();
  _register_cares_wrap();
  _register_fs_event_wrap();
  _register_js_stream();
  _register_buffer();
  _register_contextify();
  _register_crypto();
  _register_config();
  _register_fs();
  _register_http_parser();
  _register_os();
  _register_util();
  _register_v8();
  _register_zlib();
  _register_pipe_wrap();
  _register_process_wrap();
  _register_signal_wrap();
  _register_spawn_sync();
  _register_stream_wrap();
  _register_tcp_wrap();
  _register_timer_wrap();
  _register_tls_wrap();
  _register_tty_wrap();
  _register_udp_wrap();
  _register_uv();
  _register_icu();
  _register_url();
  _register_inspector();

  // my addons
  _register_myaddon();
}

Notice that I have added my addon at the end as well because it has to be registered as well.

With those 2 parts done, we create our main file, that will evoke node:

#include <node.h>
#include "external.inl"

int main(int argc, char* argv[])
{

  _register_modules();

  node::Start(argc, argv);
  return 0;
}

With this, our end of the code is ready. What is left is slight modification of nodejs code, unfortunately you cannot skip this (at least I haven't found a way how to do this without modification to node).

Nodejs modification has several steps:

1) node.h modification to create proper registration functions

2) bootstrap_node.js modification to include our module in internal variables

3) node.vcxproj modification to fix building (at the moment of writing there are bugs in gyp script which cause errors while building as static library)

1) First let's start with simple modification of src\node.h Open the file and look for define of NODE_C_CTOR, you should find something like this:

#if defined(_MSC_VER)
#pragma section(".CRT$XCU", read)
#define NODE_C_CTOR(fn)                                               \
  NODE_CTOR_PREFIX void __cdecl fn(void);                             \
  __declspec(dllexport, allocate(".CRT$XCU"))                         \
      void (__cdecl*fn ## _)(void) = fn;                              \
  NODE_CTOR_PREFIX void __cdecl fn(void)

This bit, specifically __declspec(dllexport) causes our registration functions to be omitted as they are stripped when built into static library. To fix this and to expose registration functions to linker we change that define to:

//#define NODE_C_CTOR(fn)                                               \
//  NODE_CTOR_PREFIX void __cdecl fn(void);                             \
//  __declspec(dllexport, allocate(".CRT$XCU"))                         \
//      void (__cdecl*fn ## _)(void) = fn;                              \
//  NODE_CTOR_PREFIX void __cdecl fn(void)
#define NODE_C_CTOR(fn) void fn(void)

now all registration functions will have that signature and will be visible to linker.

2) With that modification out of the way, we need to modify javascript bootstrap code to include our addon in global namespace. Fortunately it's straightforward, open lib\internal\bootstrap_node.js, find function setupGlobalVariables() and before its end add a line that will register your addon:

global.addon = process.binding('myaddon');

From now on, you can access your addon in any node script by simply using addon.hello(); without the need to use require or anything else.

3) Now it's time to build node. First we need to generate solution and project files, in command line inside your node directory type:

vcbuild static release x86 nobuild

This will generate all the project and solution files. Of course for it to work, you need to have Visual Studio 2015/2017 installed alongside Python 2.7. Python also should be in your PATH environment variable.

With the files generated, we need to modify them slightly, just so we can build it (hopefully this will get fixed soon though and this step will be obsolete). Open node.vcxproj in your favourite text editor and find the following:

    <Lib>
      <OutputFile>$(OutDir)lib\$(ProjectName)$(TargetExt)</OutputFile>
    </Lib>

This should be close to the beginning of the file. Change it to look like this:

    <Lib>
      <OutputFile>$(OutDir)lib\$(ProjectName)$(TargetExt)</OutputFile>
      <TargetMachine>MachineX86</TargetMachine>
    </Lib>

This made it buildable for Debug, now for Release find:

    <Lib>
      <AdditionalOptions>/LTCG %(AdditionalOptions)</AdditionalOptions>
      <OutputFile>$(OutDir)lib\$(ProjectName)$(TargetExt)</OutputFile>
    </Lib>

And similarily change to:

    <Lib>
      <AdditionalOptions>/LTCG %(AdditionalOptions)</AdditionalOptions>
      <OutputFile>$(OutDir)lib\$(ProjectName)$(TargetExt)</OutputFile>
      <TargetMachine>MachineX86</TargetMachine>
    </Lib>

The last change we have to do, is to remove resources from the project. For some reason there's a conflict in compiling resources, to fix that, change

  <ItemGroup>
    <ResourceCompile Include="tools\msvs\genfiles\node_etw_provider.rc"/>
    <ResourceCompile Include="tools\msvs\genfiles\node_perfctr_provider.rc"/>
    <ResourceCompile Include="src\res\node.rc"/>
  </ItemGroup>

to

  <ItemGroup>
    <ResourceCompile Include="tools\msvs\genfiles\node_etw_provider.rc"/>
    <ResourceCompile Include="tools\msvs\genfiles\node_perfctr_provider.rc"/>
  </ItemGroup>

After that save the project file and we're ready to compile:

vcbuild static release x86 noprojgen

Simply replace nobuild to noprojgen to skip project generation (it would overwrite our changes) and go straight to building. Node is quite big so it will take a while, but as a result you get node\Release\lib directory with compiled static libraries that are ready to be included in our project.

Final steps to building our project is to modify some settings in our addon. In your addon project, in Visual Studio, go to Properties -> C/C++ -> General and in Additional Include Directories input:

(node_dir)\deps\v8\include\;
(node_dir)\deps\uv\include\;
(node_dir)\src\;
(node_dir)\deps\cares\include\
(nan_dir)\;

Making sure you replace (node_dir) with a path where you cloned node and (nan_dir) with path where you cloned Nan.

In C/C++ -> Code Generation make sure to have Runtime Library set to Multi-threaded (/MT) or the debug equivalent if you run Debug version.

In Linker -> General set Additional Library Directories to (node_dir)\$(Configuration)\lib and in Linker -> Input set Additional Dependencies to

node.lib
cares.lib
gtest.lib
http_parser.lib
icudata.lib
icui18n.lib
icustubdata.lib
icutools.lib
icuucx.lib
libuv.lib
openssl.lib
v8_base_0.lib
v8_base_1.lib
v8_base_2.lib
v8_base_3.lib
v8_libbase.lib
v8_libplatform.lib
v8_libsampler.lib
v8_nosnapshot.lib
v8_snapshot.lib
zlib.lib
ws2_32.lib
shlwapi.lib
comctl32.lib
dbghelp.lib
Iphlpapi.lib
psapi.lib
wininet.lib
uxtheme.lib
winmm.lib
wintrust.lib
userenv.lib

Now you're ready to build your addon with nodejs. After the linking (which also can take a long time, it's LARGE!) you are given an exe file that you can run.

Sample run:

nodetest.exe -i
> addon
{ hello: [Function: hello] }
> addon.hello()
'hello world'

Hope this little guide will help someone in the future :)

Best regards,

Piotr