Examples of C++ modules, for Node.js

[Ryan]

Hello,

I'm learning how to write C++ modules, for Node.js. I'm sort of brute-forcing my learning by looking at examples, referencing the documentation for v8 and trial and error. The only two modules that I can seem to think of, as a reference, are hiredis-node and node.bcrypt.js. Both have helped me get to where I am now, which is able to compile a base skeleton module, using node-gyp. Are there any other modules that are written in C++ that would be simple + good to reference?

For example, I'm currently trying to write a module that simply wraps and exposes a class from an existing third-party lib, HDF5. I've got it all written to the point where I think I am supposed to initialize an object of that class I want to expose, but when I include the code to initialize it, I cannot `require` it from within Node. It will compile, but I cannot require it. I do not think it is actually compiling properly. The HDF5 lib comes with a special wrapper binary around g++ that sets up the compile environment for you, so I guess I need to figure out how to tell node-gyp to use that.

My code is here, if anyone wants to take a peek: https://github.com/ryancole/node-hdf5

Thanks

[Nikhil Marathe]

Hi Ryan,
I gave a talk at a local conference about writing bindings:
Slides: http://www.slideshare.net/nsm.nikhil/writing-native-bindings-to-nodejs-in-c
Code: https://github.com/nikhilm/jsfoo-pune-2012

In addition you can check out node-taglib
(https://github.com/nikhilm/node-taglib) which links to an external
library.
Unfortunately all of these still use node-waf and not gyp.

For gyp, looking at the gyp files for node which has bindings for
crypto and zlib might be a good idea.

Best,
Nikhil

[Nathan Rajlich]

Hey Ryan,

So ya, it looks like you're not linking against the HDF5 library, only including its header files. You need to add a "libraries" section to the binding.gyp file. See https://github.com/pkrumins/node-png-sync/blob/7fcba5ae569f23a1aec277fdbcd47da8a728f4c1/binding.gyp#L17-20 for an example. I'm guessing you'll need to put something like "-lhdf5" or whatever the library gets installed as.

Also, if you use the latest v0.7.x release of node then you'll get a more helpful error message about why the module cannot be loaded.

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[Marco Rogers]

Here are the slides from my nodeconf talk last year. Not much about building properly, but some approachable info about how a native module works in node. Hope it helps.

http://marcorogers.com/nodeconf-2011/node_addons_presentation.html

:Marco

[pconstr]

Hi Ryan,

I found this (http://kkaefer.github.com/node-cpp-modules/)
presentation helpful to get started with c++ modules.
It includes good examples (https://github.com/kkaefer/node-cpp-
modules) but covers building with node-waf only, not gyp.

-Carlos

[rhasson]

Just to add to Nathan's comments, you're missing a "libraries" that points to the the HDF5 shared library that was compiled separately.  for example:  'libraries': ['/home/user/hdf5/lib/hdf5.so']

Also, I noticed that when compiling with node-gyp (which I love btw) you need to set LD_LIBRARY_PATH environment variable to the directory where the HDF5 shared library is located.  for example:   

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/user/hdf5/lib

after you do that, try building your project again.

Roy

[Ryan Cole]

Oh, wow. I did not have email notifications enabled for responses, so I did not expect to sign in and see so many helpful responses. Thank you for these.

I'm going over them now and will apply this insight to what I'm doing. Also, per Nathan's suggestion, I think I will update from node 0.6.7 to 0.7, at least. I'll let you all know the outcome of this, soon. :)

Thanks,

Ryan

[andrew morton]

I've also been working my way up that learning curve. One problem is that there's a lot of dated examples that kept leading me a stray. 

It took me way too long to find this and I hadn't seen it mentioned yet but the  docs included with Node are really good: http://nodejs.org/api/addons.html

I'll second the suggestion to read Konstantin Käfer's slides and example code, they were a great resource for understanding async code. 

If you're working with 0.6.x and building with node-waf the waf book is handy: http://docs.waf.googlecode.com/git/book_16/single.html

[Ryan Cole]

Is there no way to specify the library search path from within the binding.gyp file? I'm just trying to see if it's possible to make `node-gyp configure` spit `-L/usr/local/hdf5/lib` out into the Makefile, or whatever it builds.

I'll be searching the docs, but I haven't run across it yet. I'll let you know if I find something.

Ryan

[Ben Noordhuis]

On Mon, Apr 23, 2012 at 05:12, Ryan Cole <ry...@rycole.com> wrote:
> Is there no way to specify the library search path from within the
> binding.gyp file? I'm just trying to see if it's possible to make `node-gyp
> configure` spit `-L/usr/local/hdf5/lib` out into the Makefile, or whatever
> it builds.

{
'ldflags': ['-L/usr/local/hdf5/lib']
}

[Ryan Cole]

I have adjusted the binding.gyp file to include the linker settings required for including the HDF5 libs I need. I believe that the current gyp file mirrors a Makefile that I am able to get working, for HDF5's stand alone compile. The only difference is that you're supposed to run the HDF5 Makefile with this custom binary, h5c++, that I think does some additional build configurations for you.

I only see two options here, I think. One option would be to compile this node lib using that h5c++ binary. The other option would be to figure out what that h5c++ binary does, and reproduce it within the binding.gyp file.

Currently, using this binding.gyp file, my node lib will compile and I assume it also links with hdf5 lib now. (https://github.com/ryancole/node-hdf5/blob/master/binding.gyp) I have updated to a node version 0.7+, and can see the error message when I try to require my node lib, now. It looks like this:

ryan@ryan-server:~/repos/node-hdf5$ node

> require('./build/Release/hdf5')

Error: libhdf5.so.7: cannot open shared object file: No such file or directory

    at Object..node (module.js:475:11)

    at Module.load (module.js:351:32)

    at Function._load (module.js:309:12)

    at Module.require (module.js:357:17)

    at require (module.js:373:17)

    at repl:1:2

    at REPLServer.eval (repl.js:110:21)

    at Interface.<anonymous> (repl.js:249:12)

    at Interface.emit (events.js:87:17)

    at Interface._onLine (readline.js:178:10)

That's with my .gyp file as-is. When I try to run this command, "node-gyp build CXX=/path/to/h5c++", I get an error saying that I need to try again using the -fPIC parameter. No matter where I specify that parameter, it just seems to tell me to try again using -fPIC. I think I may be heading down the wrong path with that -fPIC, though.

Anyway, just wanted to document my leaving-off-spot so that I can pick up with it tomorrow. Also, so that if anybody has any suggestions they could share them too! :)

Thanks all,

Ryan

[rhasson]

I found out that I needed to set up my LD_LIBRARY_PATH env variable otherwise it will not find the path to the shared library.  set it up like this:  export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/hdf5/lib

Then rebuild the project.  Once rebuilt type: ldd build/Release/hdf5.node (or whatever your module name is) and see if your HDF5 shared library you trying to link to is listed. If not than it didn't link correctly.

Roy

[Ryan Cole]

Roy,

I notice you mentioned this early in the email chain. I should have tried it earlier. It fixes the issue. I wonder why the ldflags in binding.gyp does not properly set this up, in my case? Or, is LD_LIBRARY_PATH just another required step no matter what? Also, I did not know about the ldd command - very handy!

Thanks,

Ryan

[rhasson]

I'm using node-gyp and ldflags didn't work for me either.  Maybe Nathan has an idea, but I found out that I needed to set the LD_LIBRARY_PATH otherwise my shared library wouldn't link.

[Ben Noordhuis]

On Tue, Apr 24, 2012 at 04:46, rhasson <rha...@gmail.com> wrote:
> I'm using node-gyp and ldflags didn't work for me either.  Maybe Nathan has
> an idea, but I found out that I needed to set the LD_LIBRARY_PATH otherwise
> my shared library wouldn't link.

The ldflags setting is what is passed to the linker at the time of
compilation. LD_LIBRARY_PATH affects the search path of the dynamic
linker, which is what is run when a program starts up.

What I mean to say is that a linker is not the same thing as a dynamic
linker (though they're related) and that gyp is only involved in the
compilation phase.

[rhasson]

ok that makes sense.  So the question is, once gyp compiled and the linker linked the shared library into the .node module using ldflags.  How do you require the module without needing to set the LD_LIBRARY_PATH ?

Now if I set the LD_LIBRARY_PATH I can require the module and it works.  But if I don't set it, require fails saying it can't find the shared library.

Roy

On Monday, April 23, 2012 11:03:15 PM UTC-4, Ben Noordhuis wrote:

[Ben Noordhuis]

On Tue, Apr 24, 2012 at 05:23, rhasson <rha...@gmail.com> wrote:
> ok that makes sense.  So the question is, once gyp compiled and the linker
> linked the shared library into the .node module using ldflags.  How do you
> require the module without needing to set the LD_LIBRARY_PATH ?

You don't*. The dynamic linker only searches in a set of blessed paths
unless you set LD_LIBRARY_PATH. You can dlopen() the .so and look up
symbols with dlsym() (but watch out for C++ name mangling).

* Okay, that's not entirely true. There are hacks like setting $ORIGIN
but not all linkers support that.

[rhasson]

Ben, 

When I get rid of LD_LIBRARY_PATH and include the ldflags and libraries lines in by binding.gyp file the shared library is not linked as shown by ldd.  If I set the LD_LIBRARY_PATH then it links it correctly.

Is ldflags not parsed correctly? 

Here is my gyp file:

{

  'targets': [

    {

      'target_name': 'freeling',

      'type': 'loadable_module',

      'product_extension': 'node',

      'product_prefix': '',

      'include_dirs': ['.','/home/roy/freeling/free3/include', '/home/roy/cvv8/include/cvv8'],

      'conditions': [

         ['OS=="linux"', {

          'link_settings': {

            'ldflags': ['-L/home/roy/freeling/free3/lib/'],

            'libraries': ['/home/roy/freeling/free3/lib/libfreeling.so']

            },

         }],

       ],

      'sources': ['freeling.cc', 'freeling_tokenizer.cc', 'freeling_splitter.cc', 'helper.cc'],

    },

  ],

}

On Monday, April 23, 2012 11:03:15 PM UTC-4, Ben Noordhuis wrote:

[Ben Noordhuis]

You should set it like this:

'link_settings': {
'ldflags': ['-L/home/roy/freeling/free3/lib/', '-lfreeling'],
},

[rhasson]

I found out that for Solaris ELF executables you can pass a "-R/path/to/lib" argument which will record the path and library name into the executable which will allow it to find it at runtime without needing to update the LD_LIBRARY_PATH.  Is there an equivalent argument for Linux?

On Monday, April 23, 2012 11:48:33 PM UTC-4, Ben Noordhuis wrote:

[rhasson]

I found out that for Solaris ELF executables you can pass a "-R/path/to/lib" argument which will record the path and library name into the executable which will allow it to find it at runtime without needing to update the LD_LIBRARY_PATH.  Is there an equivalent argument for Linux?

On Monday, April 23, 2012 11:48:33 PM UTC-4, Ben Noordhuis wrote:

[Ben Noordhuis]

On Tue, Apr 24, 2012 at 06:11, rhasson <rha...@gmail.com> wrote:
> I found out that for Solaris ELF executables you can pass a "-R/path/to/lib"
> argument which will record the path and library name into the executable
> which will allow it to find it at runtime without needing to update the
> LD_LIBRARY_PATH.  Is there an equivalent argument for Linux?

Yes. `man ld` and search for rpath. Restrictions may apply to setuid
root programs (which node shouldn't be).

[mdcb808]

On 04/23/12 21:11, rhasson wrote:
> I found out that for Solaris ELF executables you can pass a
> "-R/path/to/lib" argument which will record the path and library name
> into the executable which will allow it to find it at runtime without
> needing to update the LD_LIBRARY_PATH. Is there an equivalent argument
> for Linux?

I don't know what linker comes by default with Solaris nowadays - (is
there still a solaris nowadays? :) )
the GNU linker has -R or -rpath= that should work on all supported platforms
From my end though, not necessarily the best way because your 'end
product' has a path reference that might be different later or somewhere
else. Look at 'ldconfig' for system wide defaults, LD_LIBRARY_PATH will
still work if you deliberately want an override