Re: [nodejs] OpenEmbedded cross-building native libraries with node-gyp

[Ben Noordhuis]

On Thu, Feb 7, 2013 at 10:03 PM, Kevin Baker <kba...@gmail.com> wrote:
> Hello all,
>
> I am working on Node.JS's OpenEmbedded support for a project here, test
> platform is Yocto "danny"/armv7a-vfp-neon-poky-linux-gnueabi/OMAP3730.
>
> So far I have updated the recipe to v0.8.15 and it builds, compiles, and
> runs node correctly, with support for npm and pure js libraries. I plan to
> post this back to OE-core once I clean up the recipe, but I am trying to get
> native builds working to build at least node-sqlite3 for our application. I
> would really like to build the libraries in the OpenEmbedded cross-compile
> environment and not on the target device itself, like what was posted here
> http://fastr.github.com/articles/Node.js-on-OpenEmbedded.html .
>
> I have made lots of progress on this, up to the point where it cross-builds
> the library for ARM with node-gyp. This required a bit of hacking on
> node-gyp, using native gyp instead of the one included with node, and some
> differences in the way node-gyp handles paths, since they are being run out
> of the cross-compile environment directory instead of relative to / .
>
> Unfortunately the library that is getting built doesn't work in the target
> environment:
>
> root@device:/usr/lib/node_modules/npm/node_modules/sqlite3# node
>> var sqlite3 = require('./node-sqlite3.node').verbose();
> Error: /usr/lib/node_modules/npm/node_modules/sqlite3/node-sqlite3.node:
> undefined symbol: init
> at Object.Module._extensions..node (module.js:485:11)
> at Module.load (module.js:356:32)
> at Function.Module._load (module.js:312:12)
> at Module.require (module.js:362:17)
> at require (module.js:378:17)
> at repl:1:15
> at REPLServer.self.eval (repl.js:109:21)
> at rli.on.self.bufferedCmd (repl.js:258:20)
> at REPLServer.self.eval (repl.js:116:5)
> at Interface.<anonymous> (repl.js:248:12)
>>
>
> My question is, what are the steps from here? I'm not really familiar with
> the internals of node libraries or node-gyp to know what is not working. I
> can provide the built object or an armv7a node if it would help...
>
> Looking at an objdump of the module, it doesn't look like there is an
> init...
>
> venture@embedded-dev:/home/yocto-danny/poky/build/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/node-sqlite3-git-r5/git/build/Release$
> $OBJDUMP -t node_sqlite3.node | grep -i init
> 00009f50 l d .init 00000000 .init
> 000a9370 l d .init_array 00000000 .init_array
> 000a9370 l O .init_array 00000000
> __frame_dummy_init_array_entry
> 0001cf24 l F .text 00000008 sqlite3MemInit
> 0001d018 l F .text 00000008 noopMutexInit
> 0001d044 l F .text 00000008 pthreadMutexInit
> 00020f90 l F .text 000000a8 sqlite3VdbeIntegerAffinity
> 000219a8 l F .text 00000084 applyNumericAffinity
> 00021ea4 l F .text 000000b4
> sqlite3ExprNeedsNoAffinityChange
> 0002225c l F .text 00000150 sqlite3AffinityType
> 000223ac l F .text 00000080 sqlite3ExprAffinity
> 0002242c l F .text 00000044 sqlite3CompareAffinity
> 00022470 l F .text 00000058 comparisonAffinity
> 000224c8 l F .text 00000044 sqlite3IndexAffinityOk
> 0002a084 l F .text 0000009c sqlite3TableAffinityStr
> 0002ce4c l F .text 000000b4
> sqlite3IndexAffinityStr.clone.111
> 00033ad4 l F .text 000000e4 codeApplyAffinity
> 000373ac l F .text 00000074 nodeReaderInit
> 0003e670 l F .text 00000218 btreeInitPage
> 0003e888 l F .text 0000002c pageReinit
> 00046cc0 l F .text 00000060 applyAffinity
> 0004bee4 l F .text 000001c8 sqlite3Fts3InitTokenizer
> 0004d144 l F .text 00000074 pcache1Init
> 00051134 l F .text 00000054 getAndInitPage
> 000718b4 l F .text 000001b0 sqlite3InitCallback
> 00085a88 l F .text 000000ac fts3ExprTermOffsetInit
> 00077858 l F .text 000004dc rtreeInit
> 00077e74 l F .text 000004b0 sqlite3InitOne
> 00078324 l F .text 00000100 sqlite3Init
> 00081d8c l F .text 00001254 fts3InitVtab
> 0001bf04 w F .text 000000f4
> _ZNSt11_Deque_baseIPN12node_sqlite39Statement4CallESaIS3_EE17_M_initialize_mapEj
> 0004d284 g F .text 00000044 sqlite3_os_init
> 0005e464 g F .text 00000150 sqlite3_backup_init
> 00000000 *UND* 00000000 uv_async_init
> 00000000 *UND* 00000000 pthread_mutexattr_init
> 000116d8 g F .text 000001bc
> _ZN12node_sqlite39Statement4InitEN2v86HandleINS1_6ObjectEEE
> 00000000 F *UND* 00000000 pthread_mutex_init@@GLIBC_2.4
> 0000b8e0 g F .text 000001ec
> _ZN12node_sqlite38Database4InitEN2v86HandleINS1_6ObjectEEE
> 00031308 g F .text 000005dc sqlite3_initialize
> 0000f374 w F .text 000000f4
> _ZNSt11_Deque_baseIPN12node_sqlite38Database4CallESaIS3_EE17_M_initialize_mapEj
> 00009f50 g F .init 00000000 _init
>
> Thanks,
>
> Kevin Baker

The issue is that node.js looks for a symbol called "init" but it
seems your compiler / linker named it "_init". (Unless it's a
completely unrelated symbol. The fact that it's in .init instead of
.text is suggestive.)

Some things you can check:

* Is the function declared static? (It shouldn't be.)
* Does it have C linkage, i.e. extern "C" void init(...)?
* Are you compiling with -fleading-underscore? (Don't.)

Hope that helps.

[Jorge Chamorro]

On 07/02/2013, at 22:03, Kevin Baker wrote:

> Unfortunately the library that is getting built doesn't work in the target environment:
>
> root@device:/usr/lib/node_modules/npm/node_modules/sqlite3# node
> > var sqlite3 = require('./node-sqlite3.node').verbose();
> Error: /usr/lib/node_modules/npm/node_modules/sqlite3/node-sqlite3.node: undefined symbol: init
> at Object.Module._extensions..node (module.js:485:11)
> at Module.load (module.js:356:32)
> at Function.Module._load (module.js:312:12)
> at Module.require (module.js:362:17)
> at require (module.js:378:17)
> at repl:1:15
> at REPLServer.self.eval (repl.js:109:21)
> at rli.on.self.bufferedCmd (repl.js:258:20)
> at REPLServer.self.eval (repl.js:116:5)
> at Interface.<anonymous> (repl.js:248:12)
> >
>
> My question is, what are the steps from here? I'm not really familiar with the internals of node libraries or node-gyp to know what is not working. I can provide the built object or an armv7a node if it would help...
>
> Looking at an objdump of the module, it doesn't look like there is an init...

Double check that the NODE_MODULE(your_modules_name, your_modules_entry_point) macro is in its place.

Do a gcc -E to verify that it's expanding properly to something like this:

extern "C" { node::node_module_struct your_modules_name_module = { (1), __null, "/path/to/src.c", your_modules_entry_point, "your_modules_name" }; }

Also once built you *can't* rename the your_modules_name.node to anything !== than the your_modules_name you passed to NODE_MODULE.

hth,
--
Jorge.

[Kevin Baker]

Thanks for the quick responses! I wanted to make sure I was going the
right direction...

This is with the node-sqlite3 git HEAD
(https://github.com/developmentseed/node-sqlite3). I figured it would
be a safe choice to build since it seems like the most official
sqlite3 binding for node...

It looks like it possibly needs updates related to the build with
node-gyp, as all the related init methods I could find are capital-I
Init, and marked static (not in the .cc files, but in the .h files):

~/git/node-sqlite3/src$ egrep -wi init *
database.cc:void Database::Init(Handle<Object> target) {
database.h: static void Init(Handle<Object> target);
node_sqlite3.cc: Database::Init(target);
node_sqlite3.cc: Statement::Init(target);
statement.cc:void Statement::Init(Handle<Object> target) {
statement.h: static void Init(Handle<Object> target);

I don't think these are the correct init functions, though? Looking at
https://www.cloudkick.com/blog/2010/aug/23/writing-nodejs-native-extensions/
for example and from Ben's response suggests a separate init() call
that I cannot find anywhere in node-sqlite3. The last code commit to
node-sqlite3 was 8 months ago, is it possible the module calling
conventions have been updated since then? I will try adding one
according to the article...

Thanks,
Kevin

[mscdex]

On Feb 7, 8:41 pm, Kevin Baker <kba...@gmail.com> wrote:
> I don't think these are the correct init functions, though? Looking athttps://www.cloudkick.com/blog/2010/aug/23/writing-nodejs-native-exte...

> for example and from Ben's response suggests a separate init() call
> that I cannot find anywhere in node-sqlite3. The last code commit to
> node-sqlite3 was 8 months ago, is it possible the module calling
> conventions have been updated since then? I will try adding one
> according to the article...

FWIW I have a gypified version that also works on node 0.9.4+ and
Windows, with FTS support here: https://github.com/mscdex/node-sqlite-fts

[Kevin Baker]

Wow, thanks for the updated version! That version compiles and builds
as well, and it seems to work!

I believe the OE build environment might be a bit too much for node &
node-gyp... To get it to build, I needed to do this:
(warning, lots of long hacky stuff follows, if I get time to
submit a nice version to OE I will clean this up and post tickets
where applicable)

- upgrade to node 0.8.19, which fixed:
- needing to install gyp natively and patch node-gyp to use it
instead... now it finds node-gyp's version of gyp correctly.
- deploying my own 'node-gyp' binary to <sysroot>/usr/bin that fixes
the directory location (I moved it out of node's dir so it would be
picked up by the PATH, this is more of an issue with my OE packaging)
- remove ~/.node/version/common.gypi (OK, replaced with { } since
node-gyp freaks out without that file) as it was adding -m64,
side note: when cross-compiling stuff with OE there can't be any
reliance on anything in the host build environent.
The idea is that the builds are always repeatable from scratch, so
changing settings in the home dir shouldn't be possible.
Feature request: There should be a way to disable using the node
homedir in a future node-gyp.
- add LD=$(CXX) as somehow gyp picks LD=ld instead of figuring out
when to use CCLD or CXX.
I'm not entirely sure of what's happening behind the scenes with
gyp here, but I had to do the same trick to get chromium's gyp
cross-compiling with OE.
- add CXXFLAGS="$(CXXFLAGS) -fPIC" and CFLAGS="$(CFLAGS) -fPIC" - the
link step was complaining that it needed these.

After all of that, I get a binary sqlite3_bindings.node! But, when I
require it, I ran into a similar issue to:
https://github.com/joyent/node/issues/3623

> require('./sqlite3_bindings.node');
node: symbol lookup error: /home/root/sqlite3_bindings.node: undefined
symbol: _ZN2v811HandleScopeC1Ev

This was caused by the initial OE recipe I started with:
http://permalink.gmane.org/gmane.comp.handhelds.openembedded/54923
that took out -rdynamic. Since the LD=CXX trick works around this, I
removed it. So, rebuilding it all... and reinstalling it on the target
platform:

> require('./sqlite3-fts.node');
{ Database: [Function: Database],
EXEC_EMPTY: 0,
EXEC_LAST_INSERT_ID: 1,
EXEC_AFFECTED_ROWS: 2 }

Success!!! Now to test with some working code, as the documentation
for using this driver is quite out of date... (there is no execute
anymore on the driver)?

Thanks!
Kevin