node-waf: building a node plugin with a static library
Markus Sieber
i have a node plugin which uses functions from a static linked file
which is called c.o
how can I tell the node-waf build process to also include this file
when linking?
I dont have any sources for c.o i just have the compiled version of
it.
i tried [1] but this didn't work, there is no error but the generated
shared library is not linked
with c.o.
thank you!
Markus
[1]
srcdir = "."
blddir = "build"
VERSION = "0.0.1"
def set_options(opt):
opt.tool_options("compiler_cxx")
def configure(conf):
conf.check_tool("compiler_cxx")
conf.check_tool("node_addon")
conf.check(lib='c.o', uselib_store='KDBC')
def build(bld):
obj = bld.new_task_gen("cxx", "shlib", "node_addon" )
# without this, eio_custom doesn't keep a ref to the req->data
# obj.cxxflags = ["-D_FILE_OFFSET_BITS=64", "-D_LARGEFILE_SOURCE"]
obj.use = 'KDBC'
obj.target = "kdb"
obj.source = "connection.cc k2v8.cc kdb-node.cc KWorker.cc"
# obj.use = bld.srcnode.abspath() +'/c.o'
David Bond
I just worked this out, literally 5 minutes ago...
obj.add_obj_file("path/to/object/file");
as the last line under build
One line seems to work and seems to be linking properly.
Someone PLEASE correct me if I'm wrong! :)
Thanks
Dave
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
Kenneth Shaw
build command line. I'm a bit rusty on my gcc command line, but for a
project we have we have a task like this:
def build(bld):
bld.env.append_value('LINKFLAGS', '-lmylibrary')
obj = bld.new_task_gen('cxx', 'shlib', 'node_addon')
obj.cxxflags = ['-g', '-D_FILE_OFFSET_BITS=64', '-D_LARGEFILE_SOURCE', '-Wall']
obj.target = 'MYLIB'
obj.source = 'node_mylib.cc'
The important line to look at here is the bld.env.append_value() line.
Additionally, you can do it with the cxxflags variable as well ...
-Ken
Jann Horn
> Hi,
> I just worked this out, literally 5 minutes ago...
>
> obj.add_obj_file("path/to/object/file");
> as the last line under build
>
> One line seems to work and seems to be linking properly.
>
> Someone PLEASE correct me if I'm wrong! :)
Try this on a 64-bit-machine too before you start cheering. If the .o
file doesn't contain relocation data, compiling will fail there
although it works on x86. Scratched my head thinking about this stuff
for hours yesterday. :(
Kenneth Shaw
to these things somewhere ...
-Ken
David Bond
Couldn't you try detecting the arch in the waf script and use a different object? That was my plan :)
Thanks
Dave
Jann Horn
> Just got this working on x64 only need it there for now personally though.
> Couldn't you try detecting the arch in the waf script and use a different object? That was my plan :)
No - I even have the source code, but I don't understand the build
script yet. :D
Shinuza
Nathan Rajlich
favor of gyp.
In the future, module authors are expected to statically precompile
their modules for their supported platforms. Users compiling the
modules are the way of the past, especially for Windows.
Ryan Schmidt
On Jan 6, 2012, at 18:19, Nathan Rajlich wrote:
> In the future, module authors are expected to statically precompile
> their modules for their supported platforms.
Blah. That sounds like it's just going to result in a lot of modules not being available for platforms the author doesn't happen to use but on which they would work anyway.
> Users compiling the
> modules are the way of the past, especially for Windows.
This could be one of those instances where the push for Windows support will be to the detriment of users of other operating systems.
Jann Horn
> In the future, module authors are expected to statically precompile
> their modules for their supported platforms. Users compiling the
> modules are the way of the past, especially for Windows.
Ewwww. Like, just because I want to release some native module, I have
to get tools for compiling stuff for windows? Either the module user
or the registry should do this, but definitely not the module
developer.
mscdex
Users compiling the> > modules are the way of the past, especially for
systems.
Nathan Rajlich
The biggest help on this note though is gonna be some kind of
multi-platform build system in the cloud. I'm imagining a way to
specify your Git repo url and have the build system reach out to any
number of registered platforms. The different platforms would compile
the module for you and send it back over HTTP or something.
A service like that alone would make this new build system actually
quite nice IMO :)
Jann Horn
> It's not so bad guys, and I maintain quite a few native modules :)
It doesn't just mean that you or some cloud service has to compile the
code. It also means that a bunch of devs will no longer see your libs
as a bunch of code they use and can change. They'll see a black magic
box, and when there are issues, they'll rely on you to fix them even
if they could fix it themselves because they don't want to figure out
how to compile all the stuff locally.
> The biggest help on this note though is gonna be some kind of
> multi-platform build system in the cloud. I'm imagining a way to
> specify your Git repo url and have the build system reach out to any
> number of registered platforms. The different platforms would compile
> the module for you and send it back over HTTP or something.
If it really needs to be a cloud service, it should be integrated in
the registry so that there are exactly zero more things you have to do
as a developer (besides fixing bugs others won't fix anymore).
> A service like that alone would make this new build system actually
> quite nice IMO :)
You'd get a little more speed in exchange for more weird magic. If
you're new to node, you might want that it's good for you that you
don't have to install a compiler, but in the end, this will cause
users of your library to know and participate less.
IMO, one of the best things the node community has is github. A module
doesn't really work? Git clone, npm link, npm link modulename, fix,
test, commit, push, pull request. If you have to recompile, that's
just one small command. Encouraging people to use a way where it's
harder to participate seems like a really bad idea to me.
Jann Horn
> IMO, one of the best things the node community has is github. A module
> doesn't really work? Git clone, npm link, npm link modulename, fix,
> test, commit, push, pull request. If you have to recompile, that's
> just one small command. Encouraging people to use a way where it's
> harder to participate seems like a really bad idea to me.
So, basically, my opinion: Sure, building modules has to be easy. But
if we have to improve windows support, we should concentrate on an
easy way to build stuff on windows instead of precompiling for it. And
if we want deploys to be faster, npm should get a local compilation
result cache instead of precompiled tarballs.
Nathan Rajlich
users who would rather compile their own modules still can. Those
people are the kinds of people who compile node themselves as well (so
quite a lot of us). New users don't need to worry about a compiler
(until they want to), and it makes things easier for them.
So now just have the option: compile it or don't. And the workflow
doesn't change at all for us native module devs, write and compile,
except now we have the burden of learning/fixing how Windows messes up
while compiling :p
And Jann, on the note of "concentrate on an easy way to build stuff on
windows instead of precompiling for it", that's 1) completely
unnecessary on windows, and 2) users should have to install MSVC just
to *use* a native module (by thus having to compile it beforehand).
I do agree though that some sort of cloud build-system npm integration
would be ideal.
Nathan Rajlich
a native module (by thus having to compile it beforehand)."
Isaac Schlueter
some experimentation. Here's the official plan.
First, of all, let's put this into perspective. There are 347 modules
that have some kind of install or preinstall script. There are 6114
modules. That's 5.6%. It's a small minority. Yes, some of those are
very important, but I think what we've seen overall is that binary
modules are a specialized rarity, and add a bit of pain anyway; moving
it from install-time to publish time is perfectly fine.
We're going to revisit the default --bin-publish and --bindist
settings in npm. This will probably make things much nicer for users
of compiled modules, since the install will be able to skip over the
compilation step. (It'll leverage the existing install script to know
what to do to generate a binary.)
If someone tries to install a binary module, and it's not available
for their platform, it'll probably fail. This is not the end of the
world. However, modules that don't compile on all supported platforms
will have a black mark on their permanent record ;)
This means that, if you want to support windows users, you're going to
have to fire up a copy of windows and install a build toolchain.
Once this is normal, and we see how it works, we may remove the
"install" script support from some future version of npm, unless the
user opts-in to get source packages.
There are currently some uses of install/preinstall other than running
node-waf. This needs to change.
1. cake build to translate coffeescript packages into JavaScript
Please build your coffeescript packages in a `prepublish` script, not
in a `preinstall` script. Deploy JavaScript to the registry. This is
100% better in every way. It is more future proof, faster for
everyone, etc.
2. downloading a large file
We'll have to add some kind of support for a remote file to be fetched
at install time from somewhere other than the registry. This is a
reasonable use-case, easy to add npm api for, and it's not addressed
well using the preinstall script anyway.
3. building with make
I <3 make, and am allergic to python, so I can appreciate the
sentiment. However, we're going to have to standardize a bit on a
single tool for binary addons, and gyp is a good tool that works on
all the platforms node supports. Eventually, binary modules will have
to standardize on that.
Lastly, somewhat orthogonal to this effort, we're going to investigate
having a build bot that will automatically run compilation jobs on all
supported platforms and put the binary package back into the registry.
This has to be done in a trusted manner, so it's not something we can
leverage the community for.
I suspect that there will be some hardship and pain, but this plan is
the one that takes us to a better place. Here's what you can do to
make it easier:
1. If you don't really need to run a preinstall or install script,
don't. While I clearly don't have to convince most of you, please
write programs in JavaScript (or at least, *publish* them in
JavaScript) unless you absolutely need some functionality that can
only be accessed from the C layer.
2. For now, use node-waf to compile addons (not scons, or make, or
anything else). Move to node-gyp as soon as you can.
mscdex
> 1. cake build to translate coffeescript packages into JavaScript
> Please build your coffeescript packages in a `prepublish` script, not
> in a `preinstall` script. Deploy JavaScript to the registry. This is
> 100% better in every way. It is more future proof, faster for
> everyone, etc.
Ryan Schmidt
> This means that, if you want to support windows users, you're going to
> have to fire up a copy of windows and install a build toolchain.
And it means that if you want to support OS X, you have to buy a Mac to compile it there.
As a Mac user, I don't want to miss out on modules developed by Linux users who do not have a Mac.
Isaac Schlueter
Not really. This is only an issue for Windows.
If a source package exists, and the binary version for your platform
doesn't, you'll just download the source and compile it, like today.
You'll just have to have a build toolchain. The issue is that this
doesn't currently work on Windows, and probably never will.
When/if builds are done in the clown, it won't matter anyway.
Ryan Schmidt
On Jan 7, 2012, at 11:34, Isaac Schlueter wrote:
> If a source package exists, and the binary version for your platform
> doesn't, you'll just download the source and compile it, like today.
Ah. Well if that is so, then that's ok. That contradicts what was said in the original message which started this subthread; it said:
On Jan 6, 2012, at 18:19, Nathan Rajlich wrote:
> That's correct, node-waf is deprecated on the current master branch in
> favor of gyp.
>
Angel Java Lopez
Duncan Gmail
Duncan Gmail
Check the NODE_PATH...
echo $NODE_PATH
If the full path to connect is not there then...
1. Modify or create if doesn't exist... ~/.bashrc
2. Export NODE_PATH='/usr/local/lib/node_modules' - or whatever your path is.
3. Then... Source ~/.bashrc
That should do it.
Not sure that's everything you need but do a quick google for NODE_PATH and bashrc and you should find more detailed instructions.
-MRdNk
On 7 Jan 2012, at 22:33, Ryan Schmidt <googl...@ryandesign.com> wrote:
Liam
> First, of all, let's put this into perspective. There are 347 modules
> that have some kind of install or preinstall script. There are 6114
> modules. That's 5.6%. It's a small minority. Yes, some of those are
> very important, but I think what we've seen overall is that binary
> modules are a specialized rarity, and add a bit of pain anyway; moving
> it from install-time to publish time is perfectly fine.
specific, which Node users might need. And many important ones are
available via modules which are currently under-developed and lightly
tested. Serious Node users should, for at least a few years yet,
expect to write, extend, and fix C++ modules.
Can npm accommodate N platforms x M versions for any given binary
module? Each version won't necessarily support every platform.