While writing a node addon I ran into what I believe is the same
problem (with a different underlying library) that was reported by
@springmeyer in issue 436,
https://github.com/joyent/node/issues#issue/436
My problem is a result of the dlopen call in node.cc being called only
with the flag RTLD_LAZY. By default, if no other flag is specified,
dlopen assumes RTLD_LOCAL which means any symbols in the loaded file
are local to that library and not available when resolving references
from subsequently loaded libraries. The alternative, RTLD_GLOBAL,
makes loaded symbols globally available.
That seems like sensible default behavior (avoid polluting the global
namespace). However, if the file being loaded in turn uses dlopen to
load another file, and that second file has a symbol referencing the
first, an attempt to load the second file will fail.
That's exactly the problem in my case. I linked my node addon to a
third party library I am wrapping, and that library attempts to load
plugins on the fly using dlopen. There is a plugin registry in the
library and since node loads my addon with RTLD_LOCAL the registry is
not visible to the plugins loaded in the library. Failure to resolve
the plugin registry symbol properly causes a crash. If node's dlOpen
uses RTLD_GLOBAL, the addon works fine.
So in some cases the sensible default behavior can be problematic. I
liked the idea suggested by Dane in issue 436. His suggestion was to
emulate python and add a function in sys that lets you set the flags
for dlopen before loading the module. I implemented his suggestion,
putting the variable in process rather than sys, so that you can do
the following,
var c = require('constants');
var oldFlags = process.dlOpenFlags;
process.dlOpenFlags = c.RTLD_LAZY | c.RTLD_GLOBAL;
var x = require('some_dynamically_loaded_node_extension');
process.dlOpenFlags = oldFlags;
and submitted a pull request, #855:
https://github.com/joyent/node/pull/855
I think this change or one like it is necessary. Some alternatives
with the reason for rejecting:
- Take care of it in the addon itself: You're still going to have to
do use RTLD_GLOBAL which means you're potentially polluting the global
namespace, but now it's buried in the addon which makes it more
difficult for the end node user to track down problems if they occur.
- "Fix" the underlying library being wrapped: Lots of problems with
that--might make the lib incompatible with other applications, forces
an upgrade of the lib, you'd probably be modifying stable code that
changes/should be changed rarely with the risk of introducing new
bugs.
- Figure out some way to flag an addon as requiring RTLD_GLOBAL and do
that switch within the node dlOpen function (not sure of a smart way
to do this but for example you could have a ".node-global" suffix or
something instead of just '.node'): same problem as first approach,
you pollute the namespace and it's still kind of buried behind the
scenes.
- Load all addons with RTLD_GLOBAL instead of RTLD_LOCAL: that doesn't
seem like a very bright approach since most things will be fine with
the more conservative RTLD_LOCAL.
The apparent disadvantage with Dane's approach, of course, is that
every time you want to use such an addon you are required to set the
dlOpenFlags parameter before the require. However you thereby force
the issue out into the open at a level of the code making a plain node
coder aware that he or she is doing something unusual. If two such
modules are being loaded and they have a conflicting symbol I would
think it'll be much easier to track things down.
By the way if I understand Dane correctly he fixed his particular
problem by switching RTLD_LAZY to RTLD_NOW. That doesn't change the
above, it seems to me to just further support the more flexible
approach of being able to set the dlopen flags directly at the
javascript level versus handling it in some other way.
Thoughts?
