PG Extensions: Must be statically linked?
Craig A. James
libmyfuncs.so.0.0:
gcc -Wall -Wmissing-prototypes -Wpointer-arith \
-Wdeclaration-after-statement -Wold-style-definition -Wendif-labels \
-fno-strict-aliasing -fpic -shared -Wl,-soname,libmyfuncs.so.0 \
$(OBJS) \
$(MYLIB)/lib/libmylibs.a \
/usr/lib/gcc/i386-redhat-linux/3.4.2/libstdc++.a \
-o libmyfuncs.so.0.0
Is this correct? Do Postgres extension need to be fully statically linked? Or is there some configuration that will specify LD_LIBRARY_PATH (or perhaps a Postgres-specific equivalent).
The manual's instructions are good regarding writing code, but don't say much about linking.
Thanks,
Craig
---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster
Peter Eisentraut
> I'm creating user-defined server extensions, written in C per the
> manual "31.9. C-Language Functions". Everything works well, but only
> if I fully link the .so such that there are *no* unresolved external
> references at all.
What happens if you don't?
--
Peter Eisentraut
http://developer.postgresql.org/~petere/
Tom Lane
> I'm creating user-defined server extensions, written in C per the
> manual "31.9. C-Language Functions". Everything works well, but only
> if I fully link the .so such that there are *no* unresolved external
> references at all. Not even the stuff in libstdc++.a can be left out.
If you're using libstdc++.a, you are not writing C. There is no support
for C++ in the backend, and I would strongly advise not trying to use
it, as any of C++'s moderately interesting features like exceptions will
not play nicely with the backend environment.
regards, tom lane
---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?
Craig A. James
>>I'm creating user-defined server extensions, written in C per the
>>manual "31.9. C-Language Functions". Everything works well, but only
>>if I fully link the .so such that there are *no* unresolved external
>>references at all. Not even the stuff in libstdc++.a can be left out.
>
> If you're using libstdc++.a, you are not writing C. There is no support
> for C++ in the backend, and I would strongly advise not trying to use
> it, as any of C++'s moderately interesting features like exceptions will
> not play nicely with the backend environment.
Unfortunately, we're also using a second library (OpenBabel) that is written in C++. A good portion of the code I've written is a wrapper layer that hides the C++ objects and presents a simple C wrapper that works for Postgres.
Peter Eisentraut
> Unfortunately, we're also using a second library (OpenBabel) that is
> written in C++. A good portion of the code I've written is a wrapper
> layer that hides the C++ objects and presents a simple C wrapper that
> works for Postgres.
I suggest if you want to get any concrete advice out of this, post us
the commands that you execute and the error messages that you get.
--
Peter Eisentraut
http://developer.postgresql.org/~petere/
---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majo...@postgresql.org so that your
message can get through to the mailing list cleanly
John Gray
> I'm creating user-defined server extensions, written in C per the manual
[snip]
> Is this correct? Do Postgres extension need to be fully statically
> linked? Or is there some configuration that will specify LD_LIBRARY_PATH
> (or perhaps a Postgres-specific equivalent).
>
Not generally, unless your platform requires it because of restrictive
dynamic loader behaviour. contrib/xml2 uses two external libraries
(libxml and libxslt) - they are dynamically referenced by the resulting
pgxml.so. Have a look at the Makefile for contrib/xml2 for hints on
options?
> The manual's instructions are good regarding writing code, but don't say
> much about linking.
>
True - but I think the assumption has been that most
people wouldn't have any involved linking requirements. Your case doesn't
come into this category, but there are useful cases for linking to
external function libraries and exposing them to PG.
Regards
John
Craig A. James
>>Unfortunately, we're also using a second library (OpenBabel) that is
>>written in C++. A good portion of the code I've written is a wrapper
>>layer that hides the C++ objects and presents a simple C wrapper that
>>works for Postgres.
>
> I suggest if you want to get any concrete advice out of this, post us
> the commands that you execute and the error messages that you get.
Thanks for your answers -- see below.
Based on Peter's and Tom's replies regarding C++, I think you've answered my question: I should be able to do this without static linking. But the Postgres linker uses the C (not the C++) linker to resolve references, so it's not finding the C++ libraries.
My original question was misleading. I said, "I have to link everything statically...", when in fact what should have said was, "If I link statically, it works." I now realize that I'm not linking everything statically, just the OpenBable and C++ libraries, and in fact Postgres is finding the other libraries I need, like libz, libm, and so forth.
Here's what happens when I don't statically link the C++ libraries or the OpenBabel libraries:
[root]# cp libmyfuncs.so /usr/local/pgsql/lib
$ psql -d myfuncs -U postgres
Welcome to psql 8.0.3, the PostgreSQL interactive terminal.
...
myfuncs=# CREATE FUNCTION myfunc(text, text) RETURNS boolean
myfuncs-# AS '/usr/local/pgsql/lib/libmyfuncs.so', 'myfunc'
myfuncs-# LANGUAGE 'C' STRICT;
ERROR: could not load library "/usr/local/pgsql/lib/libmyfuncs.so": /usr/local/pgsql/lib/libmyfuncs.so: undefined symbol: _ZdlPv
That symbol is obviously a C++ mangled name, so it's not finding the C++ library, and the fact that it prints the mangled name suggests that it's a C linker, not a C++ linker.
So now my question is: Can I somehow add other directories/libraries to those that Postgres uses? Or is there an option for Postgres use the C++ dynamic linker? I don't mind statically linking OpenBabel, but it seems like a bad idea to put the specific version- and system-dependent location of libstdc++.a into my makefiles.
(Am I the only guy in the world who has to use a C++ library as part of a Postgres function?)
Thanks,
Craig
---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?
Martijn van Oosterhout
> Thanks for your answers -- see below.
>
> Based on Peter's and Tom's replies regarding C++, I think you've answered
> my question: I should be able to do this without static linking. But the
> Postgres linker uses the C (not the C++) linker to resolve references, so
> it's not finding the C++ libraries.
<snip>
> So now my question is: Can I somehow add other directories/libraries to
> those that Postgres uses? Or is there an option for Postgres use the C++
> dynamic linker? I don't mind statically linking OpenBabel, but it seems
> like a bad idea to put the specific version- and system-dependent location
> of libstdc++.a into my makefiles.
There are a number of ways to indicate which extra libraries to load,
but by far the easiest is by specifying them on the link line.
Your problem is that when creating a shared library, you are not
required to make sure all your external symbols are defined somewhere.
You didn't post your compile/link line but if you're using C++ you
probably need to use g++ for the linking to include the special C++
libraries.
One way to find out what's going on is using readelf:
$ readelf -a /bin/bash |grep NEEDED
0x00000001 (NEEDED) Shared library: [libncurses.so.5]
0x00000001 (NEEDED) Shared library: [libdl.so.2]
0x00000001 (NEEDED) Shared library: [libc.so.6]
These are the libraries that will be loaded when someone tries to load
your shared lib (the above works on any ELF object). The easiest is to
specify "--no-undefined" on the link line so the linker checks up front
you won't get undefined symbols at run time. Remember it's
-Wl,--no-undefined if invoking from gcc/g++.
Hope this helps,
--
Martijn van Oosterhout <kle...@svana.org> http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.
Tom Lane
> to those that Postgres uses?
This is not a Postgres problem, it's a dynamic-linker problem, and
I don't believe there is a different dynamic linker for C++ than C.
Your problem is just to get your shared library marked as needing
libstdc++.so. (ldd on the .so should show what shared libraries it
needs.) On Linux this should all happen pretty much automatically,
at least for libraries that are in the ldconfig search path. I'm
not sure what is going wrong, but you could take a look at
contrib/dblink, which dynamically includes libpq.so (and the backend
certainly does not load libpq.so by default). If that works on your
machine then try to figure out what dblink's Makefile does differently
from yours.
My concern about how nicely libstdc++ will play in the backend
environment still stands though.
regards, tom lane
Mark Dilger
> My concern about how nicely libstdc++ will play in the backend
> environment still stands though.
I have had the same concern, though never any hard evidence of a problem. If
the C++ functions are wrapped with "extern C", and all exceptions caught
(perhaps converted into error numbers which are then returned from the wrapper
functions to the plain-C calling functions), are there any remaining known
problems? I have often considered making a C++ allocator which wrapped palloc
and pfree, so that I could then use the STL within the backend...
Has anyone tried this?
mark
Andreas Seltenreich
> I have had the same concern, though never any hard evidence of a
> problem. If the C++ functions are wrapped with "extern C", and all
> exceptions caught (perhaps converted into error numbers which are then
> returned from the wrapper functions to the plain-C calling functions),
> are there any remaining known problems? I have often considered
> making a C++ allocator which wrapped palloc and pfree, so that I could
> then use the STL within the backend...
>
> Has anyone tried this?
I did some experiments on a C++ language handler last year (including
an allocator and a class loader to spare the extern "C"s/name
mangling):
<http://archives.postgresql.org/pgsql-general/2005-10/msg01570.php>
<news:87acgxj...@gate450.dyndns.org>
The remaining issue is the impedance mismatch between longjmp()ing and
exceptions.
regards,
Andreas
--