Wrapping C/C++ libraries
Anuradha Ratnaweera
I am not sure if this is the correct place to ask this, so please point
me to correct direction if it is not.
As far as I know, there is no "native" way of calling C/C++ libraries
from within Perl 5. When C/C++ API involves structures, the
corresponding Perl binding uses hashes, and an intermediate step
converts the hash to a structure, and call the C program with the
converted structure. An example of an elegent system which does this is
in gtk2-perl (http://gtk2-perl.sourceforge.net). Muppet, please correct
me if I am wrong here.
Interfacing C/C++ libs this way involves creation of intermediate
structures that duplicates lots of data. (Reminds me the the page cache
and buffer cache of the old Linux kernels.) Obviously, Perl can't
convert a hash to a structure, because the structure of the structure is
not known, although it is a structure ;-)
But if the structure of the structure _is_ known, then there is the
possibility of kind of "rearranging" the hashes and calling the C/C++
library without duplicating the related data to a seperate structure.
I like to get involved in the parrot/perl6 development, and was just
wondering if this is something to work on.
Anuradha
--
http://www.linux.lk/~anuradha/
Anuradha Ratnaweera
>
> As far as I know, there is no "native" way of calling C/C++ libraries
> from within Perl 5. When C/C++ API involves structures, the
> corresponding Perl binding uses hashes, and an intermediate step
> converts the hash to a structure, and call the C program with the
> converted structure. An example of an elegent system which does this is
> in gtk2-perl (http://gtk2-perl.sourceforge.net). Muppet, please correct
> me if I am wrong here.
Just another quick, and possibly dumb, question: Is XS the standard way
of binding Perl to a C/C++ library?
Anuradha
--
http://www.linux.lk/~anuradha/
Leopold Toetsch
> But if the structure of the structure _is_ known, then there is the
> possibility of kind of "rearranging" the hashes and calling the C/C++
> library without duplicating the related data to a seperate structure.
Parrot doesn't have a hash there, but that doesn't matter. I've started
to prepare the UnmanagedStruct PMC to describe structures. The relevant
test file and library/libpcre.imc has some examples.
Currently only integer-like structure fields and arrays of these are
down.
> I like to get involved in the parrot/perl6 development, and was just
> wondering if this is something to work on.
This would be very welcome.
> Anuradha
leo
Leopold Toetsch
> Just another quick, and possibly dumb, question: Is XS the standard way
> of binding Perl to a C/C++ library?
If your question is "... of binding Parrot to a C/C++ library?", then
no. Parrot uses NCI (native call interface). S. library/* and
t/pmc/nci.t for examples.
> Anuradha
leo
Anuradha Ratnaweera
Thanks for the quick reply, and of course the warm welcome ;-)
I think, I have to be carefully distinguish between Parrot and Perl 6,
when asking questions. Sorry about the haziness.
Let me break down the questions into more than one:
0. Is XS the standard way of binding Perl 5 to a C/C++?
1. If answer to 0 is 'yes', will it be the same for Perl 6?
2. If the ansewr to 0 or 1 is "no", will there be another standard in
Perl 6?
Anuradha
--
http://www.linux.lk/~anuradha/
Dan Sugalski
> On Tue, 2003-10-28 at 14:10, Leopold Toetsch wrote:
> > Anuradha Ratnaweera <ARatn...@virtusa.com> wrote:
> >
> > > Just another quick, and possibly dumb, question: Is XS the standard way
> > > of binding Perl to a C/C++ library?
> >
> > If your question is "... of binding Parrot to a C/C++ library?", then
> > no. Parrot uses NCI (native call interface). S. library/* and
> > t/pmc/nci.t for examples.
>
> Thanks for the quick reply, and of course the warm welcome ;-)
>
> I think, I have to be carefully distinguish between Parrot and Perl 6,
> when asking questions. Sorry about the haziness.
Just make sure to deistinguish between perl 5 and perl 6. :)
> Let me break down the questions into more than one:
>
> 0. Is XS the standard way of binding Perl 5 to a C/C++?
Mostly, yes. The Inline::C folks occasionally disagree.
> 1. If answer to 0 is 'yes', will it be the same for Perl 6?
No.
> 2. If the ansewr to 0 or 1 is "no", will there be another standard in
> Perl 6?
Yes. Whatever we do for Parrot.
Dan
--------------------------------------"it's like this"-------------------
Dan Sugalski even samurai
d...@sidhe.org have teddy bears and even
teddy bears get drunk
Leopold Toetsch
> Let me break down the questions into more than one:
> 0. Is XS the standard way of binding Perl 5 to a C/C++?
Yes, till now. Ponie (perl ~5.12) will use both XS and NCI/extend.c.
> 1. If answer to 0 is 'yes', will it be the same for Perl 6?
Perl6 will use, whatever Parrot does provide, which is NCI.
> 2. If the ansewr to 0 or 1 is "no", will there be another standard in
> Perl 6?
The no/yes doesn't apply - there is "both" too :) Parrot has its own (new)
extension system. Perl5/Ponie will continue to use XS as well.
> Anuradha
leo
Anuradha Ratnaweera
> On Tue, 28 Oct 2003, Anuradha Ratnaweera wrote:
>
> > 2. If the ansewr to 0 or 1 is "no", will there be another standard in
> > Perl 6?
>
> Yes. Whatever we do for Parrot.
Thigs are getting clearer and more confusing ;-)
I am still in the trying-to figure-out-what-the-hell-is-going-on-here
phase so please bear with me about the stupid questions ;-)
Let me ask one more, and hopefully the last, question; a specific one
this time:
Say, there is a C library x (libx.{so,a}) which defines a function foo()
which takes a single argument `struct bar'. The prototype of foo() is
defined in x.h as:
foo(struct bar);
Actual definition might be a typedef insted of struct bar itself, but
all the same.
And struct bar has three members: int a, char *b and struct morebar *c.
Now, if one wants to create a Perl[56] binding to the library x, s?he
will create a module X. A script would look like:
use X;
my %bar = ();
my %morebar = ();
$morebar{key} = 10;
$morebar{value} = "ten";
$bar{a} = 10;
$bar{b} = 'bar b';
$bar{c} = \%morebar;
foo($bar);
Now, what exactly is going to happen when it comes to the foo($bar)
call? Is $bar going to be duplicated to match struct bar? Will there
be too much overhead when compared to a C implementation?
What I am looking for is something like this: The X.pm module will have
information about struct bar. Using that, the script will be compiled
so that the hash will contain values for indices a, b and c in order and
in the same format as C does, so that foo($bar) call can happen without
minimal intermediate conversion steps.
Thanks.
Anuradha
--
http://www.linux.lk/~anuradha/
Dan Sugalski
Whatever you want to have happen. There's not goign to be anything
automatic happening here, and there's no lurking magic--if the interface
code you've written handles the conversion right, then the conversion will
happen, otherwise it won't.
Now, you could certainly write a PMC class that behaves as a struct (and
we'll likely provide one as part of the core distribution) and prototype
foo() so that it only took things of type bar, and save yourself a lot of
hassle (which is likely the route I'd take) but that's up to the person
who wrote the interface library.
Muppet
you may find something worth thinking about my description of the hoops
i have to jump through... or maybe not. ignore if you like. i am
quite open to suggestions how i should do this for perl6. replies
off-list, please, as i'm not subscribed.
On Tuesday, October 28, 2003, at 12:24 AM, Anuradha Ratnaweera wrote:
> I am not sure if this is the correct place to ask this, so please point
> me to correct direction if it is not.
>
> As far as I know, there is no "native" way of calling C/C++ libraries
> from within Perl 5.
there's the built-in ioctl, and the DynaLoader docs point you to
C::DynaLoader, but in general, yes, there's no practical way to call C
functions without some glue.
> When C/C++ API involves structures, the
> corresponding Perl binding uses hashes, and an intermediate step
> converts the hash to a structure, and call the C program with the
> converted structure. An example of an elegent system which does this
> is
> in gtk2-perl (http://gtk2-perl.sourceforge.net). Muppet, please
> correct
> me if I am wrong here.
gtk2-perl is a little picky about how structures and objects are
represented in Perl. there are two basic ways, which result in three
things happening:
1. GObjects are reference-counted beings with sophisticated lifetime
management. these get magical glue to Perl in the form of attaching
magic (containing the object pointer) to the Perl object, and sharing
references between the Perl and C objects to ensure that the combined
object lives for the right amount of time. a side effect is that
DESTROY may be called more than once, but the Perl object and C object
are persistent. for convenience, we chose a hash reference as the
object representation, because this allows Perl developers to store
arbitrary data with the object without using the type-unsafe GObject
data keys. i would like to have a way to tie the object data to hash
keys, but tie and our object wrapper don't get along.
2. GBoxed is a nifty way for GLib-based libraries to assign copy and
free functions to a type. these are used with opaque pointers, which
are nearly 100% of the time plain old C structures. there is no
instance of a GBoxed -- the type refers to the functions, and you're on
your honor to ensure that you call the proper GBoxed functions on a
given opaque pointer; this allows generic handling of black-box
structures.
gtk2-perl handles GBoxed via another vtable layer called
GPerlBoxedWrapperClass, which contains pointers to functions which
wrap, unwrap, and destroy the boxed objects. This allows for two basic
modes of handling:
a) as opaque types. default boxed wrapper class implements the Perl
wrapper as a blessed scalar reference, containing the value of a
pointer to a small wrapper structure which contains the boxed pointer
and the type identifier. members are accessed via functions, and no
data is duplicated.
b) as native perl types, via the wrap and unwrap functions. this does
result in duplication of data, but we rationalize this in a few ways:
i) it makes using the binding 1000 times easier (e.g.
GnomeCanvasPoints), ii) the user must not modify the real data in the
first place (one-way C-to-hash mappings for useful information
structures).
> Interfacing C/C++ libs this way involves creation of intermediate
> structures that duplicates lots of data.
as mentioned above, we try to avoid duplication of data except when
there are good reasons.
> But if the structure of the structure _is_ known, then there is the
> possibility of kind of "rearranging" the hashes and calling the C/C++
> library without duplicating the related data to a seperate structure.
i seriously doubt anything like this is possible. however, given
metadata that describes how each structure is laid out, you could
implement something that packs and unpacks them for you. however, the
runtime overhead of reading, processing, and/or storing those
descriptions would be greater than the codespace overhead and time to
run through specialized code for each structure (unless you had
hundreds or thousands of them).
--
muppet <scott at asofyet dot org>
smitty_one_each
Perl6.
One project pertinent to this thread is
http://www.boost.org/libs/python/doc/index.html
One wonders how feasible it would be to implement
http://www.ecma-international.org/news/ecma-TG5-PR.htm
for Parrot. Such an undertaking is about 3.7 units above my current
skill, but it's worth pointing out to see what the heads have to say.
R,
Chris