Native function calls
Dan Sugalski
mail--hopefully I'll catch up in a big dump today). Here's the sketch.
One new op:
bnc Pw, Px, Sy, Sz
which builds up a native call pmc that can be invoked. W is the new
PMC for the function (we create it), X is a handle to a dlopened
library, Y is the function name, and Z is the signature.
In this case, the signature is zero or more characters. The first
character is the return type, the remaining ones are the parameters.
The following letters apply:
v - void. Nothing. Only good for return types
c - char
C - char *
s - short
S - short *
i - int
I - int *
l - long
L - long *
f - float
F - float *
d - double
D - double *
x - long double
X - long double *
p - pointer from pmc
P - pointer to PMC
b - pointer from buffer
B - pointer to buffer
In the case of no signature characters, the signature is assumed to
be "void foo()".
The data's stored in a PMC of type NCI. The data pointer holds a
pointer to the main routine to call, while the struct value holds a
pointer to an ancillary routine. In the starting case the ancillary
routine will be the actual C function, while the main routine is one
that knows how to extract the parameters out of the various
registers, pass them in, and store the result somewhere. Later the
ancillary routine may be nonexistant if we build up the function
headers on the fly and embed the destination function into them.
(We'll see)
Parameters of type short, int, and long come from I registers.
Parameters of type char, char *, and the buffer pointers come from S registers
Parameters of type float and double come from N registers
Parameters of type P and p come from P registers.
Currently open is the situation of flags and such from more complex
calls. (Like what we do if we get back a pointer that's getting
stuffed into a PMC--do we set the type, if so what type, and what do
we set the flags to?)
I'm thinking, as part of this, that more introspection on buffers and
PMCs is in order, so we can have bytecode manipulate their flags and
types properly.
--
Dan
--------------------------------------"it's like this"-------------------
Dan Sugalski even samurai
d...@sidhe.org have teddy bears and even
teddy bears get drunk
Dan Sugalski
>One new op:
>
> bnc Pw, Px, Sy, Sz
>
>which builds up a native call pmc that can be invoked. W is the new
>PMC for the function (we create it), X is a handle to a dlopened
>library, Y is the function name, and Z is the signature.
This isn't true, it turns out. (That's what I get for not editing the
mail before sending when reconnecting) I instead altered dlfunc to
take this signature.
Other than that, this is in. Works, too, though I found that testing
this is a major pain on OS X, given some of the limitations of the
code we're using to emulate dlopen/dlsym here. :(
Right now only all-void, int return no param, double return no param,
and double return doubpe param, functions are supported, but it's a
SMOP to add more. (There are rather significant efficiency issues,
but that can be fixed)
Brent Dax
# which builds up a native call pmc that can be invoked. W is the new
# PMC for the function (we create it), X is a handle to a dlopened
# library, Y is the function name, and Z is the signature.
OK, clarification on something please. Is this essentially XS, or
something more primitive?
# The data's stored in a PMC of type NCI. The data pointer holds a
# pointer to the main routine to call, while the struct value holds a
# pointer to an ancillary routine. In the starting case the ancillary
# routine will be the actual C function, while the main routine is one
# that knows how to extract the parameters out of the various
# registers, pass them in, and store the result somewhere. Later the
# ancillary routine may be nonexistant if we build up the function
# headers on the fly and embed the destination function into them.
Oh JITters... ;^)
If this is going to be the basic XS mechanism, why don't we just ask the
user (or, more likely, the preprocessor) to write an argument-handling
routine?
--Brent Dax <bren...@cpan.org>
@roles=map {"Parrot $_"} qw(embedding regexen Configure)
"If you want to propagate an outrageously evil idea, your conclusion
must be brazenly clear, but your proof unintelligible."
--Ayn Rand, explaining how today's philosophies came to be
Leopold Toetsch
> Dan Sugalski:
> # ... Later the
> # ancillary routine may be nonexistant if we build up the function
> # headers on the fly and embed the destination function into them.
>
> Oh JITters... ;^)
jit/i386 has already code to call (specific) functions e.g. vtable funcs
or pmc_new_noinit. The main differemce is, that the function signature
(for vtables) comes from jit_info i.e. the definitions in core.ops.
The nci an is abstraction of this. *But*, what I really don't like in
nci.c are the string functions. Can't we define, the signature is plain
ascii?
Second: don't we need a signature for passing a Parrot_Interp* too?
> If this is going to be the basic XS mechanism, why don't we just ask the
> user (or, more likely, the preprocessor) to write an argument-handling
> routine?
JIT will need it anyway.
leo
Dan Sugalski
>Dan Sugalski:
># which builds up a native call pmc that can be invoked. W is the new
># PMC for the function (we create it), X is a handle to a dlopened
># library, Y is the function name, and Z is the signature.
>
>OK, clarification on something please. Is this essentially XS, or
>something more primitive?
More primitive. This is for "I have an external library that I need
to wrap" code. The sort of thing people use XS' prototypes and such
for. If you need to write parrot-accessing code in C, you'll use a
higher-level mechanism.
># The data's stored in a PMC of type NCI. The data pointer holds a
># pointer to the main routine to call, while the struct value holds a
># pointer to an ancillary routine. In the starting case the ancillary
># routine will be the actual C function, while the main routine is one
># that knows how to extract the parameters out of the various
># registers, pass them in, and store the result somewhere. Later the
># ancillary routine may be nonexistant if we build up the function
># headers on the fly and embed the destination function into them.
>
>Oh JITters... ;^)
Yep. You'll note the #define check in nci.c. :)
Dan Sugalski
>Brent Dax wrote:
>
>>Dan Sugalski:
>># ... Later the # ancillary routine may be nonexistant if we build
>>up the function # headers on the fly and embed the destination
>>function into them.
>>Oh JITters... ;^)
>
>
>jit/i386 has already code to call (specific) functions e.g. vtable
>funcs or pmc_new_noinit. The main differemce is, that the function
>signature (for vtables) comes from jit_info i.e. the definitions in
>core.ops.
>
>The nci an is abstraction of this. *But*, what I really don't like
>in nci.c are the string functions. Can't we define, the signature is
>plain ascii?
Oh, I hate those too. It's an evil, crufty, embarrassing piece of
code. It is, however, well encapsulated, so it can be ripped out,
shot, and replaced with something elegant and fast at some point in
the future.
>Second: don't we need a signature for passing a Parrot_Interp* too?
Nope. If something needs that, it ought to go through the standard
parrot extension loading mechanism and install real extension sub
PMCs for the routines.
Gopal V
> Currently open is the situation of flags and such from more complex
> calls. (Like what we do if we get back a pointer that's getting
> stuffed into a PMC--do we set the type, if so what type, and what do
> we set the flags to?)
So is it totally type unsafe ? ... (as you know I'm obsessed with type
safety) ...
"pMYPMC;pp"
Does this appeal to anybody .... [Disclaimer: idea borrowed totally
from JVM signatures "Ljava/lang/String;(Ljava/lang/String;i)"... Where
the Long 'L' was later on modified to carry type info of the pointer...]
> I'm thinking, as part of this, that more introspection on buffers and
> PMCs is in order, so we can have bytecode manipulate their flags and
> types properly.
Well that does not totally satisfy me .... I might be expecting a XPMC
when the pointer is stuffed into a YPMC and my program blows up in the
middle .... (of course throwing an exception is not technically blowing
up, it's damage control)..
But well ... I should admit , I'm just trying to make the C# to parrot
compilation as safe as IL .... (PInvoke and stuff...)
Gopal
--
The difference between insanity and genius is measured by success
Dan Sugalski
>If memory serves me right, Dan Sugalski wrote:
>> Currently open is the situation of flags and such from more complex
>> calls. (Like what we do if we get back a pointer that's getting
>> stuffed into a PMC--do we set the type, if so what type, and what do
>> we set the flags to?)
>
>So is it totally type unsafe ? ... (as you know I'm obsessed with type
>safety) ...
Oh, absolutely! 100% guaranteed type unsafe.
>"pMYPMC;pp"
>
>Does this appeal to anybody .... [Disclaimer: idea borrowed totally
>from JVM signatures "Ljava/lang/String;(Ljava/lang/String;i)"... Where
>the Long 'L' was later on modified to carry type info of the pointer...]
I do actually like it. I was shooting for simplicity with the
assumption that, since we were calling out to non-parrot-aware code,
all bets were off with respect to type safety. If you load in
libgtk.so and call functions dynamically there's not much we can do
with respect to proper security.
Having said that, now I'm getting that XML feeling. (You know the
one--"It seemed like a good idea until I thought about it" :)
The question, then, is how do we embed typesafe info into the call
PMC such that we can check reasonably efficiently?
> > I'm thinking, as part of this, that more introspection on buffers and
>> PMCs is in order, so we can have bytecode manipulate their flags and
>> types properly.
>
>Well that does not totally satisfy me .... I might be expecting a XPMC
>when the pointer is stuffed into a YPMC and my program blows up in the
>middle .... (of course throwing an exception is not technically blowing
>up, it's damage control)..
Oh, that was for other things entirely. I have this vague feeling
that we should be as close to totally introspective as we can manage.
I'm not sure if it should be everything-read/some-write or
everything-read/everything-write, but...
>But well ... I should admit , I'm just trying to make the C# to parrot
>compilation as safe as IL .... (PInvoke and stuff...)
No problem with that. I think you're not going to get it by default,
since my big thing's speed, but it does mean we need to reevaluate
things, since we need typechecking in safe code.
Gopal V
> I do actually like it. I was shooting for simplicity with the
> assumption that, since we were calling out to non-parrot-aware code,
> all bets were off with respect to type safety. If you load in
> libgtk.so and call functions dynamically there's not much we can do
> with respect to proper security.
Well IMHO you can't add in any type safety in between parrot and C ...
All I am suggesting is that ..
1) As far as possible .. have a sort of "extern" declaration in parrot
domain, so we get a warning from the compile when I try to call that
function in the .pbc file ... Sort of proxy for the native call ?..
A sort of check for type of param and number ...
2) When you are using it directly (unsafe) ... you deserve it if it blows
up ... dlopen , dlsym does it frequently with me ... ;-)
This 2 pronged approach is IMHO the ideal way ... afterall perl is all
about many ways to do the same thing ...
> >But well ... I should admit , I'm just trying to make the C# to parrot
> >compilation as safe as IL .... (PInvoke and stuff...)
>
> No problem with that. I think you're not going to get it by default,
> since my big thing's speed, but it does mean we need to reevaluate
> things, since we need typechecking in safe code.
Well if we can generate verifiable IL ... we should be able to generate
type-safe Parrot code ... That's more of a language feature than bytecode
feature ..
What I was concerned was that we wouldn't have an extern definition for
the native call to check if the actual params in C# are correct and of the
same type as the formal params of the native func... Especially if we're
going to check the calls before runtime ... (as is customary in static
typed languages)...
Hope I make myself clear,
Leopold Toetsch
> Dan Sugalski:
> # which builds up a native call pmc that can be invoked. .
> Oh JITters... ;^)
Here we go. "d" and "i" signatures currently, but it's easy to add more.
It passes the 2 (new) tests.
Currently, it's not integrated, add these lines to nci.c
#define CAN_BUILD_CALL_FRAMES
void *
Parrot_jit_build_call_func(struct Parrot_Interp *interpreter,
String *signature);
....
return Parrot_jit_build_call_func(interpreter, signature);
/* abort("Oh, no you can't!"); */
#else