Settling on FFI API version 1
Charles Oliver Nutter
some time in August. As such, we'd like to settle the FFI "version 1"
API we'll ship with.
We discussed a couple changes on IRC. Comments are welcome, especially
if you have a real case to make against one of these:
* A new function ffi_convention will be added for setting the calling
convention of a given FFI module. It will accept :default for normal
calling convention and :stdcall for stdcall (a la win32).
* set_ffi_lib will be shortened to ffi_lib
* FFI will no longer be omnipresent on all modules; instead, you will
need to extend FFI from within the module body to gain FFI capabilities:
module POSIX
extend FFI
ffi_lib 'libc'
attach_function 'getuid', :my_getuid, [], :uint
end
There are a couple open questions though:
* Is attach_foreign going to remain part of FFI API? Evan pointed out
that nothing uses it currently. I'd rather not ship a function that's
going to go away, but I understand some of the justification behind
attach_foreign
* I would like attach_function to accept either a string or a symbol for
the function name. It feels cumbersome and nonuniform to me to have the
name you're binding to be a symbol and the function you're binding to be
a string.
* This is probably a long-shot, but it also seems to me like the
parameter order should be:
attach_function [bound_name], <c func name>, <params>, <return type>
With the bound name still being optional. Then the first param is always
the name you're going to bind the function to, and the remaining args
(possibly including the first) are the three specifiers for the function
itself. So with the change to allow using symbols for the C function
name, we'd have:
attach_function :getuid, [], :uint
attach_function :my_getpid, :getpid, [], :uint
And so on.
- Charlie
Charles Oliver Nutter
Charles Oliver Nutter wrote:
> * Is attach_foreign going to remain part of FFI API? Evan pointed out
> that nothing uses it currently. I'd rather not ship a function that's
> going to go away, but I understand some of the justification behind
> attach_foreign
attach_foreign is out, since it doesn't add any new capability and it
isn't being used by any code. I think it should be physically removed,
since people have a tendency to use code whether it's "officially
supported" or not.
> * I would like attach_function to accept either a string or a symbol for
> the function name. It feels cumbersome and nonuniform to me to have the
> name you're binding to be a symbol and the function you're binding to be
> a string.
This is in, strings or symbols are both ok.
> * This is probably a long-shot, but it also seems to me like the
> parameter order should be:
>
> attach_function [bound_name], <c func name>, <params>, <return type>
This is also in, but will require a big "one shot change" to rubinius,
FFI, and libraries to support it, since there's no halfway.
- Charlie
Wayne Meissner
I have a couple of (minor?) api suggestions that allow FFI code to be
work better on the JVM, but should be trivial to implement on
rubinius:
1) :buffer, :buffer_in, :buffer_out parameter types. Rubinius
uses :string as a param type for NUL terminated strings, as well a
byte-buffer type param.
e.g.
attach_function 'fread',
[:string, :size_t, :size_t, :pointer], :size_t
attach_function 'fwrite',
[:string, :size_t, :size_t, :pointer], :size_t
The JVM doesn't allow direct access to strings/byte arrays, so they
need to be copied - and since :string doesn't denote direction, for
jruby, we first copy a the string to native memory, NUL terminate it,
then copy the memory out of native memory again after the call. i.e.
an extra memory copy which is not needed.
With :buffer_out and :buffer_in, these become:
attach_function 'fread',
[:buffer_out, :size_t, :size_t, :pointer], :size_t
attach_function 'fwrite',
[:buffer_in, :size_t, :size_t, :pointer], :size_t
When used like:
fwrite("test", 1, 4, fp)
buf = ' ' * 256
fread(buf, 1, 256, fp)
the data is only copied one way - for fread, out of native memory into
a ruby string, and for fwrite, the data only gets copied into native
memory. For bidirectional data, :buffer is used. Neither :buffer
nor :buffer_in NUL terminate the buffer.
:buffer, :buffer_in, :buffer_out also accept a MemoryPointer as an
argument.
2) Make :string mean "NUL terminated, immutable string". Same
reasoning as for :buffer. Depends on :buffer being implemented, and
any code that depends on the bidi nature of :string to be converted to
use :buffer. Maybe a new param type like :cstring would be better
than re-purposing the existing :string type.
3) Additional Struct allocation methods. For small structs, where the
struct is filled out by the native function (e.g. stat(2)), it is
cheaper to allocate the memory on the java heap, pass a temporary
native buffer to the method, and copy data out to the java heap
buffer.
e.g. for stat, we currently do:
module Posix
extend JFFI::Library
attach_function :stat, [ :string, :buffer_out ], :int
end
buf = Stat.allocate # Allocate on the heap
Posix.stat("/tmp", buf.pointer)
For rubinius, Struct#allocate() can just be an alias for Struct.new
Whilst we can get by without these changes (either by ignoring the
performance hit, or using the JFFI module instead of FFI), I want to
write a few blog posts/tutorials on how to use FFI from jruby, and it
would be nice if I could write examples that are both optimal for
jruby, and still run on rubinius.
Evan Phoenix
My gut feeling is a little uncomfortable with 3 buffer types. I can
see the reasoning, but how about a compromise? The :string type would
be as you say, NUL terminated and immutable. JRuby, that should mean
you copy it out to native memory only.
If the function you're calling is going to be manipulating the data,
you just use :pointer and pass in a MemoryPointer object you've
already created. That should in fact be more performant than the
buffer types anyway, because it doesn't require in implicit copying.
The methods on MemoryPointer would be used to copy the data from
native memory back into rubyland.
So you'd do:
attach_function 'fread',
[:pointer, :size_t, :size_t, :pointer], :size_t
attach_function 'fwrite',
[:pointer, :size_t, :size_t, :pointer], :size_t
ptr = MemoryPointer.new(256)
ptr.write_string "test\0"
fwrite(ptr, 1, 4, fp)
# you can even then reuse ptr to save on memory, C style.
fread(ptr, 1, 256, fp)
I think this works better because it matches closely the semantics of
C, where you can pass in a pointer to anything to fwrite anyway and
have the memory itself written out. Scary as that sounds (handing a
int* to fwrite), it's not uncommon.
All in all, I like this better because it doesn't require an implicit
allocations or copies. The control is in the caller entirely. It's a
little less convenient, but we can fix that easily with some shorter
apis:
fwrite(MemoryPointer.string("test"), 1, 4, fp)
The user is going to immediately see why they're doing that, because
fwrite takes a void* as it's first argument. The fact that :string
could be used anywhere any pointer type was used actually slowed
people down when they originally used the API some.
People are always going to be writing wrappers around attached
functions anyway, so the minor extra typing overhead isn't a big deal.
They'll do it in the wrapper once and be done with it.
>
>
> 3) Additional Struct allocation methods. For small structs, where the
> struct is filled out by the native function (e.g. stat(2)), it is
> cheaper to allocate the memory on the java heap, pass a temporary
> native buffer to the method, and copy data out to the java heap
> buffer.
>
> e.g. for stat, we currently do:
>
> module Posix
> extend JFFI::Library
> attach_function :stat, [ :string, :buffer_out ], :int
> end
> buf = Stat.allocate # Allocate on the heap
> Posix.stat("/tmp", buf.pointer)
>
> For rubinius, Struct#allocate() can just be an alias for Struct.new
Sure.
>
>
>
> Whilst we can get by without these changes (either by ignoring the
> performance hit, or using the JFFI module instead of FFI), I want to
> write a few blog posts/tutorials on how to use FFI from jruby, and it
> would be nice if I could write examples that are both optimal for
> jruby, and still run on rubinius.
Yeah, that makes sense. We should figure this out so we can all write
some good tutorials on using it.
- Evan
>
>
>
>
>
> >
Wayne Meissner
> My gut feeling is a little uncomfortable with 3 buffer types. I can
> see the reasoning, but how about a compromise?
>
I don't like adding a heap of types (especially ones that have special
meaning), but couldn't figure out a cleaner way of doing it. Ruby
needs annotations :-)
>
> If the function you're calling is going to be manipulating the data,
> you just use :pointer and pass in a MemoryPointer object you've
> already created. That should in fact be more performant than the
> buffer types anyway, because it doesn't require in implicit copying.
> The methods on MemoryPointer would be used to copy the data from
> native memory back into rubyland.
The answer to which is more performant is "it depends".
doing:
s = "test\0"
ptr = MemoryPointer.new(256, false) # false == don't zero the memory
ptr.write_string(s)
fwrite(ptr, 1, ptr.size, fp)
results in:
JNI call + malloc(4)
JNI call + GetByteArrayRegion
JNI call + fwrite
JNI call + free
Wheras:
s = "test\0"
fwrite(s, 1, s.length, fp)
results in:
JNI call + allocate 5 bytes on stack + GetByteArrayRegion + fwrite
Whilst JNI calls are no longer a huge overhead (at least in JDK 6/7),
they're still non-zero, and significant for some functions.
The JVM is also better at managing memory on the java heap, than it is
the C heap (which it knows nothing about) - with MemoryPointer its
possible to get into a state where a large amount of C heap memory is
allocated, but the java garbage collector hasn't run yet to clear out
the java references to it and free it.
Its a similar situation for stat() - using a C heap MemoryPointer was
about 40% slower than a java heap MemoryPointer.
These examples are a bit skewed - the jffi backend has optimizations
for small byte buffers/arrays (e.g. short strings and small structs),
so it doesn't have to incur the overhead of a malloc() call for the
temporary native memory area. For large data transfers, the JNI and
malloc overhead become a wash.
However, most of the time, small structs/chunks of data is what gets
passed into/out of C functions, so its not a bad comparison.
For fread on the other hand, it can be faster to use a C heap
MemoryPointer, since you can make it copy out _only_ the number of
bytes returned by fread(), not the whole buffer.
> The user is going to immediately see why they're doing that, because
> fwrite takes a void* as it's first argument. The fact that :string
> could be used anywhere any pointer type was used actually slowed
> people down when they originally used the API some.
Thats interesting. How did it slow them down? Was it confusing?
>
> People are always going to be writing wrappers around attached
> functions anyway, so the minor extra typing overhead isn't a big deal.
> They'll do it in the wrapper once and be done with it.
You'd be surprised at how many people _don't_ write wrappers for
native APIs - at least from what I've seen of people using JNA. (and
some ruby and python modules too)
Evan Phoenix
On Aug 1, 2008, at 1:33 AM, Wayne Meissner wrote:
>
> 2008/8/1 Evan Phoenix <ev...@fallingsnow.net>:
>
>> My gut feeling is a little uncomfortable with 3 buffer types. I can
>> see the reasoning, but how about a compromise?
>>
> I don't like adding a heap of types (especially ones that have special
> meaning), but couldn't figure out a cleaner way of doing it. Ruby
> needs annotations :-)
As always, it's a trade off.
Ok, well, what about if we just had a :buffer type, that takes a
FFI::Buffer object. The interior of FFI::Buffer is system dependent,
allowing you to use the java heap. It would just have simple methods
to pull and push data in and out as String objects.
We could also layer logic about direction into FFI::Buffer.
buf = FFI::Buffer.output(256)
fread(buf, 1, 256, fp)
p buf.string(4) # => "test"
You're underlying code could check a flag in the buffer to see that
it's output only to know whether to copy it.
That should allow you to perform Java optimizations to make this work,
right?
>
>
> Its a similar situation for stat() - using a C heap MemoryPointer was
> about 40% slower than a java heap MemoryPointer.
>
> These examples are a bit skewed - the jffi backend has optimizations
> for small byte buffers/arrays (e.g. short strings and small structs),
> so it doesn't have to incur the overhead of a malloc() call for the
> temporary native memory area. For large data transfers, the JNI and
> malloc overhead become a wash.
>
> However, most of the time, small structs/chunks of data is what gets
> passed into/out of C functions, so its not a bad comparison.
>
> For fread on the other hand, it can be faster to use a C heap
> MemoryPointer, since you can make it copy out _only_ the number of
> bytes returned by fread(), not the whole buffer.
>
>> The user is going to immediately see why they're doing that, because
>> fwrite takes a void* as it's first argument. The fact that :string
>> could be used anywhere any pointer type was used actually slowed
>> people down when they originally used the API some.
>
> Thats interesting. How did it slow them down? Was it confusing?
They didn't realize that they had to make a big string (" " * 256),
pass it in, and it would be magically populated. No ruby methods work
that way, so they were surprised.
- Evan
Wayne Meissner
>
> Ok, well, what about if we just had a :buffer type, that takes a
> FFI::Buffer object. The interior of FFI::Buffer is system dependent,
> allowing you to use the java heap. It would just have simple methods
> to pull and push data in and out as String objects.
>
> We could also layer logic about direction into FFI::Buffer.
>
> buf = FFI::Buffer.output(256)
> fread(buf, 1, 256, fp)
> p buf.string(4) # => "test"
>
> You're underlying code could check a flag in the buffer to see that
> it's output only to know whether to copy it.
>
> That should allow you to perform Java optimizations to make this work,
> right?
I kinda like it ...
So, if we extend this so FFI::Struct can also use a Buffer as storage,
we could have something like:
s = Stat.output
Posix.stat("/tmp", s.pointer)
Structs would also have:
Struct.input # heap buffer, copies to native memory
Struct.heap # bidirectional heap buffer
Struct.direct # uses a native buffer
Do we need a FFI::Buffer class, or should we repurpose MemoryPointer
to be able to use heap memory (it currently can on jruby), and add
methods to allocate it as input/output/bidi/native?
Evan Phoenix
On Aug 1, 2008, at 3:46 AM, Wayne Meissner wrote:
>
> 2008/8/1 Evan Phoenix <ev...@fallingsnow.net>:
>>
>> Ok, well, what about if we just had a :buffer type, that takes a
>> FFI::Buffer object. The interior of FFI::Buffer is system dependent,
>> allowing you to use the java heap. It would just have simple methods
>> to pull and push data in and out as String objects.
>>
>> We could also layer logic about direction into FFI::Buffer.
>>
>> buf = FFI::Buffer.output(256)
>> fread(buf, 1, 256, fp)
>> p buf.string(4) # => "test"
>>
>> You're underlying code could check a flag in the buffer to see that
>> it's output only to know whether to copy it.
>>
>> That should allow you to perform Java optimizations to make this
>> work,
>> right?
>
> I kinda like it ...
>
> So, if we extend this so FFI::Struct can also use a Buffer as storage,
> we could have something like:
>
> s = Stat.output
> Posix.stat("/tmp", s.pointer)
>
> Structs would also have:
> Struct.input # heap buffer, copies to native memory
Sure.
>
> Struct.heap # bidirectional heap buffer
> Struct.direct # uses a native buffer
I don't like these because they have no meaning outside of the JVM. A
person would never know when to use heap versus direct. JRuby should
use one of those types based on the usage pattern the user wants.
'input' and 'output' are usage patterns that they understand, because
they directly relate to the C semantics of the function.
FFI::Buffer would have 'input', 'output', and 'new' (bifi). Having
more types than that just confuses the API a lot I think.
>
>
> Do we need a FFI::Buffer class, or should we repurpose MemoryPointer
> to be able to use heap memory (it currently can on jruby), and add
> methods to allocate it as input/output/bidi/native?
FFI::Buffer could be a MemoryPointer subclass, but I don't think it
should all be in MemoryPointer. Again, the API should give the user
hints on how the classes and methods are to be used, without them
having to worry if they're in JRuby or Rubinius.
MemoryPointer is a generic pointer to a section of memory. The logic
about what should happen to the memory pointed to should be in the
FFI::Buffer subclass, since it wants specific control over that.
- Evan
>
>
> >
Charles Oliver Nutter
>> Struct.heap # bidirectional heap buffer
>> Struct.direct # uses a native buffer
>
> I don't like these because they have no meaning outside of the JVM. A
> person would never know when to use heap versus direct. JRuby should
> use one of those types based on the usage pattern the user wants.
> 'input' and 'output' are usage patterns that they understand, because
> they directly relate to the C semantics of the function.
I concur. Unless there's a compelling reason to have the distinction in
the API, it's most definitely an implementation smell leaking through.
Evan, I would put this question to you: if Rubinius started to support
"pinnning" heap memory and there was a measurable speed gain from
allocating output memory on the heap instead of directly, how would you
refer to that? Is there an impl-nonspecific abstraction we're missing here?
> FFI::Buffer would have 'input', 'output', and 'new' (bifi). Having
> more types than that just confuses the API a lot I think.
I almost wonder if having inout or bidir instead of new wouldn't help
make it clear there are more optimal choices, but I won't bikeshed on it.
>> Do we need a FFI::Buffer class, or should we repurpose MemoryPointer
>> to be able to use heap memory (it currently can on jruby), and add
>> methods to allocate it as input/output/bidi/native?
>
> FFI::Buffer could be a MemoryPointer subclass, but I don't think it
> should all be in MemoryPointer. Again, the API should give the user
> hints on how the classes and methods are to be used, without them
> having to worry if they're in JRuby or Rubinius.
>
> MemoryPointer is a generic pointer to a section of memory. The logic
> about what should happen to the memory pointed to should be in the
> FFI::Buffer subclass, since it wants specific control over that.
Also agree. Buffer is a special case of a MemoryPointer that has an
arbitrary-length block of memory attached.
- Charlie
Wayne Meissner
finalized this week.
Here it the current Buffer/Struct API in JRuby. Unless there is a
good reason to change it, its what we'll be going with.
#
# Memory is an abstract class that just defines operations on a memory object.
#
Memory
# puts an 8 bit integer value at the specified byte offset
put_int8(offset, value)
# gets an 8 bit integer value at the specified byte offset
get_int8(offset)
# put an array of 8 bit integer values at the byte offset
put_array_of_int8(offset, values)
# get a len length array of 8 bit integer values at the byte offset
get_array_of_int8(offset, len)
# similarly for uint8, int16, uint16, int32, uint32, int64, uint64,
float32, float64
# aliases for char/uchar, short/ushort, int/uint,
long_long/ulong_long, long/ulong
# as appropriate
# puts a string into the buffer at the specified byte offset.
put_string(offset, value, len = nil)
# gets a string of length len at the specified byte offset
get_string(offset, len = nil)
# puts a pointer value at the byte offset
put_pointer(offset, value)
# gets a pointer value at a byte offset
get_pointer(offset)
#
# A Buffer may be allocated either from native memory or the heap, or
# even non-existent until such time as the buffer is passed to a
native function.
#
# The memory passed in to the native function is transient, and the address
# passed to a native function can change with each invocation.
#
Buffer < Memory
# Allocate a buffer that is used as an IN param on a native func
alloc_in(size, count = nil, clear = true)
# Allocate a buffer that is used as an OUT param on a native func
alloc_out(size, count = nil, clear = true)
# Allocate a buffer that is used as both an IN & OUT param on a native func
alloc_inout(size, count = nil, clear = true)
#
# size can be either an integer (e.g. 128), or a type (e.g. :int, :long)
#
Buffer.new raises an exception that instructs the caller to use one of
the alloc functions.
#
# Struct is the same as rubinius, with the addition of the following allocation
# methods
Struct
# Allocate a Struct instance that is used as an IN param on a native func
alloc_in(clear = true)
# Allocate a Struct instance that is used as an OUT param on a native func
alloc_out(clear = true)
# Allocate a Struct instance that is used as both an IN & OUT param
on a native func
alloc_inout(clear = true)
MemoryPointer is the existing rubinius api, but with the addition of
the Memory put/get
operations.
A MemoryPointer always refers to a native allocated memory area (or NULL).
Buffer is not a subclass of MemoryPointer, since there are
MemoryPointer operations that don't make sense (e.g. null?, address,
free, autorelease=) in the context of non-native memory.
Evan Phoenix
On Aug 17, 2008, at 8:14 PM, Wayne Meissner wrote:
>
> I don't want to bikeshed this thing either, and I need to get it
> finalized this week.
>
> Here it the current Buffer/Struct API in JRuby. Unless there is a
> good reason to change it, its what we'll be going with.
>
> #
> # Memory is an abstract class that just defines operations on a
> memory object.
> #
Ok, but who has these methods? MemoryPointer? FFI::Buffer?
Also, MemoryPointer also has most of these, why aren't we just
continuing to use those? I don't want 2 methods that do the exact same
thing.
Ew. Why isn't this just new()?
>
> #
> # size can be either an integer (e.g. 128), or a type
> (e.g. :int, :long)
> #
>
> Buffer.new raises an exception that instructs the caller to use one of
> the alloc functions.
>
> #
> # Struct is the same as rubinius, with the addition of the following
> allocation
> # methods
> Struct
> # Allocate a Struct instance that is used as an IN param on a
> native func
> alloc_in(clear = true)
>
> # Allocate a Struct instance that is used as an OUT param on a
> native func
> alloc_out(clear = true)
>
> # Allocate a Struct instance that is used as both an IN & OUT param
> on a native func
> alloc_inout(clear = true)
Same, should be new().
>
>
>
> MemoryPointer is the existing rubinius api, but with the addition of
> the Memory put/get
> operations.
>
> A MemoryPointer always refers to a native allocated memory area (or
> NULL).
>
> Buffer is not a subclass of MemoryPointer, since there are
> MemoryPointer operations that don't make sense (e.g. null?, address,
> free, autorelease=) in the context of non-native memory.
Could you explain what 'non-native memory', 'heap', and 'native
memory' are?
I still don't seem to get this.
- Evan
>
>
> >
Wayne Meissner
>
> Ok, but who has these methods? MemoryPointer? FFI::Buffer?
Both have the same access methods.
> Also, MemoryPointer also has most of these, why aren't we just
> continuing to use those? I don't want 2 methods that do the exact same
> thing.
The problem is the existing API has sub-optimal performance for the
non-zero offset case.
Consider:
buf = FFI::Buffer.alloc_in(256)
(buf + 4).write_int(0xdeadbeef)
vs
buf = FFI::Buffer.alloc_in(256)
buf.put_int(4, 0xdeadbeef);
The first form results in creation of a temporary Buffer instance, vs
the second which does not.
>> # Allocate a buffer that is used as both an IN & OUT param on a
>> native func
>> alloc_inout(size, count = nil, clear = true)
>
> Ew. Why isn't this just new()?
>
>> # Allocate a Struct instance that is used as both an IN & OUT param
>> on a native func
>> alloc_inout(clear = true)
>
> Same, should be new().
The logic behind not making it new was thus:
- If you call it 'new', it will be the default
- If its the default, people will tend to use it
- Most C functions only move the data in one direction - either in or
out, not both
- Ergo, for most calls, using 'new' results in the worst performance by default.
Weighed against that:
- Calling it 'new' means that it always does right thing functionality
wise, even if it is sub-optimal in performance.
I had to choose one of those options, and I chose performance, with a
slight "Ew" factor for the least-used case where you need to allocate
for both directions.
If we backtrack a bit, adding :buffer_in, :buffer_out, :buffer_inout
as parameter types moves the ugliness to the function declaration.
One advantage to the parameter types is that it removes the need to
know which direction the function requires when allocating the buffer,
as it is handled at function declaration time, not time of function
use.
Once again, we just need to choose which is the least worst way of doing it.
> Could you explain what 'non-native memory', 'heap', and 'native
> memory' are?
>
> I still don't seem to get this.
Very roughly speaking, the JVM has two types of memory:
1) Java heap memory (aka non-native, non-direct)
- Managed by the JVM garbage collector
- Allocation is really cheap. Zeroing fresh memory allocations is free.
- Access to this memory is really cheap, as it can be accessed
directly by the JVM (i.e. accesses get inlined by the JIT)
- Can NOT be directly passed to native code. i.e. to pass to a native
function, a temporary native memory block (commonly from the C heap)
must be allocated, and the data copied from the java heap object to
the native block, then the temporary memory is passed to the native
function. After the native function call, if the function modified
the memory block (i.e. it was an OUT parameter), then the contents of
the temporary native memory must be copied back to the java heap
memory.
2) Direct memory (aka native)
- Can NOT be directly accessed by the JVM, so the contents must first
be copied to a java heap object before it can be used. i.e. accessing
from ruby code is expensive.
- Allocation and freeing are expensive.
- Access by native functions is cheap/free.