Calling varargs functions from the FFI

[David Banks]

Hi,

Is it possible to call a varargs function using the FFI? The library
I am trying to wrap has variants defined with both '...' and 'va_list'
variants, so either of those techniques would be fine. I don't need
to be able to pass arbitrary arguments to the FFI function, even.
Something like this fails, however:

(library (mosh-printf)
(export printf)
(import (rnrs)
(mosh ffi))

(define lib (open-shared-library "libc.so.6"))

(define printf
(c-function lib void printf char* char*)))

mosh> (printf "Hello, %s!\n" "world")
zsh: illegal hardware instruction mosh

This seems like quite an uncommon feature in FFIs, probably as it's
extremely unportable. The only way I can think to do it without FFI
support is to manually construct the va_list array using memory
operations, which is _extremely_ hairy. Interestingly mzscheme/Racket
handles the above case:

(define printf (get-ffi-obj "printf" #f
(_fun _string _string -> _void)))

> (printf "Hello %s!\n" "world")
Hello, world!

However Racket also does not handle the case of arbitrary arguments.
What do you think guys?

Cheers,
David

[okuoku]

Hi,

2011/3/7 David Banks <amo...@gmail.com>:

> Hi,
>
> Is it possible to call a varargs function using the FFI?

Unfortunately, no, in amd64.
In i386, your printf example will work. Thanks to their friendly C
function calling
convention... (I had checked with Win32 version of nmosh, but I think
mosh on Linux/BSDs will also work similar.)

Both mosh's own FFI backend and Racket's FFI backend (it's called
libffi) are not supporting vararg call.
So I think you are using i386 version of Racket and amd64 version of
mosh. If not, Racket has some magic :-P

Anyway, implementing vararg call is somewhat hard work. I'm now
considering 2 way to implement it:

1) using LLVM.
LLVM knows about how to call vararg functions. So we can use it to
generate calling stub.

2) Implementing some "tiny-compiler."
Now I'm implementing amd64 run-time assembler like industria library already has
( https://code.launchpad.net/~weinholt/scheme-libraries/industria ).
We may use it to generate calling stub.

But both are not coming anytime soon..

[David Banks]

Hey, I had the need to do this again today and I was wondering if
there has been any progress on this?
I noticed that some of the FFI stuff for nmosh has been tweaked a bit
over the past year, is it now possible to wrap a varargs function from
the FFI?
If it's not, have you heard of this library? http://dyncall.org/
Would it be feasible / desirable to integrate this with mosh?

Cheers,
David

[quote below for context]

On 7 March 2011 13:04, okuoku <oku...@gmail.com> wrote:
> 2011/3/7 David Banks <amo...@gmail.com>:
>> Hi,
>>
>> Is it possible to call a varargs function using the FFI?

...

> Anyway, implementing vararg call is somewhat hard work. I'm now
> considering 2 way to implement it:
>
> 1) using LLVM.
> LLVM knows about how to call vararg functions. So we can use it to
> generate calling stub.
>
> 2) Implementing some "tiny-compiler."
> Now I'm implementing amd64 run-time assembler like industria library already has
> ( https://code.launchpad.net/~weinholt/scheme-libraries/industria ).
> We may use it to generate calling stub.
>
> But both are not coming anytime soon..

--
David Banks  <amo...@gmail.com>

[okuoku]

2012/5/7 David Banks <amo...@gmail.com>:

> Hey, I had the need to do this again today and I was wondering if
> there has been any progress on this?
> I noticed that some of the FFI stuff for nmosh has been tweaked a bit
> over the past year, is it now possible to wrap a varargs function from
> the FFI?

If we don't have any floating point arguments to the FFI function, we
can call vararg functions even on amd64 like this:

(import (rnrs) (mosh ffi)
(nmosh pffi interface)
(srfi :42))

(define lib (open-shared-library "libc.so"))
(define printf7 (c-function lib int printf
;; ARGS (Up to 7, for example)
void* void* void*
void* void* void* void*))

(define (arg->pointer obj)
(cond
((bytevector? obj)
;; bytevector-pointer is from (nmosh pffi interface)
(bytevector-pointer obj))
((string? obj)
(arg->pointer (string->utf8 obj)))
((pointer? obj) obj)
((fixnum? obj) (integer->pointer obj))
(else (assertion-violation 'arg->int-pointer "invalid argument" obj))))

(define (printf . args)
(define arglen (length args))
(define call-args (list-ec (: i 7) ;; Arg count for printf7
(if (< i arglen)
(arg->pointer (list-ref args i))
(integer->pointer 0) ;; Dummy arg
)))
(write call-args)(newline)
(apply printf7 call-args))

(printf "hoge\n")
(printf "%s, %s\n" "Hello" "World")
(printf "%s %d %s\n" "Hello" 123 "World")


Points are:

- We cannot pass any double or float arguments for vararg functions on amd64
- use void* as arguments for variable portion of the function
- nmosh has "bytevector-pointer" (Create a pointer object which
references a bytevector content) in (nmosh pffi interface) library
- Fill dummy arguments with some pointers

Currently, we don't care about floating point arguments for vararg functions.
Amd64 ABI uses SSE registers to pass floating point arguments and we
don't pass its size to the function.


Hope this helps,
-- oku

> [quote below for context]
>
> On 7 March 2011 13:04, okuoku <oku...@gmail.com> wrote:
>> 2011/3/7 David Banks <amo...@gmail.com>:
>>> Hi,
>>>
>>> Is it possible to call a varargs function using the FFI?
> ...
>> Anyway, implementing vararg call is somewhat hard work. I'm now
>> considering 2 way to implement it:
>>
>> 1) using LLVM.
>> LLVM knows about how to call vararg functions. So we can use it to
>> generate calling stub.
>>
>> 2) Implementing some "tiny-compiler."
>> Now I'm implementing amd64 run-time assembler like industria library already has
>> ( https://code.launchpad.net/~weinholt/scheme-libraries/industria ).
>> We may use it to generate calling stub.
>>
>> But both are not coming anytime soon..
>
>
> --
> David Banks  <amo...@gmail.com>
>

> --
> You received this message because you are subscribed to the Google Groups "Mosh Developer Disscus" group.
> To post to this group, send email to mosh-develo...@googlegroups.com.
> To unsubscribe from this group, send email to mosh-developer-di...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mosh-developer-discuss?hl=en.
>

[okuoku]

Ah we have to patch ffi_stub_x86_64.S as:

diff --git a/src/ffi_stub_x86_64.S b/src/ffi_stub_x86_64.S
index 5c5049c..0c93f36 100644
--- a/src/ffi_stub_x86_64.S
+++ b/src/ffi_stub_x86_64.S
@@ -60,7 +60,9 @@ arguments_loop:
addq $8, %r13
jmp arguments_loop
arguments_done:
- call *%rax
+ movq %rax,%r11
+ xor %al,%al
+ call *%r11
movq %rbp, %rsp
popq %rbp
pop %r15

I will test/commit it later soon.

-- oku

2012/5/9 okuoku <oku...@gmail.com>: