Allegro CL foreign function interface

80 views
Skip to first unread message

Constantine Vetoshev

unread,
Jun 14, 2000, 3:00:00 AM6/14/00
to

Hello all,

I'm having trouble understanding how ACL's foreign functions handle
pointers. I have a C function with the following prototype:

void arg_parse(int *argc, char ***argv);

The idea is to take the main function's argc and argv arguments, pass
them to arg_parse, and modify them from arg_parse. I need to be able
to call this function from Lisp. (I use the trial version of ACL 5.0.1
on FreeBSD.)

If these were not pointer arguments, then the following works nicely:

(def-foreign-call (arg-test "arg_test")
((argc (* :int) integer)
(argv (* :char) (simple-array simple-string (*))))
:returning :void)

I have even figured out how to pass in a pointer to the integer
argument using an array:

(def-foreign-call (simple-pointer-arg-test "simple_pointer_arg_test")
((argc (* :int) (array fixnum (1))))
:returning :void)

(This will work with a void simple_pointer_arg_test(int *n) function
that C modifies.)

However, all of my attempts to modify the array of strings fail. I
tried the following definition for the arg_parse function above:

(def-foreign-call (arg-parse "arg_parse")
((argc (* :int) (array fixnum (1)))
(argv (* :char) (simple-array simple-string)))
:returning :void)

However, that results in the C function seeing some bizarre data in
the array. (But at least it doesn't segfault.)

Do I understand how Allegro's :ff works correctly? If so, what mistake
am I making here?


Thanks,
---Constantine Vetoshev

Joe Marshall

unread,
Jun 14, 2000, 3:00:00 AM6/14/00
to
Constantine Vetoshev <constantin...@dartmouth.edu> writes:

> Hello all,
>
> I'm having trouble understanding how ACL's foreign functions handle
> pointers. I have a C function with the following prototype:
>
> void arg_parse(int *argc, char ***argv);

This has a lot of indirection.

> The idea is to take the main function's argc and argv arguments, pass
> them to arg_parse, and modify them from arg_parse. I need to be able
> to call this function from Lisp. (I use the trial version of ACL 5.0.1
> on FreeBSD.)

Are you getting the actual args to main from somewhere, or do you want
to hand arg_parse some strings you have consed in lisp?

> If these were not pointer arguments, then the following works nicely:
>
> (def-foreign-call (arg-test "arg_test")
> ((argc (* :int) integer)
> (argv (* :char) (simple-array simple-string (*))))
> :returning :void)
>

> However, all of my attempts to modify the array of strings fail. I
> tried the following definition for the arg_parse function above:
>
> (def-foreign-call (arg-parse "arg_parse")
> ((argc (* :int) (array fixnum (1)))
> (argv (* :char) (simple-array simple-string)))
> :returning :void)

There isn't enough indirection there. The array of simple strings
needs to be pointed at, so you will need another wrapper around it.

> However, that results in the C function seeing some bizarre data in
> the array. (But at least it doesn't segfault.)

The C function sees a char ** argv rather than a char *** argv.

Constantine Vetoshev

unread,
Jun 14, 2000, 3:00:00 AM6/14/00
to
Joe Marshall <jmar...@alum.mit.edu> writes:

> > The idea is to take the main function's argc and argv arguments, pass
> > them to arg_parse, and modify them from arg_parse. I need to be able
> > to call this function from Lisp. (I use the trial version of ACL 5.0.1
> > on FreeBSD.)
>
> Are you getting the actual args to main from somewhere, or do you want
> to hand arg_parse some strings you have consed in lisp?

Well, I need to pass in strings from Lisp into arg_parse. (See below
on what I'm trying to do to call them.)

> There isn't enough indirection there. The array of simple strings
> needs to be pointed at, so you will need another wrapper around it.
>
> > However, that results in the C function seeing some bizarre data in
> > the array. (But at least it doesn't segfault.)
>
> The C function sees a char ** argv rather than a char *** argv.

I realize there are problems with indirection here. My latest attempt
to provide the correct indirection is this:

(def-foreign-call (arg-parse "arg_parse")
((argc (* :int) (array fixnum (1)))

(argv (* :char) (array (simple-array simple-string))))
:returning :void)

and calling the function with this:

(arg-parse
(make-array 1 :element-type 'fixnum :initial-contents '(2))
(make-array 1 :element-type 'array :initial-contents
`(,(make-array 2 :initial-contents '("lisp" "hello")))))

This results in a segfault. (I should point out that arg_parse works
fine when called from C code.)


Thanks,
---Constantine Vetoshev

Erik Naggum

unread,
Jun 14, 2000, 3:00:00 AM6/14/00
to
* Constantine Vetoshev <constantin...@dartmouth.edu>

| I realize there are problems with indirection here.
:

| This results in a segfault.

So print out the values of the "pointers" you get before you do
something with them. Compare with the memory map of the ACL
process, either from the output of (room) or /proc/<pid>/maps.
Enlightenment should ensue.

What you're trying to do is called "in out" arguments in Ada, which
is an unusually clean way to express it, but it is (of course!)
implemented in the singularly most braindamaged way in C/C++. You
are actually trying to pass two arguments into your function and
return two values from it, but conflate the two operations because
it is possible through using too pointers too much. C being what it
is lacks support for multiple return values, so the notion that it
is meaningful to pass pointers to memory objects into which any
random function may write random values without having a clue where
they point, has _not_ been debunked as the sheer idiocy it really is.

Quite often, the problems that arise from wanting a foreign function
interface that can deal with absolutely any crap any idiot is likely
to do in C can be solved with a very simple wrapper function in the
target language, as opposed to random qmagic in Common Lisp. (Note
to control freaks: You lose _less_ performance using a wrapper than
by using a more complex FFI.)

In this case, it's even worse: You are actually passing in a single
value and returning a single value: A vector of strings, but since
whoever invented C's main was probably stoned out of his wits and
figured you should get _both_ a count and zero-terminated vector for
the arguments, people everywhere have since believed in this calling
convention. *Sigh*

If you can figure out a way to pass vectors of strings in both
directions, you're all set, and can write your simple wrapper
function in no time. Both are fairly simple tasks, so as long as
you do The Right Thing and forget how you would do it in C, you
should be able to get a good grip on this.

#:Erik
--
If this is not what you expected, please alter your expectations.

Joe Marshall

unread,
Jun 14, 2000, 3:00:00 AM6/14/00
to
Constantine Vetoshev <constantin...@dartmouth.edu> writes:

> Joe Marshall <jmar...@alum.mit.edu> writes:
>
> I realize there are problems with indirection here. My latest attempt
> to provide the correct indirection is this:
>
> (def-foreign-call (arg-parse "arg_parse")
> ((argc (* :int) (array fixnum (1)))
> (argv (* :char) (array (simple-array simple-string))))
> :returning :void)
>
> and calling the function with this:
>
> (arg-parse
> (make-array 1 :element-type 'fixnum :initial-contents '(2))
> (make-array 1 :element-type 'array :initial-contents
> `(,(make-array 2 :initial-contents '("lisp" "hello")))))
>
> This results in a segfault. (I should point out that arg_parse works
> fine when called from C code.)


Right, that's because the outer array points to an inner lisp array
which points to the lisp strings. (I've never been able to get the
ffi to automatically do conversions that deep). What I would do is
something like this:

;; Convert Lisp strings to C strings.
(excl:with-native-string (string1 "lisp")
(excl:with-native-string (string2 "hello")

;; Make an array of pointers to C strings.
(ff:with-stack-fobject (string-array ....whatever syntax needed...)
(setf (ff:fslot-value-typed ...?... nil string-array 0) string1)
(setf (ff:fslot-value-typed ...?... nil string-array 1) string2)

;; Make a pointer to that array of pointers
(ff:with-stack-object (ptr-to-string-array ...whatever...)
(setf (ff:fslot-value-typed ...?.. nil ptr-to-string-array) 0 string-array)

...call foreign function here...))))

I have to admit that I find the type conversion between C and lisp to
be confusing, so I didn't put in the type declarations you need, but
maybe this will help.

Sandeep Koranne

unread,
Jun 15, 2000, 3:00:00 AM6/15/00
to
"Erik Naggum" <er...@naggum.no>
Wrote on Wednesday 14 June 2000 21:49

> A vector of strings, but since
> whoever invented C's main was probably stoned out of his wits and
> figured you should get _both_ a count and zero-terminated vector for
> the arguments, people everywhere have since believed in this calling
> convention. *Sigh*

Maybe I have not understood your comment correctly, but it seems to me that
C/C++
main fiunction does NOT get a zero terminated vector (a vector by definitin
is a single dimension array) while in C/C++ the arguments to main are,

int main (int argc, char* argv[]) : or a vector or vectors.
So we DO need both the count of argument and the Vector of Vectors (the
elements of argv are of course ZERO terminated strings).

Regards
Sandeep Koranne


Hrvoje Niksic

unread,
Jun 15, 2000, 3:00:00 AM6/15/00
to
"Sandeep Koranne" <sandeep...@nospam.philips.com> writes:

> "Erik Naggum" <er...@naggum.no>
> Wrote on Wednesday 14 June 2000 21:49
> > A vector of strings, but since
> > whoever invented C's main was probably stoned out of his wits and
> > figured you should get _both_ a count and zero-terminated vector for
> > the arguments, people everywhere have since believed in this calling
> > convention. *Sigh*
>
> Maybe I have not understood your comment correctly, but it seems to
> me that C/C++ main fiunction does NOT get a zero terminated vector
> (a vector by definitin is a single dimension array)

Oh, it does. argv[argc] is always NULL.

> (the elements of argv are of course ZERO terminated strings).

This is true, but argv is NULL-terminated itself.

Erik Naggum

unread,
Jun 15, 2000, 3:00:00 AM6/15/00
to
* "Sandeep Koranne" <sandeep...@nospam.philips.com>

| Maybe I have not understood your comment correctly, but it seems to
| me that C/C++ main fiunction does NOT get a zero terminated vector
| (a vector by definitin is a single dimension array) while in C/C++

| the arguments to main are, int main (int argc, char* argv[]) : or a
| vector or vectors. So we DO need both the count of argument and the
| Vector of Vectors (the elements of argv are of course ZERO
| terminated strings).

C's conventional variable-sized vector representation is that

(aref <vector> (length <vector>)) => nil

to put it in more familiar terms, and this holds for all vectors of
all types, with varying representations of nil, of course, but they
are generally just warped versions of the integer 0, which is, of
course, just a convenient bit pattern of all zeros. A string has a
character conflatable with the integer 0, the NUL, and a vector of
any pointer type has a void * conflatable with the integer 0, the
NULL. So it may be more proper to say "NULL-terminated" than "zero-
terminated", although the semantics of NULL is strictly that of a
machine representation of zero. Let's not give C any more credit
than it deservers, is my line.

The count is in fact unnecessary, as witnessed by the fact that the
exec* family of system calls actually transmit only the variable-
sized vector of arguments and optionally environment bindings to the
kernel, which transmits them as such to the C runtime library, which
_then_ traverses the argument list and counts the arguments and
passes them to the main function with a count, for convenience.

I happen to think that this count-as-well-as-null-termination thing
is one of C's fundamentally braindamaged design decisions, having
caused an enormous amount of problems for programmers who don't get
the idea that it's a sometime convenience, not a pattern to learn by
rote and repeat uncritically, no matter how counter-productive.

Clayton Weaver

unread,
Jun 22, 2000, 3:00:00 AM6/22/00
to
You can't even depend on argc being accurate (for a merely practical
interpretation of "accurate") for programs run by shells that
consider anything between matching quotes as a single arg:

#>printargc -a -b
2

#>printargc "-a -b"
1

Regards,


--

Clayton Weaver
<mailto:cgw...@eskimo.com>
(Seattle)

Reply all
Reply to author
Forward
0 new messages