FFI c-struct convenience classes? OO, not with-
Reini Urban
which are ACLW FFI, LWW FLI, CormanLISP FFI (my favorite), CLISP (this
is also nice) and also some scheme dialects. (found nothing better
there)
My problem is that LISP should make coding easier, using c-types and
c-structs mostly for sideffects makes lifes much harder. okay that's the
deal with a FFI. we have to deal with the dirty c or assembler details.
there are some convenience macros to simplify the necessary translation
for every lisp type to the corresponding c-type.
A c-type is easy to define, it needs only the size in bytes, a signed or
unsigned flag, and if the numeric value in the return register (let's
say eax on x86) is direct (a number) or indirect (a pointer to
something).
for a function there's more: pascal vs c linkage, if the caller or the
callee rewinds the stack. and some name mangling (ascii, unicode) and
string conversion issues (ascii, unicode), FPU and more dirty machine
details.
my problem is that no FFI package so far abstracts c-structs enough.
i'll have to use masses of c-code (AutoCAD, WinUI and COM), most of it
can be translated automatically to the implementation dependent c-type
syntax (they are all quite similar thanksfully), but when using these c
functions the problems arise.
e.g. the WinAPI uses extensively structs by side effects, you have to
malloc a struct, convert your lisp values to c-types, set the
appropriate c-slots, call the function with this struct, and afterwards
the reverse:
access the c-slots, convert them back to lisp types and free the struct
and all indirect slots (if necessary).
this is a pain, i could have stayed c programmer for doing this stupid
work. and i cannot burden plain users with this.
why not create a c-type class, and a c-struct subclass, with all the
methods to automatically initialize, get and set the c-struct slots.
any user struct just has to be subclassed from the basic c-struct class,
with nice defaults, most of the typical WinAPI slots are defaults anyway
and i would safe thousands of lines.
i only know of the corman lisp with-c-struct macro and another
corman lisp with-cref-slots macro, which is better but still too verbose
for my taste.
any other self-respecting language has "better" FFI's, really.
better = easier to write for the user.
some examples (here using corman lisp's syntax, but all are more or less
the same)
c-stuff to call a html helpwindow: (two massive structs, only one shown)
HH_AKLINK link;
link.cbStruct = sizeof(HH_AKLINK) ;
link.fReserved = FALSE ;
link.pszKeywords = "open" ;
link.pszUrl = NULL ;
link.pszMsgText = NULL ;
link.pszMsgTitle = NULL ;
link.pszWindow = NULL ;
link.fIndexOnFail = TRUE ;
HtmlHelp(GetDesktopWindow(),
"c:\\myhelp.chm",
HH_KEYWORD_LOOKUP,
(DWORD)&link);
;;; shortened and simplified lisp versions
;;; also incorrect code: in reality the struct is passed as pointer.
;;; fli:define-c-typedef / c-type:defwinstruct was omitted for brevity,
;;; just the necessary user function calls.
;;; and i don't need the getters to this struct, or only the setters.
;;; this is very simple.
;;; BTW: Corman lisp has a nice automatic c-type parser for such
;;; structs. but still awful to use.
;;; old version without uwe's with-cref-slots
(defun HH-KEYWORD-LOOKUP (file keyword)
(let ((hwnd (GetDesktopWindow))
(cfile (lisp-string-to-c-string file))
(ckey (lisp-string-to-c-string keyword))
(link (malloc (sizeof HH_AKLINK)))
(hhwin (malloc (sizeof HH_WINTYPE))))
(setf (cref HH_AKLINK link cbStruct) (sizeof HH_AKLINK))
(setf (cref HH_AKLINK link fReserved) nil)
(setf (cref HH_AKLINK link pszKeywords) ckey)
(setf (cref HH_AKLINK link pszUrl) cl::C_NULL)
(setf (cref HH_AKLINK link pszMsgTxt) cl::C_NULL)
(setf (cref HH_AKLINK link pszMsgTitle) cl::C_NULL)
(setf (cref HH_AKLINK link pszWindow) cl::C_NULL)
(setf (cref HH_AKLINK link fIndexOnFail) T)
(HtmlHelp hwnd cfile HH_KEYWORD_LOOKUP link)
(free link)(free hhwin)))
;;; this is the syntax with using with-cref-slots, a bit shorter now
(defun HH-KEYWORD-LOOKUP (file keyword)
(let ((hwnd (GetDesktopWindow))
(cfile (lisp-string-to-c-string file))
(ckey (lisp-string-to-c-string keyword))
(link (malloc (sizeof HH_AKLINK)))
(hhwin (malloc (sizeof HH_WINTYPE))))
(with-cref-slots () (link HH_AKLINK)
(setf cbStruct (sizeof HH_AKLINK))
(setf fReserved nil)
(setf pszKeywords ckey)
(setf pszUrl cl::C_NULL) ; pointer to NULL string
(setf pszMsgTxt cl::C_NULL)
(setf pszMsgTitle cl::C_NULL)
(setf pszWindow cl::C_NULL)
(setf fIndexOnFail T))
(HtmlHelp hwnd cfile HH_KEYWORD_LOOKUP link)
(free link)(free hhwin)))
other FFI's also use some automatic allocator/deallocator or convenience
macros like LWW's with-dynamic-foreign-objects.
;;; and this is now the syntax I want to have...
;;; FFI convenience classes, setting and getting is awful without.
;;; We should provide a class or metaclass (?) to provide the
;;; c-type <-> lisp-type converters and accessors (setf'able)
;;; Most the c-structs are used for side-effects only, so we only
;;; have to initialize them (malloc, lisp->c, setf cref) and then
;;; at the end read the values provided by the winapi call
;;; (cref, c->lisp)
;;; i'm a total beginner with CLOS so my suggested code is very likely
;;; incorrect or stupid.
(defclass c-type ()
;; a class for some convenient initializer, reader and converter methods
((type)))
(defgeneric set-c-type (c-type lisp-value)) ;=> sideeffect only
(defgeneric get-c-type (c-type)) ;=> lisp value
(defgeneric c-type-of (c-type) ;=> c-type such as
; (:CHAR *) or let's say (:POINTER :CHAR)
(defclass c-struct (c-type)
;; a class for some convenient initializer, reader and converter
;; methods for structs (a couple of c-types)
(usage :initform :byref :allocation :class)
;; which can be :byval or :byref,
;; either copy or pass a pointer (default)
)
(defgeneric set-c-type (c-struct slot lisp-value))
(defgeneric get-c-type (c-struct slot))
(defmethod set-c-type (c-struct slot lisp-value)
(setf (cref (c-type-of c-struct) c-struct slot)
(lisp->c (c-type-of slot) lisp-value)) ;???
....
and now in the works:
;;; definition
;;; two shared slots (same for all) and the rest is instance specific
(defclass hh-aklink (c-struct)
((cbStruct :initform (sizeof HH_AKLINK) :allocation :class)
(fReserved :initform nil :allocation :class)
(pszKeywords :initform cl::C_NULL)
(pszUrl :initform cl::C_NULL)
(pszMsgTxt :initform cl::C_NULL)
(pszMsgTitle :initform cl::C_NULL)
(pszWindow :initform cl::C_NULL)
(fIndexOnFail :initform T)))
;;; usage
(defun HH-KEYWORD-LOOKUP (file keyword)
(let ((hwnd (GetDesktopWindow))
(hhwin (make-instance 'hh-wintype))
(link (make-instance 'hh-aklink :pszKeywords "search"))
(cfile (lisp-string-to-c-string file)))
(HtmlHelp hwnd cfile HH_KEYWORD_LOOKUP link)))
before i want to nail this down, isn't there anything similar or better
out there? I found nothing. even not in commercial compilers.
there's a FFIGEN package for chez scheme which is inferior to the corman
lisp c header parser, ILU has similar converters, but i saw no useful
abstraction.
this would be nice to have. any implementation, platform and machine
dependent stuff could be hidden inside the special methods for each
c-type.
unions (another nightmare) could also be better defined with defclass.
---
Reini Urban
http://xarch.tu-graz.ac.at/autocad/news/faq/autolisp.html
Wade Humeniuk
Reini Urban <rur...@xarch.tu-graz.ac.at> wrote in message
news:374f093c.11277636@judy...
> I'm terrible dissatisfied with the FFI's I know of,
> which are ACLW FFI, LWW FLI, CormanLISP FFI (my favorite), CLISP (this
> is also nice) and also some scheme dialects. (found nothing better
> there)
>
What I would really like is Lisp to read the C/Pascal/Whatever include
file directly and produce lisp wrappers automatically. You would have to
specify some parameters like int sizes, dynamic library location, etc, but
the
result would be more declarative.
e.g.
(include-c-header-file "windows.h" :init-size 4 :dynamic-libraries
'("KERNAL32.DLL")
:convert_ '-)
and that's it. (Perhaps its wishful thinking, but hasn't this been done?)
Other problems: C Macros....
Wade
Duane Rettig
> Reini Urban <rur...@xarch.tu-graz.ac.at> wrote in message
> news:374f093c.11277636@judy...
> > I'm terrible dissatisfied with the FFI's I know of,
> > which are ACLW FFI, LWW FLI, CormanLISP FFI (my favorite), CLISP (this
> > is also nice) and also some scheme dialects. (found nothing better
> > there)
> >
>
> What I would really like is Lisp to read the C/Pascal/Whatever include
> file directly and produce lisp wrappers automatically. You would have to
> specify some parameters like int sizes, dynamic library location, etc, but
> the
> result would be more declarative.
>
> e.g.
>
> (include-c-header-file "windows.h" :init-size 4 :dynamic-libraries
> '("KERNAL32.DLL")
> :convert_ '-)
>
> and that's it. (Perhaps its wishful thinking, but hasn't this been done?)
Allegro Cl has the cbind facility for sparc, sgi, and windows. For
an overview, you can look at
ftp://ftp.franz.com/pub/cbind/5.0.beta/readme.txt
which still exists. The most current versions are on each CD, in the
cbind directory at the top of the directory structure.
--
Duane Rettig Franz Inc. http://www.franz.com/ (www)
1995 University Ave Suite 275 Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253 du...@Franz.COM (internet)
Erik Naggum
| I'm terrible dissatisfied with the FFI's I know of, which are ACLW FFI,
| LWW FLI, CormanLISP FFI (my favorite), CLISP (this is also nice) and also
| some scheme dialects. (found nothing better there)
I have found Franz Inc's new FFI very convenient. it is in Allegro CL 5
for both Windows and Unix. ACL for Windows is dead, but compatibility
support exists for old applications (and programmers :).
the only problem I have ahd with it is that since Unix is so damn focused
on using small integers for everything, it's been a hassle to define the
symbols that Unix functions expect. and more often than not, the Unix
feature is so amazingly braindamaged that providing the service is a lot
more work than the FFI, anyway. take sockets. sockets suck. making
them not suck is hard work. (thank goodness I'm not working in Windows.)
| my problem is that no FFI package so far abstracts c-structs enough.
take a look at Allegro CL 5, then.
| i'll have to use masses of c-code (AutoCAD, WinUI and COM), most of it
| can be translated automatically to the implementation dependent c-type
| syntax (they are all quite similar thanksfully), but when using these c
| functions the problems arise.
| e.g. the WinAPI uses extensively structs by side effects, you have to
| malloc a struct, convert your lisp values to c-types, set the
| appropriate c-slots, call the function with this struct, and afterwards
| the reverse:
| access the c-slots, convert them back to lisp types and free the struct
| and all indirect slots (if necessary).
| this is a pain, i could have stayed c programmer for doing this stupid
| work. and i cannot burden plain users with this.
why free the structure? use a closure for each type of call, allocate on
first call and reuse the struct once allocated. if you want reentrant
code, cons up a list of them, instead. it's a lot better for such
structs to hang around than to be freed and allocated all over again.
| why not create a c-type class, and a c-struct subclass, with all the
| methods to automatically initialize, get and set the c-struct slots.
this has been done. take a look at Allegro CL 5.
it's a shame people don't research the available techologies better
before going into serious problem-solving mode. on the other hand, it's
a shame that so many people are so aggressively opposed to information
from commercial actors in newsgroups that they think anything that
doesn't come from individuals is "marketing". however, since this
doesn't stop Harlequin from announcing everything under the sun, it
appears that Franz Inc's policy of not posting information about their
new developments may actually be a disservice to the community.
#:Erik
--
@1999-07-22T00:37:33Z -- pi billion seconds since the turn of the century
Marc Battyani
Newsgroups: comp.lang.lisp
Sent: Saturday, May 29, 1999 11:40 AM
Subject: Re: FFI c-struct convenience classes? OO, not with-
> why free the structure? use a closure for each type of call, allocate
on
> first call and reuse the struct once allocated. if you want reentrant
> code, cons up a list of them, instead. it's a lot better for such
> structs to hang around than to be freed and allocated all over again.
> this has been done. take a look at Allegro CL 5.
I like this idea. Do you mean you have a list of available structs and
you pop one on entry and push it back on exit? In that case you have
to protect the access to the stack, or am I missing a point here?
> it's a shame people don't research the available techologies better
> before going into serious problem-solving mode. on the other hand, it's
> a shame that so many people are so aggressively opposed to information
> from commercial actors in newsgroups that they think anything that
> doesn't come from individuals is "marketing". however, since this
> doesn't stop Harlequin from announcing everything under the sun, it
> appears that Franz Inc's policy of not posting information about their
> new developments may actually be a disservice to the community.
May be we should vote for a comp.lang.lisp.announce news group.
Marc Battyani
Erik Naggum
| I like this idea. Do you mean you have a list of available structs and
| you pop one on entry and push it back on exit? In that case you have
| to protect the access to the stack, or am I missing a point here?
there are many ways of doing this. one is using resources, another is
blocking interrupts and scheduling around accesses to the list, and yet
another is using process locks on the list. this is not a big issue.
| May be we should vote for a comp.lang.lisp.announce news group.
good idea, but it's a little hard in realizing right away. I think we
should see if people are annoyed here first. there's a lot of stuff out
there that I think is highly relevant to the Lisp community that is held
back because of aggressive anti-commercial attitudes that make it very
hard to write technical stuff that _isn't_ marketing, because it is not
viewed as having any commercial value, only costs, and thus are
restricted to existing or realistically prospective customers. sigh.
Craig Brozefsky
> good idea, but it's a little hard in realizing right away. I think we
> should see if people are annoyed here first. there's a lot of stuff out
> there that I think is highly relevant to the Lisp community that is held
> back because of aggressive anti-commercial attitudes that make it very
> hard to write technical stuff that _isn't_ marketing, because it is not
> viewed as having any commercial value, only costs, and thus are
> restricted to existing or realistically prospective customers. sigh.
We don't really have a traffic issue here, so I certainly would not
mind such announcements, particularly if they are tasteful and
informative. Considering the level of professionalism at most Lisp
shops, that should be second nature of course.
--
Craig Brozefsky <cr...@red-bean.com>
Less matter, more form! - Bruno Schulz
ignazz, I am truly korrupted by yore sinful tzourceware. -jb
The Osmonds! You are all Osmonds!! Throwing up on a freeway at dawn!!!
Reini Urban
> I have found Franz Inc's new FFI very convenient. it is in Allegro CL 5
> for both Windows and Unix. ACL for Windows is dead, but compatibility
> support exists for old applications (and programmers :).
>
thanks a lot!
agreed, finally found my trial acl5 linux cd and got to install it
(-o exec on mount was the trick i didn't got the first times i tried).
this is now 50% on the way to make myself happy.
i don't like acl5 cbind that much, IMHO corman lisp's c parser is easier
to use, but the acl cbind (and the underlying acl5 ffi) supports much
more functionality (and is therefore more complicated) and seems to be
more stable. hmm...
>| access the c-slots, convert them back to lisp types and free the struct
>| and all indirect slots (if necessary).
>| this is a pain, i could have stayed c programmer for doing this stupid
>| work. and i cannot burden plain users with this.
>
> why free the structure? use a closure for each type of call, allocate on
> first call and reuse the struct once allocated. if you want reentrant
> code, cons up a list of them, instead. it's a lot better for such
> structs to hang around than to be freed and allocated all over again.
of course, i hate free'ing them for sure. but i wanted to avoid closures
(for dynamic lisp compatibility, i implemented it also in plain
AutoLISP), better have some kind of system support (say gc) for
malloc'ed areas. such as acl5 has it with the :allocation keyword for
make-array ("static arrays") or allocate-fobject esp.
:foreign-static-gc.
or something like with-malloced-ftype
now this is really useful. does it use mark-and-sweep or reference
counting or stack allocation or just a simple private foreign heap (not
movable space)?
long lisp_value(index) tells me that it is probably just a private
indexed heap for callbacks, static arrays or such. this is probably the
only solution to reach lisp from c, for ff:defun-foreign-callable.
at least this is the same solution I used for my own FFI.
it is probably more efficient than mark-and-sweep or reference counting
for ftypes.
>| why not create a c-type class, and a c-struct subclass, with all the
>| methods to automatically initialize, get and set the c-struct slots.
>
> this has been done. take a look at Allegro CL 5.
> it's a shame people don't research the available techologies better
> before going into serious problem-solving mode.
i was not in serious problem-solving mode :)
it was just a short idea and thought why the hell has nobody done that
so far? common graphics must be heavily based on WinAPI calls with a
hell lot of cstructs.
but what i read from the acl5 docs I found no better cstruct abstraction
than just some typical cref convenience macros, similar to with-slots,
the convenient automatic dereferencing (returning values instead of
addresses), the cstruct is now a gc-supported static foreign array
containing a pointer to its lisp object, but there's not the
functionality I suggested.
just some nice convenience macros and a basic gc support, which i
already have in corman lisp (plus a better c-parser).
no cstruct class with simplifying readers, writers and allocators
(letting free aside). i guess i have to write that by my own. no major
problem though. at least now i know what is the current state of art.
> on the other hand, it's
> a shame that so many people are so aggressively opposed to information
> from commercial actors in newsgroups that they think anything that
> doesn't come from individuals is "marketing". however, since this
> doesn't stop Harlequin from announcing everything under the sun, it
> appears that Franz Inc's policy of not posting information about their
> new developments may actually be a disservice to the community.
this is true.
i only bought aclw2 for a lot of money, got the aclw3 trial from the net
for free, and the subsequent updates where too expensive for me, new
technical information from franz for their new releases was hard to get,
in comparison to others. but i was probably nexus who didn't react to my
trial cd orders, not to blame franz.
i finally grabbed the acl5 linux trial cd at the berkeley conference,
otherwise i would still be waiting for it.
duane is sometimes responding here, but much too seldom. never heard
anything from fritz or stephen though.
--
Reini
Martin Cracauer
>my problem is that no FFI package so far abstracts c-structs enough.
>i'll have to use masses of c-code (AutoCAD, WinUI and COM), most of it
>can be translated automatically to the implementation dependent c-type
>syntax (they are all quite similar thanksfully), but when using these c
>functions the problems arise.
>e.g. the WinAPI uses extensively structs by side effects, you have to
>malloc a struct, convert your lisp values to c-types, set the
>appropriate c-slots, call the function with this struct, and afterwards
>the reverse:
>access the c-slots, convert them back to lisp types and free the struct
>and all indirect slots (if necessary).
>this is a pain, i could have stayed c programmer for doing this stupid
>work. and i cannot burden plain users with this.
Have a look at the CMUCL alien interface. It allows you to use the C
structs directly in your Lisp code.
http://www.mindspring.com/~rtoy/software/cmu-user/node221.html
Martin
--
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Martin Cracauer <crac...@bik-gmbh.de> http://www.bik-gmbh.de/~cracauer/
"Where do you want to do today?" Hard to tell running your calendar
program on a junk operating system, eh?
Duane Rettig
> duane is sometimes responding here, but much too seldom.
Hmm, this is interesting. How often would you expect me to respond on
this newsgroup?
> never heard
> anything from fritz or stephen though.
Do they know you are expecting a response from them?
I read this newsgroup regularly, and I try to contribute to it when
I feel I have a contribution to make, usually having to do with
technical facts that might not otherwise be known. I take complaints
about our Company and product seriously, and sometimes route them to
others at Franz Inc (whether they read the newsgroup or not) but I
try never to answer support-related questions on c.l.l; we have a fairly
good bugs/problems/questions handling system and I'd rather people
took advantage of that instead. I did a quick check in our support
database, and I didn't see your name, so I assume that you hadn't
used our support system (you don't even have to be a supported
customer in order to ask a question or report a bug, although then
the level of response may vary).
With all this in mind, perhaps you can expand on what you mean by
"too seldom".
Reini Urban
i overread all the good stuff. foreign-pointer is really a CLOS class
and it is really VERY convenient being able to specialize on this class.
i found it out the hard way after having written my own ctype class
abstraction.
with acl5 I can use any clos method combination and with lww the
:wrapper ctype descriptor or define-foreign-converter which is pretty
much the same functionality. both much better than with corman lisp.
--
Reini