Improvements to "libcall" etc.

[Ron]

I have been discussing improvements to "libcall" with Bram, and I
would like to get feedback from you.

My specific issue is I want to interface with SQLite directly. I
wrote some new functions in my personal version of vim, but they are
not general enough to satisfy Bram, and he's right in this case :)

Basically, this is what is desired:

1. The ability to demand-load an arbitrary function from an arbitrary
DLL/SO, and deal with the parameters in an easy way.
2. The ability to write a 'callback' function in vimscript, also
dealing with parameters etc. in a reasonable way.

The 'callback' functionality is necessary for interfacing with lots of
DLLs - SQLite, expat and many others. Of course it is possible to
write a wrapper dll and use the existing 'libcall' interface -- and in
fact, I was doing that for years -- but the existing libcall loads and
unloads the DLL every time it's called, making stateful usage very
difficult.

I propose a demand-loading mechanism like:

call libfunc('dll_function', 'dll_name', "isi",
'VimNameForFunction')

(anyone have a better syntax in mind?)

So when 'VimNameForFunction' is called, it expects three paramters
(isi) to be passed to it, an integer a string and another integer. It
will then call the 'dll_function' in 'dll_name' (loading dll_name if
it is not already loaded). I use something similar in my Reva Forth.
This way, the dll is not loaded unless it is needed, and the binding
takes place dynamically. When vim quits, the libraries loaded would
be unloaded.

Parameter conversion should be done behind the scenes, and vim can
warn of improper parameter types or missing parameters.

Likewise, a 'callback' might be declared like:

callback MyCallback ( "ssi" )
let a = callback_param(0)
...
endcallback

I think the syntax is reasonable - what do you think?

Bigger question in my mind is how to implement callbacks in vim-
script. In Reva it's not a huge problem, as it generates native
code. But vimscript is completely interpreted, so I am guessing we
need to have a way to have a "thunk" which is passed to the C code in
the DLL, but which when called knows where to vector to.

It may be reasonable to have a limited number of callbacks available,
though that would be an unfortunate restriction. Any ideas on this?

Oh, one last thing. in the callback example mentioned above, typing
"MyCallback" should provide the address of the callback to be passed
to calling code. Vim code should never call a 'callback' directly, of
course.

[George V. Reilly]

2008/7/6 Ron <r...@ronware.org>:

I have been discussing improvements to "libcall" with Bram, and I

would like to get feedback from you.

My specific issue is I want to interface with SQLite directly.  I

wrote some new functions in my personal version of vim, but they are

not general enough to satisfy Bram, and he's right in this case :)

 

Is there a reason why you can't interface with SQLite via Vim's Python or Perl interfaces?

 

Basically, this is what is desired:

1. The ability to demand-load an arbitrary function from an arbitrary

DLL/SO, and deal with the parameters in an easy way.

2. The ability to write a 'callback' function in vimscript, also

dealing with parameters etc. in a reasonable way.

The 'callback' functionality is necessary for interfacing with lots of

DLLs - SQLite, expat and many others.  Of course it is possible to

write a wrapper dll and use the existing 'libcall' interface -- and in

fact, I was doing that for years -- but the existing libcall loads and

unloads the DLL every time it's called, making stateful usage very

difficult.

I propose a demand-loading mechanism like:

    call libfunc('dll_function', 'dll_name', "isi",

'VimNameForFunction')

 

Under Win32, there are multiple calling conventions, __stdcall, __cdecl and so on: http://www.devarticles.com/c/a/Cplusplus/DLL-Conventions-Issues-and-Solutions-Part-II/

You can corrupt the process if you call a function using the wrong convention.

 

(anyone have a better syntax in mind?)

So when 'VimNameForFunction' is called, it expects three paramters

(isi) to be passed to it, an integer a string and another integer. It

will then call the 'dll_function' in 'dll_name' (loading dll_name if

it is not already loaded).  I use something similar in my Reva Forth.

This way, the dll is not loaded unless it is needed, and the binding

takes place dynamically.  When vim quits, the libraries loaded would

be unloaded.

Parameter conversion should be done behind the scenes, and vim can

warn of improper parameter types or missing parameters.

 

Do you propose to support anything more complicated than strings and ints? Even strings are potentially complicated: ascii, utf-8, utf-16, utf-32, dbcs, BSTRs, etc.

 

Likewise, a 'callback' might be declared like:

   callback MyCallback ( "ssi" )

        let a = callback_param(0)

        ...

   endcallback

I think the syntax is reasonable - what do you think?

 

How about

  callback "ssi" MyCallback(firstName, lastName, age)

 

What does a callback return? Is there any notion of out-parameters or ref-parameters?

 

Bigger question in my mind is how to implement callbacks in vim-

script.  In Reva it's not a huge problem, as it generates native

code.  But vimscript is completely interpreted, so I am guessing we

need to have a way to have a "thunk" which is passed to the C code in

the DLL, but which when called knows where to vector to.

 

It's certainly how I'd do it in Windows. I don't know enough about SO's under Linux/Mac.

 

It may be reasonable to have a limited number of callbacks available,

though that would be an unfortunate restriction.  Any ideas on this?

Oh, one last thing.  in the callback example mentioned above, typing

"MyCallback" should provide the address of the callback to be passed

to calling code.  Vim code should never call a 'callback' directly, of

course.

 

--

/George V. Reilly geo...@reilly.org

http://www.georgevreilly.com/blog http://blogs.cozi.com/tech

[Yasuhiro MATSUMOTO]

Maybe, you can stock the instance of library with following code.

function! OpenLib(libname)
" return result of LoadLibrary() in kernel32.dll
return libcall("libcalllib.dll", "OpenLib", a:libname)
endfunction

function! CloseLib(handle)
" call FreeLibrary()
call libcall("libcalllib.dll", "CloseLib", a:handle)
endfunction

function! CallLib(handle, funcname, callargs)
" parse arguments (ex: json library?)
" get address of function by GetProcAddress
" call function.
return libcall("libcalllib.dll", "CallLib", a:handle . "," .
a:funcname . "," + string(a:callargs))
endfunction

--
- Yasuhiro Matsumoto