plans to support __stdcall?

[Mark Harviston]

I've been working on pylib7zip off and on for awhile now, slowly trying to figure out why only 64bit python works, and 32bit python would get an out of memory exception.

well, since 7zip is a COMish library it uses __stdcall calling conventions, so the stack wasn't getting cleaned up on callbacks.

If cffi supported __stdcall, I could get away with dlopen and cdef only, with having to use indirection, I need to do rather complicated indirection (for the windows version only). This may be a deal breaker in terms of cffi over ctypes since ctypes does support __stdcall.

Definition of IInStream if __stdcall was supported

typedef struct {
    HRESULT  (__stdcall *QueryInterface) (void*, GUID*, void**);
    uint32_t (__stdcall *AddRef)(void*);
    uint32_t (__stdcall *Release)(void*);
    HRESULT  (__stdcall *Read)(void* self, uint8_t *data, uint32_t size, uint32_t *processedSize);
    HRESULT  (__stdcall *Seek)(void* self, int64_t offset, uint32_t seekOrigin, uint64_t *newPosition);
} _IInStream_vtable;

typedef struct{
    _IInStream_vtable* vtable;
} IInStream;


Definition w/ in and out indirection (in practice each interface would only need one or the other, not both) (there's probably errors here since it hasn't been compiled much less run):

cdefs:

typedef struct {
    HRESULT  (*QueryInterface) (void*, GUID*, void**);
    uint32_t (*AddRef)(void*);
    uint32_t (*Release)(void*);
    HRESULT  (*Read)(void* self, uint8_t *data, uint32_t size, uint32_t *processedSize);
    HRESULT  (*Seek)(void* self, int64_t offset, uint32_t seekOrigin, uint64_t *newPosition);
} _IInStream_cb_vtable;

typedef void IInStream;

typedef struct {
    void* vtable;
    _IInStream_cb_vtable* cb_vtable;
} IInStream_impl;

HRESULT  IInStream_QueryInterface(IInStream* self, GUID* iid, void** impl);
uint32_t IInStream_AddRef(void*);
uint32_t IInStream_Release(void*);
HRESULT  IInStream_Read(void* self, uint8_t *data, uint32_t size, uint32_t *processedSize);
HRESULT  IInStream_Seek(void* self, int64_t offset, uint32_t seekOrigin, uint64_t *newPosition);

Verify:

typedef struct {
    HRESULT  (__stdcall *QueryInterface) (void*, GUID*, void**);
    uint32_t (__stdcall *AddRef)(void*);
    uint32_t (__stdcall *Release)(void*);
    HRESULT  (__stdcall *Read)(void* self, uint8_t *data, uint32_t size, uint32_t *processedSize);
    HRESULT  (__stdcall *Seek)(void* self, int64_t offset, uint32_t seekOrigin, uint64_t *newPosition);
} _IInStream_vtable;

typedef struct {
    HRESULT  (__cdecl *QueryInterface) (void*, GUID*, void**);
    uint32_t (__cdecl *AddRef)(void*);
    uint32_t (__cdecl *Release)(void*);
    HRESULT  (__cdecl *Read)(void* self, uint8_t *data, uint32_t size, uint32_t *processedSize);
    HRESULT  (__cdecl *Seek)(void* self, int64_t offset, uint32_t seekOrigin, uint64_t *newPosition);
} _IInStream_cb_vtable;

typedef struct {
    _IInStream_vtable* vtable;
    _IInStream_cb_vtable* cb_vtable;
} IInStream_impl;

HRESULT  __cdecl IInStream_QueryInterface(IInStream* self, GUID* iid, void** impl){
    return self->vtable->QueryInterface(self, iid, impl);
}

HRESULT __stdcall IInStream_QueryInterface_cb (IInStream_impl* self, GUID* iid, void** impl){
    return self->cb_vtable->QueryInterface(self, iid, impl);
}
uint32_t __stdcall IInStream_AddRef_cb(IInStream_impl*){ /* snip */
uint32_t __stdcall IInStream_Release_cb(IInStream_impl*){ /* snip */
HRESULT  __stdcall IInStream_Read_cb(IInStream_impl* self, uint8_t *data, uint32_t size, uint32_t *processedSize){
    /* snip */
}
HRESULT __stdcall IInStream_Seek_cb(IInStream_impl* self, int64_t offset, uint32_t seekOrigin, uint64_t *newPosition){
    /* snip*/
}

HRESULT  IInStream_Construct_cb(IInStream_impl* self){
    /* python/cffi will pass in pointer created with ffi.new()*/
    self.vtable.QueryInterface = &IInStream_QueryInterface;
    /* snip */
}

I can implement most of that with templates and stuff, but it's annoying, and will be totally unnecessary for the non-windows version (though it won't hurt if I do #define __stdcall).

[Armin Rigo]

Hi Mark,

On 16 March 2014 07:52, Mark Harviston <mark.ha...@gmail.com> wrote:
> If cffi supported __stdcall, I could get away with dlopen and cdef only,
> with having to use indirection, I need to do rather complicated indirection
> (for the windows version only). This may be a deal breaker in terms of cffi
> over ctypes since ctypes does support __stdcall.

The reason I didn't do it so far is that __stdcall is not a standard C
keyword supported by pycparser. However we recently implemented the
"packed" version of structs using a workaround; I guess we can reuse
the idea. It would mean that you'd use something like:

ffi.cdef(".......", callback_calling_conv="stdcall")

which would set the calling convension of any callback present in this
cdef(). (You can use several cdefs if you need various calling
convensions.)

Would that help?


A bientôt,

Armin.

[Mark Harviston]

On Sunday, March 16, 2014 11:40:35 AM UTC-7, Armin Rigo wrote:

    ffi.cdef(".......", callback_calling_conv="stdcall")

That would only work on the callbacks though? I suppose that's the major pain point, but for function calls this would be nice as well.

[Armin Rigo]

Hi Mark,

On 16 March 2014 21:42, Mark Harviston <mark.ha...@gmail.com> wrote:
> That would only work on the callbacks though? I suppose that's the major
> pain point, but for function calls this would be nice as well.

For regular function calls, you don't need to specify the calling
convention at all. The two most common possibilities, cdecl vs
stdcall, are compatible enough so that a call can be done supporting
either. This is done with just a few extra lines of assembler in the
file c/libffi_msvc/win32.c-or-win64.asm, which needs custom assembler
anyway in the first place. Regular function calls will thus continue
to support either of the two calling conventions transparently. The
issue is only about callbacks.


A bientôt,

Armin.