wrapping 32/64 bit API call?

[Oliver Wang]

hi -

I'm a newbie on cffi, and have a question on how to properly wrap and transform an api.

I think the stat() call is probably the closest general example: say "stat_t" is defined for 32-bit; stat64_t is defined for 64-bit.

If an application defines "#define INODES_64bit", then it will use stat64() version internally, yet the public interface should remain as stat(). 

I don't want to expose both stat_t and stat64_t to the outside, how can I accomplish that in cffi?

cffi.cdef("""

typedef struct {

    inode_t ino;

    ...

} stat_t;

typedef struct {

    inode64_t ino;

    ...

} stat64_t;

int stat(const char *path, struct stat_t *buf);

int stat64(const char *path, struct stat64_t buf);

""")

thanks

Oliver

 

[Armin Rigo]

Hi Oliver,

On 8 November 2016 at 03:30, Oliver Wang <pyth...@gmail.com> wrote:
> I don't want to expose both stat_t and stat64_t to the outside, how can I
> accomplish that in cffi?

It depends if you're using the ABI or the API mode.

In the recommended API mode, you get the same result as C code: you
only need to access inode_t, stat_t, and stat(), without worrying that
sometimes they are #defined to something else.

In ABI mode, there is no choice but the one you're using. If the
selected C function is called 'stat64', then in ABI mode you have to
expose it as 'lib.stat64()'. You can't rename it directly, but you
can do something outside, e.g.

if hasattr(lib, 'stat64'): stat = lib.stat64
else: stat = lib.stat


A bientôt,

Armin.

[Oliver Wang]

thanks for the reply.

I am using API mode. The library itself provides both 32 and 64-bit version, however, user (C app) can use "#define INODES_64bit" to default to one mapping (64-bit) or another. I am puzzled how can I mimick this behavior?

if I stay with 32-bit definition in cdef(), that is, by only providing stat_t and stat() declaration, and if I want to use 64-bit as default. I put 

#define INODES_64bit"

into ffibuilder.set_source(), then cffi complains about all the missing structures related to stat64_t and stat64(), which seems forcing me to provide both 32-bit and 64-bit definitions in cdef() ... 

I hope what I am describing makes sense, be glad to clarify if needed.

Thanks very much

Oliver

[Armin Rigo]

Hi Oliver,

On 8 November 2016 at 17:21, Oliver Wang <pyth...@gmail.com> wrote:
> if I stay with 32-bit definition in cdef(), that is, by only providing
> stat_t and stat() declaration, and if I want to use 64-bit as default. I put
>
> #define INODES_64bit"
>
> into ffibuilder.set_source(), then cffi complains about all the missing
> structures related to stat64_t and stat64(), which seems forcing me to
> provide both 32-bit and 64-bit definitions in cdef() ...

Sorry, I don't understand precisely. Can you give me files that mimic
what you do and the content of the .h file (or a few lines of it)?
Enough to reproduce the problem. Thanks!


A bientôt,

Armin.

[Oliver Wang]

hi Armin,

To the best of my ability, these the files that mimic what I am trying to do:

/* attr.h , both 32-bit and 64-bit version defined */


typedef int ino_t;
typedef long long ino64_t;


typedef struct attr {
    ino_t ino;
} attr_t;


typedef struct attr64 {
    ino64_t ino;
} attr64_t;


int read_attr(const char *path, struct attr_t ** buf);
int read_attr64(const char *path, struct attr64_t **buf);

Here is the client app that uses this interface, note that client only needs to deal with read_attr(), not both read_attr() and read_attr64()

/* client.c */
#include "attr.h"

int main() {
    attr_t *buf;
    char *path = "/home/test";
    read_attr(path, &buf);
}

from client point of view, if compile:

gcc client.c  -o client

32-bit version is generated (call read_attr() )

if I compile with:

gcc -DINODES_64bit client.c  -o client

then, it will use read_attr64() internally.

The guess is the library itself did some internal mapping to hide the difference, which I don't have access to the source. I am trying to mimic that in the Python.

Does it help to clarify?

thanks!

Oliver

[Armin Rigo]

Hi Oliver,

On 10 November 2016 at 14:38, Oliver Wang <pyth...@gmail.com> wrote:
> The guess is the library itself did some internal mapping to hide the
> difference, which I don't have access to the source. I am trying to mimic
> that in the Python.
>
> Does it help to clarify?

No completely. If you compile with ``-DINODES_64bit`` then there must
be ``#ifdef INODES_64bit`` in the header file, which you should be
able to see without looking in the closed source code for the
library---that's the only way such an option can have an effect. Is
it something like this?

#ifdef INODES_64bit
# define attr_t attr64_t
# define read_attr read_attr64
#endif

If I'm wrong, then I fear we need to understand exactly how ``gcc
-DINODES_64bit client.c -o client`` can still give a result that
differs from ``gcc client.c -o client``.


A bientôt,

Armin.

[Oliver Wang]

hi Armin,

A slight correction here (my bad) is, client.c can enable 64-bit behavior by (instead of compile time define):

/* client.c */
#include "attr.h"

#define INODES_64bit

int main() {
    attr_t *buf;
    char *path = "/home/test";
    read_attr(path, &buf);
}

So yes, I think you are right that there must be someplace in the library that did exactly what you wrote:

    #ifdef INODES_64bit 
    # define attr_t attr64_t 
    # define read_attr read_attr64 
    #endif

What I did was to replicate this behavior by passing above #ifdef #endif to ffibuilder.set_source().

However, how do I give such a "#define INODES_64bit" knob to Python users? I can't seem to expose it through cdef() by:

   

   #define INODES_64bit ...

There are compile errors

_lib.c:1601:12: error: ‘INODES_64bit’ undeclared (first use in this function)

   int n = (INODES_64bit) <= 0;

Thanks

Oliver

[Armin Rigo]

Hi Oliver,

On 11 November 2016 at 13:14, Oliver Wang <pyth...@gmail.com> wrote:
> /* client.c */
> #include "attr.h"
> #define INODES_64bit

I think the "#define" must be before the "#include", otherwise it
can't have an effect.

> So yes, I think you are right that there must be someplace in the library
> that did exactly what you wrote:
>
> #ifdef INODES_64bit
> # define attr_t attr64_t
> # define read_attr read_attr64
> #endif

No, I meant these lines must be inside "attr.h". If they were inside
some .c source file, they wouldn't have the effect that you want.
If you want to dig more, I'd recommend that you read more about
#define and #ifdef in C first. But:

> However, how do I give such a "#define INODES_64bit" knob to Python users?

This is a compilation switch. If you call set_source() and compile()
only once, then you can only get one of the two behaviors, compiled
inside the produced .so. This is because this .so is produced from
auto-generated C code, which will itself either use the #define or
not. There is no reasonable way to have both behaviors inside one
.so.

You could build two .so's, but what I think you should try to do
instead: just forget about the whole #define issue and expose both
read_attr and read_attr64 and both attr_t and attr64_t. Then, write
regular Python functions that will use one variant or the other,
depending on some Python switch. Something like:

def read_attr(args):
if use_inodes_64:
p = ffi.new("attr64_t")
lib.read_attr64(p, args)
return p.ino
else:
p = ffi.new("attr_t")
lib.read_attr(p, args)
return p.ino


A bientôt,

Armin.

[Oliver Wang]

hi Armin,

You are right, the #ifdef ... #endif is indeed in the very end of this big header file, I missed it when I first read through.

I think what you suggested make sense, make the switch in python - I think I got it work, It is a great learning experience. 

Your help and patience are much appreciated.

Oliver