Help with unions inside structs?

[Michael Karlesky]

Thanks for such a great tool. CFFI really is a bit of magic.

I've been working on wrapping a C-based device communication SDK for a short while. I've figured out everything I need from the CFFI documentation and examples—except for one troublesome problem.

The SDK I'm wrapping is for interacting with a USB-connected proprietary device. One dataset the device communicates over the wire includes some byte padding for certain members of a memory block. I do not know exactly why the padding exists, nor am I able to change it. The device SDK handles the padding with struct-embedded nameless unions that encompass two items, the desired element and a “throwaway” array of 8 unsigned chars that provides the padding.

No matter what I've tried, I cannot find a resolution to this sort of error:

ffi.error: <struct name>: wrong size for field '<field>' (cdef says 4, but C compiler says 8). fix it or use "...;" as the last field in the cdef…

Here is a representative version of the troublesome struct/union in the SDK:

// SDK header file 

typedef struct
{
  union
  {
    SOURCE_t source; // An enum
    unsigned char padding1[8];
  };

  union
  {
    time_t time;
    unsigned char padding2[8];
  };

  char some_data[512];  
  char more_data[64];
  char another_array_field[64];
  short simple_field;
} SDK_DATA_t;

I've tried variations / permutations of:

• Adding ...; as the final field of the unions in the CDEF.
• Adding ...; as the final field of the containing struct in the CDEF.
  
• Omitting the padding character arrays of the unions in the CDEF.
• Naming the union in the source header file and CDEF.
• Extracting dedicated typedefs for the unions and using them within the structs in the source header file and CDEF.

Nothing has worked so far. And, I'd like to avoid modifying the SDK header file (some of my experiments have modified the header file).

How do I help CFFI figure out that the field within the union is a simple type but that there's possible padding bytes to ignore? I'm guessing CFFI is more than capable at handling this, but I've not found the secret sauce.

Any guidance or troubleshooting is much appreciated!

[Armin Rigo]

Hi Michael,

On Mon, 20 Nov 2023 at 18:14, Michael Karlesky
<michael....@thingamabyte.com> wrote:
> ffi.error: <struct name>: wrong size for field '<field>' (cdef says 4, but C compiler says 8). fix it or use "...;" as the last field in the cdef…

Note that the error says that the given field has a wrong *size*, not
that its offset inside the struct is wrong. What field is that, and
what is its type? If it's always the `time_t` field, then the problem
might be that the C compiler and the cdef() declaration disagree about
the size of `time_t`.


Armin Rigo

[Michael Karlesky]

Ah ha. Yes. That was it. Thank you, Armin!

I was heading down this path today already but did not realize it. I have simultaneously been trying to figure out how best to handle time_t. It is only defined as an arithmetic type in the C standard. There is no width, encoding, or format specified in C. The Python wrapper I'm building will ultimately work on multiple platforms (with a prebuilt shared library and Python extension for each). It so happens that time_t can be 32 bits even on 64 bit systems. 

What is the best way to allow CFFI and the C compiler to discover the width of time_t without having to hardcode it per platform build? typedef ... time_t; leads to an error regarding an opaque type within the struct that opened this thread.

Thank you again.

--
-- python-cffi: To unsubscribe from this group, send email to python-cffi...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/python-cffi?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "python-cffi" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/python-cffi/8v3irRVVZ7Y/unsubscribe.
To unsubscribe from this group and all its topics, send an email to python-cffi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python-cffi/CAMSv6X1L-x3OvcmoQ7zEOai%2BcNYZvck7jokfh5ShR-DiX5D7-Q%40mail.gmail.com.

[Armin Rigo]

Hi!

On Tue, 21 Nov 2023 at 01:30, Michael Karlesky
<michael....@thingamabyte.com> wrote:
> What is the best way to allow CFFI and the C compiler to discover the width of time_t without having to hardcode it per platform build? typedef ... time_t; leads to an error regarding an opaque type within the struct that opened this thread.

There's this slightly obscure trick that was implemented specifically
for this kind of cases: ``typedef int... time_t;``. It means that
``time_t`` is an arithmetic type, but its size and signedness is found
by using the C compiler. The syntax ``int...`` only works in this
exact location in a typedef, as far as I remember.

Armin Rigo

[Matěj Cepl]

On Tue Nov 21, 2023 at 8:05 AM CET, Armin Rigo wrote:
> There's this slightly obscure trick that was implemented specifically
> for this kind of cases: ``typedef int... time_t;``. It means that
> ``time_t`` is an arithmetic type, but its size and signedness is found
> by using the C compiler. The syntax ``int...`` only works in this
> exact location in a typedef, as far as I remember.

Is this cffi thing or the C compiler thing? Is there some
documentation on it? I am just right now struggling with `time_t`
definitions on various platforms with M2Crypto and this could
help me a lot.

Thank you for hinting me in hopefully right direction,

Matěj

--
http://matej.ceplovi.cz/blog/, @mc...@floss.social
GPG Finger: 3C76 A027 CA45 AD70 98B5 BC1D 7920 5802 880B C9D8

Extremism in the defense of liberty is no vice; moderation in
the pursuit of justice is no virtue.
-- Barry Goldwater (actually written by Karl Hess)

[Armin Rigo]

Hi,

On Tue, 21 Nov 2023 at 09:24, Matěj Cepl <mc...@cepl.eu> wrote:
> On Tue Nov 21, 2023 at 8:05 AM CET, Armin Rigo wrote:
> > There's this slightly obscure trick that was implemented specifically
> > for this kind of cases: ``typedef int... time_t;``. It means that
> > ``time_t`` is an arithmetic type, but its size and signedness is found
> > by using the C compiler. The syntax ``int...`` only works in this
> > exact location in a typedef, as far as I remember.
>
> Is this cffi thing or the C compiler thing? Is there some
> documentation on it? I am just right now struggling with `time_t`
> definitions on various platforms with M2Crypto and this could
> help me a lot.

That's a CFFI thing, where you can add ``...`` at some places in the
``cdef()``. It is documented there:
https://cffi.readthedocs.io/en/stable/cdef.html#letting-the-c-compiler-fill-the-gaps


A bientôt,

Armin

[Michael Karlesky]

Bingo. That worked. Thank you again, Armin!

--
-- python-cffi: To unsubscribe from this group, send email to python-cffi...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/python-cffi?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "python-cffi" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/python-cffi/8v3irRVVZ7Y/unsubscribe.
To unsubscribe from this group and all its topics, send an email to python-cffi...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python-cffi/CAMSv6X02Kv0BT%3DkP%2BcEzuMxxtRr5ji7xpSSMNng4R3Z-k6mApg%40mail.gmail.com.