Bitfield conflict between Construct and C

[John Kalter]

I have a project where I need to decode and display various size bitfield data stored in both big endian and little endian format. The python Construct package looked to be the perfect tool for the problem. Where the BitStructs are chunked into groupings of 16 bits (2 bytes), this seems to work fine. However, when dealing with larger sizes, I have run into a problem when a Bitfield crosses a word boundary. My example is on a 64 bit Linux intel machine (little endian only).

 

Here is an example C program that stores a structure of bit fields including a counter that increments:

==============================================================================

#include <stdio.h>

typedef struct
{
  unsigned int id     : 8;
  unsigned int count  : 12;
  unsigned int spare  : 12;
} TestCounter;

int main()
{
  FILE* fh;
  int i;
  TestCounter tc;
  tc.id = 194;
  tc.spare = 0;

  fh = fopen("count.dat", "wb");

  for (i=0; i < 16; i++) {
    tc.count = i;
    fwrite(&tc, 1, sizeof(TestCounter), fh);
  }
  fclose(fh);

  return 0;
}
================================================================================

 

The resulting dat file looks like this viewed with hexdump:

 

0000000 00c2 0000 01c2 0000 02c2 0000 03c2 0000
0000010 04c2 0000 05c2 0000 06c2 0000 07c2 0000
0000020 08c2 0000 09c2 0000 0ac2 0000 0bc2 0000
0000030 0cc2 0000 0dc2 0000 0ec2 0000 0fc2 0000

I have a python script using Construct to decode and display the results:

==================================================================================

from construct import BitStruct, BitField

TestCounter = BitStruct('Testdata',
  BitField('id',      8),
  BitField('count',  12),
  BitField('spare',  12),
)

fh = open('count.dat', 'rb')

for i in range(16):
  chunk = fh.read(4)
  tc = TestCounter.parse(chunk)
  print tc

fh.close()
==================================================================================

 

This is the output of the python script (beginning and end):

 

Container:
    id = 194
    count = 0
    spare = 0
Container:
    id = 194
    count = 16
    spare = 0
Container:
    id = 194
    count = 32
    spare = 0
Container:
    id = 194
    count = 48
    spare = 0

       .  .  .

 

Container:
    id = 194
    count = 224
    spare = 0
Container:
    id = 194
    count = 240
    spare = 0

 

The counter increments by 16 rather than one. It looks like C puts the MSB fragment in the second word while Construct assumes that the LSB fragment is in the second word? Is there a workaround for this situation?

 

 

 

[Basil Peace]

On Tuesday, January 28, 2014 at 4:22:51 PM UTC+3, John Kalter wrote:

It looks like C puts the MSB fragment in the second word while Construct assumes that the LSB fragment is in the second word? Is there a workaround for this situation?

C behavior for bit fields is compiler-dependent, see e.g. http://stackoverflow.com/a/6044223/2576070. Construct could have special tuning, but anyway full compatibility can't be guaranteed. Different C compiler could give you different results.