Endianness and Record Specification

[awdorrin]

I have been reading over google searches all day and I feel like I'm missing something here.

Is there a straight forward method to use Standard'Default_Bit_Order to define a specification for a record, that will work on either a Little Endian or Big Endian system?

For instance, I have a 16-bit field that is comprised of 5 bit-fields:
Reserved: 1 bit
Error Flag: 1 bit
Word Count: 6 bits
Bus Id: 1 bit
Message Type: 7 bits

So, 0x0401 is [0000 0100 0000 0001] or [0][0][000100][0][0000001]
Word Count = 4, Message Type = 1

On a Big Endian system the record is defined as:

type TW_REC is record
Message : Integer;
BusId : Integer;
WordCount : Integer;
Error : Integer;
Reserved : Integer;
end record
for TW_REC use record
Reserved at 0 range 0..0;
Error at 0 range 1..1;
WordCount at 0 range 2..7;
BusId at 1 range 0..0;
Message at 1 range 1..7;
end record;

Now, I know I can rewrite this on a Little Endian system to:
for TW_REC'Bit_Order use System.High_Order_First;
for TW_REC use record
Reserved at 1 range 0..0;
Error at 1 range 1..1;
WordCount at 1 range 2..7;
BusId at 0 range 0..0;
Message at 0 range 1..7;
end record;

Taking advantage of the High_Order_First flag to keep the bit ordering the same,
but is there a way to either calculate the byte position, or 'conditionally compile' the right record specification, depending on the system on which it is being compiled?

At first I was thinking I could do:

if Standard'Default_Bit_Order = 1 then
-- defined Little endian rec spec
else
-- define Bit endian rec spec
end if;

But, the compiler doesn't let me do that... :-)

I would think I should be able to somehow use the Default_Bit_Order to define some sort of relative byte position, depending on the system endianness, but I'm drawing a blank and hoping someone might have a quick example they could provide.

Thanks!
-Al

[awdorrin]

I think I found the cause of some of my confusion.
The Gnat compiler treats the record specifications differently depending on if the code is compiled as Ada95 or Ada2005...

Under '05 it produces the following output:
mytest.adb:26:22: info: reverse bit order in machine scalar of length 16
mytest.adb:26:22: info: little-endian range for component "Res" is 15 .. 15
mytest.adb:27:22: info: reverse bit order in machine scalar of length 16
mytest.adb:27:22: info: little-endian range for component "Error" is 14 .. 14
mytest.adb:28:22: info: reverse bit order in machine scalar of length 16
mytest.adb:28:22: info: little-endian range for component "WordCnt" is 8 .. 13
mytest.adb:29:22: info: reverse bit order in machine scalar of length 16
mytest.adb:29:22: info: little-endian range for component "BusId" is 7 .. 7
mytest.adb:30:22: info: reverse bit order in machine scalar of length 16
mytest.adb:30:22: info: little-endian range for component "Message" is 0 .. 6

And the big endian specification is reordered. Under Ada95 this does not happen.

Hmm...

[Simon Wright]

awdorrin <awdo...@gmail.com> writes:

> At first I was thinking I could do:
>
> if Standard'Default_Bit_Order = 1 then
> -- defined Little endian rec spec
> else
> -- define Bit endian rec spec
> end if;
>
> But, the compiler doesn't let me do that... :-)
>
> I would think I should be able to somehow use the Default_Bit_Order to
> define some sort of relative byte position, depending on the system
> endianness, but I'm drawing a blank and hoping someone might have a
> quick example they could provide.

The way I've approached this is to declare an unrepresented type for the
main program to use, and then make a local, derived, represented type
inside a bit-order-dependent conditional.

Below, the type Local_Status is very local, but it works with a wider
scope, and may lead to improved performance by avoiding
packing/unpacking.

This assumes it's OK to have the conversion only at the edges of the
program.

Big_Endian : constant Boolean
:= System."=" (System.Default_Bit_Order, System.High_Order_First);

function To_Stream_Element (S : Status) return Ada.Streams.Stream_Element
is
type Local_Status is record
LI : Leap_Indicator;
VN : Version;
M : Mode;
end record;
V : constant Local_Status := (LI => S.LI, VN => S.VN, M => S.M);
begin
if Big_Endian then
declare
type Host_Status is new Local_Status;
for Host_Status use record
LI at 0 range 0 .. 1;
VN at 0 range 2 .. 4;
M at 0 range 5 .. 7;
end record;
for Host_Status'Size use 8;
function Convert
is new Ada.Unchecked_Conversion (Host_Status,
Ada.Streams.Stream_Element);
begin
return Convert (Host_Status (V));
end;
else
declare
type Host_Status is new Local_Status;
for Host_Status use record
LI at 0 range 6 .. 7;
VN at 0 range 3 .. 5;
M at 0 range 0 .. 2;
end record;
for Host_Status'Size use 8;
function Convert
is new Ada.Unchecked_Conversion (Host_Status,
Ada.Streams.Stream_Element);
begin
return Convert (Host_Status (V));
end;
end if;
end To_Stream_Element;

function To_Status (S : Ada.Streams.Stream_Element) return Status
is
begin
if Big_Endian then
declare
type Host_Status is record
LI : Leap_Indicator;
VN : Version;
M : Mode;
end record;
for Host_Status use record
LI at 0 range 0 .. 1;
VN at 0 range 2 .. 4;
M at 0 range 5 .. 7;
end record;
for Host_Status'Size use 8;
function Convert
is new Ada.Unchecked_Conversion (Ada.Streams.Stream_Element,
Host_Status);
V : constant Host_Status := Convert (S);
begin
return (LI => V.LI, VN => V.VN, M => V.M);
end;
else
declare
type Host_Status is record
LI : Leap_Indicator;
VN : Version;
M : Mode;
end record;
for Host_Status use record
LI at 0 range 6 .. 7;
VN at 0 range 3 .. 5;
M at 0 range 0 .. 2;
end record;
for Host_Status'Size use 8;
function Convert
is new Ada.Unchecked_Conversion (Ada.Streams.Stream_Element,
Host_Status);
V : constant Host_Status := Convert (S);
begin
return (LI => V.LI, VN => V.VN, M => V.M);
end;
end if;
end To_Status;

[Stephen Leake]

awdorrin <awdo...@gmail.com> writes:

> I think I found the cause of some of my confusion.
> The Gnat compiler treats the record specifications differently
> depending on if the code is compiled as Ada95 or Ada2005...
>
> Under '05 it produces the following output:
> mytest.adb:26:22: info: reverse bit order in machine scalar of length 16
> mytest.adb:26:22: info: little-endian range for component "Res" is 15 .. 15
>

> And the big endian specification is reordered. Under Ada95 this does not happen.
>
> Hmm...

Yes. Ada 2005 finally got bit-endianness right; all you have to do is
specify what the bit numbers in your record spec mean, by giving
'Bit_Order. Then the compiler does the Right Thing.

See http://www.ada-auth.org/ai-files/grab_bag/bitorder.pdf

So these two record specs represent the same bit layout:

type Other_Date_And_Time_Type is record
Years_Since_1980 : Unsigned_7;
Month : Unsigned_4;
Day_Of_Month : Unsigned_5;
Hour : Unsigned_5;
Minute : Unsigned_6;
Seconds : Unsigned_5;
end record;

for Other_Date_And_Time_Type'Bit_Order use System.High_Order_First;
for Other_Date_And_Time_Type use record
Years_Since_1980 at 0 range 0 .. 6;
Month at 0 range 7 .. 10;
Day_Of_Month at 0 range 11 .. 15;
Hour at 2 range 0 .. 4;
Minute at 2 range 5 .. 10;
Seconds at 2 range 11 .. 15;
end record;

type Date_And_Time_Type is record
Years_Since_1980 : Unsigned_7;
Month : Unsigned_4;
Day_Of_Month : Unsigned_5;
Hour : Unsigned_5;
Minute : Unsigned_6;
Seconds : Unsigned_5;
end record;
for Date_And_Time_Type'Bit_Order use System.Low_Order_First;
for Date_And_Time_Type use record
Years_Since_1980 at 0 range 9 .. 15;
Month at 0 range 5 .. 8;
Day_Of_Month at 0 range 0 .. 4;
Hour at 2 range 11 .. 15;
Minute at 2 range 5 .. 10;
Seconds at 2 range 0 .. 4;
end record;

GNAT is a little paranoid about this and outputs all kinds of "helpful"
informational messages. I think there are now ways to turn them all off,
once you are comfortable that the compiler is doing the right thing.


In Ada 95, the best you could expect was that the compiler would
complain that the endianness was "wrong". I used to have a work-around
for Ada 95, that looked something like this:

for Good_Date_And_Time_Type use record
Years_Since_1980 at 0 range
Low_Bit_First * (LSBit_16 + Bit_Order * 9) + High_Bit_First * (LSBit_32 + Bit_Order * 15) ..
High_Bit_First * (LSBit_16 + Bit_Order * 9) + Low_Bit_First * (LSBit_32 + Bit_Order * 15);
end record;

But I have deleted the spec that defines LSBit_16 and LSBit_32; the
depend on the endianness of the target.

--
-- Stephe

[Quentin Ochem]

Side note on this topic, there's been a recent enhancement to GNAT to specify endianness as representation clauses:

http://www.adacore.com/developers/development-log/NF-71-L207-008-gnat/
http://docs.adacore.com/gnat-unw-docs/html/gnat_rm_3.html#SEC181

Will do the thing that you expect automatically ;-)

[Robin Vowels]

This won't help your problem, but
PL/I bit strings are stored in correct order regardless
of big-endian or little-endian machine on which the program runs.
The memory image is identical for both machine types.

[Dmitry A. Kazakov]

On Sat, 22 Sep 2012 02:32:17 -0700 (PDT), Robin Vowels wrote:

> This won't help your problem, but
> PL/I bit strings are stored in correct order regardless
> of big-endian or little-endian machine on which the program runs.
> The memory image is identical for both machine types.

This does not make any sense.

1. Anything a language stores it does in the correct order, unless the
compiler in broken. Just per definition, the correct order is the order
used by the language.

2. As for logical order of bits, that depends exclusively on the hardware.
As an example consider an A/D converter card which maps its registers onto
certain physical addresses of the host machine. The compiler has no
whatsoever influence on the bit, byte, word etc orders of the register as
perceived by the host machine hardware.

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

[Robin Vowels]

On Sep 22, 7:59 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
wrote:

> On Sat, 22 Sep 2012 02:32:17 -0700 (PDT), Robin Vowels wrote:
> > This won't help your problem, but
> > PL/I bit strings are stored in correct order regardless
> > of big-endian or little-endian machine on which the program runs.
> > The memory image is identical for both machine types.
>
> This does not make any sense.
>
> 1. Anything a language stores it does in the correct order, unless the
> compiler in broken. Just per definition, the correct order is the order
> used by the language.

I pointed out that in PL/I the order in which the bits are stored
is the same regardless of endian-ness.
Endian-ness came many years after the introduction of PL/I.

It shouldn't be an issue with Ada either.

> 2. As for logical order of bits, that depends exclusively on the hardware.
> As an example consider an A/D converter card which maps its registers onto
> certain physical addresses of the host machine. The compiler has no
> whatsoever influence on the bit, byte, word etc orders of the register as
> perceived by the host machine hardware.

That isn't relevant in determining the endian-ness.
An A-D converter isn't the processor, it's an attachment.

[Simon Wright]

Robin Vowels <robin....@gmail.com> writes:

> On Sep 22, 7:59 pm, "Dmitry A. Kazakov" <mail...@dmitry-kazakov.de>
> wrote:
>> On Sat, 22 Sep 2012 02:32:17 -0700 (PDT), Robin Vowels wrote:
>> > This won't help your problem, but
>> > PL/I bit strings are stored in correct order regardless
>> > of big-endian or little-endian machine on which the program runs.
>> > The memory image is identical for both machine types.
>>
>> This does not make any sense.
>>
>> 1. Anything a language stores it does in the correct order, unless the
>> compiler in broken. Just per definition, the correct order is the order
>> used by the language.
>
> I pointed out that in PL/I the order in which the bits are stored
> is the same regardless of endian-ness.
> Endian-ness came many years after the introduction of PL/I.
>
> It shouldn't be an issue with Ada either.

I no longer have access to a big-endian processor to check this out on,
but in Ada a packed array of boolean will be accessed in index order
exactly as you would expect.

But it is definitely the case that GNAT will map such an array (0..7) on a
big-endian machine so that Arr(0) accesses the MSB and Arr(7) accesses
the LSB.

>> 2. As for logical order of bits, that depends exclusively on the hardware.
>> As an example consider an A/D converter card which maps its registers onto
>> certain physical addresses of the host machine. The compiler has no
>> whatsoever influence on the bit, byte, word etc orders of the register as
>> perceived by the host machine hardware.
>
> That isn't relevant in determining the endian-ness.
> An A-D converter isn't the processor, it's an attachment.

It is highly relevant to the way the host code is written.

Consider a little-endian device attached by VME to a big-endian host.

If a short command 16#ff00# is to be given, the device expects the first
byte of the command to be 16#00# and the second to be 16#ff#; so what
the host has to prepare for writing to the command register is 16#00ff#.

The reverse happens every day (every millisecond!) in your network
stack; network headers are in network byte order, ie big-endian, which
is why the library functions htonl(3), htons(3), ntohl(3), ntohs(3) are
provided (host-to-network-long, host-to-network-short,
network-to-host-long, network-to-host-short).

[awdorrin]

I finally have had a chance to get back into this code (been side-tracked on another part of this migration effort.

What version of GNAT/GCC support this new Scalar_Storage_Order functionality?
Sounds like it could save me a lot of effort if I can simply define records as 'big endian format' and let the compiler generate all the byte swapping code for me.

Thanks

[Stephen Leake]

awdorrin <awdo...@gmail.com> writes:

>> http://www.adacore.com/developers/development-log/NF-71-L207-008-gnat/

>
> What version of GNAT/GCC support this new Scalar_Storage_Order functionality?

The date on that announcement is Aug 4, 2012, so it's only available in
current wavefronts from AdaCore; the most recent release of GNAT Pro
7.0.2 was in July 2012.

--
-- Stephe

[awdorrin]

On Wednesday, October 24, 2012 6:20:05 AM UTC-4, Stephen Leake wrote:

It seems that the functionality may be available in a more recent version of GCC/GNAT - but I can't find much information on it. Just too new I suppose.

Perhaps time to setup a VirtualBox and play around. :)

[Anh Vo]

On Friday, November 2, 2012 6:13:33 AM UTC-7, awdorrin wrote:
> On Wednesday, October 24, 2012 6:20:05 AM UTC-4, Stephen Leake wrote: > awdorrin writes: > > > > >> http://www.adacore.com/developers/development-log/NF-71-L207-008-gnat/ > > > > > > What version of GNAT/GCC support this new Scalar_Storage_Order functionality? > > > > The date on that announcement is Aug 4, 2012, so it's only available in > > current wavefronts from AdaCore; the most recent release of GNAT Pro > > 7.0.2 was in July 2012. > > > > -- > > -- It seems that the functionality may be available in a more recent version of GCC/GNAT - but I can't find much information on it. Just too new I suppose. Perhaps time to setup a VirtualBox and play around. :)

As indicated by Stephen, the gnat-7.1-preview has implemened this feature. Indeed, I tested it out and it worked as advertised. This is one of the third best features added to Ada 2012. The Default Initial Values is the best one while pre/post conditions are the second best.

A. Vo

[Niklas Holsti]

It's not an Ada 2012 feature, AIUI, but a GNAT feature.

--
Niklas Holsti
Tidorum Ltd
niklas holsti tidorum fi
. @ .

[Anh Vo]

<<As indicated by Stephen, the gnat-7.1-preview has implemened this feature. Indeed, I tested it out and it worked as advertised. This is > one of the third best features added to Ada 2012.>>

<It's not an Ada 2012 feature, AIUI, but a GNAT feature>

You are right this feature is not part of Ada 2012 addition. Since this feature is extremely useful in practice, I wish it was the feature of Ada 2012.

[Randy Brukardt]

"Anh Vo" <anhvo...@gmail.com> wrote in message
news:8c726f83-c972-4435...@googlegroups.com...

Then propose it for the next version of Ada (whenever that will be). Send a
message to ada-comment explaining why you find the feature "extremely
useful". If no one asks, the odds that it will be included are quite
small...

Randy.

[Anh Vo]

< On Tuesday, December 4, 2012 3:31:51 PM UTC-8, Randy Brukardt wrote:
Then propose it for the next version of Ada (whenever that will be). Send a
message to ada-comment explaining why you find the feature "extremely
useful". If no one asks, the odds that it will be included are quite
small... >

Sound like it has been proposed before but it was rejected. Frankly, I have never submitted any change proposal before. Thus, it may be a big task for me. However, I am willing to send in the change proposal for the next Ada revision if needed.

This reminds me the time when I first study Ada in earnest. After the first look at System.Bit_Order type from reading Mil-Std 1815-A, I thought it was used to handle Endian feature. But it did turn out the way I thought. Now more than 20 years later, GNAT-7.1 could be the first compiler to implement it.

A. Vo

[Jeffrey Carter]

On 12/04/2012 05:12 PM, Anh Vo wrote:
>
> This reminds me the time when I first study Ada in earnest. After the first
> look at System.Bit_Order type from reading Mil-Std 1815-A, I thought it was
> used to handle Endian feature. But it did turn out the way I thought. Now
> more than 20 years later, GNAT-7.1 could be the first compiler to implement
> it.

System.Bit_Order isn't defined in ANSI/MIL-STD-1815A (ARM 83). It first shows up
in Ada 95.

--
Jeff Carter
"The time has come to act, and act fast. I'm leaving."
Blazing Saddles
36

[Anh Vo]

System.Bit_Order isn't defined in ANSI/MIL-STD-1815A (ARM 83). It first shows up
in Ada 95.

I just downloaded a PDF version of MIL-STD-1815A since I misplace my copy. You are right System.Bit_Order does not exist in it. I guess my brain has not functioned properly.

It is interesting to note that APEX Ada 83 compiler implemented this type. Below is a section of package System containing this type that I just copy strait from APEX.

Memory_Size : constant := 16#100000000#;
function "<" (Left, Right : in Address) return Boolean;
function "<=" (Left, Right : in Address) return Boolean;
function ">" (Left, Right : in Address) return Boolean;
function ">=" (Left, Right : in Address) return Boolean;
subtype Address_Sized_Integer is Integer;
function To_Address (Value : in Address_Sized_Integer) return Address;
function To_Integer (Value : in Address) return Address_Sized_Integer;
type Address_Sized_Unsigned is mod 16#100000000#;
function "+" (I : in Address_Sized_Unsigned) return Address;
function Memory_Address (I : in Address_Sized_Unsigned) return Address
renames "+";
function "+" (Left : in Address; Right : in Integer) return Address;
function "+" (Left : in Integer; Right : in Address) return Address;
function "-" (Left : in Address; Right : in Integer) return Address;
function "-" (Left : in Address; Right : in Address) return Integer;
function "+" (Left : in Address; Right : in Long_Integer) return Address;
function "+" (Left : in Long_Integer; Right : in Address) return Address;
function "-" (Left : in Address; Right : in Long_Integer) return Address;
function "-" (Left : in Address; Right : in Address) return Long_Integer;
type Bit_Order is (High_Order_First, Low_Order_First);
Default_Bit_Order : constant Bit_Order := High_Order_First;
type Byte_Order_T is (Little_Endian, Big_Endian);
Byte_Order : constant Byte_Order_T := Big_Endian;
subtype Any_Priority is Integer range 0 .. 58;
subtype Priority is Any_Priority range 0 .. 57;
subtype Interrupt_Priority is Any_Priority range 58 .. 58;
Default_Priority : constant Priority := 28;
Address_Zero : constant Address;
No_Addr : constant Address;