[Boost-users] Convert big endian to little endian (floating point values)

[Martin Vymazal]

Hello,

I am trying to convert a floating point value from big endian to little
endian, but the code below does not compile:

double x = 10.0;

const double x_big_endian = boost::endian::native_to_big(x);
boost::endian::native_to_big_inplace(x);

and the compiler (gcc 5.2) only offers overloads for integer types. This is
with boost 1.59. Are floating point values currently supported (the website of
boost::endian says so)? Am I doing something wrong?

Thank you.

Martin
_______________________________________________
Boost-users mailing list
Boost...@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[Aaron_...@selinc.com]

> From: Martin Vymazal <martin....@vki.ac.be>

> Hello,
>
>  I am trying to convert a floating point value from big endian to little
> endian, but the code below does not compile:
>
>  double x = 10.0;
>
>  const double x_big_endian = boost::endian::native_to_big(x);        
>  boost::endian::native_to_big_inplace(x);  

Be very careful how you swap floats. They aren't like integer data types where the swapped representation can fit nicely into the same type. Not all swapped floats are valid floats, and you'll find that the bits will be messaged into a valid float, thus changing your value without telling you. You can only swap a float into an integer. You can't swap a float into a float.

[Martin Vymazal]

Hi,

ok, here's the whole story. We have a code that computes some data,
compresses them through zlib and saves as binary. We also use our own
utility to postprocess the compressed file and convert it to
human-readable format.

What happened recently is that the code ran on a big-endian machine and
since zlib preserves the endian-ess of the data, it generated a binary
with big-endian doubles. We would like to have the possibility of
bringing that file on a little-endian machine (say user's laptop) and
postprocess there, instead of doing everything on the big-endian
machine. Do you have an alternative suggestion how to approach this
problem? Let's assume that writing the output data in any other format
than the current one is not possible.

I am also still interested in knowing whether boost (1.59) supports
conversions between endian representations of floating-point values.

[Maarten de Vries]

On 8 December 2015 at 22:14, Martin Vymazal <martin....@vki.ac.be> wrote:

Hi,

 ok, here's the whole story. We have a code that computes some data, compresses them through zlib and saves as binary. We also use our own utility to postprocess the compressed file and convert it to human-readable format.

 What happened recently is that the code ran on a big-endian machine and since zlib preserves the endian-ess of the data, it  generated a binary with big-endian doubles. We would like to have the possibility of bringing that file on a little-endian machine (say user's laptop) and postprocess there, instead of doing everything on the big-endian machine. Do you have an alternative suggestion how to approach this problem? Let's assume that writing the output data in any other format than the current one is not possible.

I am also still interested in knowing whether boost (1.59) supports conversions between endian representations of floating-point values.

If you absolutely must, the easiest way to do this is to read the data as if it's an unsigned integer type (of the correct size), convert endianness, then reinterpret_cast (or something similarly horrible) to the floating point type.

Of course, not relying on the platform specific bit representation of primitive data types when serializing data is a much better idea.

-- Maarten

[Martin Vymazal]

What would be a possible approach to save the data without 'relying on
the platform specific bit representation of primitive data types' as you
put it? I can not save the data as plain ascii text, because the files
would simply be huge (the're already big as they are). Is there some
more intelligent solution? I'm not a software engineer, so I'd be happy
to hear what is the common practice here.

I will also try one last time: does boost support conversion of floating
point types or not? The website says so, but it doesn't work for me. I
am not asking because I plan to use this functionality, I'm simply
curious since I already started playing with it.

[Jeff Garland]

Not currently -- right from the docs under floating point support it sez...

http://www.boost.org/doc/libs/1_59_0/libs/endian/doc/index.html

Support for floating point types was removed from Boost 1.58.0 because there was not enough time to resolve reliability concerns. It is expected that floating point support will be available in Boost 1.59.0.

Since those are the 1.59 docs I'd infer that it wasn't fixed in 1.59.

Jeff

[Paolo Bolzoni]

On Wed, Dec 9, 2015 at 7:18 AM, Martin Vymazal <martin....@vki.ac.be> wrote:
> What would be a possible approach to save the data without 'relying on the
> platform specific bit representation of primitive data types' as you put it?

What about using boost integer to find an unsigned integer (let's call
its type Lint) longer or equal than the float (type Float) you have
and make an union?

union Lint_Float_t {
Lint store;
Float read;
};
static_assert(sizeof(Lint) >= sizeof(Float), "");

To store you copy the float in the union, and save the integer. To
load you load the integer and copy from the union the float.

It is definitely not portable (reading and writing different types in
a union is at most implementation dependant) and you keep the original
bits, but it should work at least on machine of the same type.

Cheers,
Paolo

[Maarten de Vries]

On 8 December 2015 at 23:18, Martin Vymazal <martin....@vki.ac.be> wrote:

On 2015-12-08 22:48, Maarten de Vries wrote:

Of course, not relying on the platform specific bit representation of
primitive data types when serializing data is a much better idea.

-- Maarten

What would be a possible approach to save the data without 'relying on the platform specific bit representation of primitive data types' as you put it? I can not save the data as plain ascii text, because the files would simply be huge (the're already big as they are). Is there some more intelligent solution? I'm not a software engineer, so I'd be happy to hear what is the common practice here.

Martin

​
If you're interested:

You can still use a binary format, though it depends on your needs. You could look at boost serialization [1]. I've never worked with it, so I can't comment on performance (either in terms of speed or serialized size). But in theory it solves the problem of saving and loading data to/from files (and more).

Another option might be Binn [2]. It's C but it looks simple and clean enough (although I also never worked with it).

[1] http://www.boost.org/doc/libs/1_59_0/libs/serialization/doc/index.html
[2] https://github.com/liteserver/binn​

[Agustín K-ballo Bergé]

On 12/8/2015 11:32 PM, Paolo Bolzoni wrote:
> On Wed, Dec 9, 2015 at 7:18 AM, Martin Vymazal <martin....@vki.ac.be> wrote:
>> What would be a possible approach to save the data without 'relying on the
>> platform specific bit representation of primitive data types' as you put it?
>
> What about using boost integer to find an unsigned integer (let's call
> its type Lint) longer or equal than the float (type Float) you have
> and make an union?

Ugh, this again? Third time this week... just no.

> union Lint_Float_t {
> Lint store;
> Float read;
> };
> static_assert(sizeof(Lint) >= sizeof(Float), "");
>
> To store you copy the float in the union, and save the integer. To
> load you load the integer and copy from the union the float.
>
> It is definitely not portable (reading and writing different types in
> a union is at most implementation dependant) and you keep the original
> bits, but it should work at least on machine of the same type.

Reading from an inactive member of a union is plain and simply undefined
behavior. To read and write the bits of the `float` in a well defined
way use `std::memcpy`.

Regards,
--
Agustín K-ballo Bergé.-
http://talesofcpp.fusionfenix.com

[Ion Gaztañaga]

> Reading from an inactive member of a union is plain and simply undefined
> behavior. To read and write the bits of the `float` in a well defined
> way use `std::memcpy`.

Curiously, it is undefined in C++, whereas well-defined in C:

"6.5.2.3 Structure and union members

If the member used to read the contents of a union object is not the
same as the member last used to store a value in the object, the
appropriate part of the object representation of the value is
reinterpreted as an object representation in the new type as described
in 6.2.6 (a process sometimes called ‘‘type punning’’). This might be a
trap representation."

Since C++ includes C as a normative reference it would be a good idea to
allow it also in C++. memcpy is fine if the compiler optimizes it as an
intrinsic, but inefficient in some implementations to copy only a couple
of bytes.

Best,

Ion

[Agustín K-ballo Bergé]

On 12/11/2015 4:54 PM, Ion Gaztañaga wrote:
>> Reading from an inactive member of a union is plain and simply undefined
>> behavior. To read and write the bits of the `float` in a well defined
>> way use `std::memcpy`.
>
> Curiously, it is undefined in C++, whereas well-defined in C:
>
> "6.5.2.3 Structure and union members
>
> If the member used to read the contents of a union object is not the
> same as the member last used to store a value in the object, the
> appropriate part of the object representation of the value is
> reinterpreted as an object representation in the new type as described
> in 6.2.6 (a process sometimes called ‘‘type punning’’). This might be a
> trap representation."
>
> Since C++ includes C as a normative reference it would be a good idea to
> allow it also in C++. memcpy is fine if the compiler optimizes it as an
> intrinsic, but inefficient in some implementations to copy only a couple
> of bytes.

C has *vastly* weaker aliasing rules than C++ (to the point that several
C implementations choose to follow the C++ rules instead).
For more details see the thread at
http://lists.boost.org/Archives/boost/2015/11/226663.php

If your implementation of choice gives you poor codegen for `memcpy`
then I'll suggest you file a bug against it. We would all benefit from that.

Regards,
--
Agustín K-ballo Bergé.-
http://talesofcpp.fusionfenix.com