Who can explain this bug?

[mathog]

Platform: gcc 4.6.3 on AMD Athlon(tm) II X2 245 Processor on
Ubuntu 12.04

The bug I thought was due to a particularly hard to find memory access
problem (see other thread I started today) seems to be even odder than
that. It was tracked down to a particular conditional, which seems to
optimize in such a way that the result is not always correct. (But
possibly only when embedded in the rest of the program.)

The skeletal elements of the problem code are:

double function(double *ymax){

double yheight;
double baseline;
double tmp=0.0;
double fs;
double yll, boff; /* these have values */
/* bbox is a struct that includes yMin and yMax */

/* set fs */

/* start of loop */
/* set yll, boff, bbox */
yheight = bbox.yMax - bbox.yMin;
printf("baseline %lf tmp %20.17lf %16llX ymax %20.17lf %16llX \n",
baseline, tmp, *(uint64_t *)&tmp,
(ymax ? *ymax: 666), (ymax ? *(uint64_t *)ymax: 666));


if(ymax){
tmp = fs * (((double)bbox.yMax)/yheight);
//the next line would "fix" the code when compiled in:
// printf("DEBUG ymax %lf tmp %lf\n", *ymax, tmp);fflush(stdout);
if(*ymax <= tmp){ // <--------THIS IS THE PROBLEM
*ymax = tmp;
baseline = yll - boff;
}
}
/* end of loop */
return(baseline);
}

When this is compiled with -O2 or -O3, AND when it is part of the larger
program, the indicated test is failing for the equals case, the
output from the printf statements is:

baseline 0.000000 tmp 0.00000000000000000 0 ymax
0.00000000000000000 0
baseline 12.680004 tmp 6.85406955898171422 401B6A9135E157A9 ymax
6.85406955898171422 401B6A9135E157A9
baseline 12.680004 tmp 6.85406955898171422 401B6A9135E157A9 ymax
6.85406955898171422 401B6A9135E157A9
baseline 12.680004 tmp 6.85406955898171422 401B6A9135E157A9 ymax
6.85406955898171422 401B6A9135E157A9

which is wrong, the baseline value should have been changing because
yll and boff were different on each loop iteration.

Compile it with -O0 (or with -O2 or -O3 and not be part of the larger
program, or with the commented out printf uncommented) and
instead it emits:

baseline 0.000000 tmp 0.00000000000000000 0 ymax
0.00000000000000000 0
baseline 12.680004 tmp 6.85406955898171422 401B6A9135E157A9 ymax
6.85406955898171422 401B6A9135E157A9
baseline 13.480004 tmp 6.85406955898171422 401B6A9135E157A9 ymax
6.85406955898171422 401B6A9135E157A9
baseline 17.480006 tmp 6.85406955898171422 401B6A9135E157A9 ymax
6.85406955898171422 401B6A9135E157A9

which is the correct result.

So, bizarrely, there is a problem with the conditional "<=" not giving
the right answer when the values of *ymax and tmp are equal when the
code is compiled in certain ways. And they are equal, down to the last
bit, as shown by the print statements!

This modification does not resolve the issue (when compiled in such a
way that it would be an issue):
if(*ymax <= (tmp * 1.0000000000000001)){

but this one does:
if(*ymax <= (tmp * 1.00000000001)){

So, what sort of "optimization" is it that gives the wrong result for
the "<=" condition when *ymax and tmp are equal????

Thanks,

David Mathog

[Gordon Burditt]

> So, what sort of "optimization" is it that gives the wrong result for
> the "<=" condition when *ymax and tmp are equal????

An optimization that keeps excess precision in a calculation.

A typical floating point unit uses registers that are 80 bits wide,
and only narrows it down to 64 bits when you store in a double.
The code may not bother storing the result, then re-loading it
before doing a comparison.

See the GCC option -ffloat-store option.

Comparing for floating-point equality is treading on dangerous
ground. Where calculations are involved, there's almost always an
excuse for at least one bit of roundoff. Also, very few non-integer
decimal numbers can be exactly represented in finite-precision
binary floating point.

[mathog]

Gordon Burditt wrote:
>> So, what sort of "optimization" is it that gives the wrong result for
>> the "<=" condition when *ymax and tmp are equal????
>
> An optimization that keeps excess precision in a calculation.
>
> A typical floating point unit uses registers that are 80 bits wide,
> and only narrows it down to 64 bits when you store in a double.
> The code may not bother storing the result, then re-loading it
> before doing a comparison.

That would do it. In the example tmp is calculated and probably lives
in a register but the ymax value is stored. (If after the calculation
tmp is not stored to ymax, it is never used again, so there is no reason
it would not just be in the register.) So tmp may differ in the extra
16 bits out of the 80 bits that were not stored into ymax.

> See the GCC option -ffloat-store option.

Right. But I want this to work always - code behavior generally should
not depend on obscure compiler flags being set one way or the other.
That flag cannot be the one causing troubles here in any case, because
it is disabled for -O0 and -O3. Actually comparing O0 and O3 it was
not immediately evident which of the many changed flags might have
caused this.

>
> Comparing for floating-point equality is treading on dangerous
> ground. Where calculations are involved, there's almost always an
> excuse for at least one bit of roundoff. Also, very few non-integer
> decimal numbers can be exactly represented in finite-precision
> binary floating point.

The odd thing here is effectively what is happening is

b = 10000;
a = 1.0/9.0; /* a calculation with no exact representation */
if(b <= a) b= a; /* comparison is true */
a = 1.0/9.0; /* the SAME calculation */
if(b <= a) b= a; /* comparison is false */

Seems like a compiler bug to me that a value _known to have been stored_
in memory at 64 bits would be compared directly to a recent calculation
at 80 bits. Possibly faster to do it that way, but, as here, it can
give the wrong answer.

Thanks,

David Mathog

[James Kuyper]

On 04/18/2013 04:41 PM, mathog wrote:
> Gordon Burditt wrote:
>>> So, what sort of "optimization" is it that gives the wrong result for
>>> the "<=" condition when *ymax and tmp are equal????
>>
>> An optimization that keeps excess precision in a calculation.
>>
>> A typical floating point unit uses registers that are 80 bits wide,
>> and only narrows it down to 64 bits when you store in a double.
>> The code may not bother storing the result, then re-loading it
>> before doing a comparison.
>
> That would do it. In the example tmp is calculated and probably lives
> in a register but the ymax value is stored. (If after the calculation
> tmp is not stored to ymax, it is never used again, so there is no reason
> it would not just be in the register.) So tmp may differ in the extra
> 16 bits out of the 80 bits that were not stored into ymax.
>
>> See the GCC option -ffloat-store option.
>
> Right. But I want this to work always - code behavior generally should
> not depend on obscure compiler flags being set one way or the other.

You're out of luck - the standard doesn't mandate the existence of any
feature that would deal with this. There's no portable way to do this,
just a variety of non-portable ways.

If you can restrict the use of your code to fully conforming
implementations of C99 or later, you can examine the value of
FLT_EVAL_METHOD (defined in <float.h>) to determine whether or not a
given compiler carries excess precision when evaluating expressions of
type float or double. However, if that value is -1, the evaluation
method is indeterminable, and if it's any other negative value, the
meaning is implementation-defined. If FLT_EVAL_METHOD >= 0, then float_t
and double_t (defined in <math.h>) are guaranteed to be wide enough that
intermediate expressions are evaluated in the same type, but the
standard imposes no requirements on float_t or double_t when
FLT_EVAL_METHOD < 0.

> That flag cannot be the one causing troubles here in any case, because
> it is disabled for -O0 and -O3. Actually comparing O0 and O3 it was
> not immediately evident which of the many changed flags might have
> caused this.

I couldn't find any documentation of a connection between --float-store
and -O0 or -O3. It also seems odd that something would be disabled for
those two levels, but not for -O1 or -O2. Can you explain that comment a
little more?

...

> The odd thing here is effectively what is happening is
>
> b = 10000;
> a = 1.0/9.0; /* a calculation with no exact representation */
> if(b <= a) b= a; /* comparison is true */
> a = 1.0/9.0; /* the SAME calculation */
> if(b <= a) b= a; /* comparison is false */

It might seem to you that both comparisons are identical, but they can
look very different to an optimizer. One side in one of the comparisons
might be a value that the optimizer has decided to keep in a
high-precision register; the corresponding side in the other comparison
might be a value that it has decided to retrieve from a lower-precision
memory location.

> Seems like a compiler bug to me that a value _known to have been stored_
> in memory at 64 bits would be compared directly to a recent calculation
> at 80 bits. Possibly faster to do it that way, but, as here, it can
> give the wrong answer.

That's why --float-store was invented, so you could express your
preference for "more consistent" vs. "faster and with greater but
inconsistent precision". That's precisely why the project I'm working on
has mandated --float-store when compiling using gcc (SGIs MIPS-PRO
compiler used to be our main compiler, and we used to use several
others, but for several years now we've been using gcc exclusively - it
makes portability testing a bit difficult).

[Nobody]

On Thu, 18 Apr 2013 13:41:14 -0700, mathog wrote:

>> See the GCC option -ffloat-store option.
>
> Right. But I want this to work always - code behavior generally should
> not depend on obscure compiler flags being set one way or the other.

Tough; floating-point calculations are effectively non-deterministic
without certain additional compiler flags.

[Tim Rentsch]

mathog <dma...@gmail.com> writes:

> Platform: gcc 4.6.3 on AMD Athlon(tm) II X2 245 Processor on
> Ubuntu 12.04
>
> The bug I thought was due to a particularly hard to find memory
> access problem (see other thread I started today) seems to be
> even odder than that. It was tracked down to a particular
> conditional, which seems to optimize in such a way that the
> result is not always correct. (But possibly only when embedded
> in the rest of the program.)
>

> The skeletal elements of the problem code [white space added] are:
>
> double function( double *ymax ){

> double yheight;
> double baseline;
> double tmp = 0.0;
> double fs;
> double yll, boff; /* these have values */
> /* bbox is a struct that includes yMin and yMax */
>
> /* set fs */
>
> /* start of loop */
> /* set yll, boff, bbox */
> yheight = bbox.yMax - bbox.yMin;
> printf( "baseline %lf tmp %20.17lf %16llX ymax %20.17lf %16llX \n",
> baseline, tmp, *(uint64_t *)&tmp,
> (ymax ? *ymax : 666), (ymax ? *(uint64_t *)ymax : 666) );
>
> if( ymax ){
> tmp = fs * ( ((double) bbox.yMax) / yheight );

> // the next line would "fix" the code when compiled in:

> // printf( "DEBUG ymax %lf tmp %lf\n", *ymax, tmp); fflush(stdout);

> if( *ymax <= tmp ){ // <--------THIS IS THE PROBLEM

> *ymax = tmp;
> baseline = yll - boff;
> }
> }
> /* end of loop */
> return(baseline);
> }
>

> [program behaves differently compiled with no optimization
> and -O2 or -O3]

>
> So, bizarrely, there is a problem with the conditional "<=" not
> giving the right answer when the values of *ymax and tmp are
> equal when the code is compiled in certain ways.

> [snip elaboration]

The problem is gcc, which is non-conforming in how it handles
floating point. Changing the program with

volatile double tmp = 0.0;

or, if you prefer a more delicate touch, with

if( *ymax <= *(volatile double *)&tmp ){ ...

should remove the offending non-conforming behavior.

[Tim Rentsch]

gordon...@burditt.org (Gordon Burditt) writes:

>> So, what sort of "optimization" is it that gives the wrong result for
>> the "<=" condition when *ymax and tmp are equal????
>
> An optimization that keeps excess precision in a calculation.
>
> A typical floating point unit uses registers that are 80 bits wide,
> and only narrows it down to 64 bits when you store in a double.
> The code may not bother storing the result, then re-loading it

> before doing a comparison. [snip]

Only in non-conforming implementations. What happens in a
conforming implementation must be as if the value were actually
stored and re-fetched, and the semantics of assignment imcludes
removing any extra range and precision (footnote, 6.3.1.8 p2).
What gcc does is not a legitimate optimization but non-conforming
behavior.

[Tim Rentsch]

James Kuyper <james...@verizon.net> writes:

> On 04/18/2013 04:41 PM, mathog wrote:
>> Gordon Burditt wrote:
>>>> So, what sort of "optimization" is it that gives the wrong result for
>>>> the "<=" condition when *ymax and tmp are equal????
>>>
>>> An optimization that keeps excess precision in a calculation.
>>>
>>> A typical floating point unit uses registers that are 80 bits wide,
>>> and only narrows it down to 64 bits when you store in a double.
>>> The code may not bother storing the result, then re-loading it
>>> before doing a comparison.
>>
>> That would do it. In the example tmp is calculated and probably lives
>> in a register but the ymax value is stored. (If after the calculation
>> tmp is not stored to ymax, it is never used again, so there is no reason
>> it would not just be in the register.) So tmp may differ in the extra
>> 16 bits out of the 80 bits that were not stored into ymax.
>>
>>> See the GCC option -ffloat-store option.
>>
>> Right. But I want this to work always - code behavior generally should
>> not depend on obscure compiler flags being set one way or the other.
>
> You're out of luck - the standard doesn't mandate the existence of any
> feature that would deal with this. There's no portable way to do this,

> just a variety of non-portable ways. [snip elaboration]

This explanation is all messed up. There is a portable way of
doing what he wants to do, for _conforming_ implementations, and
he is doing it: storing the result of a floating point
calculation into a variable (ie, by assignment) is obliged to
remove any extra range and precision, which would eliminate the
unexpected behavior. That gcc does not behave this way is a bug
in gcc, not behavior that the Standard admits in conforming
implementations.

[Tim Rentsch]

Floating-point operations can be effectively non-deterministic.
But in C, assigning and accessing floating-point variables is not
as non-deterministic as you seem to think it is. The culprit
here is gcc, which cheerfully generates non-conforming code for
floating point expressions involving assignment and subsequent
access.

[glen herrmannsfeldt]

mathog <dma...@gmail.com> wrote:
> Gordon Burditt wrote:
>>> So, what sort of "optimization" is it that gives the wrong result for
>>> the "<=" condition when *ymax and tmp are equal????

It is commonly said, for good reasons, not to depend on equality
in floating point comparisons. Not even if you just assigned the
same value to a variable.

>> An optimization that keeps excess precision in a calculation.

>> A typical floating point unit uses registers that are 80 bits wide,
>> and only narrows it down to 64 bits when you store in a double.
>> The code may not bother storing the result, then re-loading it
>> before doing a comparison.

> That would do it. In the example tmp is calculated and probably lives
> in a register but the ymax value is stored. (If after the calculation
> tmp is not stored to ymax, it is never used again, so there is no reason
> it would not just be in the register.) So tmp may differ in the extra
> 16 bits out of the 80 bits that were not stored into ymax.

Actually, I believe it is 11 bits, you also get 4 more exponent bits.
The latter is convenient in not having to worry about overflow in
intermediate calculations when the result will fit. (and no hidden 1,
like in the other forms.)

>> See the GCC option -ffloat-store option.

> Right. But I want this to work always - code behavior generally should
> not depend on obscure compiler flags being set one way or the other.
> That flag cannot be the one causing troubles here in any case, because
> it is disabled for -O0 and -O3. Actually comparing O0 and O3 it was
> not immediately evident which of the many changed flags might have
> caused this.

The x87 floating point processor has eight registers, each 80 bits
wide. The original design was for a virtual stack such that when
it overflowed an interrupt routine would write some out to memory,
and on underflow read them back. Programs could be written without
worry about stack size. After the hardware (8087) was built, it
turned out that it was not possible to write such a routine.
As I remember it, you couldn't restore all the status bits back
to the appropriate values.

There are control bits that affect the precision of some operations,
but not all of them. Some systems (runtime start-up code) set those
bits, others don't.

>> Comparing for floating-point equality is treading on dangerous
>> ground. Where calculations are involved, there's almost always an
>> excuse for at least one bit of roundoff. Also, very few non-integer
>> decimal numbers can be exactly represented in finite-precision
>> binary floating point.
>
> The odd thing here is effectively what is happening is

> b = 10000;
> a = 1.0/9.0; /* a calculation with no exact representation */
> if(b <= a) b= a; /* comparison is true */
> a = 1.0/9.0; /* the SAME calculation */
> if(b <= a) b= a; /* comparison is false */

If you really have a compiler where 10000 <= 1.0/9.0 then
it is a bug.

As far as I know, it is usual for -O0 to keep intermediates in
register during evaluation of an expression, but store to the actual
variable in the end. With -O3, most try to keep values in registers
between statements, until they run out of registers.

In any case, the complications of writing a register allocator
for arbitrarily complicated expressions and eight registers means
that often you will be surprised with the results.

> Seems like a compiler bug to me that a value _known to have been stored_
> in memory at 64 bits would be compared directly to a recent calculation
> at 80 bits. Possibly faster to do it that way, but, as here, it can
> give the wrong answer.

If you don't depend on equality, or even being close, for floating
point then it isn't a problem. For even more fun, note that there
was (maybe still are) Cray computers where:

if(a*b!=b*a) printf("Ooops!\n");

might surprise you. That is, the floating point multiplier gave
different answers depending on the order of the operands.

The normal problem with the inconsistent extra precision is that it
causes some calculations designed to avoid problems with precision
loss to fail.

Another common problem is that constant expressions evaluated at
compile time give different results from the same expression,
(with variables) evaluated at run time.

-- glen

[glen herrmannsfeldt]

I believe that is correct. Fortran is more lenient in what it allows
for floating point expression evaluation. But storing every intermediate
value can slow things down a lot!

Well, they should go to cache instead of all the way to main memory,
but it is still slow.

Does gcc always store intermediate values with -O0?

-- glen

[gwowen]

On Apr 19, 12:05 am, Tim Rentsch <t...@alumni.caltech.edu> wrote:
> The culprit here is gcc, which cheerfully generates non-conforming code for
> floating point expressions involving assignment and subsequent
> access.

If you ask for conformance, with --std=c99, say, you get conformance.
I don't know of any compiler that is strictly conforming without being
specifically asked to be.

[Noob]

Tim Rentsch wrote:

> Floating-point operations can be effectively non-deterministic.
> But in C, assigning and accessing floating-point variables is not
> as non-deterministic as you seem to think it is. The culprit
> here is gcc, which cheerfully generates non-conforming code for
> floating point expressions involving assignment and subsequent
> access.

You're quick to castigate gcc!

Can you provide the test-case showing the problem you speak of?

Regards.

[James Kuyper]

According to its documentation, gcc enables -fexcess-precision=standard
automatically if you set -std=c99. Wording that addresses the excess
precision issue was only added in that version, so that seems reasonable.

I would be very surprised if gcc generates any code which fails to
conform to the floating point accuracy requirements of the C standard,
when invoked with the options that promise such conformance. That's not
because gcc's developers are unusually competent, but because the C
standard imposes so few accuracy requirements on floating point code.
gcc's developers would have to be unusually incompetent to mis-handle
the few cases where it does impose requirements.

5.2.4.2.2p6 says: "The accuracy of the floating-point operations (+, -,
*, /) and of the library functions in <math.h> and <complex.h> that
return floating-point results is implementation defined, as is the
accuracy of the conversion between floating-point internal
representations and string representations performed by the library
functions in <stdio.h>, <stdlib.h>, and <wchar.h>. The implementation
may state that the accuracy is unknown."

If __STDC_IEC_559__ is pre-defined by the implementation, the accuracy
requirements of Annex F apply, which are essentially the same as those
of IEC 60559:1989 (== IEEE 754:1985). However, if that macro is not
pre-defined. about the only significant floating point accuracy
requirements imposed by the C Standard are those on the conversion of
floating point constants to floating point values (6.4.4.2p3-5), and of
conversions to floating point types (6.3.1.4p2, 6.3.1.5p1).

The standard does recommend that "The translation-time conversion of
floating constants should match the execution-time conversion of
character strings by library functions, such as strtod, given matching
inputs suitable for both conversions, the same result format, and
default execution-time rounding." (6.4.4.2p7). It also recommends that
the printf() family be fairly accurate when performing the opposite
conversion (7.21.6.1p12-13). However, in both cases, that's in the
"Recommended Practice" section, it's not an actual requirement.
--
James Kuyper

[Keith Thompson]

A quibble: the phrase "strictly conforming" applies to a subset
of conforming *programs*, not implementations. An implementation,
as far as the standard is concerned, is either conforming or not.

I wouldn't complain about the otherwise reasonable use of "strictly"
to modify "conforming" if not for the fact that the standard actually
uses "strictly conforming" as a defined technical term.

--
Keith Thompson (The_Other_Keith) ks...@mib.org <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

[mathog]

Tim Rentsch wrote:

> The culprit
> here is gcc, which cheerfully generates non-conforming code for
> floating point expressions involving assignment and subsequent
> access.
>

This little test program shows the same issue:

#include <stdio.h>
#include <stdlib.h>

void atest(double *ymax, double a, double b){
double tmp;
tmp=a/b;
if(*ymax <= tmp){
*ymax = tmp;
printf("True\n");
}
else {
printf("False\n");
}
}

int main(void){
double ymax=0;
double a, b;

while(1){
(void) fscanf(stdin,"%lf %lf",&a,&b);
if(!b)break;
atest(&ymax,a,b);
}
}

> gcc -O3 -o test test.c
> ./test
11 23
True
11 23
False
11 19
True
11 19
False
etc.

As suggested elsewhere in this thread, making tmp "volatile" does
resolve the issue. Volatile is supposed to prevent compilers from
optimizing away calculations that need to be repeated. Here for some
reason it forces the result of tmp into a variable. Why? The compiler
then knows that 'tmp" may change unpredictably between where it is set
and where it is read, but why does that imply that the desired value at
the second position is the calculated value rather than something
written into tmp from a register in a hardware device (or whatever)
outside of this program's control?

Regards,

David Mathog

[James Kuyper]

If compiled with -fexcess-precision=standard (which is the default if
you choose -std=c99), then the results become consistent. All you need
to do, in order get standard-conforming behavior, is to request
standard-conforming behavior (at least, in this case).
--
James Kuyper

[Noob]

mathog wrote:

> Tim Rentsch wrote:
>
>> The culprit
>> here is gcc, which cheerfully generates non-conforming code for
>> floating point expressions involving assignment and subsequent
>> access.

Tim: In your haste to blame gcc, you forget that gcc is NOT
a conforming compiler by default (it compiles gnuc89).

http://gcc.gnu.org/onlinedocs/gcc/C-Dialect-Options.html

If you want conforming, it is the operator's responsibility
to explicitly ask for it.

gcc -std=c99 -pedantic-errors

There have been, literally, entire books written on the subject
of floating-point arithmetic. If you follow comp.arch, you can read
old-timers (mostly 70s and 80s engineers) criticize several aspects
of IEEE 754, or even the very concept of "floating-point" arithmetic.

You have barely scratched the surface of the complexity in FP.

Maybe you'll enjoy David Goldberg's article:
http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

Regards.

[Tim Rentsch]

This may shock you, but I didn't just fall off the turnip
truck. :)

My tests with gcc were done using -std=c99 -pedantic-errors
(which is pretty much my default these days).

[Tim Rentsch]

Noob <root@localhost> writes:

> mathog wrote:
>
>> Tim Rentsch wrote:
>>
>>> The culprit
>>> here is gcc, which cheerfully generates non-conforming code for
>>> floating point expressions involving assignment and subsequent
>>> access.
>
> Tim: In your haste to blame gcc, you forget that gcc is NOT
> a conforming compiler by default (it compiles gnuc89).

Oh, excuse me, but I forgot no such thing.

> http://gcc.gnu.org/onlinedocs/gcc/C-Dialect-Options.html
>
> If you want conforming, it is the operator's responsibility
> to explicitly ask for it.
>
> gcc -std=c99 -pedantic-errors

And that is just how I invoked gcc when running my tests.

[Tim Rentsch]

James Kuyper <james...@verizon.net> writes:

> On 04/19/2013 05:09 AM, Noob wrote:
>> Tim Rentsch wrote:
>>
>>> Floating-point operations can be effectively non-deterministic.
>>> But in C, assigning and accessing floating-point variables is not
>>> as non-deterministic as you seem to think it is. The culprit
>>> here is gcc, which cheerfully generates non-conforming code for
>>> floating point expressions involving assignment and subsequent
>>> access.
>>
>> You're quick to castigate gcc!
>>
>> Can you provide the test-case showing the problem you speak of?
>
> According to its documentation, gcc enables -fexcess-precision=standard
> automatically if you set -std=c99. Wording that addresses the excess

> precision issue was only added in that version, [snip elaboration].

There's an implication there (perhaps unintended) that discarding
excess precision wasn't required until C99. That conclusion
doesn't jibe with other, pre-C99, documents.

In particular, if we look at comments in Defect Reports, and
also how the wording in this area has changed over time (note
for example n843.htm, and compare n1256 to C99), it's clear that
narrowing behavior was intended all along.

What's more, assignment has _always_ implied removing any extra
range and precision, because in the abstract machine we actually
store the value in an object, and then on subsequent accesses
convert the stored representation to a value. Under the as-if
rule, any information beyond what the object representation can
represent must be lost.

Bottom line: assignment of floating point types always requires
removing any extra range and precision, even in C90.

[Tim Rentsch]

Noob <ro...@127.0.0.1> writes:

> Tim Rentsch wrote:
>
>> Floating-point operations can be effectively non-deterministic.
>> But in C, assigning and accessing floating-point variables is not
>> as non-deterministic as you seem to think it is. The culprit
>> here is gcc, which cheerfully generates non-conforming code for
>> floating point expressions involving assignment and subsequent
>> access.
>
> You're quick to castigate gcc!

Only after running some test cases.

> Can you provide the test-case showing the problem you speak of?

I ran several simple test cases, including the ones
involving use of volatile that I showed else-thread. The
version of gcc I was using is not as recent as the one
mentioned up-thread, but recent enough so extrapolation
seemed reasonable. (And I note with satisfaction that my
advice about using volatile was proved correct.) I think if
you try to generate some test cases yourself they will be
easy enough to find, although that may depend on exactly
which version of gcc is used. It's possible newer versions
of gcc have eliminated this class of problems entirely;
however, knowing that gcc has had this problem historically,
and verifying a particular test case myself, was I thought
sufficient evidence to focus attention on that possibility.

[Tim Rentsch]

The other answers you got weren't especially informative,
so I'll take a shot at it.

First, I recommend forgetting what you think you know about
volatile. It may be right or it may be wrong, but either
way it isn't helping you understand what is happening here.

In a nutshell, what is happening is this. The variable 'tmp'
has a location in storage. The code generator notices that a
value is stored (from a register) into tmp, then later loaded
again (let's say into the same register). The optimizer trys
to improve the code by not bothering to load the value again,
since it's already there. This kind of optimization is very
common, and happens all the time with other variable types.
(Probably you know all this part already, but I thought I
would put it in for context.)

However, for floating point types, including in particular
the type double, the register is wider than the place in
memory where 'tmp' resides. So the register can retain
extra information which could not be retained if the value
were put into the 'tmp' memory location. So re-using the
value in the register can behave differently than storing
and reloading.

Normally the difference between these choices is small enough
so it usually doesn't matter. Moreoever, on the x86, the
cost of doing "the right thing" is high enough so there is a
signifcant motivation to "cut corners", as it were, especially
since "it hardly ever matters". That kind of reasoning is
what led gcc to make the decisions it did regarding how to
optimize this sort of code.

To return to your question -- the reason 'volatile' makes a
difference is that volatile INSISTS the compiler deal with the
actual memory location for 'tmp', no matter what the compiler
may think otherwise. There are various reasons for having
'volatile' in the language, and also various reasons for using
it in certain circumstances, but as far as the compiler goes
the primary consequence of a volatile access is this: the
generated code must reference the actual memory location
involved, no matter _what_ the compiler might think about
what other code might have "the same effect".

So that's why using 'volatile' forces the value to be (stored
and then) fetched from the variable rather than being kept in
a register.

[glen herrmannsfeldt]

Tim Rentsch <t...@alumni.caltech.edu> wrote:

(snip on keeping extra precision)

> There's an implication there (perhaps unintended) that discarding
> excess precision wasn't required until C99. That conclusion
> doesn't jibe with other, pre-C99, documents.

> In particular, if we look at comments in Defect Reports, and
> also how the wording in this area has changed over time (note
> for example n843.htm, and compare n1256 to C99), it's clear that
> narrowing behavior was intended all along.

> What's more, assignment has _always_ implied removing any extra
> range and precision, because in the abstract machine we actually
> store the value in an object, and then on subsequent accesses
> convert the stored representation to a value. Under the as-if
> rule, any information beyond what the object representation can
> represent must be lost.

I agree for -O0, and probably -O1, but at the higher optimization
levels one of the things you expect is to keep values in registers
throughout loops, and in general don't evaluate expressions more
often than necessary.

One idea behind optimization is that the compiler knows more than
the programmer, such that the programmer can write things in the
most readable form without worrying about how slow it is.

> Bottom line: assignment of floating point types always requires
> removing any extra range and precision, even in C90.

-- glen

[Tim Rentsch]

glen herrmannsfeldt <g...@ugcs.caltech.edu> writes:

> Tim Rentsch <t...@alumni.caltech.edu> wrote:
>
> (snip on keeping extra precision)
>
>> There's an implication there (perhaps unintended) that discarding
>> excess precision wasn't required until C99. That conclusion
>> doesn't jibe with other, pre-C99, documents.
>>
>> In particular, if we look at comments in Defect Reports, and
>> also how the wording in this area has changed over time (note
>> for example n843.htm, and compare n1256 to C99), it's clear that
>> narrowing behavior was intended all along.
>>
>> What's more, assignment has _always_ implied removing any extra
>> range and precision, because in the abstract machine we actually
>> store the value in an object, and then on subsequent accesses
>> convert the stored representation to a value. Under the as-if
>> rule, any information beyond what the object representation can
>> represent must be lost.
>
> I agree for -O0, and probably -O1, but at the higher optimization
> levels one of the things you expect is to keep values in registers
> throughout loops, and in general don't evaluate expressions more
> often than necessary.

Whatever you might expect, such optimizations (with wide
floating-point registers) are not allowed in conforming
implementations of ISO C, and that has been true since 1990.

> One idea behind optimization is that the compiler knows
> more than the programmer, such that the programmer can
> write things in the most readable form without worrying
> about how slow it is.

Even if the qualifying assumption is true, the reasoning is
irrelevant, because optimized code is still required to
observe the abstract semantics of the language. The
"optimization" you are talking about is not faithful to the
abstract semantics of ISO C, and ergo is not conforming.

[glen herrmannsfeldt]

Tim Rentsch <t...@alumni.caltech.edu> wrote:

(snip)

>>> What's more, assignment has _always_ implied removing any extra
>>> range and precision, because in the abstract machine we actually
>>> store the value in an object, and then on subsequent accesses
>>> convert the stored representation to a value. Under the as-if
>>> rule, any information beyond what the object representation can
>>> represent must be lost.

>> I agree for -O0, and probably -O1, but at the higher optimization
>> levels one of the things you expect is to keep values in registers
>> throughout loops, and in general don't evaluate expressions more
>> often than necessary.

> Whatever you might expect, such optimizations (with wide
> floating-point registers) are not allowed in conforming
> implementations of ISO C, and that has been true since 1990.

I suppose reordering statements is also disallowed by the
standard. Moving things outside loops, and such. Fine, then
don't use the high optimization modes.

>> One idea behind optimization is that the compiler knows
>> more than the programmer, such that the programmer can
>> write things in the most readable form without worrying
>> about how slow it is.

> Even if the qualifying assumption is true, the reasoning is
> irrelevant, because optimized code is still required to
> observe the abstract semantics of the language. The
> "optimization" you are talking about is not faithful to the
> abstract semantics of ISO C, and ergo is not conforming.

I first knew about optimizing compilers in Fortran, where they
were doing it since before C existed. Now, Fortran does seem
to be less restrictive on floating point, but much of it people
live with because the speed-up is worthwhile. If it reduces the
run time from six days to five days, people will do it.

Some people like to do scientific programming in C that would
otherwise be done in Fortran. They also expect reasonable optimization.

Seems to me that compilers should state that with -O3 that it doesn't
conform, and you have the choice to use it or not.

-- glen

[David Brown]

I don't know the ins and outs of the standards here, but the intention
is that gcc will follow the standards for floating point regardless of
the -O level, unless you specifically tell it that you are happy with
non-conforming behaviour through "-ffast-math" or related flags.
Optimisation level flags in themselves should never change the behaviour
of a program - just its size and speed. (The new "-Ofast" flag enables
-ffast-math, which I think is wrong in this way - even though "-Ofast"
is likely to be a good choice for most programs.)

[Stephen Sprunk]

On 22-Apr-13 02:48, David Brown wrote:
> I don't know the ins and outs of the standards here, but the
> intention is that gcc will follow the standards for floating point
> regardless of the -O level, unless you specifically tell it that you
> are happy with non-conforming behaviour through "-ffast-math" or
> related flags.

It has been known for a decade or two that GCC does this even without
-ffast-math, yet nobody has fixed it, which makes your statement of
intent suspect.

> Optimisation level flags in themselves should never change the
> behaviour of a program - just its size and speed.

It is also well-known that optimization often results in different
manifestations of undefined behavior. ITYM that optimization shouldn't
affect conformance, which I think we can all agree with.

S

--
Stephen Sprunk "God does not play dice." --Albert Einstein
CCIE #3723 "God is an inveterate gambler, and He throws the
K5SSS dice at every possible opportunity." --Stephen Hawking

[James Kuyper]

On 04/20/2013 06:26 PM, glen herrmannsfeldt wrote:
> Tim Rentsch <t...@alumni.caltech.edu> wrote:
>
> (snip on keeping extra precision)
>
>> There's an implication there (perhaps unintended) that discarding
>> excess precision wasn't required until C99. That conclusion
>> doesn't jibe with other, pre-C99, documents.

I've never owned a copy of C89; they didn't become cheap enough to
justify buying one until after I'd already switched to C99, and
therefore no longer needed a copy of C89. Therefore, I can't be sure
whether or not C89 can be read as permitting or prohibiting excess
precision. However, I'm fairly certain, from what I've read, that it did
not explicitly address the issue. That would explain why Tim has to use
such subtle arguments as he has, rather than being able to simply cite a
specific clause that would clearly be violated by retaining excess
precision.

I'd be interested in seeing an example of code that, when compiled by
gcc with -std=c99 (and without -fexcess_precision=fast), fails to
satisfy the C99 standard's very lax requirements on the accuracy of
floating point operations, due to incorrect use of excess precision. gcc
does not pre#define __STDC_IEC_559__, even when compiling for hardware
compliant with IEC 60559, thereby exempting it from all but the most
basic of the standard's accuracy requirements. I would be surprised if
gcc's developers mishandled any case that was simple enough to be
meaningfully constrained by such lax requirements.
--
James Kuyper

[David Brown]

On 22/04/13 14:45, Stephen Sprunk wrote:
> On 22-Apr-13 02:48, David Brown wrote:
>> I don't know the ins and outs of the standards here, but the
>> intention is that gcc will follow the standards for floating point
>> regardless of the -O level, unless you specifically tell it that you
>> are happy with non-conforming behaviour through "-ffast-math" or
>> related flags.
>
> It has been known for a decade or two that GCC does this even without
> -ffast-math, yet nobody has fixed it, which makes your statement of
> intent suspect.

Well, "intent" and "reality" can be two different things. And there are
certainly a few things in gcc that the developers would like to fix or
improve, but haven't done so - like every other program there are
priorities about what gets done.

But the documentation for the "-fexcess-precision" flag indicates that
the intention is to follow the standards when you have specified strict
conformance.


In the case of the standard's requirements about precision, I expect it
would be a significant effort trying to follow these requirements while
still generating reasonable code. The only way that would work reliably
on all hardware that supports higher internal precision is to have lots
of extra stores to memory then re-load the values. Obviously this is
going to mean a serious hit on floating point performance.

So what you have here is a standards requirement to produce poorer
quality results that means more work for the compiler to generate bigger
and slower code. You can argue that following the standards gives a bit
more consistency in results across architectures, but as far as I
understand it the IEEE specs for floating point do not require
bit-perfect repeatability in all cases - so even with full standards
conformance you do not have 100% consistency.

In my opinion (which may not count for much, as I seldom have need for
high precision floating point work), code that relies on limiting
precision or "unreliable" operations such as floating point equality
tests, is risky and should be avoided. If you need such features, you
are better of using a library such as gnu mpfr (of which I have no
experience).

>
>> Optimisation level flags in themselves should never change the
>> behaviour of a program - just its size and speed.
>
> It is also well-known that optimization often results in different
> manifestations of undefined behavior. ITYM that optimization shouldn't
> affect conformance, which I think we can all agree with.
>
> S
>

Undefined behaviour is, by definition, undefined. It should not be a
surprise if the actual effect varies according to optimisation level -
it should not be a surprise if it varies from run to run, or in any
other way.

But we do agree that optimisation levels should not affect the
observable behaviour of well-defined code.

[mathog]

Tim Rentsch wrote:
> There are various reasons for having
> 'volatile' in the language, and also various reasons for using
> it in certain circumstances, but as far as the compiler goes
> the primary consequence of a volatile access is this: the
> generated code must reference the actual memory location
> involved, no matter _what_ the compiler might think about
> what other code might have "the same effect".

That was the explanation I was after.

Thanks,

David Mathog

[James Kuyper]

On 04/22/2013 11:25 AM, David Brown wrote:
> On 22/04/13 14:45, Stephen Sprunk wrote:
>> On 22-Apr-13 02:48, David Brown wrote:

...

>>> Optimisation level flags in themselves should never change the
>>> behaviour of a program - just its size and speed.
>>
>> It is also well-known that optimization often results in different
>> manifestations of undefined behavior. ITYM that optimization shouldn't
>> affect conformance, which I think we can all agree with.
>>
>> S
>>
>
> Undefined behaviour is, by definition, undefined. It should not be a
> surprise if the actual effect varies according to optimisation level -
> it should not be a surprise if it varies from run to run, or in any
> other way.
>
> But we do agree that optimisation levels should not affect the
> observable behaviour of well-defined code.

In general, all unspecified behavior can change with optimization level
- which is the whole point, since one key thing that the standards
leaves unspecified is the speed with which your code executes. However,
unspecified behavior also includes implementation-defined behavior.
There's a lot of room for unexpected consequences, without having to
invoke undefined behavior. In principle, INT_MAX could change with
optimization level. This all follows from the simple fact that the
standard doesn't talk about optimization levels.
--
James Kuyper

[Tim Rentsch]

glen herrmannsfeldt <g...@ugcs.caltech.edu> writes:

> Tim Rentsch <t...@alumni.caltech.edu> wrote:
>
> (snip)
>>>> What's more, assignment has _always_ implied removing any extra
>>>> range and precision, because in the abstract machine we actually
>>>> store the value in an object, and then on subsequent accesses
>>>> convert the stored representation to a value. Under the as-if
>>>> rule, any information beyond what the object representation can
>>>> represent must be lost.
>>>
>>> I agree for -O0, and probably -O1, but at the higher optimization
>>> levels one of the things you expect is to keep values in registers
>>> throughout loops, and in general don't evaluate expressions more
>>> often than necessary.
>>
>> Whatever you might expect, such optimizations (with wide
>> floating-point registers) are not allowed in conforming
>> implementations of ISO C, and that has been true since 1990.
>
> I suppose reordering statements is also disallowed by the
> standard. Moving things outside loops, and such. Fine, then
> don't use the high optimization modes.

Rather than opine in ignorance, why not read what the language
definition actually admits in this regard?

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf
http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf

Not official standard documents, but either is close enough
to give accurate answers about source transformations.

>>> One idea behind optimization is that the compiler knows
>>> more than the programmer, such that the programmer can
>>> write things in the most readable form without worrying
>>> about how slow it is.
>>
>> Even if the qualifying assumption is true, the reasoning is
>> irrelevant, because optimized code is still required to
>> observe the abstract semantics of the language. The
>> "optimization" you are talking about is not faithful to the
>> abstract semantics of ISO C, and ergo is not conforming.
>
> I first knew about optimizing compilers in Fortran, where they
> were doing it since before C existed.

The original Fortran didn't have a defining document. Whatever
the compiler did was how the language worked. The orginal C was
that way too, and compounded by the lack of a standard, because
of the variety of choices made by different implementations. If
you yearn for "the good old days", you might read Fred Brooks's
comments in The Mythical Man-Month about System/360 and its
architectural definition, contrasted with the previous generation
of IBM machines.

> Now, Fortran does seem to be less restrictive on floating
> point, but much of it people live with because the speed-up is
> worthwhile. If it reduces the run time from six days to five
> days, people will do it.

What you mean is some people will do it. No doubt some will.
Others want more reliable guarantees.

> Some people like to do scientific programming in C that would
> otherwise be done in Fortran. They also expect reasonable
> optimization.

Another over-generalization. Plus you're implying that what
optimization ISO C does allow is not "reasonable". It seems
more likely that your comment springs from ignorance of just
what ISO C does allow, and also why.

> Seems to me that compilers should state that with -O3 that it
> doesn't conform, and you have the choice to use it or not.

As it is stated, this suggestion is hopelessly naive.

[Tim Rentsch]

James Kuyper <james...@verizon.net> writes:

> On 04/20/2013 06:26 PM, glen herrmannsfeldt wrote:
>> Tim Rentsch <t...@alumni.caltech.edu> wrote:
>>
>> (snip on keeping extra precision)
>>
>>> There's an implication there (perhaps unintended) that discarding
>>> excess precision wasn't required until C99. That conclusion
>>> doesn't jibe with other, pre-C99, documents.
>
> I've never owned a copy of C89; they didn't become cheap enough
> to justify buying one until after I'd already switched to C99, and
> therefore no longer needed a copy of C89. Therefore, I can't be
> sure whether or not C89 can be read as permitting or prohibiting
> excess precision. However, I'm fairly certain, from what I've
> read, that it did not explicitly address the issue. That would
> explain why Tim has to use such subtle arguments as he has, rather
> than being able to simply cite a specific clause that would

> clearly be violated by retaining excess precision. [snip]

A few comments..

C90 is explicit in identifying operations that behave differently
with respect to extra range and precision. What it is not is clear
in spelling out the consequences of that distinction in plain,
impossible-to-misunderstand language of the kind that M. Kuyper
prefers.

Second, part of my comments were complicated by having to look at
other documents, because a larger issue is being addressed, namely,
the general question of conversion, not just storage. Here the C90
document is more ambiguous, and to produce support for the general
statement I think other sources need to be looked at besides just
the C90 document. In that sense I agree with the statements above,
at least in spirit, for the issue of conversion generally. But
only for that, not for what happens when storing during assignment.

Third, specifically with regard to storing and accessing, actually
my reasoning there was more subtle than it needed to be. Only
values and the results of expressions fall under the umbrella of
possibly having extra range and precision; objects do not. Because
storing into an object is not part of the universe of cases that
might have extra range and precision, it simply cannot happen,
and there is no reason to redundantly exclude it.

[Tim Rentsch]

David Brown <da...@westcontrol.removethisbit.com> writes:

> On 22/04/13 14:45, Stephen Sprunk wrote:
>> On 22-Apr-13 02:48, David Brown wrote:
>>> I don't know the ins and outs of the standards here, but the
>>> intention is that gcc will follow the standards for floating point
>>> regardless of the -O level, unless you specifically tell it that you
>>> are happy with non-conforming behaviour through "-ffast-math" or
>>> related flags.
>>
>> It has been known for a decade or two that GCC does this even without
>> -ffast-math, yet nobody has fixed it, which makes your statement of
>> intent suspect.
>

> [snip]

>
> In the case of the standard's requirements about precision, I expect
> it would be a significant effort trying to follow these requirements
> while still generating reasonable code. The only way that would work
> reliably on all hardware that supports higher internal precision is to
> have lots of extra stores to memory then re-load the values.
> Obviously this is going to mean a serious hit on floating point
> performance.

These comments seem overly simplistic. Given everything else a
good optimizing compiler has to do, putting in a few extra forced
register spills shouldn't be very hard. Furthermore extra stores
would be necessary only on processors that don't have a suitable
rounding instruction -- any serious compiler that targets multiple
platforms will do processor-specific code generation. The idea
that the performance cost will be "serious" may not even be true,
let alone obvious. I think you're assuming that the cost of doing
the data movement is what will dominate, but I don't think that is
necessarily true; local caches cut the cost of data movement way
down, and the rounding operation itself is not a trivial operation
(mainly I think because some edge cases need some care).

Also it's worth pointing out that when these things (necessarily)
happen is subject to programmer control, by changing how the code
in question is written.

> So what you have here is a standards requirement to produce poorer
> quality results that means more work for the compiler to generate
> bigger and slower code. You can argue that following the standards
> gives a bit more consistency in results across architectures, but as
> far as I understand it the IEEE specs for floating point do not
> require bit-perfect repeatability in all cases - so even with full
> standards conformance you do not have 100% consistency.

I think you're operating from a couple of false assumptions here.
One is that enforcing a standard precision (as opposed to some
unknown "extended" precision) on assignment necessarily gives
poorer quality results. The other is that conformance to floating
point standards is motivated (primarily) by reproducibility. Often
times it is not nearly as important that calculations "do the right
thing" as that they "do NOT do the wrong thing". The rule for
assignment may not produce a more _accurate_ result, but I believe
it produces a more _dependable_ result.

> In my opinion (which may not count for much, as I seldom have need
> for high precision floating point work), code that relies on
> limiting precision or "unreliable" operations such as floating point
> equality tests, is risky and should be avoided. If you need such
> features, you are better of using a library such as gnu mpfr (of
> which I have no experience).

I think you've learned the wrong lesson. Operations on floating
point values don't behave the same way as they do in mathematics
on real values, but they do have rules for what behavior(s) are
allowed. The people who decided how these things should work
aren't stupid people: the rules are not arbitrary but are there
to allow computation that is fast, accurate, and dependable. To
get that, however, it's better to learn the rules, and work within
that framework, rather than treating floating point operations as
inherently mysteriously fuzzy.

>>> Optimisation level flags in themselves should never change the
>>> behaviour of a program - just its size and speed.
>>
>> It is also well-known that optimization often results in different
>> manifestations of undefined behavior. ITYM that optimization
>> shouldn't affect conformance, which I think we can all agree with.
>

> Undefined behaviour is, by definition, undefined. It should not be a
> surprise if the actual effect varies according to optimisation level -
> it should not be a surprise if it varies from run to run, or in any
> other way.
>
> But we do agree that optimisation levels should not affect the
> observable behaviour of well-defined code.

The comment about conformance is slightly different, and in
particular a slightly stronger statement. That is so because
defined-ness is not a binary condition. Do you see that?

[David Brown]

Yes, I can see that difference. And I have also had it pointed out to
me that there are things defined by the standards, things declared as
undefined, and things that are "implementation-defined" - i.e., they are
well-defined and the compiler must be consistent in generating correct
code for them, but the standards don't give all the details of /how/
they are defined.


Regarding floating point, I realise the IEEE folks made the rules the
way they are for reasons (presumably, but not necessarily, for good
technical reasons). And I understand that for some types of work, it is
more important that an implementation follows a given set of rules
exactly than what these rules actually are.

But for other types of work, you do want to treat floating point as
"mysteriously fuzzy". It's a "best effort" sort of thing. You don't
care about NaNs, denormals, rounding modes, etc. You just want the best
approximation you can reasonably get at reasonable speed, and you accept
that there can be rounding errors on the last couple of digits on big
calculations. When you want more accurate results, you switch to "long
doubles", when you want faster results you use "single" (depending on
the platform, of course). You get used to oddities like "2.0 * 0.5"
being not quite equal to "1.0". And if you want to do complex work
where rounding, order of calculations, etc., can have a significant
effect (such as inverting a large matrix), you use a pre-written library
by someone who understands these details.

I believe that most floating point code, and most programmers, fall into
this category - and would get better results with "-ffast-math" than
with strict IEEE conformance.

But of course I also think the compiler should be able to produce code
that follows the C and IEEE rules as closely as possible.

The question is what should the compiler do by default, in the absence
of explicit flags? Should it make "fast and easy" code to suit most
uses? Should it follow the C standards as closely as possible? Should
it find a reasonable middle ground, and make everyone equally unhappy?

[Tim Rentsch]

James Kuyper <james...@verizon.net> writes:

> On 04/22/2013 11:25 AM, David Brown wrote:
>> On 22/04/13 14:45, Stephen Sprunk wrote:
>>> On 22-Apr-13 02:48, David Brown wrote:
> ...
>>>> Optimisation level flags in themselves should never change the
>>>> behaviour of a program - just its size and speed.
>>>
>>> It is also well-known that optimization often results in different
>>> manifestations of undefined behavior. ITYM that optimization shouldn't
>>> affect conformance, which I think we can all agree with.
>>

>> Undefined behaviour is, by definition, undefined. It should not be a
>> surprise if the actual effect varies according to optimisation level -
>> it should not be a surprise if it varies from run to run, or in any
>> other way.
>>
>> But we do agree that optimisation levels should not affect the
>> observable behaviour of well-defined code.
>
> In general, all unspecified behavior can change with optimization
> level - which is the whole point, since one key thing that the
> standards leaves unspecified is the speed with which your code
> executes. However, unspecified behavior also includes
> implementation-defined behavior. There's a lot of room for
> unexpected consequences, without having to invoke undefined
> behavior. In principle, INT_MAX could change with optimization
> level. This all follows from the simple fact that the standard
> doesn't talk about optimization levels.

The comment about changing INT_MAX is silly. INT_MAX must be
constant across a given implementation: a different value of
INT_MAX means a different implementation. Certainly a compiler
could take an "optimization" flag to mean a different value of
INT_MAX and become a different implementation, but that is no
different from saying it could take an "optimization" flag and
choose to compile Pascal rather than C. Neither one has anything
to do with variability due to implementation-defined behavior.
Unspecified (but not implementation-defined) behavior may change
within an implementation. Implementation-defined behavior may
change between implementations, but not within a particular
implementation.

[Tim Rentsch]

David Brown <da...@westcontrol.removethisbit.com> writes:

> On 25/04/13 07:58, Tim Rentsch wrote:
>> David Brown <da...@westcontrol.removethisbit.com> writes:

>>> [snip]

>>>
>>> But we do agree that optimisation levels should not affect the
>>> observable behaviour of well-defined code.
>>
>> The comment about conformance is slightly different, and in
>> particular a slightly stronger statement. That is so because
>> defined-ness is not a binary condition. Do you see that?
>
> Yes, I can see that difference. And I have also had it pointed out to
> me that there are things defined by the standards, things declared as
> undefined, and things that are "implementation-defined" - i.e., they are
> well-defined and the compiler must be consistent in generating correct
> code for them, but the standards don't give all the details of /how/
> they are defined.

I don't know how to make sense of the last clause there. Behavior
of C programs (as delineated by the ISO standard) falls into one of
four categories: undefined, unspecified, implementation-defined,
defined. Undefined behavior imposes no requirements. Defined
behavior is completely specified. Implementation-defined behavior
means the implementation must make a choice from a set of allowed
(and defined) behaviors, and then stick to that choice. Unspecified
behavior means there is a set of allowed, and defined behaviors,
and any of those behaviors may be adopted in any given instance.
Do you mean the ISO standard specifies only behavior, not how the
behavior is to be carried out? That is true, but that's the point
of language definition - to define the behavior of programs, not
how that behavior will be actualized.

(I snipped most of the earlier context, thinking it's better
to respond to these comments standalone.)

> Regarding floating point, I realise the IEEE folks made the rules
> the way they are for reasons (presumably, but not necessarily, for
> good technical reasons). And I understand that for some types of
> work, it is more important that an implementation follows a given
> set of rules exactly than what these rules actually are.
>
> But for other types of work, you do want to treat floating point
> as "mysteriously fuzzy". It's a "best effort" sort of thing. You
> don't care about NaNs, denormals, rounding modes, etc. You just
> want the best approximation you can reasonably get at reasonable
> speed, and you accept that there can be rounding errors on the
> last couple of digits on big calculations. When you want more
> accurate results, you switch to "long doubles", when you want
> faster results you use "single" (depending on the platform, of
> course). You get used to oddities like "2.0 * 0.5" being not
> quite equal to "1.0". And if you want to do complex work where
> rounding, order of calculations, etc., can have a significant
> effect (such as inverting a large matrix), you use a pre-written
> library by someone who understands these details.
>
> I believe that most floating point code, and most programmers,
> fall into this category - and would get better results with
> "-ffast-math" than with strict IEEE conformance.

I understand what you're saying. I don't agree with it. More to
the point though, you haven't offered any kind of evidence or
supporting arguments for why anyone else should agree with it.
Are you offering anything more than an opinion? Why should
anyone accept your opinion of how floating point should behave
over, say, the opinions of the IEEE 754 group? To offer an
analogy, your views sound like saying novice drivers shouldn't
bother with seatbelts. On the contrary, it is only people who
know what they are doing who should work outside what are thought
(by experts) to be best practices.

> But of course I also think the compiler should be able to produce
> code that follows the C and IEEE rules as closely as possible.
>
> The question is what should the compiler do by default, in the
> absence of explicit flags? Should it make "fast and easy" code to
> suit most uses? Should it follow the C standards as closely as
> possible? Should it find a reasonable middle ground, and make
> everyone equally unhappy?

Most developers are better served by compilers that are completely
conforming rather than partially conforming. For floating-point,
the benefits of more portable and more reliable behavior far
outweigh the difference in performance or accuracy (taking 64-bit
doubles and 80-bit internals as representative). Don't take my
word for it -- I encourage anyone who is interested to do some
experiments measuring how different compiler options affect
accuracy and/or performance. (It's much harder to measure the
costs of using using non-standard behavior. Often though the
effects in such cases are closer to "catastrophic" than "minor".)

Consequent to the above, I believe the community is better served
by compilers that are completely conforming -- not partially
conforming -- in their default mode. If someone wants to work
outside the standard language definition, then by all means, more
power to them, and I'm all in favor of compiler options to give a
variety of non-standard choices. But for the community as a
whole, it's better if the choice of using non-standard behavior
is the exception, not the rule.

[glen herrmannsfeldt]

Tim Rentsch <t...@alumni.caltech.edu> wrote:
> David Brown <da...@westcontrol.removethisbit.com> writes:

>> On 25/04/13 07:58, Tim Rentsch wrote:

(snip)

Cray sold many machines for many millions of dollars that gave
approximate answers to many floating point operations.

The even sold one that had a non-commutative floating point multiply.

Now, I suppose if you add up the value of all the IEEE 754 compatible
machines it will total more than all the Cray machines, but how many
are actually used for floating point number crunching?

> Why should
> anyone accept your opinion of how floating point should behave
> over, say, the opinions of the IEEE 754 group? To offer an
> analogy, your views sound like saying novice drivers shouldn't
> bother with seatbelts. On the contrary, it is only people who
> know what they are doing who should work outside what are thought
> (by experts) to be best practices.

OK, how about the opinion of the market? See what people actually
buy, and assume that is related to what they want.

(snip)

-- glen

[Tim Rentsch]

Irrelevant to the point under discussion, which has to do with
conformance to the C standard, not IEEE 754 specifically (that
is an example, but only an example).

>> Why should
>> anyone accept your opinion of how floating point should behave
>> over, say, the opinions of the IEEE 754 group? To offer an
>> analogy, your views sound like saying novice drivers shouldn't
>> bother with seatbelts. On the contrary, it is only people who
>> know what they are doing who should work outside what are thought
>> (by experts) to be best practices.
>
> OK, how about the opinion of the market? See what people actually
> buy, and assume that is related to what they want.

Based on that we should all use the Microsoft dog**** compilers.
No thanks.

[glen herrmannsfeldt]

Tim Rentsch <t...@alumni.caltech.edu> wrote:

(snip)

>>> I understand what you're saying. I don't agree with it. More to
>>> the point though, you haven't offered any kind of evidence or
>>> supporting arguments for why anyone else should agree with it.
>>> Are you offering anything more than an opinion?

(snip, then I wrote)

>> Cray sold many machines for many millions of dollars that gave
>> approximate answers to many floating point operations.

>> The even sold one that had a non-commutative floating point multiply.

>> Now, I suppose if you add up the value of all the IEEE 754 compatible
>> machines it will total more than all the Cray machines, but how many
>> are actually used for floating point number crunching?

> Irrelevant to the point under discussion, which has to do with
> conformance to the C standard, not IEEE 754 specifically (that
> is an example, but only an example).

It isn't just that they aren't IEEE 754.

http://ed-thelen.org/comp-hist/CRAY-1-HardRefMan/CRAY-1-HRM.html

read especially about the floating point multiply and reciprocal
approximation. (There is no divide.)

Instead of generating the full product and rounding, they only generate
part of the product and round. Reasonably often there is no carry from
the part not generated, but not always.

As the product term truncation is not symmetrical, mutliply might not
be commutative.

(There is only 64 bit single precision with a 48 bit significand in
hardware. Double precision with 95 bits is available in software.)

Reciprocal approximate is accurate to 30 bits. An additional refinement
step gets to 47 bits. As that is one less than the 48 bits available, it
seems that the last bit is sometimes wrong.

-- glen

[Tim Rentsch]

glen herrmannsfeldt <g...@ugcs.caltech.edu> writes:

> Tim Rentsch <t...@alumni.caltech.edu> wrote:
>
> (snip)
>>>> I understand what you're saying. I don't agree with it. More to
>>>> the point though, you haven't offered any kind of evidence or
>>>> supporting arguments for why anyone else should agree with it.
>>>> Are you offering anything more than an opinion?
>
> (snip, then I wrote)
>
>>> Cray sold many machines for many millions of dollars that gave
>>> approximate answers to many floating point operations.
>>>
>>> The even sold one that had a non-commutative floating point multiply.
>>>
>>> Now, I suppose if you add up the value of all the IEEE 754 compatible
>>> machines it will total more than all the Cray machines, but how many
>>> are actually used for floating point number crunching?
>
>> Irrelevant to the point under discussion, which has to do with
>> conformance to the C standard, not IEEE 754 specifically (that
>> is an example, but only an example).
>
> It isn't just that they aren't IEEE 754.
>
> http://ed-thelen.org/comp-hist/CRAY-1-HardRefMan/CRAY-1-HRM.html
>
> read especially about the floating point multiply and reciprocal

> approximation. (There is no divide.) [snip elaboration]

Still irrelevant to the point under discussion: what matters is
not what the hardware does, but what C compilers do.

Incidentally, the Cray-1 came out before K&R was published,
which is to say 14 years before the first ISO C standard.

The Cray-1 was a marvel of design and floating-point performance
when it came out. But it isn't just a coincidence this approach
to doing floating-point has fallen by the wayside.

[Seebs]

On 2013-04-20, Tim Rentsch <t...@alumni.caltech.edu> wrote:
> However, for floating point types, including in particular
> the type double, the register is wider than the place in
> memory where 'tmp' resides. So the register can retain
> extra information which could not be retained if the value
> were put into the 'tmp' memory location. So re-using the
> value in the register can behave differently than storing
> and reloading.

There was a really lovely example of this I encountered recently. Someone
had come up with a test case which failed only on one or two CPUs at a
particular optimization level.

It ultimately turned out that the problem was that a test was computing
a value that was roughly equal to x / (x * 20) for some smallish x, thus, very
close to 0.05, then multiplying it by 200, which might or might not yield
10. Then this value was subtracted from 15 and converted to int. The result
was usually 5, but occasionally 4. The problem, it turns out, is that the
value was more than .05 by an amount small enough that it got rounded away
by the store.

So if you changed it from 15 - N == 5 to 25 - N == 15, it "worked". And I
suspect that if the test case had been changed from 15 - N == 5 to 10 - N ==
0, it would have "failed" on many more systems.

But fundamentally, the test case was broken. This was completely obvious
once I'd spent an hour or so studying it.

-s
--
Copyright 2013, all wrongs reversed. Peter Seebach / usenet...@seebs.net
http://www.seebs.net/log/ <-- lawsuits, religion, and funny pictures
http://en.wikipedia.org/wiki/Fair_Game_(Scientology) <-- get educated!
I am not speaking for my employer, although they do rent some of my opinions.

[David Brown]

Thanks for your clarification of the terms. I think my biggest
misunderstanding here was that I thought compilers made more leniency
with "implementation-defined" behaviour. And re-reading what I wrote, I
can see I did not express myself very well.

I offer only my opinion here - I have no direct evidence. It's an
opinion on what I think applies to many programs, not just my own, but I
have done no research to back it up. So if you think it is not even
worth the traditional 2 cents, that's fair enough. This is a Usenet
group - exchanging opinions is part of what goes on here, and I don't
expect anyone to read more into it than that. You and anyone else can
agree with it or disagree with it, and anyone from the IEEE 754 group
listening in can dismiss it or embrace it.

I agree on what you say about experts. /I/ can choose to use
"-ffast-math" with my code, and I can offer that as an /opinion/ to
other people. But you would not want me to decide on the default policy
on gcc compiler flags!

>> But of course I also think the compiler should be able to produce
>> code that follows the C and IEEE rules as closely as possible.
>>
>> The question is what should the compiler do by default, in the
>> absence of explicit flags? Should it make "fast and easy" code to
>> suit most uses? Should it follow the C standards as closely as
>> possible? Should it find a reasonable middle ground, and make
>> everyone equally unhappy?
>
> Most developers are better served by compilers that are completely
> conforming rather than partially conforming. For floating-point,
> the benefits of more portable and more reliable behavior far
> outweigh the difference in performance or accuracy (taking 64-bit
> doubles and 80-bit internals as representative). Don't take my
> word for it -- I encourage anyone who is interested to do some
> experiments measuring how different compiler options affect
> accuracy and/or performance. (It's much harder to measure the
> costs of using using non-standard behavior. Often though the
> effects in such cases are closer to "catastrophic" than "minor".)
>

This all depends on the type of program and the type of target in
question. For the sorts of systems I work with, /enforcing/ conforming
behaviour in floating point work would be catastrophic. I work on
embedded systems, and don't often use floating point. But when I do use
it, speed and code size is vital - following IEEE standards regarding
rounding, or support for NaNs, or matching the target accuracy for
irrational functions, is all unnecessary. Many microcontrollers have no
hardware floating point support at all, and many that do only support a
limited subset. So as long as the compiler can assume that floats "x"
and "y" are normal numbers, "x * y" might be done in a single cycle -
but if it has to check for NaNs and signed zeros, it's a library call
and a hundred times slower.


For more mainstream programming, I think it is common to treat floating
point as approximate numbers with a wide dynamic range. For such
applications, the differences in functionality between strictly
conforming floating point and "-ffast-math" are going to be very minor -
unless there are a lot of calculations, in which case the speed
difference might be relevant. (But as noted above, this is just an
opinion.)

Obviously there are other types of application for which non-conforming
behaviour could be catastrophic. If there were no need of the full set
of IEEE floating point rules, then the rules would not be there in the
first place.

[Tim Rentsch]

David Brown <da...@westcontrol.removethisbit.com> writes:

> On 25/04/13 19:49, Tim Rentsch wrote:
>> David Brown <da...@westcontrol.removethisbit.com> writes:
>>
>>> On 25/04/13 07:58, Tim Rentsch wrote:
>>>> David Brown <da...@westcontrol.removethisbit.com> writes:

>>>>> [substantial snipping]
>>
>> Are you offering anything more than an opinion? [snip]

>
> I offer only my opinion here - I have no direct evidence.

> [snip elaboration]

I'm all for people offering their opinions (distinguished as
such). I do so myself. But people who offer only their opinion,
and nothing in the way of evidence or some sort of supporting
reasoning, just aren't very interesting, because they don't
really contribute anything to the discussion.

> I agree on what you say about experts. /I/ can choose to use
> "-ffast-math" with my code, and I can offer that as an
> /opinion/ to other people. But you would not want me to decide
> on the default policy on gcc compiler flags!

I'm pretty sure I agree with that last sentence. :)

>>> But of course I also think the compiler should be able to produce
>>> code that follows the C and IEEE rules as closely as possible.
>>>
>>> The question is what should the compiler do by default, in the
>>> absence of explicit flags? Should it make "fast and easy" code to
>>> suit most uses? Should it follow the C standards as closely as
>>> possible? Should it find a reasonable middle ground, and make
>>> everyone equally unhappy?
>>
>> Most developers are better served by compilers that are completely
>> conforming rather than partially conforming. For floating-point,
>> the benefits of more portable and more reliable behavior far
>> outweigh the difference in performance or accuracy (taking 64-bit
>> doubles and 80-bit internals as representative). Don't take my
>> word for it -- I encourage anyone who is interested to do some
>> experiments measuring how different compiler options affect
>> accuracy and/or performance. (It's much harder to measure the
>> costs of using using non-standard behavior. Often though the
>> effects in such cases are closer to "catastrophic" than "minor".)
>
> This all depends on the type of program and the type of target in
> question. For the sorts of systems I work with, /enforcing/
> conforming behaviour in floating point work would be catastrophic.
> I work on embedded systems, and don't often use floating point. But
> when I do use it, speed and code size is vital - following IEEE

> standards regarding rounding, [snip elaboration]

You are using 'conforming' here in a way that makes me think you
don't understand the different ways the Standard uses the term.
A conforming _implementation_ is one that follows all the rules
laid out by the ISO standard. These rules allow great latitude;
they don't require IEEE floating point, for example. There are
two uses of 'conforming' as applied to programs. There is a very
restricted class of programs called 'strictly conforming'. A C
program being strictly conforming basically means it will produce
identical output on ANY conforming implementation. This class is
very narrow. In fact it is so narrow that I'm not sure it is
even possible to write a strictly conforming program that uses
floating-point in any non-trivial way, and if it is possible it
certainly isn't easy. At the other end of the spectrum is a
'conforming' program (ie, without the 'strictly' adverb). A
conforming program is one that is accepted by SOME conforming
implemention (not all, but at least one). This class is very
wide, and admits things that look almost nothing like C programs,
to say nothing about what their outputs might be. For example,
the program source

#define __PASCAL__ 1

... here the source continues written in the Pascal
programming language ...

could be a conforming C program. (It isn't, because no conforming
implementation accepts it, but an implementation could be written
that accepts it, and still is a conforming implementation.)

My comment above is about conforming _implementations_. Your
comment is about _programs_. These two things (ignoring the
semantic glitch around "conforming") are not incompatible.
For example, your program source could have

#pragma floating point ala David Brown

and then do floating-point operations however you want, yet still
be within the bounds of what a conforming implementation is allowed
to do. Do you see how important this difference is? My comment is
about the behavior of implementations, not the programs they
translate; we can use only conforming implementations without
being forced to use one floating-point system or another, or even
_any_ pre-defined set of floating-point systems.

> For more mainstream programming, I think it is common to treat
> floating point as approximate numbers with a wide dynamic
> range. For such applications, the differences in functionality
> between strictly conforming floating point and "-ffast-math"
> are going to be very minor - unless there are a lot of
> calculations, in which case the speed difference might be
> relevant. (But as noted above, this is just an opinion.)

Here again I think you are confusing the ideas of an implementation
being conforming and a program being "conforming". Depending on IEEE
floating-point (not counting depending on it in a way that makes no
difference) is guaranteed to make a program NOT be 'strictly
conforming", as the Standard defines the term. An implementation can
be conforming yet still offer a wide variety of floating-point
systems.

> Obviously there are other types of application for which
> non-conforming behaviour could be catastrophic. If there were
> no need of the full set of IEEE floating point rules, then the
> rules would not be there in the first place.

>> Consequent to the above, I believe the community is better served
>> by compilers that are completely conforming -- not partially
>> conforming -- in their default mode. If someone wants to work
>> outside the standard language definition, then by all means, more
>> power to them, and I'm all in favor of compiler options to give a
>> variety of non-standard choices. But for the community as a
>> whole, it's better if the choice of using non-standard behavior
>> is the exception, not the rule.

Now that I've written out this more careful differentiation of
how "conforming" is used, it might be helpful to go back and
re-read the earlier comments. Probably that will clear up
most of the confusion but if some further thoughts come up
I'd be interested to hear them.

[Tim Rentsch]

Seebs <usenet...@seebs.net> writes:

> On 2013-04-20, Tim Rentsch <t...@alumni.caltech.edu> wrote:
>> However, for floating point types, including in particular
>> the type double, the register is wider than the place in
>> memory where 'tmp' resides. So the register can retain
>> extra information which could not be retained if the value
>> were put into the 'tmp' memory location. So re-using the
>> value in the register can behave differently than storing
>> and reloading.
>
> There was a really lovely example of this I encountered recently.
> Someone had come up with a test case which failed only on one or

> two CPUs at a particular optimization level. [...]

Nice example -- thanks.

[88888 Dihedral]

glen herrmannsfeldt於 2013年4月19日星期五UTC+8上午7時27分39秒寫道：
> Tim Rentsch <t...@alumni.caltech.edu> wrote:
>
> > gordon...@burditt.org (Gordon Burditt) writes:
>
>
>
> >>> So, what sort of "optimization" is it that gives the wrong result for
>
> >>> the "<=" condition when *ymax and tmp are equal????
>
>
>
> >> An optimization that keeps excess precision in a calculation.
>
>
>
> >> A typical floating point unit uses registers that are 80 bits wide,
>
> >> and only narrows it down to 64 bits when you store in a double.
>
> >> The code may not bother storing the result, then re-loading it
>
> >> before doing a comparison. [snip]
>
>
>
> > Only in non-conforming implementations. What happens in a
>
> > conforming implementation must be as if the value were actually
>
> > stored and re-fetched, and the semantics of assignment imcludes
>
> > removing any extra range and precision (footnote, 6.3.1.8 p2).
>
> > What gcc does is not a legitimate optimization but non-conforming
>
> > behavior.
>
>
>
> I believe that is correct. Fortran is more lenient in what it allows
>
> for floating point expression evaluation. But storing every intermediate
>
> value can slow things down a lot!
>
>
>
> Well, they should go to cache instead of all the way to main memory,
>
> but it is still slow.
>
>
>
> Does gcc always store intermediate values with -O0?
>
>
>
> -- glen

The C-language standard does specify the requirements
of the precisions of floating numbers and doubles.

In arm or mips the doubles are not the same as the 80 bits
in the Intel format which has to ensure the backward
compatibility since 199X.

Bounding floating point rounding errors in several
operations in C programs requires some disciplines.

This is not an easy problem.

[David Brown]

I think when I have used the term "conforming", I have meant "strictly
conforming" (and usually without making clear distinction between
compiler implementations and programs).

But it strikes me that "strictly conforming" is so tight that it is
close to useless, if you can't write real-world programs that are
strictly conforming. Certainly you can't do anything useful in the
embedded world (where I work) while remaining strictly conforming -
there are always parts of the code that are specific to the target
and/or particular compiler, so that on other conforming compilers you
would get compiler errors or at least different run-time behaviour.

On the other hand, "conforming" is so lose that it is close to useless.
If I write a compiler that accepts Pascal as well as C, but is
otherwise a conforming C compiler, then Pascal programs suddenly become
conforming C code.

Isn't this a bit black-or-white? I suppose that "conforming" and
"strictly conforming" are still useful in discussions, even if they
don't match real-world tools and programs 100% - I just need to be more
careful in the terms I use.

> My comment above is about conforming _implementations_. Your
> comment is about _programs_. These two things (ignoring the
> semantic glitch around "conforming") are not incompatible.
> For example, your program source could have
>
> #pragma floating point ala David Brown
>
> and then do floating-point operations however you want, yet still
> be within the bounds of what a conforming implementation is allowed
> to do. Do you see how important this difference is? My comment is
> about the behavior of implementations, not the programs they
> translate; we can use only conforming implementations without
> being forced to use one floating-point system or another, or even
> _any_ pre-defined set of floating-point systems.

I can't honestly say that I am entirely clear on this - but I am getting
there... At the very least, I have a better understanding of what I am
missing, and what I've been getting wrong.

>
>> For more mainstream programming, I think it is common to treat
>> floating point as approximate numbers with a wide dynamic
>> range. For such applications, the differences in functionality
>> between strictly conforming floating point and "-ffast-math"
>> are going to be very minor - unless there are a lot of
>> calculations, in which case the speed difference might be
>> relevant. (But as noted above, this is just an opinion.)
>
> Here again I think you are confusing the ideas of an implementation
> being conforming and a program being "conforming". Depending on IEEE
> floating-point (not counting depending on it in a way that makes no
> difference) is guaranteed to make a program NOT be 'strictly
> conforming", as the Standard defines the term. An implementation can
> be conforming yet still offer a wide variety of floating-point
> systems.

This is one of my mistakes. Basically, I thought an implementation
/had/ to follow IEEE fully (including all the awkward bits like NaNs) to
be "conforming". But I suppose the requirements for conforming floating
point behaviour are loser than that.

>
>> Obviously there are other types of application for which
>> non-conforming behaviour could be catastrophic. If there were
>> no need of the full set of IEEE floating point rules, then the
>> rules would not be there in the first place.
>
>>> Consequent to the above, I believe the community is better served
>>> by compilers that are completely conforming -- not partially
>>> conforming -- in their default mode. If someone wants to work
>>> outside the standard language definition, then by all means, more
>>> power to them, and I'm all in favor of compiler options to give a
>>> variety of non-standard choices. But for the community as a
>>> whole, it's better if the choice of using non-standard behavior
>>> is the exception, not the rule.
>
> Now that I've written out this more careful differentiation of
> how "conforming" is used, it might be helpful to go back and
> re-read the earlier comments. Probably that will clear up
> most of the confusion but if some further thoughts come up
> I'd be interested to hear them.
>

Yes, I am definitely learning here. I have some twenty years experience
in C programming, with a wide variety of tools and targets - but there
is always scope to learn more. There is lots more to the standards and
"language lawyer" aspects of the language, and lots of other types of C
programming. That's why I hang out here - to learn.

Thanks,

David

[James Kuyper]

On 05/03/2013 03:53 AM, David Brown wrote:
....

> I think when I have used the term "conforming", I have meant "strictly
> conforming" (and usually without making clear distinction between
> compiler implementations and programs).

I'd like to point out one thing that you might or might not be aware of:
the standard assigns a meaning to the term "strictly conforming" only
when applied to programs. A C implementation either conforms or doesn't
conform. My first draft of this paragraph said "There's only one level
of conformance for implementations", but that's not true: it can conform
as either a freestanding or hosted implementation, and the latest
version of the standard made many features optional, so there's several
levels of conformance - but none of them is labeled "strict".

> But it strikes me that "strictly conforming" is so tight that it is
> close to useless, if you can't write real-world programs that are
> strictly conforming. Certainly you can't do anything useful in the
> embedded world (where I work) while remaining strictly conforming -
> there are always parts of the code that are specific to the target
> and/or particular compiler, so that on other conforming compilers you
> would get compiler errors or at least different run-time behaviour.

Strictly conforming code has some uses; failure to accept such code can
be used to prove an implementation non-conforming. Any new feature that
would break even strictly conforming conforming code has to be very
strongly motivated in order for the committee to be willing to accept it.

> On the other hand, "conforming" is so lose that it is close to useless.

As applied to code, it is exactly useless; there's nothing "close" about it.

> If I write a compiler that accepts Pascal as well as C, but is
> otherwise a conforming C compiler, then Pascal programs suddenly become
> conforming C code.

A compiler that has an option to process either Pascal or C code does
not, in itself, make that Pascal code qualify as conforming C code. Only
if the compiler accepts Pascal code when invoked in a mode where it is a
conforming implementation of C would it have that effect. In most cases,
a Pascal program will violate either C's syntax rules or constraints; if
so, a conforming implementation of C must generate at least one
diagnostic message - that message could be "Pascal code detected -
switching to Pascal mode". I don't know if it's possible to write a
Pascal program that could also be parsed as a correct C program, but if
so, the behavior required upon translating and accepting such a program
must be the C behavior, even if that's incompatible with the required
Pascal behavior.

...

> This is one of my mistakes. Basically, I thought an implementation
> /had/ to follow IEEE fully (including all the awkward bits like NaNs) to
> be "conforming". But I suppose the requirements for conforming floating
> point behaviour are loser than that.

An implementation can per-#define __STDC_IEC_60559__, in which case it
must satisfy all the requirements of Appendix F, which are based on and
very close to (but not identical with) those of IEC 60559 (== IEEE 754).
Otherwise, there are VERY few meaningful requirements on floating point
operations in C - those requirements are not, for instance, strong
enough to guarantee that 1.0-1.0 != DBL_MAX.
--
James Kuyper

[David Brown]

On 03/05/13 14:07, James Kuyper wrote:
> On 05/03/2013 03:53 AM, David Brown wrote:
> ....
>> I think when I have used the term "conforming", I have meant "strictly
>> conforming" (and usually without making clear distinction between
>> compiler implementations and programs).
>
> I'd like to point out one thing that you might or might not be aware of:
> the standard assigns a meaning to the term "strictly conforming" only
> when applied to programs. A C implementation either conforms or doesn't
> conform.

Yes, I am aware of that now - after Tim managed to bash it through my
thick skull...

> My first draft of this paragraph said "There's only one level
> of conformance for implementations", but that's not true: it can conform
> as either a freestanding or hosted implementation, and the latest
> version of the standard made many features optional, so there's several
> levels of conformance - but none of them is labeled "strict".
>
>> But it strikes me that "strictly conforming" is so tight that it is
>> close to useless, if you can't write real-world programs that are
>> strictly conforming. Certainly you can't do anything useful in the
>> embedded world (where I work) while remaining strictly conforming -
>> there are always parts of the code that are specific to the target
>> and/or particular compiler, so that on other conforming compilers you
>> would get compiler errors or at least different run-time behaviour.
>
> Strictly conforming code has some uses; failure to accept such code can
> be used to prove an implementation non-conforming. Any new feature that
> would break even strictly conforming conforming code has to be very
> strongly motivated in order for the committee to be willing to accept it.
>

Fair enough.

>> On the other hand, "conforming" is so lose that it is close to useless.
>
> As applied to code, it is exactly useless; there's nothing "close" about it.
>
>> If I write a compiler that accepts Pascal as well as C, but is
>> otherwise a conforming C compiler, then Pascal programs suddenly become
>> conforming C code.
>
> A compiler that has an option to process either Pascal or C code does
> not, in itself, make that Pascal code qualify as conforming C code. Only
> if the compiler accepts Pascal code when invoked in a mode where it is a
> conforming implementation of C would it have that effect. In most cases,
> a Pascal program will violate either C's syntax rules or constraints; if
> so, a conforming implementation of C must generate at least one
> diagnostic message - that message could be "Pascal code detected -
> switching to Pascal mode". I don't know if it's possible to write a
> Pascal program that could also be parsed as a correct C program, but if
> so, the behavior required upon translating and accepting such a program
> must be the C behavior, even if that's incompatible with the required
> Pascal behavior.

That makes a little more sense. A conforming compiler not only has to
accept all strictly conforming code, it also has to reject syntactically
incorrect C code with a diagnostic message.

>
> ...
>> This is one of my mistakes. Basically, I thought an implementation
>> /had/ to follow IEEE fully (including all the awkward bits like NaNs) to
>> be "conforming". But I suppose the requirements for conforming floating
>> point behaviour are loser than that.
>
> An implementation can per-#define __STDC_IEC_60559__, in which case it
> must satisfy all the requirements of Appendix F, which are based on and
> very close to (but not identical with) those of IEC 60559 (== IEEE 754).
> Otherwise, there are VERY few meaningful requirements on floating point
> operations in C - those requirements are not, for instance, strong
> enough to guarantee that 1.0-1.0 != DBL_MAX.
>

So as long as the compiler does not define __STDC_IEC_60559__ (and thus
claim to follow the "Appendix F" rules), it can be "conforming" despite
doing almost anything it wants with floating point?

Some embedded compilers I have 4-byte doubles (identical to their
single-precision floats), which is not unreasonable for 8-bit
microcontrollers. Can such a compiler be "conforming", or must the
doubles be bigger? I'm getting the impression that a conforming
compiler would need 8-byte doubles, but could throw away the extra bits
and do the calculations as floats.

[James Kuyper]

On 05/03/2013 08:56 AM, David Brown wrote:
> On 03/05/13 14:07, James Kuyper wrote:
>> On 05/03/2013 03:53 AM, David Brown wrote:
...

> That makes a little more sense. A conforming compiler not only has to
> accept all strictly conforming code, it also has to reject syntactically
> incorrect C code with a diagnostic message.

Close, but not quite: it must diagnose syntactically incorrect C code,
but it need not reject it. The only kind of program that a conforming
implementation is required to reject is one that contains a #error
directive that survives conditional compilation (#if, #ifdef, etc.).
Ironically, this implies, among other things, that in order to guarantee
that a program must be rejected, the code must NOT contain any syntax
errors or constraint violations that apply during translation phases
1-4, since translation phase 4 is the one where conditional compilation
and implementation of #error directives occurs.

That's what makes the definition of "conforming" code so useless: code
is conforming if there is any conforming implementation, anywhere, which
can accept it. Therefore, the only kind of program that is guaranteed to
be non-conforming is code which contains a #error directive that is
guaranteed to survive conditional compilation.

...

> So as long as the compiler does not define __STDC_IEC_60559__ (and thus
> claim to follow the "Appendix F" rules), it can be "conforming" despite
> doing almost anything it wants with floating point?

There are some requirements: conversions of floating point constants and
strings to floating point numbers, and back again, are tightly
constrained. However, there are not enough guarantees to make any
practical use of floating point.

> Some embedded compilers I have 4-byte doubles (identical to their
> single-precision floats), which is not unreasonable for 8-bit
> microcontrollers. Can such a compiler be "conforming", or must the
> doubles be bigger? I'm getting the impression that a conforming
> compiler would need 8-byte doubles, but could throw away the extra bits
> and do the calculations as floats.

The <float.h> standard header must #define a constant named DBL_EPSILON,
which is "the difference between 1 and the least value greater than 1
that is representable in the given floating point type", and that
difference cannot be greater than 1e-9 (5.2.4.2.2p13). The standard
imposes no accuracy requirements on the basic arithmetic operations +,
-, *, and /, nor on the floating point functions from <math.h> and
<complex.h> (5.2.4.2.2p6), which means it's not possible to reliably
test DBL_EPSILON by calculating the defining difference: nextafter(1.0,
2.0)-1.0.

However, it is possible to test it indirectly. The most extreme possible
case occurs if DBL_EPSILON is exactly 1e-9, which requires that
FLT_RADIX be a power of 10. In that case the floating point constant
1.000000002 could be represented exactly, but the standard does not
require it to be. It could be converted into a floating point value as
low as 1.000000001, but no lower, or as high as 1.000000003, but no
higher (6.4.4.2p3). The comparison operators <, >, <=, >=, ==, and !=
are not covered by 5.2.4.2.2p6, so you can use them to reliably test
whether or not this requirement has been met: 1.00000000 <= 1.000000002
&& 1.000000002 <= 1.000000004.
--
James Kuyper

[James Kuyper]

On 05/03/2013 10:09 AM, James Kuyper wrote:
...

> However, it is possible to test it indirectly. The most extreme possible
> case occurs if DBL_EPSILON is exactly 1e-9, which requires that
> FLT_RADIX be a power of 10. In that case the floating point constant
> 1.000000002 could be represented exactly, but the standard does not
> require it to be. It could be converted into a floating point value as
> low as 1.000000001, but no lower, or as high as 1.000000003, but no
> higher (6.4.4.2p3). The comparison operators <, >, <=, >=, ==, and !=
> are not covered by 5.2.4.2.2p6, so you can use them to reliably test
> whether or not this requirement has been met: 1.00000000 <= 1.000000002
> && 1.000000002 <= 1.000000004.

I just realized that comparisons that could come up equal don't properly
test conformance. I need comparisons for which inequality is guaranteed,
despite the inaccurate conversions allowed by 6.4.4.2p3. Therefore, go for
1.00000000 < 1.000000003 && 1.000000003 < 1.0000000006
--
James Kuyper

[Marcin Grzegorczyk]

David Brown wrote:
[...]

> Some embedded compilers I have 4-byte doubles (identical to their
> single-precision floats), which is not unreasonable for 8-bit
> microcontrollers. Can such a compiler be "conforming", or must the
> doubles be bigger? I'm getting the impression that a conforming
> compiler would need 8-byte doubles, but could throw away the extra bits
> and do the calculations as floats.

Indeed, the minimum requirements given in 5.2.4.2.2 for DBL_DIG,
DBL_MIN_10_EXP and DBL_MAX_10_EXP cannot be met by an implementation
that stores double in 32 bits. However, 8 bytes are not necessary; 6
are enough. (If both the exponent and the significand are stored in
binary, then (unless I made a mistake in the calculations) the exponent
requires min. 8 bits and the significand requires min. 34 bits, of which
the most significant one need not be stored. Together with the sign
bit, that gives 42 bits total.)

(Side note: IIRC, at least one C implementation for Atari ST (which did
not have an FPU) did use 6-byte doubles. Unlike the IEEE formats, the
exponent was stored after the significand, so that they could be
separated for further processing with only AND instructions, no shifts.)

Still, as James Kuyper wrote in this thread, an implementation could
pretend (via the <float.h> constants) to provide more precision than it
really did, and probably still claim conformance. It would be a rather
perverse thing to do, though.
--
Marcin Grzegorczyk

[Robert Wessel]

On Sun, 05 May 2013 00:55:06 +0200, Marcin Grzegorczyk
<mgrz...@poczta.onet.pl> wrote:

>David Brown wrote:
>[...]
>> Some embedded compilers I have 4-byte doubles (identical to their
>> single-precision floats), which is not unreasonable for 8-bit
>> microcontrollers. Can such a compiler be "conforming", or must the
>> doubles be bigger? I'm getting the impression that a conforming
>> compiler would need 8-byte doubles, but could throw away the extra bits
>> and do the calculations as floats.
>
>Indeed, the minimum requirements given in 5.2.4.2.2 for DBL_DIG,
>DBL_MIN_10_EXP and DBL_MAX_10_EXP cannot be met by an implementation
>that stores double in 32 bits. However, 8 bytes are not necessary; 6
>are enough. (If both the exponent and the significand are stored in
>binary, then (unless I made a mistake in the calculations) the exponent
>requires min. 8 bits and the significand requires min. 34 bits, of which
>the most significant one need not be stored. Together with the sign
>bit, that gives 42 bits total.)
>
>(Side note: IIRC, at least one C implementation for Atari ST (which did
>not have an FPU) did use 6-byte doubles. Unlike the IEEE formats, the
>exponent was stored after the significand, so that they could be
>separated for further processing with only AND instructions, no shifts.)

To be pedantic, IEEE-754 does not specify the internal representation
of the float. Field order, bit order, and the (non)presence of
internal padding are all unspecified. There is a specified
*interchange* format (which uses the common sign/exponent/mantissa
format), but there is not requirement that a CPU implement that as its
internal format. It would be perfectly acceptable for an
implementation of doubles to put the exponent in last eleven bits, if
the ISA architects found that convenient.

OTOH, I've not seen an actual implementation that didn't use the
interchange format (endian issues aside) internally, although I could
see a software implementation taking some liberties for the same
reasons the Atari implementation did (perhaps on an embedded C
implementation where external communications of binary float images is
unlikely).

[glen herrmannsfeldt]

Robert Wessel <robert...@yahoo.com> wrote:

(snip, someone wrote)

>>Indeed, the minimum requirements given in 5.2.4.2.2 for DBL_DIG,
>>DBL_MIN_10_EXP and DBL_MAX_10_EXP cannot be met by an implementation
>>that stores double in 32 bits. However, 8 bytes are not necessary; 6
>>are enough.

(snip)

> To be pedantic, IEEE-754 does not specify the internal representation
> of the float. Field order, bit order, and the (non)presence of
> internal padding are all unspecified. There is a specified
> *interchange* format (which uses the common sign/exponent/mantissa
> format), but there is not requirement that a CPU implement that as its
> internal format. It would be perfectly acceptable for an
> implementation of doubles to put the exponent in last eleven bits, if
> the ISA architects found that convenient.

Yes. Well, normally you can't say much about the actual order of bits
inside the processor. There is no reason to give them an order until
they are written to memory.

As an example, the internal format of the x87 is the 80 bit temporary
real format. The order is known, as it is possible to write them out
in that form.

> OTOH, I've not seen an actual implementation that didn't use the
> interchange format (endian issues aside) internally, although I could
> see a software implementation taking some liberties for the same
> reasons the Atari implementation did (perhaps on an embedded C
> implementation where external communications of binary float images is
> unlikely).

It would complicate things. You would expect, for example, the Fortran
UNFORMATTED I/O to use the interchange format, though normally it is
expected to just copy the bits. It is even less obvious that fwrite()
could write the bits in a different order than they were stored in
memory.

-- glen

[Robert Wessel]

It's neither required to store the interchange format in memory, or
write it to a file. But your odds of reading it from a file on a
different IEEE-754 supporting system are better if you do write the
interchange format. And even that's of somewhat limited value, as
most people prefer a text format for exchanging data between systems
(not that I necessarily agree with that, but that seems to be the
world's preference).

[Malcolm McLean]

I wrote this function to write IEEE floats portably.

/*
* write a double to a stream in ieee754 format regardless of host
* encoding.
* x - number to write
* fp - the stream
* bigendian - set to write big bytes first, elee write litle bytes
* first
* Returns: 0 or EOF on error
* Notes: different NaN types and negative zero not preserved.
* if the number is too big to represent it will become infinity
* if it is too small to represent it will become zero.
*/
static int fwriteieee754(double x, FILE *fp, int bigendian)
{
int shift;
unsigned long sign, exp, hibits, hilong, lowlong;
double fnorm, significand;
int expbits = 11;
int significandbits = 52;

/* zero (can't handle signed zero) */
if(x == 0)
{
hilong = 0;
lowlong = 0;
goto writedata;
}
/* infinity */
if(x > DBL_MAX)
{
hilong = 1024 + ((1<<(expbits-1)) - 1);
hilong <<= (31 - expbits);
lowlong = 0;
goto writedata;
}
/* -infinity */
if(x < -DBL_MAX)
{
hilong = 1024 + ((1<<(expbits-1)) - 1);
hilong <<= (31-expbits);
hilong |= (1 << 31);
lowlong = 0;
goto writedata;
}
/* NaN - dodgy because many compilers optimise out this test, but
*there is no portable isnan() */
if(x != x)
{
hilong = 1024 + ((1<<(expbits-1)) - 1);
hilong <<= (31 - expbits);
lowlong = 1234;
goto writedata;
}

/* get the sign */
if(x < 0) {sign = 1; fnorm = -x;}
else {sign = 0; fnorm = x;}

/* get the normalized form of f and track the exponent */
shift = 0;
while(fnorm >= 2.0) { fnorm /= 2.0; shift++; }
while(fnorm < 1.0) { fnorm *= 2.0; shift--; }

/* check for denormalized numbers */
if(shift < -1022)
{
while(shift < -1022) {fnorm /= 2.0; shift++;}
shift = -1023;
}
/* out of range. Set to infinity */
else if(shift > 1023)
{
hilong = 1024 + ((1<<(expbits-1)) - 1);
hilong <<= (31-expbits);
hilong |= (sign << 31);
lowlong = 0;
goto writedata;
}
else
fnorm = fnorm - 1.0; /* take the significant bit off mantissa */

/* calculate the integer form of the significand */
/* hold it in a double for now */

significand = fnorm * ((1LL<<significandbits) + 0.5f);


/* get the biased exponent */
exp = shift + ((1<<(expbits-1)) - 1); /* shift + bias */

/* put the data into two longs (for convenience) */
hibits = (long) ( significand / 4294967296);
hilong = (sign << 31) | (exp << (31-expbits) ) | hibits;
x = significand - hibits * 4294967296;
lowlong = (unsigned long) (significand - hibits * 4294967296);

writedata:
/* write the bytes out to the stream */
if(bigendian)
{
fputc( (hilong >> 24) & 0xFF, fp);
fputc( (hilong >> 16) & 0xFF, fp);
fputc( (hilong >> 8) & 0xFF, fp);
fputc( hilong & 0xFF, fp);

fputc( (lowlong >> 24) & 0xFF, fp);
fputc( (lowlong >> 16) & 0xFF, fp);
fputc( (lowlong >> 8) & 0xFF, fp);
fputc( lowlong & 0xFF, fp);
}
else
{
fputc( lowlong & 0xFF, fp);
fputc( (lowlong >> 8) & 0xFF, fp);
fputc( (lowlong >> 16) & 0xFF, fp);
fputc( (lowlong >> 24) & 0xFF, fp);

fputc( hilong & 0xFF, fp);
fputc( (hilong >> 8) & 0xFF, fp);
fputc( (hilong >> 16) & 0xFF, fp);
fputc( (hilong >> 24) & 0xFF, fp);
}
return ferror(fp);
}

[Ike Naar]

What's the purpose of the above statement?
It looks like x is not used in the rest of the function.

[Ian Collins]

Malcolm McLean wrote:
> On Sunday, May 5, 2013 7:30:49 AM UTC+1, robert...@yahoo.com wrote:
>> On Sun, 5 May 2013 05:42:32 +0000 (UTC), glen herrmannsfeldt
>>
>> <g...@ugcs.caltech.edu> wrote:
>>
>>
>>
>>> Robert Wessel <robert...@yahoo.com> wrote:
>>
>>
>>> It would complicate things. You would expect, for example, the Fortran
>>> UNFORMATTED I/O to use the interchange format, though normally it is
>>> expected to just copy the bits. It is even less obvious that fwrite()
>>> could write the bits in a different order than they were stored in
>>> memory.
>>
>>
>>
>> It's neither required to store the interchange format in memory, or
>> write it to a file. But your odds of reading it from a file on a
>> different IEEE-754 supporting system are better if you do write the
>> interchange format. And even that's of somewhat limited value, as
>> most people prefer a text format for exchanging data between systems
>> (not that I necessarily agree with that, but that seems to be the
>> world's preference).
>
>
> I wrote this function to write IEEE floats portably.

Handy.

I would make "writedata" a function and remove the gotos!

Sun cc flags this as an integer overflow, which looks to be correct.

--
Ian Collins

[Tim Rentsch]

David Brown <da...@westcontrol.removethisbit.com> writes:

> [snipping a lot]

>
> But it strikes me that "strictly conforming" is so tight that it
> is close to useless, if you can't write real-world programs that

> are strictly conforming. [...] On the other hand, "conforming"

> is so lose that it is close to useless.

The point of these terms (and their definitions) is to aid
discussion of implementations, not of programs. Sometimes
people forget that the main reason for having a C standard
is to define requirements for implementors, not to help
developers.

[Tim Rentsch]

David Brown <da...@westcontrol.removethisbit.com> writes:

> [..snip..]

>
> Some embedded compilers I have 4-byte doubles (identical to their
> single-precision floats), which is not unreasonable for 8-bit
> microcontrollers. Can such a compiler be "conforming", or must
> the doubles be bigger? I'm getting the impression that a
> conforming compiler would need 8-byte doubles, but could throw
> away the extra bits and do the calculations as floats.

You'll get better answers to questions like this, and a better
understanding of what is being said, if you first read what
the Standard says and try to discover the answer yourself:

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf
http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf

For this question you should look at what's written in section
5.2.4.2.2 'Characteristics of floating types'.

[Tim Rentsch]

Marcin Grzegorczyk <mgrz...@poczta.onet.pl> writes:

> David Brown wrote:
> [...]
>> Some embedded compilers I have 4-byte doubles (identical to their
>> single-precision floats), which is not unreasonable for 8-bit
>> microcontrollers. Can such a compiler be "conforming", or must the
>> doubles be bigger? I'm getting the impression that a conforming
>> compiler would need 8-byte doubles, but could throw away the extra bits
>> and do the calculations as floats.
>
> Indeed, the minimum requirements given in 5.2.4.2.2 for DBL_DIG,
> DBL_MIN_10_EXP and DBL_MAX_10_EXP cannot be met by an implementation
> that stores double in 32 bits. However, 8 bytes are not necessary; 6
> are enough. (If both the exponent and the significand are stored in
> binary, then (unless I made a mistake in the calculations) the
> exponent requires min. 8 bits and the significand requires min. 34
> bits, of which the most significant one need not be stored. Together
> with the sign bit, that gives 42 bits total.)

The formula given for DBL_DIG in 5.2.4.2.2 p11 implies, for b == 2,
a lower bound of 35 significand bits, giving 43 bits altogether (ie,
the rest of the calculation is right).

However, if b == 10 is used, and the exponent and significand are
encoded into a single (binary) numeric quantity, this can be
reduced to just 41 bits. This representation also gives a
noticeably larger exponent range (though entailing, of course, a
loss of almost two bits of precision, plus an unmentionably
large cost converting to/from the stored representation to do
floating point arithmetic).

> (Side note: IIRC, at least one C implementation for Atari ST (which
> did not have an FPU) did use 6-byte doubles. Unlike the IEEE formats,
> the exponent was stored after the significand, so that they could be
> separated for further processing with only AND instructions, no
> shifts.)
>
> Still, as James Kuyper wrote in this thread,

Either I missed this posting, or I understood its contents as
saying something different from the rest of the sentence.

> an implementation could pretend (via the <float.h> constants) to
> provide more precision than it really did, and probably still
> claim conformance. It would be a rather perverse thing to do,
> though.

Are you saying that, for example, an implementation could define
DBL_DIG to be 10, even when the stored representation would make
it impossible to convert any 10 decimal digit floating point
number to the stored representation and back again (so as to get
the original 10 decimal digits)? AFAICS such an implementation
could no more be conforming than one that defined INT_MAX as 15.
Have I misunderstood what you're saying? If not, can you explain
how you arrive at this conclusion, or what interpretation might
allow such a result?

[Keith Thompson]

I'm not sure I agree with that last statement. I see a language
standard as a contract between implementers and developers, applying
equally to both.

--
Keith Thompson (The_Other_Keith) ks...@mib.org <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

[James Kuyper]

On 05/04/2013 06:55 PM, Marcin Grzegorczyk wrote:
...

> Still, as James Kuyper wrote in this thread, an implementation could
> pretend (via the <float.h> constants) to provide more precision than it
> really did, and probably still claim conformance. It would be a rather
> perverse thing to do, though.

I didn't really intend to imply that, though there's a certain amount of
truth to it. The standard does fail to impose any accuracy requirements
whatsoever on floating point arithmetic or on the functions in the
<math.h> and <complex.h> (5.2.4.2.2p6). That fact makes it difficult to
write any code involving floating point operations that is guaranteed by
the standard to do anything useful. As a special case of that general
statement, it makes it difficult to prove that a defective <float.h> is
in fact defective.

However, it was not my point that <float.h> could be defective. In fact,
what I basically demonstrated is that there's a limit on how much
imprecision can be covered up by this means. The inaccuracy allowed by
the standard for conversion of floating point constants to floating
point values is sufficient to make it difficult to prove non-conformance
if the correct value for DBL_EPSILON were as large as twice the
standard's maximum value. However, anything more than twice that minimum
would be provably non-conforming, despite what it says in 5.2.4.2.2p6.
That's not enough to fake conformance with the standard's requirements
while using 32-bit floating point for double, which was the issue I was
addressing.

[Marcin Grzegorczyk]

Tim Rentsch wrote:
> Marcin Grzegorczyk <mgrz...@poczta.onet.pl> writes:
[...]

>> (If both the exponent and the significand are stored in
>> binary, then (unless I made a mistake in the calculations) the
>> exponent requires min. 8 bits and the significand requires min. 34
>> bits, of which the most significant one need not be stored. Together
>> with the sign bit, that gives 42 bits total.)
>
> The formula given for DBL_DIG in 5.2.4.2.2 p11 implies, for b == 2,
> a lower bound of 35 significand bits, giving 43 bits altogether (ie,
> the rest of the calculation is right).

Checked that again, you're right.
I did make a mistake in the calculations, then :-)
--
Marcin Grzegorczyk

[Tim Rentsch]

Keith Thompson <ks...@mib.org> writes:

> Tim Rentsch <t...@alumni.caltech.edu> writes:
>> David Brown <da...@westcontrol.removethisbit.com> writes:
>>
>>> [snipping a lot]
>>>
>>> But it strikes me that "strictly conforming" is so tight that it
>>> is close to useless, if you can't write real-world programs that
>>> are strictly conforming. [...] On the other hand, "conforming"
>>> is so lose that it is close to useless.
>>
>> The point of these terms (and their definitions) is to aid
>> discussion of implementations, not of programs. Sometimes
>> people forget that the main reason for having a C standard
>> is to define requirements for implementors, not to help
>> developers.
>
> I'm not sure I agree with that last statement. I see a
> language standard as a contract between implementers and
> developers, applying equally to both.

I don't disagree on your fundamental point, but there's a
subtle distinction between what the two statements are
considering. The main reason for having C be standardized is
just as you say, to form a contract between implementors and
developers, and this is of interest to both. However, once the
terms of the contract are decided, the main reason for having a
defining document is to serve as a reference for implementors.
To be sure, some developers will read the defining document,
and even benefit from reading it, but it isn't necessary to be
a successful C developer, whereas reading and understanding the
C Standard _is_ pretty much necessary to write a conforming
implementation. So I don't think it's wrong to say the main
reason for having a C standard -- in the sense of the actual
formal document -- is to define requirements for implementors.
C developers do very much benefit from having the language
be standardized, but for the most part they do not benefit
(directly) from the actual formal document.

[glen herrmannsfeldt]

Tim Rentsch <t...@alumni.caltech.edu> wrote:
> Keith Thompson <ks...@mib.org> writes:

(snip)

>> I'm not sure I agree with that last statement. I see a
>> language standard as a contract between implementers and
>> developers, applying equally to both.

> I don't disagree on your fundamental point, but there's a
> subtle distinction between what the two statements are
> considering. The main reason for having C be standardized is
> just as you say, to form a contract between implementors and
> developers, and this is of interest to both. However, once the
> terms of the contract are decided, the main reason for having a
> defining document is to serve as a reference for implementors.
> To be sure, some developers will read the defining document,
> and even benefit from reading it, but it isn't necessary to be
> a successful C developer, whereas reading and understanding the
> C Standard _is_ pretty much necessary to write a conforming
> implementation.

Well, to me a good language can be understood by developers
using a simpler set of rules, rarely having to look up the
fine details in the standard.

PL/I includes features for scientific, commercial, and systems
programming, and people from one group will rarely need ones
specific for the others. (Many features are used by more than
one group.)

As for precision and range of data types, often the limitations
of an implementation are passed along to users of the program.
That might be more true in floating point, but still true
in fixed point.

One consequence of that is that undetected overflow may surprise
the user of a program.

> So I don't think it's wrong to say the main
> reason for having a C standard -- in the sense of the actual
> formal document -- is to define requirements for implementors.
> C developers do very much benefit from having the language
> be standardized, but for the most part they do not benefit
> (directly) from the actual formal document.

I agree.

-- glen