Precedence of operators and FMA

[Andrea Arteaga]

The C++ standard defines a strict operator precedence and associativity, so for instance a+b+c must be interpreted by the compiler as (a+b)+c and not as a+(b+c); a compiler which interprets the expression according to the second way is not strictly standard compliant (a few compilers, like GCC and ICC, allow to specify a flag which disables all those optimizations which can cause reassociation like in this case).

With C++11 the new mathematical function FMA is introduced, which allows to perform operations like a*b+c in a single hardware instruction. The proper language construct to use a FMA is the function std::fma(a, b, c), but, to my knowledge, the compiler can also interpret the expression a*b+c as an FMA and generate the same instruction. In this case the group */+ used in that expression can be considered as a single ternary operator instead of a couple of binary operators.

The question is whether such an interpretation is allowed, prescribed or regulated by the standard. What happen in the case of the expression a*b + c*d? Is the compiler allowed to reformulate it as fma(a, b, c*d) and as fma(c, d, a*b)?

I'm particularly interested because suddenly, with the introduction of FMAs, the very same C++ code can be interpreted in different ways by the compilers leading to potentially different results depending on which instruction set is selected at compile time. Before, the strict operators precedence and associativity rules prevented such differences due to floating point arithmetic.

Please let me know if some of my assumptions are wrong or if my understanding of the standard is misleading.

[Richard Smith]

On Tue, Mar 22, 2016 at 5:15 PM, Andrea Arteaga <andys...@gmail.com> wrote:
> The C++ standard defines a strict operator precedence and associativity, so
> for instance a+b+c must be interpreted by the compiler as (a+b)+c and not as
> a+(b+c); a compiler which interprets the expression according to the second
> way is not strictly standard compliant (a few compilers, like GCC and ICC,
> allow to specify a flag which disables all those optimizations which can
> cause reassociation like in this case).

The compiler is permitted to interpret the expression the second way
if it doesn't change the observable behavior of the program.

> With C++11 the new mathematical function FMA is introduced, which allows to
> perform operations like a*b+c in a single hardware instruction. The proper
> language construct to use a FMA is the function std::fma(a, b, c), but, to
> my knowledge, the compiler can also interpret the expression a*b+c as an FMA
> and generate the same instruction. In this case the group */+ used in that
> expression can be considered as a single ternary operator instead of a
> couple of binary operators.
>
> The question is whether such an interpretation is allowed, prescribed or
> regulated by the standard. What happen in the case of the expression a*b +
> c*d? Is the compiler allowed to reformulate it as fma(a, b, c*d) and as
> fma(c, d, a*b)?

Yes. Per [expr]/12:

"The values of the floating operands and the results of floating
expressions may be represented in greater precision and range than
that required by the type"

... and since fma differs from a*b+c by providing extra precision, it
falls under this rule (note that this rule also allows use of 80-bit
x87 floating point computations for storing the intermediate values of
64-bit double computations, etc).

> I'm particularly interested because suddenly, with the introduction of FMAs,
> the very same C++ code can be interpreted in different ways by the compilers
> leading to potentially different results depending on which instruction set
> is selected at compile time. Before, the strict operators precedence and
> associativity rules prevented such differences due to floating point
> arithmetic.
>
> Please let me know if some of my assumptions are wrong or if my
> understanding of the standard is misleading.
>

> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "ISO C++ Standard - Discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to std-discussio...@isocpp.org.
> To post to this group, send email to std-dis...@isocpp.org.
> Visit this group at
> https://groups.google.com/a/isocpp.org/group/std-discussion/.

[Andrea Arteaga]

On Wed, Mar 23, 2016 at 3:03 AM, Richard Smith <ric...@metafoo.co.uk> wrote:

On Tue, Mar 22, 2016 at 5:15 PM, Andrea Arteaga <andys...@gmail.com> wrote:
> The C++ standard defines a strict operator precedence and associativity, so
> for instance a+b+c must be interpreted by the compiler as (a+b)+c and not as
> a+(b+c); a compiler which interprets the expression according to the second
> way is not strictly standard compliant (a few compilers, like GCC and ICC,
> allow to specify a flag which disables all those optimizations which can
> cause reassociation like in this case).

The compiler is permitted to interpret the expression the second way
if it doesn't change the observable behavior of the program.

This sentence is crucial for my research project. Could you please link to the specific parts of the standard which express this concept? I know Fortran allows this kind of reinterpretation which does not change the mathematical meaning of the expression, but I was really assuming that the strict operator precedence of C++ was preventing such a behavior.

 

> With C++11 the new mathematical function FMA is introduced, which allows to
> perform operations like a*b+c in a single hardware instruction. The proper
> language construct to use a FMA is the function std::fma(a, b, c), but, to
> my knowledge, the compiler can also interpret the expression a*b+c as an FMA
> and generate the same instruction. In this case the group */+ used in that
> expression can be considered as a single ternary operator instead of a
> couple of binary operators.
>
> The question is whether such an interpretation is allowed, prescribed or
> regulated by the standard. What happen in the case of the expression a*b +
> c*d? Is the compiler allowed to reformulate it as fma(a, b, c*d) and as
> fma(c, d, a*b)?

Yes. Per [expr]/12:

"The values of the floating operands and the results of floating
expressions may be represented in greater precision and range than
that required by the type"

... and since fma differs from a*b+c by providing extra precision, it
falls under this rule (note that this rule also allows use of 80-bit
x87 floating point computations for storing the intermediate values of
64-bit double computations, etc).

Well, my question is actually different. (a*b)+c*d is a different application of the operators than a+b*(c*d). Does the standard defines which one is the "correct" one, i.e. (assuming your statement above is correct, i.e. the compiler is allowed to reassociate without changing the mathematical meaning) the one according to which the abstract meaning is deducted?

 

> I'm particularly interested because suddenly, with the introduction of FMAs,
> the very same C++ code can be interpreted in different ways by the compilers
> leading to potentially different results depending on which instruction set
> is selected at compile time. Before, the strict operators precedence and
> associativity rules prevented such differences due to floating point
> arithmetic.
>
> Please let me know if some of my assumptions are wrong or if my
> understanding of the standard is misleading.
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "ISO C++ Standard - Discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to std-discussio...@isocpp.org.
> To post to this group, send email to std-dis...@isocpp.org.
> Visit this group at
> https://groups.google.com/a/isocpp.org/group/std-discussion/.

--

---

You received this message because you are subscribed to a topic in the Google Groups "ISO C++ Standard - Discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/a/isocpp.org/d/topic/std-discussion/wqb6unxwmJI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to std-discussio...@isocpp.org.

[Columbo]

On Wednesday, March 23, 2016 at 10:02:48 AM UTC+1, Andrea Arteaga wrote:

On Wed, Mar 23, 2016 at 3:03 AM, Richard Smith <ric...@metafoo.co.uk> wrote:

On Tue, Mar 22, 2016 at 5:15 PM, Andrea Arteaga <andys...@gmail.com> wrote:
> The C++ standard defines a strict operator precedence and associativity, so
> for instance a+b+c must be interpreted by the compiler as (a+b)+c and not as
> a+(b+c); a compiler which interprets the expression according to the second
> way is not strictly standard compliant (a few compilers, like GCC and ICC,
> allow to specify a flag which disables all those optimizations which can
> cause reassociation like in this case).

The compiler is permitted to interpret the expression the second way
if it doesn't change the observable behavior of the program.

This sentence is crucial for my research project. Could you please link to the specific parts of the standard which express this concept? I know Fortran allows this kind of reinterpretation which does not change the mathematical meaning of the expression, but I was really assuming that the strict operator precedence of C++ was preventing such a behavior.

See [intro.execution]/1:

This International Standard places no requirement on the structure of conforming implementations. In particular, they need not copy or emulate the structure of the abstract machine. Rather, conforming implementations are required to emulate (only) the observable behavior of the abstract machine as explained below. 5
Footnote 5) This provision is sometimes called the “as-if” rule, because an implementation is free to disregard any requirement of this
International Standard as long as the result is as if the requirement had been obeyed, as far as can be determined from the
observable behavior of the program. For instance, an actual implementation need not evaluate part of an expression if it can
deduce that its value is not used and that no side effects affecting the observable behavior of the program are produced. 

[Andrea Arteaga]

Thanks.

From this I understand that the standard defines an abstract machine which follows a well-defined set of rules. The implementations do not have to mimic the very same behavior as long as the actual outcome is the same ("conforming implementations are required to emulate (only) the observable behavior of the abstract machine"). Since (a+b)+c and a+(b+c) result into two different observable behaviors, I would assume this sentence prohibits this kind of reassociation.

Otherwise put, the operators prevedence dictates to interpret a+b+c as (a+b)+c, and computing instead a+(b+c) changes the observable behavior (according to the IEEE754 standard), so I would assume such a translation is forbidden.

[Columbo]

On Wednesday, March 23, 2016 at 10:33:57 AM UTC+1, Andrea Arteaga wrote:

On Wed, Mar 23, 2016 at 10:11 AM, Columbo <r....@gmx.net> wrote:

On Wednesday, March 23, 2016 at 10:02:48 AM UTC+1, Andrea Arteaga wrote:

On Wed, Mar 23, 2016 at 3:03 AM, Richard Smith <ric...@metafoo.co.uk> wrote:

On Tue, Mar 22, 2016 at 5:15 PM, Andrea Arteaga <andys...@gmail.com> wrote:
> The C++ standard defines a strict operator precedence and associativity, so
> for instance a+b+c must be interpreted by the compiler as (a+b)+c and not as
> a+(b+c); a compiler which interprets the expression according to the second
> way is not strictly standard compliant (a few compilers, like GCC and ICC,
> allow to specify a flag which disables all those optimizations which can
> cause reassociation like in this case).

The compiler is permitted to interpret the expression the second way
if it doesn't change the observable behavior of the program.

This sentence is crucial for my research project. Could you please link to the specific parts of the standard which express this concept? I know Fortran allows this kind of reinterpretation which does not change the mathematical meaning of the expression, but I was really assuming that the strict operator precedence of C++ was preventing such a behavior.

See [intro.execution]/1:

This International Standard places no requirement on the structure of conforming implementations. In particular, they need not copy or emulate the structure of the abstract machine. Rather, conforming implementations are required to emulate (only) the observable behavior of the abstract machine as explained below. 5
Footnote 5) This provision is sometimes called the “as-if” rule, because an implementation is free to disregard any requirement of this
International Standard as long as the result is as if the requirement had been obeyed, as far as can be determined from the
observable behavior of the program. For instance, an actual implementation need not evaluate part of an expression if it can
deduce that its value is not used and that no side effects affecting the observable behavior of the program are produced. 

Thanks.

From this I understand that the standard defines an abstract machine which follows a well-defined set of rules. The implementations do not have to mimic the very same behavior as long as the actual outcome is the same ("conforming implementations are required to emulate (only) the observable behavior of the abstract machine"). Since (a+b)+c and a+(b+c) result into two different observable behaviors, I would assume this sentence prohibits this kind of reassociation.

No, the observable behavior of a program is not the precise set of instructions and operands executed. It is actually defined in [intro.execution]/8.
However, you are right in that the value could be different, eventually altering the observable behavior. Nonetheless, some implementations do support optimizations that are nonconforming. This specific example is mentioned in the description of GCC's -funsafe-math-optimizations flag:

This mode enables optimizations that allow arbitrary reassociations and transformations with no accuracy guarantees. It also does not try to preserve the sign of zeros. […] Due to roundoff errors the associative law of algebra do not necessary hold for floating point numbers and thus expressions like (x + y) + z are not necessary equal to x + (y + z). 

[Andrea Arteaga]

Yes, I'm aware of a few nonconforming optimizations performed by some compilers. But I'm mostly interested on the standard itself. And actually, [into.execution]/9 seems to be exactly what I need:

Operators can be regrouped according to the usual mathematical rules only where the operators really are associative or commutative.

So, apparently the compilers are not allowed to reassociate the expressions involving floating point operations.

[Andrea Arteaga]

I still can't find any mentions to the FMA instructions. From what I read, generating FMA instructions for expressions such as a*b+c is nonconforming, since this would give different results from the canonical interpretation of the expression, which is (a*b)+c.

[Sergey Zubkov]

I still can't find any mentions to the FMA instructions. From what I read, generating FMA instructions for expressions such as a*b+c is nonconforming, since this would give different results from the canonical interpretation of the expression, which is (a*b)+c.

FMA is covered by C's 6.5/8 "Expressions"

"A floating expression may be contracted, that is, evaluated as though it were a single operation, thereby omitting rounding errors implied by the source code and the expression evaluation method. The FP_CONTRACT pragma in <math.h> provides a way to disallow contracted expressions. Otherwise, whether and how expressions are contracted is implementation-defined"

with a footnote "This license is specifically intended to allow implementations to exploit fast machine instructions that combine multiple C operators."

Whether this normative part of the C standard applies to C++ has been debated by language lawyers

[Mathias Gaunard]

On 23 March 2016 at 09:02, Andrea Arteaga <andys...@gmail.com> wrote:

> With C++11 the new mathematical function FMA is introduced, which allows to
> perform operations like a*b+c in a single hardware instruction. The proper
> language construct to use a FMA is the function std::fma(a, b, c), but, to
> my knowledge, the compiler can also interpret the expression a*b+c as an FMA
> and generate the same instruction. In this case the group */+ used in that
> expression can be considered as a single ternary operator instead of a
> couple of binary operators.
>
> The question is whether such an interpretation is allowed, prescribed or
> regulated by the standard. What happen in the case of the expression a*b +
> c*d? Is the compiler allowed to reformulate it as fma(a, b, c*d) and as
> fma(c, d, a*b)?

Yes. Per [expr]/12:

"The values of the floating operands and the results of floating
expressions may be represented in greater precision and range than
that required by the type"

... and since fma differs from a*b+c by providing extra precision, it
falls under this rule (note that this rule also allows use of 80-bit
x87 floating point computations for storing the intermediate values of
64-bit double computations, etc).

Well, my question is actually different. (a*b)+c*d is a different application of the operators than a+b*(c*d). Does the standard defines which one is the "correct" one, i.e. (assuming your statement above is correct, i.e. the compiler is allowed to reassociate without changing the mathematical meaning) the one according to which the abstract meaning is deducted?

(a*b)+c*d is not the same as a+b*(c*d), I assume you meant something different?

In any case, for a*b+c*d, both fma(a, b, c*d) and fma(c, d, a*b) are valid transformations since + is commutative.

As was said earlier, the language allows any sequence of operations to be evaluated with extra precision, and you cannot predict which ones. (you can however prevent it by going through volatile memory or using compiler pragmas).

[Andrea Arteaga]

On Wed, Mar 23, 2016 at 2:54 PM, Mathias Gaunard <mat...@gaunard.com> wrote:

On 23 March 2016 at 09:02, Andrea Arteaga <andys...@gmail.com> wrote:

> With C++11 the new mathematical function FMA is introduced, which allows to
> perform operations like a*b+c in a single hardware instruction. The proper
> language construct to use a FMA is the function std::fma(a, b, c), but, to
> my knowledge, the compiler can also interpret the expression a*b+c as an FMA
> and generate the same instruction. In this case the group */+ used in that
> expression can be considered as a single ternary operator instead of a
> couple of binary operators.
>
> The question is whether such an interpretation is allowed, prescribed or
> regulated by the standard. What happen in the case of the expression a*b +
> c*d? Is the compiler allowed to reformulate it as fma(a, b, c*d) and as
> fma(c, d, a*b)?

Yes. Per [expr]/12:

"The values of the floating operands and the results of floating
expressions may be represented in greater precision and range than
that required by the type"

... and since fma differs from a*b+c by providing extra precision, it
falls under this rule (note that this rule also allows use of 80-bit
x87 floating point computations for storing the intermediate values of
64-bit double computations, etc).

Well, my question is actually different. (a*b)+c*d is a different application of the operators than a+b*(c*d). Does the standard defines which one is the "correct" one, i.e. (assuming your statement above is correct, i.e. the compiler is allowed to reassociate without changing the mathematical meaning) the one according to which the abstract meaning is deducted?

(a*b)+c*d is not the same as a+b*(c*d), I assume you meant something different?

I meant a*b+(c*d). Sorry for the typo, thanks for pointing out.

In any case, for a*b+c*d, both fma(a, b, c*d) and fma(c, d, a*b) are valid transformations since + is commutative.

+ is commutative even in floating point algebra, but + and * are not associative, so the result changes (or may change).

 

As was said earlier, the language allows any sequence of operations to be evaluated with extra precision, and you cannot predict which ones. (you can however prevent it by going through volatile memory or using compiler pragmas).

Can you point to the place in the standard where this is stated?

[Robert Haberlach]

Actually, this was covered by CWG 238, resolved 2015/08:

> It is not clear what constraints are placed on a floating point implementation by the wording of the Standard. For instance, is an implementation
permitted to generate a "fused multiply-add" instruction if the result would be different from what would be obtained by performing the operations
separately? To what extent does the "as-if" rule allow the kinds of optimizations (e.g., loop unrolling) performed by FORTRAN compilers?

The resolution was just the note in [basic.fundamental]:

> [Note: This International Standard imposes no requirements on the accuracy of floating-point operations; see also 18.3.2 [limits]. —end note]

I don't know if this is enough though, because these kinds of optimizations could effectively change the behavior of the exact same operation (i.e.
identical operator and operands) under different circumstances.

On 03/23/2016 11:12 AM, Andrea Arteaga wrote:
>
>
> On Wednesday, March 23, 2016 at 11:05:10 AM UTC+1, Andrea Arteaga wrote:
>

> On Wed, Mar 23, 2016 at 10:56 AM, Columbo <r....@gmx.net <mailto:r....@gmx.net>> wrote:
>
>
>
> On Wednesday, March 23, 2016 at 10:33:57 AM UTC+1, Andrea Arteaga wrote:
>
> On Wed, Mar 23, 2016 at 10:11 AM, Columbo <r....@gmx.net> wrote:
>
>
>
> On Wednesday, March 23, 2016 at 10:02:48 AM UTC+1, Andrea Arteaga wrote:
>
>
> On Wed, Mar 23, 2016 at 3:03 AM, Richard Smith <ric...@metafoo.co.uk> wrote:
>
> On Tue, Mar 22, 2016 at 5:15 PM, Andrea Arteaga <andys...@gmail.com> wrote:
> > The C++ standard defines a strict operator precedence and associativity, so
> > for instance a+b+c must be interpreted by the compiler as (a+b)+c and not as
> > a+(b+c); a compiler which interprets the expression according to the second
> > way is not strictly standard compliant (a few compilers, like GCC and ICC,
> > allow to specify a flag which disables all those optimizations which can
> > cause reassociation like in this case).
>
> The compiler is permitted to interpret the expression the second way
> if it doesn't change the observable behavior of the program.
>
>
> This sentence is crucial for my research project. Could you please link to the specific parts of the standard which express this
> concept? I know Fortran allows this kind of reinterpretation which does not change the mathematical meaning of the expression, but
> I was really assuming that the strict operator precedence of C++ was preventing such a behavior.
>
>

> See [intro.execution]/1 <http://eel.is/c++draft/intro.execution#1>:

>
> This International Standard places no requirement on the structure of conforming implementations. In particular, they need not
> copy or emulate the structure of the abstract machine. Rather, conforming implementations are required to emulate (only) the
> observable behavior of the abstract machine as explained below. 5
> Footnote 5) This provision is sometimes called the “as-if” rule, because an implementation is free to disregard any requirement of
> this
> International Standard as long as the result is as if the requirement had been obeyed, as far as can be determined from the
> observable behavior of the program. For instance, an actual implementation need not evaluate part of an expression if it can
> deduce that its value is not used and that no side effects affecting the observable behavior of the program are produced.
>
>
> Thanks.
> From this I understand that the standard defines an abstract machine which follows a well-defined set of rules. The implementations do not
> have to mimic the very same behavior as long as the actual outcome is the same ("conforming implementations are required to emulate (only)
> the observable behavior of the abstract machine"). Since (a+b)+c and a+(b+c) result into two different observable behaviors, I would
> assume this sentence prohibits this kind of reassociation.
>
>
> No, the observable behavior of a program is not the precise set of instructions and operands executed. It is actually defined in

> [intro.execution]/8 <http://eel.is/c++draft/intro.execution#8>.

> However, you are right in that the value could be different, eventually altering the observable behavior. Nonetheless, some implementations do
> support optimizations that are nonconforming. This specific example is mentioned in the description of GCC's -funsafe-math-optimizations flag

> <https://gcc.gnu.org/wiki/FloatingPointMath>:

>
> This mode enables optimizations that allow arbitrary reassociations and transformations with no accuracy guarantees. It also does not try
> to preserve the sign of zeros. […] Due to roundoff errors the associative law of algebra do not necessary hold for floating point numbers
> and thus expressions like (x + y) + z are not necessary equal to x + (y + z).
>
>
>
> Yes, I'm aware of a few nonconforming optimizations performed by some compilers. But I'm mostly interested on the standard itself. And actually,

> [into.execution]/9 <http://eel.is/c++draft/intro.execution#9> seems to be exactly what I need:

>
> Operators can be regrouped according to the usual mathematical rules only where the operators really are associative or commutative.
>
>
> So, apparently the compilers are not allowed to reassociate the expressions involving floating point operations.
>
>
> I still can't find any mentions to the FMA instructions. From what I read, generating FMA instructions for expressions such as a*b+c is nonconforming,
> since this would give different results from the canonical interpretation of the expression, which is (a*b)+c.
>

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Discussion" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to std-discussio...@isocpp.org

> <mailto:std-discussio...@isocpp.org>.
> To post to this group, send email to std-dis...@isocpp.org <mailto:std-dis...@isocpp.org>.

[Mathias Gaunard]

On 23 March 2016 at 14:17, Andrea Arteaga <andys...@gmail.com> wrote:

In any case, for a*b+c*d, both fma(a, b, c*d) and fma(c, d, a*b) are valid transformations since + is commutative.

+ is commutative even in floating point algebra, but + and * are not associative, so the result changes (or may change).

Well, you have the two following possible chains of transformations:
 - a*b+c*d -> fma(a, b, c*d)
 - a*b+c*d -> c*d+a*b -> fma(c, d, a*b)

As you can see, it doesn't matter how you construct the fma.

 

 

As was said earlier, the language allows any sequence of operations to be evaluated with extra precision, and you cannot predict which ones. (you can however prevent it by going through volatile memory or using compiler pragmas).

Can you point to the place in the standard where this is stated?

I don't understand, wasn't that already given by Richard higher in the thread? [expr]/12.

[Myriachan]

On Wednesday, March 23, 2016 at 11:23:07 AM UTC-7, Mathias Gaunard wrote:

On 23 March 2016 at 14:17, Andrea Arteaga <andys...@gmail.com> wrote:

In any case, for a*b+c*d, both fma(a, b, c*d) and fma(c, d, a*b) are valid transformations since + is commutative.

+ is commutative even in floating point algebra, but + and * are not associative, so the result changes (or may change).

Well, you have the two following possible chains of transformations:
 - a*b+c*d -> fma(a, b, c*d)
 - a*b+c*d -> c*d+a*b -> fma(c, d, a*b)

As you can see, it doesn't matter how you construct the fma.

 

Wikipedia quotes an example in which fused multiply-add would break the observable behavior compared to an abstract machine:  (x*x) - (y*y) becoming std::fma(x, x, -(y * y)) can result in a negative answer even if x == y under certain conditions, possibly breaking a subsequent square root.

https://en.wikipedia.org/wiki/Multiply%E2%80%93accumulate_operation

Melissa

[christophe...@gmail.com]

The C++ standard defines a strict operator precedence and associativity, so for instance a+b+c must be interpreted by the compiler as (a+b)+c and not as a+(b+c); a compiler which interprets the expression according to the second way is not strictly standard compliant

I'm relatively new to this particular field and I'm following this interesting discussion because I'm a bit lost regarding the OP statement, the answers and I've got a hard time to understand the standard. At first I had the same view as OP regarding reassociation, but if I get it right, the standard states that the compiler is allowed to do reassociation if it doesn't change the observable behaviour of the program. Hence for a subset of programs, reassociation is allowed. Right?

However, I found this report from the Intel Software Solutions Group (p. 4):
https://software.intel.com/sites/default/files/article/164389/fp-consistency-102511.pdf

In which the authors state:

The ANSI C and C++ language standards do not permit reassociation by the compiler; even in the absence of parentheses, floating point expressions are to be evaluated from left to right.

So what should I assume ? That for a C++ compiler following the standard, reassociation may occurs ? Is there a mistake in this report ?

If somebody can help me to have a more clearer of view of the answers and the C++ standard, this would be great. Thank you.

[Christopher Jefferson]

On 29 March 2016 at 10:09, <christophe...@gmail.com> wrote:
>
> I'm relatively new to this particular field and I'm following this
> interesting discussion because I'm a bit lost regarding the OP statement,
> the answers and I've got a hard time to understand the standard. At first I
> had the same view as OP regarding reassociation, but if I get it right, the
> standard states that the compiler is allowed to do reassociation if it
> doesn't change the observable behaviour of the program. Hence for a subset
> of programs, reassociation is allowed. Right?

Yes, the compiler can always do anything transformation which doesn't

change the observable behaviour of the program.

>

> However, I found this report from the Intel Software Solutions Group (p. 4):
> https://software.intel.com/sites/default/files/article/164389/fp-consistency-102511.pdf
>
> In which the authors state:
>>
>> The ANSI C and C++ language standards do not permit reassociation by the
>> compiler; even in the absence of parentheses, floating point expressions are
>> to be evaluated from left to right.
>
>
> So what should I assume ? That for a C++ compiler following the standard,
> reassociation may occurs ? Is there a mistake in this report ?

This is a slight simplification. What they here is that the compiler
does not have permission to rearrange expressions such that different
results would be produced. The usual "don't change observable
behaviour" rule applies, but is in general almost useless when it
comes to rearranging floating point expressions, as it is very hard to
prove any transformation of a floating point expression will not
change the result, except in very limited cases (such as the floating
point numbers are known to be small and integer, for example).

[Mathias Gaunard]

On 28 March 2016 at 19:17, Myriachan <myri...@gmail.com> wrote:

Wikipedia quotes an example in which fused multiply-add would break the observable behavior compared to an abstract machine:  (x*x) - (y*y) becoming std::fma(x, x, -(y * y)) can result in a negative answer even if x == y under certain conditions, possibly breaking a subsequent square root.

https://en.wikipedia.org/wiki/Multiply%E2%80%93accumulate_operation

The result is changed by using higher precision, so the change is observable. The rule that allows the compiler to change anything so long as it doesn't lead to observable behaviour is not relevant here.

The relevant rule is the one that allows a compiler to arbitrarily execute operations with more precision than asked.

Yes it can break code, doesn't change the fact that the standard allows it. If you care about exact results in floating-point, you need to be careful.

[kalum...@ghs.nsw.edu.au]

On Wednesday, March 23, 2016 at 11:15:53 AM UTC+11, Andrea Arteaga wrote:

The C++ standard defines a strict operator precedence and associativity, so for instance a+b+c must be interpreted by the compiler as (a+b)+c and not as a+(b+c); a compiler which interprets the expression according to the second way is not strictly standard compliant (a few compilers, like GCC and ICC, allow to specify a flag which disables all those optimizations which can cause reassociation like in this case).

With C++11 the new mathematical function FMA is introduced, which allows to perform operations like a*b+c in a single hardware instruction. The proper language construct to use a FMA is the function std::fma(a, b, c), but, to my knowledge, the compiler can also interpret the expression

a*b+c as an FMA and generate the same instruction. In this case the group */+ used in that expression can be considered as a single ternary operator instead of a couple of binary operators.
udloyurui8turc252

The question is whether such an interpretation is allowed, prescribed or regulated by the standard. What happen in the case of the expression

a*b + c*d? Is the compiler allowed to reformulate it as fma(a, b, c*d) and as fma(c, d, a*b)?

[kalum...@ghs.nsw.edu.au]

hrurset6yfdx

[kalum...@ghs.nsw.edu.au]

On Wednesday, March 23, 2016 at 11:15:53 AM UTC+11, Andrea Arteaga wrote:

[Faseeh X]

--

---
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-discussio...@isocpp.org.

[Faseeh X]

Hello !

Can some one answer that question (http://stackoverflow.com/questions/36331304/got-an-error-error-function-definition-does-not-declare-parameters)?

Thanks

--

---
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-discussio...@isocpp.org.

[Andrea Arteaga]

Dear Faseeh X,

your question has nothing to do with the topic of this discussion. We are trying to keep the internet an ordered place where it is easy to find things, and your message does not help.

I see you already have some answers in that stackoverflow page, so I would redirect you, well... to your stackoverflow question page. Or to /dev/null...

Don't take this as an unhelpful behavior. I'm trying to help you by keeping the discussions clean of off-topics comments. Here and everywhere on the internet ;-)

[Hyman Rosen]

@Andrea, Surprise!  Faseeh X's off-topic question was, in fact, the last disordered portion of the internet, and thanks to you, the internet is now entirely in order!  Good work, and you can now take a well-deserved vacation.  I am offering you a free cruise to the Bahamas.  If you're interested, please click here: http://tinyurl.com/3uarh.

[Andrea Arteaga]

Just one more relevant question (in spite of the many spammers here).

The fortran standard explicitely specifies that reassociation and other reinterpretations of the code due to abstract arithmetic are allowed. In the F2008 draft [1] you can find this on page 141, section 7.1.5.2.4-2:

"Once the interpretation of a numeric intrinsic operation is established, the processor may evaluate any mathematically equivalent expression, provided that the integrity of parentheses is not violated."

Does the C++ standard have anything as explicit, either in favor of reinterpretation or against?

[1] http://www.j3-fortran.org/doc/year/10/10-007.pdf

On Mon, Apr 4, 2016 at 2:42 AM, Kalum Moore <kalum...@ghs.nsw.edu.au> wrote:

hgjgdgtumnbbgjcwsai

On Mon, Apr 4, 2016 at 10:34 AM, Kalum Moore <kalum...@ghs.nsw.edu.au> wrote:

hfhfg

On Mon, Apr 4, 2016 at 10:29 AM, Kalum Moore <kalum...@ghs.nsw.edu.au> wrote:

djgskr

[Mathias Gaunard]

On 5 April 2016 at 08:43, Andrea Arteaga <andys...@gmail.com> wrote:

Just one more relevant question (in spite of the many spammers here).

The fortran standard explicitely specifies that reassociation and other reinterpretations of the code due to abstract arithmetic are allowed. In the F2008 draft [1] you can find this on page 141, section 7.1.5.2.4-2:

"Once the interpretation of a numeric intrinsic operation is established, the processor may evaluate any mathematically equivalent expression, provided that the integrity of parentheses is not violated."

Does the C++ standard have anything as explicit, either in favor of reinterpretation or against?

[1] http://www.j3-fortran.org/doc/year/10/10-007.pdf

Not as far as I know, though I think that some compilers might implement that behaviour as an extension (e.g. fast math mode). 

[Kalum Moore]

tyrr😘

On Mon, Apr 4, 2016 at 10:42 AM, Kalum Moore <kalum...@ghs.nsw.edu.au> wrote:

hgjgdgtumnbbgjcwsai

On Mon, Apr 4, 2016 at 10:34 AM, Kalum Moore <kalum...@ghs.nsw.edu.au> wrote:

hfhfg

On Mon, Apr 4, 2016 at 10:29 AM, Kalum Moore <kalum...@ghs.nsw.edu.au> wrote:

djgskr

On Fri, Apr 1, 2016 at 7:15 PM, Faseeh X <fase...@gmail.com> wrote:

[Kalum Moore]

hsdsdad

[Kalum Moore]

uty8if

g

[Kalum Moore]

siejsnddjskj3wwofdxjajrdkalksakedogajkrdskzkswskfdk dfjdkawsskfsdkadxkdxkzkdsskskddkskdjdddjasjrdjajdsjsjddjsjsjsjsfddhshedususududxsjsddsuddssureushdsuedusududududsiuedudueudsufdusuedufdusudwurdusufududufjejdsjdsjdsiwskgcjzndxksjfdiussjgvnswsnfrduyhbv835jfdj

On Mon, Apr 4, 2016 at 10:30 AM, Kalum Moore <kalum...@ghs.nsw.edu.au> wrote:

dmewj2qjd

To unsubscribe from this group and all its topics, send an email to std-discussion+unsubscribe@isocpp.org.