On Saturday, April 30, 2022 at 8:31:04 AM UTC-7, Skybuck Flying wrote:
> When exception masks are all enabled to stop the processor from throwing floating point exceptions the following calculation produces a somewhat strange result:
> 0.0 / 0.0 = -nan
Totally expected. 0 / 0 is undefined. This should've been covered in middle school.
> (At least in Delphi).
> For now I will assume this is the case in C/C++ as well and with that I mean on x86/x64 which should and seems to be following IEEE 754 floating-point format.
> I am a little bit surprised by this and I want/need to know more. Where is this defined that 0.0 / 0.0 should be -NAN ?!?
> Problem is with the code, example:
> T := 0;
> D := 0.0 / 0.0;
> P := T * D;
> This screws up P. instead of P being zero, P is now also -NAN ?!?
What you're seeing is on purpose and by design.
Suppose you had sqrt(-1e-30) in place of 0/0, that is, your
computations had some rounding errors and the square root
got a negative argument instead of zero or a small positive argument.
If we're staying away from complex values, square root of
negative argument is undefined as well.
But the CPU doesn't know that it may, for example, ignore this error
in the context of your formula and simply substitute 0 for sqrt(-1e-30).
So, instead it returns a NAN for sqrt(-1e-30) and poisons with it
all further computations by propagating the NAN all the way through
all operations to the final result.
This way the programmer may find out that their result had an
anomaly during computation and something needs to be done with it.
You can also think of it in terms of "how do I hide a boolean or
an if statement in a pure math formula/expression". This is how.