AvoidFloatingPointEqualityRule for Decimal
fafa
In AvoidFloatingPointEqualityRule Rule documentation (http://www.mono-
project.com/
Gendarme.Rules.Correctness#AvoidFloatingPointEqualityRule), it's
indicated to use decimal comparaison as good example. But, With
Decimal, there could have the same problem. See
http://msdn.microsoft.com/en-us/library/system.decimal.aspx in block
Remarks :
decimal dividend = Decimal.One;
decimal divisor = 3;
// The following displays 0.9999999999999999999999999999 to the
console
Console.WriteLine(dividend/divisor * divisor);
So, the use of Decimal only minimizes rounding errors. So, it's maybe
also a defect with severity medium ?
Regards,
Fabien
Jesse Jones
The rule should definitely be checking decimal numbers. And, while
decimal numbers may have smaller errors than floating point numbers,
they will still have errors so they cannot be trivially compared so I
don't think they should have a reduced severity. I also think the good
example that uses decimal math is completely wrong.
I'm also dubious about the `Math.Abs (f1 - f2) < delta` good example.
It says "this will work with known value but in real-life you may hit
[Positive|Negative]Infinity and NaN" which is true enough for
infinities, but NaNs will work fine. It also elides what I think is
the more important problem in that it's difficult to use that method
to compare numbers with large or small magnitudes.
-- Jesse
Sebastien Pouliot
> Hello,
>
> In AvoidFloatingPointEqualityRule Rule documentation (http://www.mono-
> project.com/
> Gendarme.Rules.Correctness#AvoidFloatingPointEqualityRule), it's
> indicated to use decimal comparaison as good example. But, With
> Decimal, there could have the same problem.
The underlying problem is a different but you're totally right that
decimals, while better, are not a complete solution, nor the right to
promote.
> See
> http://msdn.microsoft.com/en-us/library/system.decimal.aspx in block
> Remarks :
>
> decimal dividend = Decimal.One;
> decimal divisor = 3;
> // The following displays 0.9999999999999999999999999999 to the
> console
> Console.WriteLine(dividend/divisor * divisor);
> So, the use of Decimal only minimizes rounding errors.
It does not minimize them, it removes them. 1 / 3 = 0.333... up to your
precision and 0.333... * 3 = 0.999... (again up to your precision).
There's _no_ error in this (unlike what you can get with FP values) but
it's not what people expect either ;-)
Since precision is not infinite you'll still have to deal with it - and
it does affect comparison. The "nice" thing is that you know it will
only affect the last digit (per calculation) unlike regular FP values.
> So, it's maybe
> also a defect with severity medium ?
It should be reported. Unsure about severity, it may be less likely but
decimals are generally used, with great performance impact, when
precision is important (like financial software) so the defects should
not be downplayed.
I will have a look at it soon (but that could be after my vacations,
next week ;-)
Thanks for the report!
Sebastien
Sebastien Pouliot
On Sun, 2009-02-22 at 20:39 -0800, Jesse Jones wrote:
>
> On Feb 22, 2009, at 1:07 PM, fafa wrote:
>
> >
> > Hello,
> >
> > In AvoidFloatingPointEqualityRule Rule documentation (http://www.mono-
> > project.com/
> > Gendarme.Rules.Correctness#AvoidFloatingPointEqualityRule), it's
> > indicated to use decimal comparaison as good example. But, With
> > Decimal, there could have the same problem. See
> > http://msdn.microsoft.com/en-us/library/system.decimal.aspx in block
> > Remarks :
> >
> > decimal dividend = Decimal.One;
> > decimal divisor = 3;
> > // The following displays 0.9999999999999999999999999999 to the
> > console
> > Console.WriteLine(dividend/divisor * divisor);
> >
> > So, the use of Decimal only minimizes rounding errors. So, it's maybe
> > also a defect with severity medium ?
>
> The rule should definitely be checking decimal numbers. And, while
> decimal numbers may have smaller errors than floating point numbers,
> they will still have errors so they cannot be trivially compared so I
> don't think they should have a reduced severity. I also think the good
> example that uses decimal math is completely wrong.
>
> I'm also dubious about the `Math.Abs (f1 - f2) < delta` good example.
AFAIK this should work unless you compare NaN with a NaN - but that
would return false too using == (or Equals).
> It says "this will work with known value but in real-life you may hit
> [Positive|Negative]Infinity and NaN" which is true enough for
> infinities, but NaNs will work fine. It also elides what I think is
> the more important problem in that it's difficult to use that method
> to compare numbers with large or small magnitudes.
Got some test cases where this leads to different results (with
constants and a small* delta) than == or Equals ?
*lower than 0 and bigger than epsilon
If so please share them. I'm not married with the technique but this is
a very common one so if something is wrong with it then maybe we should
have a rule for it :-)
Sebastien
Jesse Jones
On Feb 23, 2009, at 5:09 PM, Sebastien Pouliot wrote:
>
> It does not minimize them, it removes them. 1 / 3 = 0.333... up to
> your
> precision and 0.333... * 3 = 0.999... (again up to your precision).
> There's _no_ error in this (unlike what you can get with FP values)
> but
> it's not what people expect either ;-)
>
> Since precision is not infinite you'll still have to deal with it -
> and
> it does affect comparison. The "nice" thing is that you know it will
> only affect the last digit (per calculation) unlike regular FP values.
They are only nice because they are base10 so they work more like we
expect numbers to work (and because they have more precision). And, of
course, because the precision is finite errors will accumulate just
like regular floating point errors so you cannot naively compare them.
-- Jesse
Jesse Jones
On Feb 23, 2009, at 5:23 PM, Sebastien Pouliot wrote:
>
> Hello Jesse,
>
> On Sun, 2009-02-22 at 20:39 -0800, Jesse Jones wrote:
>>
>>> The rule should definitely be checking decimal numbers. And, while
>> decimal numbers may have smaller errors than floating point numbers,
>> they will still have errors so they cannot be trivially compared so I
>> don't think they should have a reduced severity. I also think the
>> good
>> example that uses decimal math is completely wrong.
>>
>> I'm also dubious about the `Math.Abs (f1 - f2) < delta` good example.
>
> AFAIK this should work unless you compare NaN with a NaN - but that
> would return false too using == (or Equals).
The NaN case works as expected: f1 - f2 will return a Nan if f1 and/or
f2 is a NaN and the comparison will return false. So, NaN won't
compare equal to anything, not even itself, which is correct.
Otoh if f1 and f2 are both +infinity then it will not work: infinity -
infinity is a NaN so the compare will fail which is incorrect since
infinity is an equatable value.
>
>
>> It says "this will work with known value but in real-life you may hit
>> [Positive|Negative]Infinity and NaN" which is true enough for
>> infinities, but NaNs will work fine. It also elides what I think is
>> the more important problem in that it's difficult to use that method
>> to compare numbers with large or small magnitudes.
>
> Got some test cases where this leads to different results (with
> constants and a small* delta) than == or Equals ?
>
> *lower than 0 and bigger than epsilon
>
> If so please share them. I'm not married with the technique but this
> is
> a very common one so if something is wrong with it then maybe we
> should
> have a rule for it :-)
It's common because it's easy and because it normally works. The
problem is (again) the limited precision of floating point (and
decimal) numbers. They are formatted like this: mantissa *
base^exponent. Now if you subtract two nearly equal numbers with large
exponents you are going to get a number with a mantissa having zero in
most of its bits, but the same large exponent. This is not going to
work with a delta like 0.001. To do this right you need to scale your
delta using the magnitudes of the numbers you're comparing.
Here's an illustration. This code:
double x = 1.0e20;
double y = x + x/1.0e10;
Console.WriteLine("x = {0}", x);
Console.WriteLine("y = {0}", y);
Console.WriteLine("x - y = {0}", Math.Abs(x - y));
Prints this for me:
x = 1E+20
y = 1.0000000001E+20
x - y = 10000007168
-- Jesse
Sebastien Pouliot
That's a good example. Most of the time I've been confronted with
comparison with 0 (most critical) and 1 (libgdiplus) and not with large
values. Can you suggest something as the replacement ?
Thanks,
Sebastien
p.s. I'm on vacations until March 9th so it will take some time before I
can reply. Don't hesitate to discuss without me :-)
Jesse Jones
Well instead of:
/// Comparing floating points values isn't easy, because simple
values, such as 0.2,
/// cannot be precisely represented. This rule ensures the code
doesn't contains
/// floating point [in]equality comparison for <c>Single</c> and
<c>Double</c> values.
I would say something like:
/// Comparing floating points values is tricky because not all values
can be represented
/// precisely, because errors accumulate due to rounding errors, and
because of special
/// values like NaN, infinity, and +/- zero. Because of these problems
it is normally a
/// mistake to compare <c>Single</c> and <c>Double</c> values using
operator ==
/// or !=.
And instead of:
const float delta = 0.000001;
void AMethod ()
{
float f1 = 0.1;
float f2 = 0.001 * 100;
if (Math.Abs (f1 - f2) <= delta) {
// this will work with known value but in real-life
// you may hit [Positive|Negative]Infinity and NaN
}
}
I would have:
public static class DoubleExtensions
{
// Returns true if the two floating point numbers are close to
each other.
// Note that if very large values are compared delta should also
be made larger.
public static bool NearlyEqual(this double x, double y, double
delta = 0.0000001)
{
if (double.IsInfinity(x) || double.IsInfinity(y))
return x == y;
return Math.Abs (x - y) <= delta
}
}
I do have a version of NearlyEqual I cooked up for Smokey that scales
the delta automagically, but I'm not sure it's worth including it: few
people are in the habit of comparing really big numbers.
-- Jesse