Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Decimal Precision - short code, strange problem

0 views
Skip to first unread message

saar...@gmail.com

unread,
Feb 6, 2006, 12:09:06 PM2/6/06
to
Hi

Could anyone help me to understand why in the following code sum1 is
different from sum2?
Basically it's the same calculation done. Keep in mind the x is 28
digits so it's within the range of decimal.

<codesnip>
decimal x = (-0.0084682975822291150192357277M);
decimal sum1 , sum2;

sum2 = (3500.91M * x) + (x * 3500M + 822m);
sum1 = (3500.91M * x) + (x * 3500M) + 822m;

Console.WriteLine (sum1);
Console.WriteLine (sum2);
</codesnip>

Thank you,
Saar Carmi

Jon Skeet [C# MVP]

unread,
Feb 6, 2006, 12:39:55 PM2/6/06
to

I believe I understand the reason for the difference. Suppose you have
a type which is accurate to 3 significant digits. Now consider the
difference between:

100 + 1.49 + 0.03
and
100 + (1.49 + 0.03)

The first would be evaluated as:

(100 + 1.49) + 0.03
=> 101 + 0.03
=> 101

The second is:

100 + (1.49 + 0.03)
=> 100 + 1.52
=> 102

Do you follow why the order is important?

I haven't checked whether or not that's what's happening in your case,
but I suspect it is. You could print out each part of the sum and do it
manually to verify it.

--
Jon Skeet - <sk...@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

saar...@gmail.com

unread,
Feb 7, 2006, 4:06:05 AM2/7/06
to
Seems you're right.
Thank you.

0 new messages