IEEE-754 DP-FP rounding question
VK
My question came from the FAQ discussion on comp.lang.javascript
related to IEEE-754 double-precision floating-point standard: IEEE-754-
DP-FP any further, if no one is against of a lingo shortcut.
Currently I'm trying to disambiguate the native IEEE-754-DP-FP
behavior out of internal convenience neuristic added atop of it in
different ECMAScript implementations. Some points are successfully
cleared out, but one thing puzzles me - and sorry if I'm wrong in my
reasoning.
For instance, decimal float 1.035 is not representable as a dyadic
fraction, so no way can be represented as a finite binary sequence, so
no way can be stored in exact form in IEEE-754-DP-FP
The production check seems confirming that, as decimal 1.035 in
IEEE-754-DP-FP emulator being stored as:
0 01111111111 0000100011110101110000101000111101011100001010001111
S EEEEEEEEEEE FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
There is no need to add hidden bit and to resolve mantissa to see that
it is far from anything like 1.035/out
At the same time a primitive practical check like:
var probe = 1.035;
window.alert(probe); // displays "1.035"
shows that somehow toString method knows what the original input was -
at least until further math operations are made with the number. I
would understand some rounding done before display, but somehow the
system knows _what_ rounding to do, so it doesn't display 1.034999 or
1.3499 - but 1.35, just like in the input source.
I'm not asking to explain ECMAScript engine mechanics - I'm wondering
if this behavior is coming out from IEEE-754-DP-FP itself or is it
some added neuristic to look for.
If I was wrong with my reasoning, pointing to my error is highly
appreciated as well.
Terje Mathisen
> Hello everyone,
>
> My question came from the FAQ discussion on comp.lang.javascript
> related to IEEE-754 double-precision floating-point standard: IEEE-754-
> DP-FP any further, if no one is against of a lingo shortcut.
>
> Currently I'm trying to disambiguate the native IEEE-754-DP-FP
> behavior out of internal convenience neuristic added atop of it in
> different ECMAScript implementations. Some points are successfully
> cleared out, but one thing puzzles me - and sorry if I'm wrong in my
> reasoning.
>
> For instance, decimal float 1.035 is not representable as a dyadic
> fraction, so no way can be represented as a finite binary sequence, so
> no way can be stored in exact form in IEEE-754-DP-FP
> The production check seems confirming that, as decimal 1.035 in
> IEEE-754-DP-FP emulator being stored as:
> 0 01111111111 0000100011110101110000101000111101011100001010001111
> S EEEEEEEEEEE FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
> There is no need to add hidden bit and to resolve mantissa to see that
> it is far from anything like 1.035/out
Huh?
Isn't that fractional binary number exactly what it should be, i.e. the
closest possible binary representation of your decimal input value?
There's a couple of classic papers about converting binary fp numbers
to/from decimal representation, try Google.
Terje
--
- <Terje.M...@hda.hydro.com>
"almost all programming can be viewed as an exercise in caching"
VK
> > IEEE-754-DP-FP emulator being stored as:
> > 0 01111111111 0000100011110101110000101000111101011100001010001111
> > S EEEEEEEEEEE FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
> > There is no need to add hidden bit and to resolve mantissa to see that
> > it is far from anything like 1.035/out
>
> Huh?
>
> Isn't that fractional binary number exactly what it should be, i.e. the
> closest possible binary representation of your decimal input value?
Yes, it _is_ exactly what it should be in IEEE-754-DP-FP conforming
format. I did not question that. My question was... OK... in some more
visual form:
Imagine we have a machine that stretches whatever you put into it and
then it makes a hole on one end. So I put a piece of plastic into into
it - huuur...chomp... - I'm taking out the piece of bigger size and
with hole in it. Everything as expected. Now I put a piece of plastic
into into it - huuur...chomp... - I'm taking out the intact piece as
never nothing - though while pulling out I clearly saw that the
plastic was as deformed as the first one. A miracle one would say.
Now let's us take IEEE-754-DP-FP as the "deformator". I put 1.35 into
it:
var probe = 1.35;
huuur...chomp... the rest of "plastic" was already shown, is correct
and predictable. But: right on the next step I'm taking it out:
window.alert(probe); // it shows "1.35" as if "huuur...chomp..."
never happened.
So my question was: is this "miracle" defined somewhere in IEEE-754-DP-
FP itself or it must be an atop heuristic added to keep the rest of
programmers' sanity? :-)
Nick Maclaren
In article <1171473735....@l53g2000cwa.googlegroups.com>,
"VK" <school...@yahoo.com> writes:
|>
|> So my question was: is this "miracle" defined somewhere in IEEE-754-DP-
|> FP itself or it must be an atop heuristic added to keep the rest of
|> programmers' sanity? :-)
It's an atop heuristic that deludes naive users into believing that
floating-point arithmetic is exact when it isn't.
There is nothing in IEEE 754 that mandates or even encourages such
misleading behaviour.
Regards,
Nick Maclaren.
VK
> In article <1171473735.350804.15...@l53g2000cwa.googlegroups.com>,"VK" <schools_r...@yahoo.com> writes:
>
> |>
> |> So my question was: is this "miracle" defined somewhere in IEEE-754-DP-
> |> FP itself or it must be an atop heuristic added to keep the rest of
> |> programmers' sanity? :-)
>
> It's an atop heuristic that deludes naive users into believing that
> floating-point arithmetic is exact when it isn't.
Thanks for confirming this. For records: I'm a programmer but I'm not
IEEE-754-DP-FP naive. You may mark somewhere that such species exist.
:-)
> There is nothing in IEEE 754 that mandates or even encourages such
> misleading behaviour.
Thanks.
robert...@yahoo.com
> For instance, decimal float 1.035 is not representable as a dyadic
> fraction, so no way can be represented as a finite binary sequence, so
> no way can be stored in exact form in IEEE-754-DP-FP
> The production check seems confirming that, as decimal 1.035 in
> IEEE-754-DP-FP emulator being stored as:
> 0 01111111111 0000100011110101110000101000111101011100001010001111
> S EEEEEEEEEEE FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
> There is no need to add hidden bit and to resolve mantissa to see that
> it is far from anything like 1.035/out
>
> At the same time a primitive practical check like:
>
> var probe = 1.035;
> window.alert(probe); // displays "1.035"
>
> shows that somehow toString method knows what the original input was -
> at least until further math operations are made with the number. I
> would understand some rounding done before display, but somehow the
> system knows _what_ rounding to do, so it doesn't display 1.034999 or
> 1.3499 - but 1.35, just like in the input source.
>
> I'm not asking to explain ECMAScript engine mechanics - I'm wondering
> if this behavior is coming out from IEEE-754-DP-FP itself or is it
> some added neuristic to look for.
IEEE-745 doesn't specify how number should be displayed.
I would assume that windows.alert() simply rounds to some internally
defined target. For example, in C, printf("%f", x); will round to six
decimal places (although you can override that by specifying a
precision), no matter what type of floating point arithmetic the
platform implements (IEEE or not). I don't know of any "double"
formats that wouldn't round back to 1.035 when rounded at the sixth
decimal place.
BTW, the word is "heuristic."
Nick Maclaren
In article <1171501192....@a34g2000cwb.googlegroups.com>,
"robert...@yahoo.com" <robert...@yahoo.com> writes:
|> On Feb 14, 8:31 am, "VK" <schools_r...@yahoo.com> wrote:
|> >
|> > I'm not asking to explain ECMAScript engine mechanics - I'm wondering
|> > if this behavior is coming out from IEEE-754-DP-FP itself or is it
|> > some added neuristic to look for.
|>
Actually, I like "neuristic" - English is a language in which it is quite
permissible to invent nonce words, and many people have done and do so.
Regards,
Nick Maclaren.
Terje Mathisen
> On Feb 14, 8:31 am, "VK" <schools_r...@yahoo.com> wrote:
>> I'm not asking to explain ECMAScript engine mechanics - I'm wondering
>> if this behavior is coming out from IEEE-754-DP-FP itself or is it
>> some added neuristic to look for.
>
> IEEE-745 doesn't specify how number should be displayed.
>
> I would assume that windows.alert() simply rounds to some internally
> defined target. For example, in C, printf("%f", x); will round to six
I believe the real answer is even simpler (or maybe not?): Pretend that
you're first generating an infinitely accurate decimal version of your
binary number, then inspect the result:
If it is less or equal to half a binary ulp away from an exactly
representable number, then you should return that number, particularly
if this results in a much shorter representation.
I.e. you should always try to return the shortest possible decimal value
which when converted back into binary will result in the original number.
Nick Maclaren
In article <266ea4-...@osl016lin.hda.hydro.com>,
Terje Mathisen <terje.m...@hda.hydro.com> writes:
|>
|> I believe the real answer is even simpler (or maybe not?): Pretend that
|> you're first generating an infinitely accurate decimal version of your
|> binary number, then inspect the result:
|>
|> If it is less or equal to half a binary ulp away from an exactly
|> representable number, then you should return that number, particularly
|> if this results in a much shorter representation.
|>
|> I.e. you should always try to return the shortest possible decimal value
|> which when converted back into binary will result in the original number.
That is what quite a few of those systems do, and has been known to be
a BAD idea since time immemorial. The reason is that most naive users
interpret a short decimal string as meaning that the number is exactly
that. The removal of trailing zeroes should be done only if they are
truly zero.
The alternative of having an explicit inexactness flag disappeared some
decades ago, though I have no idea why.
Regards,
Nick Maclaren.
Terje Mathisen
> In article <266ea4-...@osl016lin.hda.hydro.com>,
> Terje Mathisen <terje.m...@hda.hydro.com> writes:
> |>
> |> I believe the real answer is even simpler (or maybe not?): Pretend that
> |> you're first generating an infinitely accurate decimal version of your
> |> binary number, then inspect the result:
> |>
> |> If it is less or equal to half a binary ulp away from an exactly
> |> representable number, then you should return that number, particularly
> |> if this results in a much shorter representation.
> |>
> |> I.e. you should always try to return the shortest possible decimal value
> |> which when converted back into binary will result in the original number.
>
> That is what quite a few of those systems do, and has been known to be
> a BAD idea since time immemorial. The reason is that most naive users
> interpret a short decimal string as meaning that the number is exactly
> that. The removal of trailing zeroes should be done only if they are
> truly zero.
Nick, this is getting awfully close to interval arithmetic!
IA is a good idea,at least in principle, the problem is to figure out
exactly how good/bad the final answer should be shown to be.
I.e. there are many algorithms with quadratic convergence, where the
exact representation doesn't matter at all before the last couple of
iterations, but where naive IA would say that you had no significance at
all.
>
> The alternative of having an explicit inexactness flag disappeared some
> decades ago, though I have no idea why.
It is still there in binary FP, except nobody uses it, right?
Nick Maclaren
In article <vugea4-...@osl016lin.hda.hydro.com>,
Terje Mathisen <terje.m...@hda.hydro.com> writes:
|> > |>
|> > |> I.e. you should always try to return the shortest possible decimal value
|> > |> which when converted back into binary will result in the original number.
|> >
|> > That is what quite a few of those systems do, and has been known to be
|> > a BAD idea since time immemorial. The reason is that most naive users
|> > interpret a short decimal string as meaning that the number is exactly
|> > that. The removal of trailing zeroes should be done only if they are
|> > truly zero.
|>
|> Nick, this is getting awfully close to interval arithmetic!
Oh, I know of no realistic objection to displaying fewer digits; it is
specifically the idea that values that 'look like' simple digit strings
should be displayed as if they were and other ones displayed otherwise.
That deludes the naive into avoidable mistakes. I really am not making
up the fact that the above approach was backed off in the 1970s, because
experience showed it to do more harm than good. That doesn't stop it
being reinvented, because there are damn few of us left who remember
those days ....
|> IA is a good idea,at least in principle, the problem is to figure out
|> exactly how good/bad the final answer should be shown to be.
|>
|> I.e. there are many algorithms with quadratic convergence, where the
|> exact representation doesn't matter at all before the last couple of
|> iterations, but where naive IA would say that you had no significance at
|> all.
Yes, indeed.
|> > The alternative of having an explicit inexactness flag disappeared some
|> > decades ago, though I have no idea why.
|>
|> It is still there in binary FP, except nobody uses it, right?
Is it? That's news to me! Methinks that you need to recheck. I was,
however, referring to its use in output, such as:
1.25 ~4.56 ~1.25
It is in the Python interface I am implementing, but that is choc-full
of heresies :-)
Regards,
Nick Maclaren.
VK
> I would assume that windows.alert() simply rounds to some internally
> defined target. For example, in C, printf("%f", x); will round to six
> decimal places
Very true but it doesn't explain that 1.034xxxxxxxxxxxxxxx get's
rounded back to 1.035 (as it was "typed in") and not as say 1.034xxx
Terje Mathisen's post was the light bolb I guess.
> BTW, the word is "heuristic"
Sorry for that - though I like this term from pseudo-neuronic systems
studies better.
VK
> |> which when converted back into binary will result in the original number.
> That is what quite a few of those systems do, and has been known to be
> a BAD idea since time immemorial.
I dare to say that this question has to be taken diachronically to be
answered properly. It possible _was_ a bad idea at the times of
(international) business machines and "why would a private individual
need a computer?!". It seems as not as such a bad idea - or at least
something absolutely required either is it bad or not - after the PC
revolution. Mary Smith and John Doe came to do their everyday stuff
and 99.9% of software is currently written for these individuals. And
the amount of care these individuals pay to real math vs computer math
issues is measurable in fractions of ulp - if presented at all ;-) By
the time one will finish to explain that 0.06+0.01 != 0.07 and it
should be so: by that time she will loose all clients moved to other
product with "proper" results. This way I see that further
vulgarization of computer math is inevitable - because it affects
noticeably on the merchantability of your product. In the particular
IEEE-754-DP-FP I believe will be kept patched with different heuristic
and screw on math itself wherever it is technically possible.
You may find interesting to read as well:
<http://blogs.msdn.com/ericlippert/archive/2005/01/26/361041.aspx>
I am no way a mathematician - my last mutual :-) tortures with the
classical math ended way over a decade ago together with math
bachelory studies (in linguistics, not in CS). Ever since I programmed
rather a lot, but I doubt I resolved a single quadratic equation ever
since.
The current interest to the subject originated from the discussion
over comp.lang.javascript FAQ such as:
<http://www.jibbering.com/faq/#FAQ4_6>
4.6 How do I convert a Number into a String with exactly 2 decimal
places?
<http://www.jibbering.com/faq/#FAQ4_7>
Why does 5 * 1.015 != 5.075 or 0.06+0.01 != 0.07?
First one user complained that 1.035 rounded to two sign after radix
"still gives 1.03 while it must be 1.04". From the argumentation what
should it really be with 1.35 - from that I became that curious cat
you see right now :-) Funny - or sadly - enough this question seems
having by now two answers: one what it must be by IEEE-754-DP-FP and
the other what the majority of end user want it to be: so an extra
switch may need to be added to the final program.
P.S. Seems OT but really related in some way: any decimal floating
number in form of
x.125
x.375
x.625
x.875
can be presented as a dyadic fraction so as a finite binary sequence.
I tried with xy.125 and it is still true. Is it true that any decimal
number ending with 125, 375, 625 or 875 is representable as a dyadic
fraction. Sounds awfully non-mathematical as a statement: but the lack
of knowledge prevents me to see the general internal pattern.
Nick Maclaren
In article <1171571364.5...@l53g2000cwa.googlegroups.com>,
"VK" <school...@yahoo.com> writes:
|>
|> > |> I.e. you should always try to return the shortest possible decimal value
|> > |> which when converted back into binary will result in the original number.
|>
|> > That is what quite a few of those systems do, and has been known to be
|> > a BAD idea since time immemorial.
|>
|> I dare to say that this question has to be taken diachronically to be
|> answered properly. It possible _was_ a bad idea at the times of
|> (international) business machines and "why would a private individual
|> need a computer?!". It seems as not as such a bad idea - or at least
|> something absolutely required either is it bad or not - after the PC
|> revolution.
On the contrary, as you would know if you had been active in the numeric
area before then. It is TODAY, when the average numeric programmer does
not understand numerical analysis or computer arithmetic, that needs the
protection. It has nothing to do with the IBM mindset.
|> Mary Smith and John Doe came to do their everyday stuff
|> and 99.9% of software is currently written for these individuals. And
|> the amount of care these individuals pay to real math vs computer math
|> issues is measurable in fractions of ulp - if presented at all ;-)
Why not just print the required number of random digits? Look, even
those people need to know the exact amount of money in their bank
account, because the bank assuredly will not clear a cheque because it
is only just above their limit and their limit and the amount are
the same to N decimal places.
|> You may find interesting to read as well:
|> <http://blogs.msdn.com/ericlippert/archive/2005/01/26/361041.aspx>
Not at all. I have dealt with delusions like that for many decades,
and my tolerance isn't increasing as I approach retirement. The issue
isn't what he was claiming, but whether the following should occur:
print x, y
9.2 9.2
print x >= y
False
|> The current interest to the subject originated from the discussion
|> over comp.lang.javascript FAQ such as:
|> <http://www.jibbering.com/faq/#FAQ4_6>
|> 4.6 How do I convert a Number into a String with exactly 2 decimal
|> places?
|>
|> <http://www.jibbering.com/faq/#FAQ4_7>
|> Why does 5 * 1.015 != 5.075 or 0.06+0.01 != 0.07?
Precisely. And, by hiding the reality of floating-point from its
users, you are merely making it harder for them to understand the
reasons.
Regards,
Nick Maclaren.
VK
> those people need to know the exact amount of money in their bank
> account, because the bank assuredly will not clear a cheque because it
> is only just above their limit and their limit and the amount are
> the same to N decimal places.
That is called "bank rounding" and it is the least regulated by IEEE
standards. It is regulated on the governmental level so say 1.035 of
the main monetary unit rounded to X digits after radix in one country
is not the same as in another. So your argument was not the best -
though I understand fully the logic behind it. Please don't take me as
arguing with you and advocating some "lousy math" path. For me let the
math be as correct as it should. The problem is that it is often not a
market option. From your other sample, should the following occur:
print x, y
9.2 9.2
print x >= y
False
paradoxically _yes_: the experience shows that end users are much
happier with that while anything more proper leads to complains and
"bug fixes" requests. It is safer to give them what they want and move
on something like
print ((x - y) <= 0.00001)
instead of
print x >= y
Regards.
robert...@yahoo.com
> On Feb 15, 3:59 am, "robertwess...@yahoo.com"
>
> > I would assume that windows.alert() simply rounds to some internally
> > defined target. For example, in C, printf("%f", x); will round to six
> > decimal places
>
> Very true but it doesn't explain that 1.034xxxxxxxxxxxxxxx get's
> rounded back to 1.035 (as it was "typed in") and not as say 1.034xxx
Sure it does. 1.0349999999 rounded at the sixth decimal place is
1.035000. Truncate a few trailing zeros, and voila.
Now if you implementation rounded 1.034789 at the sixth decimal place
and then displayed 1.035, I'd would be seriously broken.
robert...@yahoo.com
> That is called "bank rounding" and it is the least regulated by IEEE
> standards. It is regulated on the governmental level so say 1.035 of
> the main monetary unit rounded to X digits after radix in one country
> is not the same as in another. So your argument was not the best -
> though I understand fully the logic behind it. Please don't take me as
> arguing with you and advocating some "lousy math" path. For me let the
> math be as correct as it should. The problem is that it is often not a
> market option. From your other sample, should the following occur:
> print x, y
> 9.2 9.2
> print x >= y
> False
> paradoxically _yes_: the experience shows that end users are much
> happier with that while anything more proper leads to complains and
> "bug fixes" requests. It is safer to give them what they want and move
> on something like
> print ((x - y) <= 0.00001)
> instead of
> print x >= y
You can't really do that unless you implement variable precision
fractional representations, with all the overhead that implies. Even
then that breaks down if you ever do something that can result in a
non-rational, or if you have to store an intermediate result (or even
just displaying it to the user) in a non-fractional or limited length
format.
Many years ago I had to patch up a system that printed invoices, on
which, for whatever reason, the customers wanted things like sales tax
and discounts listed on the line items as well as for the totals.
That requirement requires that one of two errors exists. Either the
total of the line-item tax fields will not match the actual tax
amount, due to rounding errors, or at least some of the line items
will have incorrect tax amounts (because to the need to adjust them to
balance out the rounding errors to make the total correct).
We got calls from end users complaining about the errors in both
scenarios.
In short, the problem is "fixable" only in fairly trivial cases.
Which is not to say that fractional-bignum arithmetic might not be the
best choice for programming interfaces exposed to most users, and the
overhead be damned. Yet Nick is quite correct that there is far less
awareness of the limitations of computer arithmetic now than there was
30 years ago. Back when numerical analysis on boxes like the early
Crays was a full contact sport, everyone understood that there were
limitations to how computers do arithmetic, and most competent folks
tried to figure out what that meant for their applications. These
days those same problems exist, although many trivial operations
result in apparently correct results, which lulls people into thinking
that all the results will be acceptable, which they will not (and
cannot) be.
Nick Maclaren
In article <1171579433....@j27g2000cwj.googlegroups.com>,
"robert...@yahoo.com" <robert...@yahoo.com> writes:
|>
|> Many years ago I had to patch up a system that printed invoices, on
|> which, for whatever reason, the customers wanted things like sales tax
|> and discounts listed on the line items as well as for the totals.
|> That requirement requires that one of two errors exists. Either the
|> total of the line-item tax fields will not match the actual tax
|> amount, due to rounding errors, or at least some of the line items
|> will have incorrect tax amounts (because to the need to adjust them to
|> balance out the rounding errors to make the total correct).
|>
|> We got calls from end users complaining about the errors in both
|> scenarios.
Somehow, that fails to surprise me :-(
|> In short, the problem is "fixable" only in fairly trivial cases.
|> Which is not to say that fractional-bignum arithmetic might not be the
|> best choice for programming interfaces exposed to most users, and the
|> overhead be damned. Yet Nick is quite correct that there is far less
|> awareness of the limitations of computer arithmetic now than there was
|> 30 years ago. Back when numerical analysis on boxes like the early
|> Crays was a full contact sport, everyone understood that there were
|> limitations to how computers do arithmetic, and most competent folks
|> tried to figure out what that meant for their applications. These
|> days those same problems exist, although many trivial operations
|> result in apparently correct results, which lulls people into thinking
|> that all the results will be acceptable, which they will not (and
|> cannot) be.
Precisely.
Regards,
Nick Maclaren.
Dik T. Winter
> On Feb 15, 3:59 am, "robertwess...@yahoo.com"
> > I would assume that windows.alert() simply rounds to some internally
> > defined target. For example, in C, printf("%f", x); will round to six
> > decimal places
>
> Very true but it doesn't explain that 1.034xxxxxxxxxxxxxxx get's
> rounded back to 1.035 (as it was "typed in") and not as say 1.034xxx
The number 1.035 rounded to DP binary is:
1.0000100011110101110000101000111101011100001010001111
(as your original posting did show), that is exactly:
1.0349999999999999200639422269887290894985198974609375
in decimal. Rounding that to 15 or fewer decimals after the decimal
point yields 1.035, you need at least 16 decimals after the point to
get something different.
--
dik t. winter, cwi, kruislaan 413, 1098 sj amsterdam, nederland, +31205924131
home: bovenover 215, 1025 jn amsterdam, nederland; http://www.cwi.nl/~dik/
glen herrmannsfeldt
(snip)
> Many years ago I had to patch up a system that printed invoices, on
> which, for whatever reason, the customers wanted things like sales tax
> and discounts listed on the line items as well as for the totals.
> That requirement requires that one of two errors exists. Either the
> total of the line-item tax fields will not match the actual tax
> amount, due to rounding errors, or at least some of the line items
> will have incorrect tax amounts (because to the need to adjust them to
> balance out the rounding errors to make the total correct).
Taxes are funny that way. I do remember the Maryland sales tax
from some years ago, which tended to round up much more often than
you might expect.
I remember some years ago after a winery tour, in line to buy some
wine. I had figure out from watching those in front of me that it
cost one cent less to buy two bottles separately, than to buy them
together. I presume, then, that sometimes your rounding will give
different tax amounts to two items that have the same price.
-- glen
robert...@yahoo.com
wrote:
> Taxes are funny that way. I do remember the Maryland sales tax
> from some years ago, which tended to round up much more often than
> you might expect.
>
> I remember some years ago after a winery tour, in line to buy some
> wine. I had figure out from watching those in front of me that it
> cost one cent less to buy two bottles separately, than to buy them
> together. I presume, then, that sometimes your rounding will give
> different tax amounts to two items that have the same price.
Correct. Consider an invoice with three lines items, each for 33
cents. Then assume 10% tax. So the tax on the invoice is going to be
10 cents. If you try to expand that on each line item you either get
three cents on each line item (and you're obviously missing a penny
someplace), or two of the 33 cent line items get three cents of tax
and the other 33 cent line item gets four cents.
The obvious answer is "don't do that," but it's certainly not
unreasonable that a customer would want to know how much tax he paid
on the widgets in line item #3.
glen herrmannsfeldt
(snip)
> Correct. Consider an invoice with three lines items, each for 33
> cents. Then assume 10% tax. So the tax on the invoice is going to be
> 10 cents. If you try to expand that on each line item you either get
> three cents on each line item (and you're obviously missing a penny
> someplace), or two of the 33 cent line items get three cents of tax
> and the other 33 cent line item gets four cents.
> The obvious answer is "don't do that," but it's certainly not
> unreasonable that a customer would want to know how much tax he paid
> on the widgets in line item #3.
Not so different, though, if I buy items that are, say, 3 for $1
they come out on the receipt as
0.34, 0.33, 0.33.
It doesn't seem so far off, then, to assign each item the
marginal tax, which could be very different than the appropriate
percentage of the items price. So again, say 10% tax then it
might go something like:
item price tax total
first 0.14 0.01 0.15
second 0.01 0.01 0.17
third 0.09 0.00 0.26
assuming usual rounding.
-- glen
Mike Cowlishaw
>> > defined target. For example, in C, printf("%f", x); will round to six
>> > decimal places
>>
>> Very true but it doesn't explain that 1.034xxxxxxxxxxxxxxx get's
>> rounded back to 1.035 (as it was "typed in") and not as say 1.034xxx
>
>
> Sure it does. 1.0349999999 rounded at the sixth decimal place is
> 1.035000. Truncate a few trailing zeros, and voila.
But if you are rounding to two decimal places (pennies or cents) using
Banker's Rounding (round to nearest, ties to even) you would have got the
wrong answer, quite possibly. See:
http://www2.hursley.ibm.com/decimal/decifaq1.html#inexact
item #2 for a specific example.
mfc
Mike Cowlishaw
"Nick Maclaren" <nm...@cus.cam.ac.uk> wrote in message
news:er1r7r$fke$1...@gemini.csx.cam.ac.uk...
> |> I.e. you should always try to return the shortest possible decimal
> value
> |> which when converted back into binary will result in the original
> number.
>
> That is what quite a few of those systems do, and has been known to be
> a BAD idea since time immemorial. The reason is that most naive users
> interpret a short decimal string as meaning that the number is exactly
> that. The removal of trailing zeroes should be done only if they are
> truly zero.
I agree .. although probably one trailing zero is a good compromise between
brevity and delusion. For example, if 0.1 is assigned to a double and then
printed using default formatting (Java's Double.toString, for example) then
a display string of "0.10" would alert the reader to a possible inaccuracy.
[And it evokes a different convention, in which adding a trailing zero to an
integer often indicates that the result should be a binary floating-point
value.] A representation such as "0.1?" might be even better.
Steele & White are probably responsible for more hours of confused
programmer time than most.
mfc
Nick Maclaren
In article <er4mvo$gel$1$830f...@news.demon.co.uk>,
"Mike Cowlishaw" <m...@uk.ibm.com> writes:
|>
|> [And it evokes a different convention, in which adding a trailing zero to an
|> integer often indicates that the result should be a binary floating-point
|> value.] A representation such as "0.1?" might be even better.
Where did that one come from? I am familiar with the convention of
adding a '.0' to indicate floating-point rather than integral, but
I can't remember any circumstance in which it implied any particular
base.
Regards,
Nick Maclaren.
glen herrmannsfeldt
PL/I does them with a trailing B, but the significand is also in binary,
though the exponent is decimal.
1100100.1000011111101101010100010e-5B
compilers are usually good at converting FLOAT DECIMAL constants
to binary, so it is rarely done.
-- glen