Rounding Errors
0 views
Skip to first unread message
Mark
Jun 16, 2011, 12:28:37 AM6/16/11
to BYU CS 142 (Spring 2011) [McCarthy]
Hey,
I'm working on the array problem on 20.6, and when I do the math,
exactly one member of the whole thing is having rounding errors. As
in, it's supposed to be 2.5, but instead it's 2.4999999999997. Is this
a good enough answer? I imagine not. What do I do to fix this kind of
thing?
-Mark
I'm working on the array problem on 20.6, and when I do the math,
exactly one member of the whole thing is having rounding errors. As
in, it's supposed to be 2.5, but instead it's 2.4999999999997. Is this
a good enough answer? I imagine not. What do I do to fix this kind of
thing?
-Mark
Jay McCarthy
Jun 16, 2011, 7:31:02 AM6/16/11
to byu-cs-jm-14...@googlegroups.com
That's fine. Remember that Java numbers are not the reals, so Math is a bit off.
Jay
2011/6/16 Mark <Ma...@morrowlife.net>:
--
Jay McCarthy <j...@cs.byu.edu>
Assistant Professor / Brigham Young University
http://faculty.cs.byu.edu/~jay
"The glory of God is Intelligence" - D&C 93
Reply all
Reply to author
Forward
0 new messages