JStat and JS floating point errors?

81 views
Skip to first unread message

muh...@gmail.com

unread,
Jan 9, 2013, 8:24:33 AM1/9/13
to jstat-...@googlegroups.com
I've tried searching and reading the docs, but don't find anything on this. Does JStat use BigNumber or BigDecimal to handle Javascript floating point errors? We're doing a lot of math where we don't see differences until the 8th or 10th decimal place.

Thanks!

Trevor Norris

unread,
Jan 10, 2013, 12:41:29 PM1/10/13
to jstat-...@googlegroups.com, muh...@gmail.com
No external libraries are used in jStat. We've been using and modifying algorithms specifically for the numeric precision of javascript's IEEE754 floating point values. This allows for much quicker calculations when you only need around 8 digits of accuracy.

Though there is a case where iterating over the same value can accumulate error. It hasn't been of concern yet, but if it does become a problem I would like to know.
Reply all
Reply to author
Forward
0 new messages