Re: Mathematical accuracy in xpath

[Martin Honnen]

kinobo wrote:
> If I execute the following xslt I get the result given below.
> I'm no mathematician but 100 times 2500.01 does not equal 250001.00000000003

Well XPath 1.0 uses double floating point numbers like JavaScript or
Java do too and with these there is limited precision, in particular
when there is no finite binary representation of a number.

With XSLT 1.0 you can use format-number to ensure you get a certain
number of decimal positions. Or you need to change to XPath 2.0 (as
implemented by Saxon 9) and use xs:decimal numbers instead of double
numbers.

--

Martin Honnen --- MVP XML
http://msmvps.com/blogs/martin_honnen/

[kinobo]

If I execute the following xslt I get the result given below.
I'm no mathematician but 100 times 2500.01 does not equal 250001.00000000003

I can find very little about this on the web and no suggested fixes or
solutions.
This is causing me a big problem as my entire architecture hangs around xslt.
Using VS2008 and have IE8 installed.
Can anyone help? Please?

<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform" >
<xsl:template match="/">
<res>
<xsl:value-of select="2500.01 * 100"/>
</res>
</xsl:template>
</xsl:stylesheet>

<?xml version="1.0" encoding="utf-8"?>
<res>250001.00000000003</res>

[Thomas Braun]

Martin Honnen wrote:

> kinobo wrote:
>> If I execute the following xslt I get the result given below.
>> I'm no mathematician but 100 times 2500.01 does not equal 250001.00000000003
>
> Well XPath 1.0 uses double floating point numbers like JavaScript or
> Java do too and with these there is limited precision, in particular
> when there is no finite binary representation of a number.

For more details, he (she?) might also look here:

http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems

Thomas

[kinobo]

Thanks Martin for your speedy any useful reply.