Floating point computation issue?

[ejlis...@yahoo.fr]

Hi all,

Does anyone know why JSDB (win version, did not check others) and SQLite
do not get the same result in the following situation?

---

R:\>jsdb.exe
js>var db = new SQLite();
js>db.exec("select 999901/10.0", function(r){println(r.toString())});
999901/10.0=99990.1000000001
true
js>exit

R:\>SQLite.exe NUL "select 999901/10.0;"
99990.1

---

Regards,
Etienne

P.S. Keep up the good work Shanti!

[Jason S]

R:\>jsdb.exe

js>var db = new SQLite();
js>db.exec("select 999901/10.0", function(r){println(r.toString())});
999901/10.0=99990.1000000001
true
js>exit

R:\>SQLite.exe NUL "select 999901/10.0;"
99990.1

The former goes through Javascript (the call of r.toString()); the latter does not.

[Shanti Rao]

That's what I thought at first, but it actually turned out to be quite subtle. A 32-bit float with a rounding error is promoted to double by the compiler in a call to sqlite3_snprintf(). The fix wouls be to change how JSDB converts SQLite values to text in sqlite_exec().

Dies anyone else need thus fixed?

Shanti