Decimal value precision

26 views
Skip to first unread message

ogaq...@gmail.com

unread,
Apr 2, 2015, 6:14:32 AM4/2/15
to open...@googlegroups.com
Hi,

I read this in the online documentation :
<<
...... Currently all floating point values are stored on 4 bytes, single-precision, with support for 8 bytes planned for a future release......
>>

In which release do you plan to support 64 bits precision ?
I installed the 2.1RC1 version and it seems to still be stored on 4 bits.
Indeed, when I store 224.15212, I get the following value back : 224.15211486816406

Thanks

ManOLamancha

unread,
Apr 3, 2015, 8:05:07 PM4/3/15
to open...@googlegroups.com
We haven't started storing double precision points yet. If anyone wants to take a crack at the code to detect when we should store on 4 or 8 bytes, that'd be great :) 

ogaq...@gmail.com

unread,
Apr 22, 2015, 4:47:18 AM4/22/15
to open...@googlegroups.com
Hi,

I've created a patch for the double precision management when storing values (TextImporter and API REST).
How could i post and where ? (diff files)

ManOLamancha

unread,
Apr 23, 2015, 8:09:34 PM4/23/15
to open...@googlegroups.com
On Wednesday, April 22, 2015 at 1:47:18 AM UTC-7, ogaq...@gmail.com wrote:
Hi,

I've created a patch for the double precision management when storing values (TextImporter and API REST).
How could i post and where ? (diff files)

Preferably Github, otherwise you can post here. 
Reply all
Reply to author
Forward
0 new messages