Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

Error in numeric values ​​equal

496 views
Skip to first unread message

NICO

unread,
Nov 23, 2012, 2:24:39 PM11/23/12
to multisoft...@googlegroups.com
Anyone know because when you compare two numerical variables FlagShip 7 says that its contents are different, both are declared as numeric but one of them is as EXPORT and another as INSTANCE, this same comparison FlagShip 4 not difficult, thanks in advance.

Paul

unread,
Aug 26, 2020, 6:33:59 AM8/26/20
to multisoft...@googlegroups.com
There are two different kind of numbers in computer industry: integer and floating point. Integer (or long integer) is without decimal fraction and is always precise, but has limited range (approx. +/- 2 billions). Floating point has nearly unlimited range, but may be imprecise. This means, the number 5.0 may be stored as floating point 5.000...000 or 4.9999...999 or 5.0000...001 (the decimal fraction is up to 15 digits). To be sure to compare two floating point values (or integer with floating point), best to use the ROUND() function. See further details and hints in the FlagShip on-line manual (fsman) section LNG.2.6.4

Paul
Reply all
Reply to author
Forward
0 new messages