floating point: too many significant digits

[Laurent MANCHON]

--Hi,

TAWK (v6.7) display warning with more than 14 digits:

awkw "BEGIN{var=1.9224832057952844;printf("%.8f",var);exit}"
awk: warning in program line 1: floating point: too many significant digits
1.92248321

[J Naman]

Have you tried coding constants as strings, which will be converted to f.p. by gawk|tawk|mawk|awk?
pi_str="3.14159265358979323846264338327950288419716939937510"; # 50 decimal digits
var="1.9224832057952844"; # ???
I do not know anything about TAWK beyond Ver 5, which was 32-bit, but I don't remember any warning such as you got.
Maybe consider switching to one of the *awk's that are up-to-date?

[Laurent MANCHON]

nothing change:
awkw "BEGIN{var=sprintf(\"%.8f\","3.14159265358979323846264338327950288419716939937510");print var; exit 1}"

awk: warning in program line 1: floating point: too many significant digits

3.14159265

strange behaviour, and it doesn't occur with traditional awk, maybe a bug in coding long float

[Janis Papanagnou]

And what is your question or problem? Or do you just want to inform us
about that nice and useful tawk feature to get a warning?

Janis

PS: Don't top-post!

[Laurent MANCHON]

just to say: Why this warning message ?
when you have a file with millions of big floating numbers, TAWK print this message !
Is there an option or parameter to avoid this warning ?

[Janis Papanagnou]

On 08.10.2021 10:22, Laurent MANCHON wrote:
> Le vendredi 8 octobre 2021 à 09:02:46 UTC+2, Janis Papanagnou a écrit :
>> On 07.10.2021 14:24, Laurent MANCHON wrote:
>>> --Hi,
>>>
>>> TAWK (v6.7) display warning with more than 14 digits:
>>>
>>> awkw "BEGIN{var=1.9224832057952844;printf("%.8f",var);exit}"
>>> awk: warning in program line 1: floating point: too many significant digits
>>> 1.92248321
>> And what is your question or problem? Or do you just want to inform us
>> about that nice and useful tawk feature to get a warning?
>>
>> Janis
>>
>> PS: Don't top-post!
>
> just to say: Why this warning message ?

Just as an educated guess; if a user explicitly specifies many digits
he may intend to do arithmetics with such a high accuracy, but if the
underlying data type doesn't support that accuracy he may get numbers
with digits at the far end that are effectively just random. Say, you
calculate pi and get 3.141592653198765465133 the result is useless or
misleading because only the leading digits are correct.

> when you have a file with millions of big floating numbers, TAWK print this message !
> Is there an option or parameter to avoid this warning ?

That question should be answered by the Tawk manual or man page.

Practically, if suppressing the message is not possible, you may try
to redirect the output channel (standard error?) where the message is
created (and hopefully it's not the same as the standard output where
regular output is placed). Another workaround is to read that value as
string and strip the data string after the supported number of digits
before converting it to a number or using it in arithmetic expressions.

Ideally it should be possible to activate (and suppress) individual
warning messages as desired in Awks.

Janis

[Ben Bacarisse]

Laurent MANCHON <manch...@gmail.com> writes:

> Le vendredi 8 octobre 2021 à 09:02:46 UTC+2, Janis Papanagnou a écrit :
>> On 07.10.2021 14:24, Laurent MANCHON wrote:
>> > --Hi,
>> >
>> > TAWK (v6.7) display warning with more than 14 digits:
>> >
>> > awkw "BEGIN{var=1.9224832057952844;printf("%.8f",var);exit}"
>> > awk: warning in program line 1: floating point: too many significant digits
>> > 1.92248321
>> And what is your question or problem? Or do you just want to inform us
>> about that nice and useful tawk feature to get a warning?

> just to say: Why this warning message ?
> when you have a file with millions of big floating numbers, TAWK print
> this message !

Does it happen when processing input? If so, I can see how it would be
very annoying.

> Is there an option or parameter to avoid this warning ?

I don't know much about tawk, but I do know it's old an unmaintained.
Are you committed to using that version of awk? If not, this warning
could be the prompt you need to switch to a maintained version...

--
Ben.

[Laurent MANCHON]

Le vendredi 8 octobre 2021 à 11:40:27 UTC+2, Ben Bacarisse a écrit :
> Laurent MANCHON <manch...@gmail.com> writes:
>
> > Le vendredi 8 octobre 2021 à 09:02:46 UTC+2, Janis Papanagnou a écrit :
> >> On 07.10.2021 14:24, Laurent MANCHON wrote:
> >> > --Hi,
> >> >
> >> > TAWK (v6.7) display warning with more than 14 digits:
> >> >
> >> > awkw "BEGIN{var=1.9224832057952844;printf("%.8f",var);exit}"
> >> > awk: warning in program line 1: floating point: too many significant digits
> >> > 1.92248321
> >> And what is your question or problem? Or do you just want to inform us
> >> about that nice and useful tawk feature to get a warning?
> > just to say: Why this warning message ?
> > when you have a file with millions of big floating numbers, TAWK print
> > this message !
> Does it happen when processing input? If so, I can see how it would be
> very annoying.

yes, try to put this list of numbers in a file and try to print them:
1.941359281539914
1.965880632400514
1.977476954460144
2.168184041976924
2.213710546493534
1.984231114387514
1.944460153579714
2.156220912933344
1.908116817474364
1.963439345359804
2.109183073043824
1.921808719635004
2.107269763946534
2.151268005371094
1.991624832153324

> > Is there an option or parameter to avoid this warning ?
> I don't know much about tawk, but I do know it's old an unmaintained.
> Are you committed to using that version of awk? If not, this warning
> could be the prompt you need to switch to a maintained version...

with traditional *awk this warning never occurs
>
> --
> Ben.

[Janis Papanagnou]

On 08.10.2021 11:45, Laurent MANCHON wrote:
> Le vendredi 8 octobre 2021 à 11:40:27 UTC+2, Ben Bacarisse a écrit :
>> Does it happen when processing input? If so, I can see how it would be
>> very annoying.
>
> yes, try to put this list of numbers in a file and try to print them:
> 1.941359281539914
> 1.965880632400514

> [...]

On which platform are you working? If you are on some Unix system I'd
try

1) redirecting stderr (hoping the warning message uses that channel)

tawk '...' a_file 2>/dev/null

2) stripping the input data to the supported accuracy, say, using a
string length of only 8

cut -c -8 < a_file | tawk '...'

3) using (as suggested by Ben) another Awk (here GNU Awk)

gawk '...' a_file


Janis

[Laurent MANCHON]

i work on windows
I got around the problem as suggested by using: $1=substr($1,1,10)

[Janis Papanagnou]

On 08.10.2021 12:10, Laurent MANCHON wrote:
>
> I got around the problem as suggested by using: $1=substr($1,1,10)

That means the warning message appears not when the data is read in
but later when it is processed. Good. Glad the workaround helped.

Janis

[Kaz Kylheku]

Your 17-digit value 1.9224832057952844 is not the result of converting
any IEEE 754 64 bit double to decimal.

When this value is converted to double, and then back to 17 decimal
digits, the reprsentation 1.9224832057952843 is produced.

If tawk does not warn about 1.9224832057952843, then that is the
issue. Just the wording of the diagnostic is poor.

A 64 bit double has two properties, in relation to decimal
representations:

- it can store up to 15 decimal digits of mantissa precision.
If more decimal digits are specified, some of them may be changed
or truncated away.

- 17 decimal digits are required to exactly specify/preserve
a double's value in decimal form.

A programming language must accept up to 17 digits in a constant
without complaining, otherwise it becomes a nuisance to those who
need to specify an exact number down to the last mantissa bit.

However, not all 17 digit numbers are distinct.

The 64 bit double type contains a value which is very close to
1.9224832057952843. The next one (next one bit increment in the
mantissa) rounds to 1.9224832057952845.

The constant 1.9224832057952844 is ambiguous: it isn't clear
which of those two points you want to specify.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal

[J Naman]

If you are VERY particular, consider d11=substr($1,11,1), examine it and decide if you need to round up, then possibly add 1, such as: d10=substr($1,10,1)+(d11>4); THEN $1=substr($1,1,9) d10;

[Markus Gnam]

You can turn off the "too many significant digits" message with the FLOATMASK variable only for a *string* being converted to a float, see TAWK 5 manual page 200:

BEGIN {
FLOATMASK = and(FLOATMASK,not(0x400))
var = "1.9224832057952844"
printf("%.8f",var)
}

=> 1.92248321

[Laurent MANCHON]

to Kaz:
awkw "BEGIN{var=\"1.9224832057952843\";printf(\"%.8f\n\",var);exit 1}"
awk: warning in program(line 1) (NR=0):

floating point: too many significant digits
1.92248321

1.9224832057952843 is a right representation of decimal, so it is not an issue.

now with:
awkw "BEGIN{FLOATMASK = and(FLOATMASK,not(0x400));var=\"1.9224832057952843\";printf(\"%.8f\n\",var);exit 1}"

it works fine, this is a good issue, thank you Markus.