bin/29226: /etc/daily pipes >32bit integers from netstat to awk, which can't deal with them
ar...@selonen.org
>Category: bin
>Synopsis: /etc/daily pipes >32bit integers from netstat to awk, which can't deal with them
>Confidential: no
>Severity: non-critical
>Priority: low
>Responsible: bin-bug-people
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Fri Feb 04 08:17:00 +0000 2005
>Originator: Arto Selonen
>Release: NetBSD-current, 2.99.15 from ~20050131
>Organization:
>Environment:
NetBSD blah 2.99.15 NetBSD 2.99.15 (BLAH) #77: Mon Jan 31 10:27:44 EET 2005 blah@blah:/obj/sys/arch/i386/compile/BLAH i386
>Description:
/etc/daily's check_network may pipe 'netstat -inv' output to awk
for further processing. However, netstat may produces numbers larger
than 32 bits, and awk seems to only handle up to 32bit signed integers.
Here is a sample 'netstat -inv' that triggers the problem:
(I have edited it to fit better, and replaced addresses; the problem
is in the size of the packet counters)
Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Colls
wm0 1500 <Link> re:pl:ac:ed 6333370853 0 7202788621 0 0
wm0 1500 10.2.3/24 10.2.3.4 6333370853 0 7202788621 0 0
And this is what /etc/daily check_network reports:
(again output white space edited for readability; prolem is
in the number's representation, ie. overflow)
Name Ipkts Ierrs Opkts Oerrs Colls
wm0 -2147483648 0 -2147483648 0 0
>How-To-Repeat:
% echo "9876543210" | awk '{ printf("%10d\n", $1);}'
-2147483648
% echo "9876543210" | awk '{ printf("%12lu\n", $1);}'
2147483648
% echo "9876543210" | awk '{ printf("%12qu\n", $1);}'
awk: weird printf conversion %12q
input record number 1, file
source line number 1
%12q9876543210u
>Fix:
(1) Teach awk 64bit number handling, including quad conversions.
(2) Fix printf field formatting in /etc/daily accordingly.
or
(1) Replace awk in /etc/daily with something that can deal with
large numbers
This is how a Mandrake 10.0 Linux awk (presumably GNU awk) handles it:
% echo "9876543210" | awk '{ printf("%12lu\n", $1);}'
9.87654e+09
This is really a awk/64bit problem, but as it has so far only surfaced
in /etc/daily usage, I could argue that the problem is not awk's
integer sizes, but rather using *awk* in /etc/daily.
Fixing awk instead of just /etc/daily will probably benefit other
instances of awk usage as well.
Klaus Klein
From: Klaus Klein <kle...@mibh.de>
To: gnats...@netbsd.org
Cc:
Subject: Re: bin/29226: /etc/daily pipes >32bit integers from netstat to awk, which can't deal with them
Date: Fri, 4 Feb 2005 14:46:58 +0100
On Fri, Feb 04, 2005 at 08:17:00AM +0000, ar...@selonen.org wrote:
> (1) Teach awk 64bit number handling, including quad conversions.
Shell utilities seem a bad place to instill knowledge of explicit
integer size conversions.
> (1) Replace awk in /etc/daily with something that can deal with
> large numbers
I wouldn't mind awk(1) using intmax_t internally; however, there is
a showstopper in IEEE Std 1003.1-2001 section 1.7.2:
"Integer variables and constants, including the values of
operands and option-arguments, used by the standard
utilities [...] shall be implemented as equivalent to
the ISO C standard _signed long_ data type [...]"
Having an alternate, conforming version in, say, /usr/posix/bin and
adjusting _CS_PATH accordingly might do the trick for this one (and
a couple of similar issues).
- Klaus
Christos Zoulas
-- Subject: Re: bin/29226: /etc/daily pipes >32bit integers from netstat to a
Or a flag, if that does not make the code too difficult to handle.
christos
Ben Harris
From: Ben Harris <bj...@netbsd.org>
To: gnats...@netbsd.org
Cc: Klaus Klein <kle...@mibh.de>
Subject: Re: bin/29226: /etc/daily pipes >32bit integers from netstat to awk, which can't deal with them
Date: Fri, 04 Feb 2005 14:02:40 +0000
In article <2005020413530...@narn.netbsd.org> you write:
> I wouldn't mind awk(1) using intmax_t internally; however, there is
> a showstopper in IEEE Std 1003.1-2001 section 1.7.2:
>
> "Integer variables and constants, including the values of
> operands and option-arguments, used by the standard
> utilities [...] shall be implemented as equivalent to
> the ISO C standard _signed long_ data type [...]"
I think you're missing the fact that overflow behaviour of signed integer
types in C is undefined, so the application can do anything it wants in case
of overflow, including magically continuing to return arithmetically correct
results. XRAT explicitly mentions that this was intended:
# The behavior on overflow is undefined for ISO C standard arithmetic.
# Therefore, the standard utilities can use "bignum'' representation for
# integers so that there is no fixed maximum unless otherwise stated in the
# utility description. Similarly, standard utilities can use
# infinite-precision representations for floating-point arithmetic, as long
# as these representations exceed the ISO C standard requirements.
<http://www.opengroup.org/onlinepubs/009695399/xrat/xcu_chap01.html
#tag_02_01_07_04>
> Having an alternate, conforming version in, say, /usr/posix/bin and
> adjusting _CS_PATH accordingly might do the trick for this one (and
> a couple of similar issues).
Please, let's not turn into Solaris. At least, not in that regard.
--
Ben Harris <bj...@NetBSD.org>
Portmaster, NetBSD/acorn26 <URL:http://www.NetBSD.org/Ports/acorn26/>
Christos Zoulas
From: chri...@zoulas.com (Christos Zoulas)
To: gnats...@netbsd.org, gnats...@netbsd.org,
netbs...@netbsd.org
Cc:
Subject: Re: bin/29226: /etc/daily pipes >32bit integers from netstat to awk, which can't deal with them
Date: Fri, 4 Feb 2005 09:14:41 -0500
On Feb 4, 1:53pm, kle...@mibh.de (Klaus Klein) wrote:
-- Subject: Re: bin/29226: /etc/daily pipes >32bit integers from netstat to a
| The following reply was made to PR bin/29226; it has been noted by GNATS.
|
| From: Klaus Klein <kle...@mibh.de>
| To: gnats...@netbsd.org
| Cc:
| Subject: Re: bin/29226: /etc/daily pipes >32bit integers from netstat to awk, which can't deal with them
| Date: Fri, 4 Feb 2005 14:46:58 +0100
|
| On Fri, Feb 04, 2005 at 08:17:00AM +0000, ar...@selonen.org wrote:
|
| > (1) Teach awk 64bit number handling, including quad conversions.
|
| Shell utilities seem a bad place to instill knowledge of explicit
| integer size conversions.
|
| > (1) Replace awk in /etc/daily with something that can deal with
| > large numbers
|
| I wouldn't mind awk(1) using intmax_t internally; however, there is
| a showstopper in IEEE Std 1003.1-2001 section 1.7.2:
|
| "Integer variables and constants, including the values of
| operands and option-arguments, used by the standard
| utilities [...] shall be implemented as equivalent to
| the ISO C standard _signed long_ data type [...]"
|
| Having an alternate, conforming version in, say, /usr/posix/bin and
| adjusting _CS_PATH accordingly might do the trick for this one (and
| a couple of similar issues).