Problems with awk/nbawk on m68k
Michael L. Hitch
builds (to execersize the system). I'm pretty sure I've seen the same
problems some time back, and it just killed by build twice recently.
The first time was while building postfix:
/home/NetBSD-5/tools.m68k/bin/nbawk: trying to access out of range field -1
input record number 513, file
/home/NetBSD-5/src/external/ibm-public/postfix/dist/src/postconf/../milter/milter.c
source line number 29
The second one was a failure while checking the set lists:
====== 1 missing files in DESTDIR ========
Files in flist but missing from DESTDIR.
File wasn't installed ?
------------------------------------------
./usr/X11R7/lib/X11/fonts/100dpi/timB24-ISO8859-4.pcf.gz
======== end of 1 missing files ==========
*** [checkflist] Error code 1
It's not obvious that this was due to nbawk, but that was my suspicion at
the time, and I have verfied that it was nbawk. It was rather odd that it
complained about a /usr/X11R7 file being missing, since the amiga is not
using Xorg yet.
I was able to isolate the checkflist to a simple test for this. A simple
find would generate a list of the installed files and directories, and I
was able to generate a list of the expected files using
src/distrib/sets/makeflist. I can now replicate both the above nbawk
problems with a simple command line.
for N in 0 1 2 3 4 5 6 7 8 9
do
MKX11=yes AWK=awk sh distrib/sets/makeflist -b -a m68k -m amiga| sort -u > /tmp/flist.$N
echo -n `md5 /tmp/flist.$N`
wc -l /tmp/flist.$N
done
This will generate the list of files from the set/lists data 10 times,
with and md5 sum and wc -l of each run.
On an idle system, I am usually able to run that script with the same
result for all 10 output files:
MD5 (/tmp/flist.0) = df65d1ef5d67ce0144bb726985ce2fde 42145 /tmp/flist.0
MD5 (/tmp/flist.1) = df65d1ef5d67ce0144bb726985ce2fde 42145 /tmp/flist.1
MD5 (/tmp/flist.2) = df65d1ef5d67ce0144bb726985ce2fde 42145 /tmp/flist.2
MD5 (/tmp/flist.3) = df65d1ef5d67ce0144bb726985ce2fde 42145 /tmp/flist.3
MD5 (/tmp/flist.4) = df65d1ef5d67ce0144bb726985ce2fde 42145 /tmp/flist.4
MD5 (/tmp/flist.5) = df65d1ef5d67ce0144bb726985ce2fde 42145 /tmp/flist.5
MD5 (/tmp/flist.6) = df65d1ef5d67ce0144bb726985ce2fde 42145 /tmp/flist.6
MD5 (/tmp/flist.7) = df65d1ef5d67ce0144bb726985ce2fde 42145 /tmp/flist.7
MD5 (/tmp/flist.8) = df65d1ef5d67ce0144bb726985ce2fde 42145 /tmp/flist.8
MD5 (/tmp/flist.9) = df65d1ef5d67ce0144bb726985ce2fde 42145 /tmp/flist.9
If I do something like a du -s /home/* while running that script, it will
almost always fail. This is the output when I start the du after the
first iteration of the test script:
MD5 (/tmp/flist.0) = df65d1ef5d67ce0144bb726985ce2fde 42145 /tmp/flist.0
/tmp/usr/bin/awk: trying to access out of range field -1
input record number 8154, file
source line number 25
xargs: cat terminated by SIGPIPE
MD5 (/tmp/flist.1) = 15800048dbd6453ee466516d92d5dc84 6314 /tmp/flist.1
/tmp/usr/bin/awk: trying to access out of range field -1
input record number 8214, file
source line number 25
xargs: cat terminated by SIGPIPE
MD5 (/tmp/flist.2) = 1f41c56016e8323fff319167e386e81e 6351 /tmp/flist.2
/tmp/usr/bin/awk: trying to access out of range field -1
input record number 11213, file
source line number 54
xargs: cat terminated by SIGPIPE
MD5 (/tmp/flist.3) = 46ea89ed4ae41fd3c881a998a7ffb659 8218 /tmp/flist.3
/tmp/usr/bin/awk: trying to access out of range field -1
input record number 113167, file
source line number 25
xargs: cat terminated by SIGPIPE
MD5 (/tmp/flist.4) = 52dbdae43cbf699534a02a0fc2fd1625 37486 /tmp/flist.4
MD5 (/tmp/flist.5) = df65d1ef5d67ce0144bb726985ce2fde 42145 /tmp/flist.5
MD5 (/tmp/flist.6) = cbea14a873825db74869d5b0e60d846c 42145 /tmp/flist.6
/tmp/usr/bin/awk: trying to access out of range field -1
input record number 7679, file
source line number 25
xargs: cat terminated by SIGPIPE
MD5 (/tmp/flist.7) = 42fb1df910c3ae1ce2d4a9a3bc044c84 5988 /tmp/flist.7
MD5 (/tmp/flist.8) = df65d1ef5d67ce0144bb726985ce2fde 42145 /tmp/flist.8
/tmp/usr/bin/awk: trying to access out of range field -1
input record number 41043, file
source line number 54
xargs: cat terminated by SIGPIPE
MD5 (/tmp/flist.9) = 12eb420e8ecf5ac284adb85defda3f77 6392 /tmp/flist.9
Note that /tmp/flist.6 didn't error out, but the md5 sum doesn't match.
The diff between flist.5 and flist.6:
0a1
>
17129d17129
< ./usr/share/i18n/esdb/MISC/ATARIST.esdb
I'm not quite sure how to debug this, but at least I now can reliably
reproduce the failure. I think I may still have a 4.? version installed
on that amiga, so I will be trying that to see if the same thing happens
there as well.
--
Michael L. Hitch mhi...@montana.edu
Computer Consultant
Information Technology Center
Montana State University Bozeman, MT USA
--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-...@muc.de
Frank Wille
> I was able to isolate the checkflist to a simple test for this. A
> simple find would generate a list of the installed files and
> directories, and I was able to generate a list of the expected files
> using src/distrib/sets/makeflist. I can now replicate both the above
> nbawk problems with a simple command line.
>
> for N in 0 1 2 3 4 5 6 7 8 9
> do
> MKX11=yes AWK=awk sh distrib/sets/makeflist -b -a m68k -m amiga|
> sort -u > /tmp/flist.$N echo -n `md5 /tmp/flist.$N`
> wc -l /tmp/flist.$N
> done
Yes, I can confirm that!
As soon as I run a long "du -s" in background the flists start losing lines
or getting new ones. Besides wrong md5 checksums I got the following
errors:
awk: trying to access out of range field -1
awk: out of space in makefields 4160740
Then I made a second test with running a Whetstone benchmark in background.
The problem is less frequent here (maybe 2 or 3 flists out of 10), but it
can still be reproduced! So I hope it has nothing to do with DMA or a disk
driver.
For comparison: I was running on an A3000 with CSPPC 68060/50, 128MB, using
a disk attached to cbiiisc(4).
I could not reproduce it on i386 or macppc.
--
Frank Wille
Michael L. Hitch
> Yes, I can confirm that!
I'm kind of glad it's not jut me :)
> Then I made a second test with running a Whetstone benchmark in background.
> The problem is less frequent here (maybe 2 or 3 flists out of 10), but it
> can still be reproduced! So I hope it has nothing to do with DMA or a disk
> driver.
I was doing some tests running 'openssl speed' and would get a few
errors.
> For comparison: I was running on an A3000 with CSPPC 68060/50, 128MB, using
> a disk attached to cbiiisc(4).
I'm currently running an A4000 with CyberStorm III.
I also did some tests doing a du -s and dd on the IDE drive. That also
causes the same types of errors. The IDE driver doesn't use DMA, so it
makes it less likely to be DMA problems (although my source tree was still
on the SCSI drive).
I've also seen the same thing on a 4.0 system, and I even had a 3.0
system with a 3.1 kernel - which does the same thing.
I was thinking this morning that the list creation process is probably
using pipes; after find a pipe(2) problem on vax a while back, that could
potentially be a source of the problem. Also, to totally eliminate the
possiblity of DMA and/or SCSI driver problems, I'll try putting enough of
the tree on the IDE drive and run entirely off the IDE drive.
Mike
Ignatios Souvatzis
I haven't looked closely at the recent pmap changes, but: is branch
target cache flushing still intact? Because if not, I would expect
occasional erratic behaviour of that sort.
-is