logrotate and NULLs after rotating
betelge87
we are detecting a strange problem after truncating a log file with
logrotate (version 3.6.9).
Tests are being done with solaris 5.8 but similar problem seen on
5.10.
We have a "very simple" shell script test.sh which is writing
periodically to log file.
This log file is rotated through logrotate, but after this we see that
log file is not really
truncated (we use copytruncate option in logrotate).
See the sequence of events:
test.log size 65 bytes
-> now logrotate rotates test.log
test.log size 0 bytes
-> then shell script writes again to test file, other new 65 chars
test.log size 65+65 = 130
if you edit the test.log file, you may see that you have 65 visible
chars and 65 NULLs.
test.log" 1 line, 130 characters (65 null)
Do you have seen any similar behaviour? Where do these NULLs come
from?
We thought that copytruncate was able to truncate log files.
Thanks in advance,
Additional data
==============
$ cat test.sh
#!/bin/sh
while true
do
echo "hellohellohellohello...."
sleep 60
done
$
and we run:
$ test.sh > test.log
->> logrotate config
test.log {
compress
rotate 1
size=10
missingok
copytruncate
}
hume.sp...@bofh.ca
> Do you have seen any similar behaviour? Where do these NULLs come
> from?
> We thought that copytruncate was able to truncate log files.
It sounds as though something still has the log file open and is writing
to it during the truncation. If you have a process write 1000 bytes to a
file, and in ANOTHER process truncates the file, then when the first process
writes again the file will be backfilled with NULLs up until whatever point
in the file the process was at.
You'll need to track down whatever process still has the file open and force
it to close the file before you do the truncation.
--
Brandon Hume - hume -> BOFH.Ca, http://WWW.BOFH.Ca/
Casper H.S. Dik
>Do you have seen any similar behaviour? Where do these NULLs come
>from?
>We thought that copytruncate was able to truncate log files.
The file is opened without using O_APPEND; the fd will then remember
the file offset and the next output will written at the same
offset. Bytes before that offset will show up as NULLs.
Create the file with O_APPEND and you'll be fine.
Use ">>" from your shell and make sure you don't use csh or sh;
use /usr/xpg4/bin/sh, or ksh/ksh93/bash/tcsh/zsh
Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
betelge87
you were completely right, problem was due to process test.sh had the
fd
opened and after logrotate truncated test.log file, when test.sh wrote
again
all previous byte content in test.log was converted into NULLs and
added.
Root cause was that test.sh was usign sh, we changed that and with
ksh problem vanished.
Do you know the reason of that? Is it a bug on sh?
I didnt know about these differences with fd handling between
different shells.
Thanks again for your help,
On 20 oct, 15:06, Casper H.S. Dik <Casper....@Sun.COM> wrote:
Casper H.S. Dik
>Casper, Brandon,
>you were completely right, problem was due to process test.sh had the
>fd
>opened and after logrotate truncated test.log file, when test.sh wrote
>again
>all previous byte content in test.log was converted into NULLs and
>added.
>Root cause was that test.sh was usign sh, we changed that and with
>ksh problem vanished.
>Do you know the reason of that? Is it a bug on sh?
Not using O_APPEND in the first case and using it properly in
the working case.
>I didnt know about these differences with fd handling between
>different shells.
"Historical differences".
Casper
Sven Mascheck
> betelge87 writes:
>>Do you know the reason of that? Is it a bug on sh?
>
> Not using O_APPEND in the first case and using it properly in
> the working case.
>
>>I didnt know about these differences with fd handling between
>>different shells.
>
> "Historical differences".
Instead of calling open(2) with O_APPEND, the shell jumps
to the end with lseek(2). A few other shells, like csh(1)
and very early ksh(1), behave similar:
O_APPEND had not been introduced until System III or
4.2BSD (4.1c.2), respectively.
Another difference about fd handling in the Bourne
shell: if the shell is called with a script as argument
(this also applies to the #! mechanism), then it reads
the file through fd 19. (This is even not of concern for
redirections, because fd > 9 are not directly accessible,
but other programs might have opened this fd before
invoking the shell.)
--
http://www.in-ulm.de/~mascheck/bourne/common.html