Overrunning syslog: logging too fast?

[Russell Holt]

Hello all,

The following short test program demostrates a problem that I am
having with my application, which can generate a huge amount of logging
very quickly in some cases. (running Solaris 2.5 on a Sparc 5 - 96M ram)

#include "syslog.h"

main()
{
int i;

openlog("TEST", LOG_PID, LOG_LOCAL1);
for(i=0; i<5000; i++)
syslog(LOG_DEBUG,
"TESTING %d * * * * * * * * * * * * * * * * *", i);
}

This example simply sends 5000 lines to the syslog, very fast.
It turns out that only the first 50 lines and the last 80 are actually
written to the log file, cutting out thousands of lines in between.
Loosing log data is not good.

I tried putting a delay into the example loop with usleep() for 50, 100,
and 500 microseconds, and the number of omitted lines dropped to about
2400 (instead of 3900 or so).

I need massive logging in some circumstances, and I'm now questioning
the use of syslog. Is this normal syslog behavior? Is it a bug? Perhaps
some config issue with syslog I'm unaware of? Does anyone have any
references that discuss issues like this - whether to use syslog or
not, & if not, how to approach massive logging over a network from
multiple machines/processes... fun stuff like that..

Any help is appreciated.
thanks

-Russell

--
=(Russell Holt)====-===-==-=-=--=---=---- --- -- - - - - - -
hol...@destinyusa.com http://www.destinyusa.com/russ/
"hypertext web site.. hypertext web site.. am I interactive yet?" -Zippy

[Gregory Rowan]

Russell Holt (hol...@raja.destinyusa.com) wrote:

> This example simply sends 5000 lines to the syslog, very fast.
> It turns out that only the first 50 lines and the last 80 are actually
> written to the log file, cutting out thousands of lines in between.
> Loosing log data is not good.

> I tried putting a delay into the example loop with usleep() for 50, 100,
> and 500 microseconds, and the number of omitted lines dropped to about
> 2400 (instead of 3900 or so).

> I need massive logging in some circumstances, and I'm now questioning
> the use of syslog. Is this normal syslog behavior? Is it a bug? Perhaps
> some config issue with syslog I'm unaware of? Does anyone have any
> references that discuss issues like this - whether to use syslog or
> not, & if not, how to approach massive logging over a network from
> multiple machines/processes... fun stuff like that..

This behavior is normal (although not always appreciated). I can't remember
the details, but it has something to do with some buffer syslog uses. This
behavior of losing lines only occurs if you are giving lots of data to syslog
in a very short time. I think it occurs on both SunOS and HPUX.
Options? Have your program open your own "syslog" file with the O_APPEND flag
and put their debug messages in there.

GREG