High processor usage and bandwidth usage from Windows Agent

[Sean Brown]

We have just started to roll out OSSec here, so we're starting with
OSSec 1.5 on everything. We run a mix of Linux, Solaris, OS X and
Windows servers that we will monitor. So far in a little over a week, we
only have a few problems with the Windows Agent.

First, we have a domain controller at another site, connected to our
main site by a 768k connection. When we launched the ossec agent on this
system, it saturated that connection and brought it down. Is there a way
to throttle or somehow prevent ossec from bringing down that connection?
I wasn't expecting the agent to be sending that much data in the first
place.

Our second problem is the amount of processor time the Windows agent is
using. On the Windows servers we are running, the moment the agent
launches the processor gets pegged at 100%. This is especially a problem
on our VMWare Server machines where several Windows servers are running.
Is there a guide or something for tuning OSSec?

-Thank you.

[Daniel Cid]

Hi Sean,

When OSSEC starts it sends all the integrity checking messages to the
server (basically all the
monitored file names and checksums), so it can use a lot of bandwidth.
So make sure it runs
the integrity checking slowly, take a look at:

http://www.ossec.net/wiki/index.php/Know_How:Syscheck_Perf

Specially changing the values of syscheck.sleep and sleep_after to
something like:

syscheck.sleep=5
syscheck.sleep_after=5

Should use much less CPU/bandwidth.

*btw, which version of Windows are you using? It should not be using
100% of CPU at all...

Thanks,

--
Daniel B. Cid
dcid ( at ) ossec.net

[Sean Brown]

On Fri, 2008-05-16 at 14:42 -0300, Daniel Cid wrote:
> Hi Sean,
>
> When OSSEC starts it sends all the integrity checking messages to the
> server (basically all the
> monitored file names and checksums), so it can use a lot of bandwidth.
> So make sure it runs
> the integrity checking slowly, take a look at:
>
> http://www.ossec.net/wiki/index.php/Know_How:Syscheck_Perf
>
> Specially changing the values of syscheck.sleep and sleep_after to
> something like:
>
> syscheck.sleep=5
> syscheck.sleep_after=5
>
> Should use much less CPU/bandwidth.
>
> *btw, which version of Windows are you using? It should not be using
> 100% of CPU at all...
>

I'll try these on Monday.

We're using Windows Server 2003. The machine that took down our 768k
line was 2003 R2. They all have Service Pack 2 installed. All of them
pegged the CPU from the moment the service started to the moment it
could be shut off.

[Sean Brown]

On Fri, 2008-05-16 at 14:42 -0300, Daniel Cid wrote:

> Hi Sean,
>
> When OSSEC starts it sends all the integrity checking messages to the
> server (basically all the
> monitored file names and checksums), so it can use a lot of bandwidth.
> So make sure it runs
> the integrity checking slowly, take a look at:
>
> http://www.ossec.net/wiki/index.php/Know_How:Syscheck_Perf
>
> Specially changing the values of syscheck.sleep and sleep_after to
> something like:
>
> syscheck.sleep=5
> syscheck.sleep_after=5
>
> Should use much less CPU/bandwidth.

Am I correct in assuming that editing the internal_options.conf on the
server, all the agents will recieve the updated settings?

[Daniel Cid]

Hi Sean,

Actually, no, the internal_options file is unique from each agent...
However, it sounds like a good
feature request for next version.

Thanks,

--
Daniel B. Cid
dcid ( at ) ossec.net

[Sean Brown]

Ok, setting the following:

syscheck.sleep=20
syscheck.sleep_after=15

There is a brief spike of CPU usage, then it seems to behave properly
for a bit, but within 5 minutes (it's random, sometimes it's much
sooner) the CPU is pegged at 100% until the service is stopped.

Setting Debug to 2 does not log any errors. I really have no idea what
to do as OSSec does not react this way on our Linux or Solaris machines.
It is however seeming like we can not run the Windows agent, which quite
frankly sucks.

[Daniel Cid]

Hi Sean,

That should not be happening at all. At least, it never happened in
any of my systems before. What
version of Windows do you have? Can you update to our latest snapshot
just to see if it changes
anything (remember to re set the internal_options.conf after):

http://www.ossec.net/files/snapshots/ossec-win32-080520.exe

If none of this helps, let me know and I will generate a debug version
of the agent for you
to try (which hopefully will show us what is going on).

Thanks,

--
Daniel B. Cid
dcid ( at ) ossec.net

[Sean Brown]

No changes with the snapshot.

This is running on Windown 2003 standard. We also have Windows 2003 SBS
and Windows 2000 in addition to Linux and Solaris.

The agent is well behaved on Linux, Solaris and Windows 2000. It's only
on 2003 that it seems to freak out. Single processor systems are pegged
at 100%, a quad-core machine we have averages around 75% when idling
with OSSec running. Both fall to 1-2% when the OSSec service is stopped.

[Tim Meader]

Hello all,

I'm trying to use Ossec in a very rudimentary process monitoring
capacity (similar to Nagios or Big Brother). However, everything I've
read so far only applies to process monitoring on Windows systems,
nothing in the manual or wiki references Linux/Unix process monitoring
at all, even though the system_audit_rcl.txt file clearly lists "p" as
one of the types available. Is it possible to use Ossec (1.5) on the
client side to monitor for a particular process running?

If so, is the check basically done via "ps -ef | grep <process> | grep
-v grep"... where Ossec will just look for any returned lines and accept
those as a positive check? What I'm trying to do is send an alert when a
process does not exist (ie - isn't running). Is there any example out
there for Linux systems that I could look at? I'm quite surprised that
the wiki and manual (and even the book, which we've purchased) make no
mention of such a monitoring scenario.

Specifically, we need stunnel running on our logging server, and need to
be notified when the process is no longer running.

Thanks in advance for any and all help.
--

Timothy Meader
L-3 Communications, NASA EOS Security Operations
Timothy....@nasa.gov
(301) 614-6371

[Daniel Cid]

Hi Tim,

OSSEC can be used as that, but the policy monitoring is not executed
frequently enough by default,
so you will need to change the rootcheck frequency to a lower value
(probably every 10 or 20 minutes,
depending on how quickly do you want to be notified).

As far as unix process monitoring, it works very similar to the Windows one.

For example, if you want to be alerted whenever ossec-analysis is
running, you can add:

[Process ossec-analysisd running] [any] []
p:r:ossec-analysisd;

Or if you want to be alerted whenever it is NOT running, do:

[Alert! Process ossec-analysisd NOT running] [any] []
p:!r:ossec-analysisd;

Hope it helps..

*yes, we need to update the wiki with this information... Anyone
interested in doing that? :)

--
Daniel B. Cid
dcid ( at ) ossec.net

[Will Metcalf]

monit is good for this sort of monitoring...

http://www.tildeslash.com/monit/

[Sean Brown]

We use (or are trying to use ;) ) OSSec as a log monitoring app, we use
Zabbix (www.zabbix.com) for availability monitoring.

[Daniel Cid]

Hi Sean,

I was thinking about your issue today and have a few questions:

-Do you have object access auditing enabled on the 2003 system (for
success operations)?
I remember someone saying a while ago that if you have this enabled on
2003, every time ossec
tries to monitor/read the registry, it will generate an audit event.
Depending on your system it can
create quite a few events.

-How big is your security log? (related to question above)

-If that's not it, can you share the log you have with the windows
debug to 2? It might
reveal something ..

Thanks,

--
Daniel B. Cid
dcid ( at ) ossec.net

[Sean Brown]

Thank you for the reply

On Mon, 2008-06-09 at 12:36 -0300, Daniel Cid wrote:
> Hi Sean,
>

> I was thinking about your issue today and have a few questions:
>
> -Do you have object access auditing enabled on the 2003 system (for
> success operations)?

This was our problem. Switching to failures only (I have no idea why it
was logging successes as well anyway) has OSSec agents behaving
themselves.