Disk usage monitor not working in RHEL5

[Robert Micallef]

For anyone who encounters this issue where disk usage alerts are not working on Redhat 5, the issue is that in RHEL5 'df -h' output is multiline.

You can easily fix it by modifying the ossec agent conf. Modify the 'df -h' to 'df -Pkh' and add an alias.

  <localfile>
    <log_format>command</log_format>
    <command>df -Pkh</command>
    <alias>df -h</alias>
  </localfile>

[Pedro S]

Thanks! nice work-around.

[Daniel Cid]

Curious how was that not working. Can you give some details?

By default, it will send each line as a separated log message and we
have rules to alert if any of the entries
are over 95% utilization. Have the original running here on Centos 5,6
and 7 without any issues.

thanks,

> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "ossec-list" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ossec-list+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Robert Micallef]

I tested it on CentOS 5 and the output of df is as expected (Single line).

We don't have a lot of RHEL5 but this happens on every 1 I tried so far (I tried 7).

Here is the output of df -h on RHEL5:

Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                       23G   16G  5.4G  75% /
/dev/hda1              99M   13M   82M  14% /boot
tmpfs                 4.9G     0  4.9G   0% /dev/shm

Here is the output of a CentOS 5 machine:

Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3             1.9T  1.7T  104G  95% /
/dev/sda1              99M   36M   58M  39% /boot
tmpfs                 3.9G     0  3.9G   0% /dev/shm

So the CentOS is a single line and OSSEC picks that log perfectly. But RHEL5 it will see 2 logs:

ossec: output: 'df -h': /dev/mapper/VolGroup00-LogVol00
ossec: output: 'df -h':                        23G   16G  5.4G  75% /

And doesn't work. Tested in RHEL 5.8 and 5.11.

[Santiago Bassett]

Out of curiosity, what is the rule supposed to trigger the alert?  The one is see by default looks for full partitions...

https://github.com/ossec/ossec-hids/blob/a7ca63d6d074f2f6bdb49f4bc79a054c31dcafc7/etc/rules/ossec_rules.xml#L137

--

[Robert Micallef]

I added custom rules to alert if space is over 90%.

--

---
You received this message because you are subscribed to a topic in the Google Groups "ossec-list" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ossec-list/A8ekjtycKY4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ossec-list+...@googlegroups.com.

[theresa mic-snare]

cool, would you mind sharing those custom rules with us? the threshold (over 90%) one is specifically appealing to me :)

[Robert Micallef]

Sure. Current rule:

  <rule id="531" level="7" ignore="7200">
    <if_sid>530</if_sid>
    <match>ossec: output: 'df -h': /dev/</match>
    <regex>100%</regex>
    <description>Partition usage reached 100% (disk space monitor).</description>
    <group>low_diskspace,</group>
  </rule>

Leave that rule for 100% (so you don't modify the original rules).

In local_rules add:
 <rule id="xxxxxx" level="7" ignore="7200">
    <if_sid>530</if_sid>
    <match>ossec: output: 'df -h': /dev/</match>
    <regex>9\d%</regex>
    <description>Partition usage over 90% (disk space monitor).</description>
    <group>low_diskspace,</group>
  </rule>

[Victor Fernandez]

I think the problem is the option "-h", because it introduces arbitrary line feeds in order to be more readable by people, but it makes more difficult to decode. This happened in RHEL5 but it can happen at any system with a long filesystem path.

"-P" is an interesting option to preserve compatibility. So I think the best solution is to use only "df" or "df -P" at "ossec.conf" and rule 531.

Thank you for letting us know!

Regards.

[Robert Micallef]

No problem. Thanks to you for OSSEC. It is a lifesaver. Just had this happen in RHEL 6 too, so as you said it could happen on other operating systems.

[dan (ddp)]

Do you want to open a PR on the ossec github for this? It seems like a
decent change to make.

> Regards.

[Robert Micallef]

Hi Dan,

Created PR #819. I hope I chose the correct branches..

[dan (ddp)]

On Mon, May 2, 2016 at 9:57 AM, Robert Micallef <rober...@gmail.com> wrote:
> Hi Dan,
>
> Created PR #819. I hope I chose the correct branches..
>

Thanks for submitting that, but I left a comment in the PR. New
changes should be against MASTER, and the "h" and "P" flags are
incompatible together on OpenBSD.

[Robert Micallef]

I'm sorry about that. I never used github before. I didn't know P and h are incompatible for OpenBSD. But I think the rule should still work by just modifying <match>.

So how can I fix the PR? Do I close it and try opening it again? I cannot find where to edit anything apart from the title.

[dan (ddp)]

On Mon, May 2, 2016 at 10:12 AM, Robert Micallef <rober...@gmail.com> wrote:
> I'm sorry about that. I never used github before. I didn't know P and h are
> incompatible for OpenBSD. But I think the rule should still work by just
> modifying <match>.
>

No worries, we'll figure it all out.

> So how can I fix the PR? Do I close it and try opening it again? I cannot
> find where to edit anything apart from the title.
>

I'm not sure, but I'd probably close it and open a new one against MASTER.

[Robert Micallef]

Third time's the charm. Is PR #821 ok?

[dan (ddp)]

On Mon, May 2, 2016 at 10:36 AM, Robert Micallef <rober...@gmail.com> wrote:
> Third time's the charm. Is PR #821 ok?
>

At a glance, I don't see the df changes in it. I'll try to take a
better look in a bit.

[dan (ddp)]

On Mon, May 2, 2016 at 10:49 AM, dan (ddp) <ddp...@gmail.com> wrote:
> On Mon, May 2, 2016 at 10:36 AM, Robert Micallef <rober...@gmail.com> wrote:
>> Third time's the charm. Is PR #821 ok?
>>
>
> At a glance, I don't see the df changes in it. I'll try to take a
> better look in a bit.
>

The steps to submit the PR should basically be the following:
1. Fork the repository on github (fork button in the top right of the page)
2. Clone your fork (git clone https://github.com/USER/ossec-hids.git)
3. OPTIONAL: Create a branch for your changes (git checkout -b df_issue)
4. Make your changes (modify install.sh and etc/rules/ossec_rules.xml)
5. Add and commit your changes (git add install.sh
etc/rules/ossec_rules.xml && git commit)
6. Push your changes to a new branch on your fork on github (git push
--set-upstream origin df_issue)
7. Open a pull request on the ossec/ossec-hids repo.
a. Click "New pull request" at https://github.com/ossec/ossec-hids
b. Click "compare across forks" link
c. Ensure the left hand drop down boxes say "base fork:
ossec/ossec-hids" and "base: master"
d. In the right hand drop boxes select "head fork:
USERNAME/ossec-hids" and "compare: BRANCH"
e. Review the changes and write a brief title and comment
f. Click "Create pull request"

I've opened #822 with the proposed change as kind of a demonstration
(since I made did the above steps while documenting them).

[Robert Micallef]

Good to know for next time maybe. Thanks a lot.