Linux Audit?

[Mark Moorcroft]

Is anyone moving linux auditd data, and have you done any work with interpreting it? Possibly combining regex with linux audit tools? Or possibly just using fluentd to pipe all the audit logs to a directory on the server for compliance?

#> tail /var/log/audit/audit.log

type=USER_AUTH msg=audit(1408929267.794:74242): user pid=19126 uid=1853945932 auid=1853945932 ses=391 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:authentication acct="user" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'

type=SYSCALL msg=audit(1408929267.798:74243): arch=c000003e syscall=59 success=yes exit=0 a0=7f1486c9bd18 a1=7fff580bcb40 a2=7f1486e9e350 a3=a items=2 ppid=19126 pid=19133 auid=1853945932 uid=0 gid=504 euid=0 suid=0 fsuid=0 egid=504 sgid=504 fsgid=504 tty=pts0 ses=391 comm="unix_chkpwd" exe="/sbin/unix_chkpwd" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="privileged"

type=EXECVE msg=audit(1408929267.798:74243): argc=3 a0="/sbin/unix_chkpwd" a1="user" a2="chkexpiry"

type=CWD msg=audit(1408929267.798:74243):  cwd="/home/user"

type=PATH msg=audit(1408929267.798:74243): item=0 name="/sbin/unix_chkpwd" inode=4333 dev=fd:00 mode=0104755 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:chkpwd_exec_t:s0 nametype=NORMAL

type=PATH msg=audit(1408929267.798:74243): item=1 name=(null) inode=130610 dev=fd:00 mode=0100755 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:ld_so_t:s0 nametype=NORMAL

type=USER_ACCT msg=audit(1408929267.802:74244): user pid=19126 uid=1853945932 auid=1853945932 ses=391 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:accounting acct="user" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'

type=USER_CMD msg=audit(1408929267.802:74245): user pid=19126 uid=1853945932 auid=1853945932 ses=391 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='cwd="/home/user" cmd="-bash" terminal=pts/0 res=success'

type=CRED_ACQ msg=audit(1408929267.803:74246): user pid=19126 uid=0 auid=1853945932 ses=391 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=failed'

type=USER_START msg=audit(1408929267.816:74247): user pid=19126 uid=0 auid=1853945932 ses=391 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:session_open acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'

[Masahiro Nakagawa]

HI,

Auditd log can't seem to parse via Regexp.

You can implement own parser instead of Regexp.

This is one parser plugin example:

https://github.com/repeatedly/fluent-plugin-multi-format-parser/blob/master/lib/fluent/plugin/parser_multi_format.rb

Following code is parser plugin template. filename is parser_foo.rb:

require 'fluent/parser'

module Fluent
  class TextParser
    class FooParser
      include Configurable

      def initialize
        super
      end

      def configure(conf)
        super
      end

      def call(text)
        # text is type=USER_AUTH msg=audit(1408929267.794:74242): user pid=19126 uid=1853945932 auid=1853945932 ses=391 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:authentication acct="user" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'

        # parse text implementation

        time = parse_time(text)

        record = parse_body(text)

        # can parse. Return time and record
        if time && record
          if block_given?
            yield time, record
            return
          else
            return time, record
          end
        end

        # can't parse. Return nil
        if block_given?
          yield nil, nil
        else
          return nil, nil
        end
      end
    end

    register_template('foo', Proc.new { FooParser.new })
  end
end

Sorry, we have not added Parser plugin document to docs.fluentd.org yet.
Adding Parser and Formatter plugin documents is our TODOs...

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kiyoto Tamura]

Masa-

Thanks for the example, the following pages have been added to the docs: docs.fluentd.org/articles/parser-plugin-overview and http://docs.fluentd.org/articles/plugin-development#parser-plugins-v01046-and-above

Mark-

Is this working for you?

Kiyoto

--

Check out Fluentd, the open source data collector to unify log management.

[Mark Moorcroft]

Several other disasters diverted my attention for the moment. Still can't get CentOS5/syslog to produce any forwarded data. Not sure where to look, other than wait for the CentOS6 upgrades with rsyslog.

--
You received this message because you are subscribed to a topic in the Google Groups "Fluentd Google Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/fluentd/yMooaGMbGj8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to fluentd+u...@googlegroups.com.

[Kiyoto Tamura]

Mark-

>Still can't get CentOS5/syslog to produce any forwarded data.

I see. Do you have any further insights? For example:

- what are you seeing in /etc/td-agent/td-agent.log (with -vv option)?

- how is syslog being forwarded to td-agent? Is td-agent at least receiving events from rsyslog?

Thanks,

Kiyoto

[Mark Moorcroft]

I have this in syslog.conf

#Fluentd

*.* @127.0.0.1:5140

And no, I see nothing in td-agent log with -vv other than:

2014-08-27 12:23:49 -0700 [trace]: plugin/out_secure_forward.rb:135:block in node_watcher: in node health watcher

2014-08-27 12:23:49 -0700 [trace]: plugin/out_secure_forward.rb:138:block (2 levels) in node_watcher: node health watcher for xxx.xxx.nasa.gov

[Mark Moorcroft]

So, let me tell you what I think I know.

The CentOS 5 syslog will not accept a port number argument to the IP address. It seems to just do nothing, and you get no sent data at all. If you take out the port argument it works over 514. I can find no evidence online that the native CentOS 5 syslog accepts the port argument, and definitely not in the man page.

The td-agent client "might" allow you to use 514, but not running as root means it can't get at 514.

You might be able to use iptables to redirect 514 to 5140, but I'm not sure if this will work, or if it's even a good idea. These systems are "Rocks clusters" that are CentOS based, BUT they use a database system for dealing with iptables, so I can't just edit /etc/sysconfig/iptables directly.

Presumably I could replace syslog with something that supports the port argument, but I have been trying to avoid that up to now. Especially since the CentOS 6 upgrade looms near.

So, if my findings are correct, the advise you gave me for forwarding via syslog are incorrect for CentOS 5.

[Kiyoto Tamura]

Hi Mark,

Thank you for investigating this issue further. I did some googling myself, and it does sound like certain versions of syslogd does not support forwarding to non-514 ports. For example, this page on Splunk's doc says, "If syslogd is your only option (as is the case with some router or network devices), first ensure that your version of syslog supports sending data to a custom port number (other than UDP port 514)."

So, i would say the best options are either upgrading syslogd or using rsyslogd instead of syslogd.

Kiyoto

[Mark Moorcroft]

So, at the moment I am collecting syslog via secure->gelf, and I'm using audisp-remote to collect audit logs (unencrypted :-(. I am not piping audit logs to fluent/graylog because there is no "good" parser so far, and I have yet to look at regex. I am running my OSSEC server on the same system, and so far not piping that to fluentd either. At least for compliance we have a central collection point that will soon be very limited access.

[Mark Moorcroft]

Hey all, I still have not had the time to look at any options for parsing/tagging audit data. BUT, I stumbled into something I should probably have found long ago.

http://wiki.rsyslog.com/index.php/Centralizing_the_audit_log

Once you have syslog secure forwarding with fluentd, it is dead simple to turn on the audisp syslog plugin. At that point your audit records will accompany your syslog data with no additional effort, and nicely encrypted. And looking at the records I see so far I get "ident" = "audisp", and since I already have source tagging working this slots in very nicely. For compliance I must say this seems to fit the bill well. I can probably get more out of this (in Graylog2) by doing more with the incoming data. But at the very least my archiving requirements are handled.

I must say I am disappointed that nobody suggested this by now ;-)

[Mark Moorcroft]

Hmm, I just realized that one big problem is shoving the audit records into mongo prevents the native audit tools from reaching them. I hate to keep 2 copies of the records.. BUT

Do we believe the audisp syslog plugin can produce records that linux audit tools can use? And can fluentd produce an unmolested output file that audit tools can also read, in parallel with feeding them to mongo?