Parsing auditd log file

142 views

Skip to first unread message

Alan Evangelista

unread,

Mar 10, 2021, 11:10:33 AM3/10/21

to flu...@googlegroups.com

I'm checking if I could use fluentd to parse auditd log file. I want to get its multiline records, transform it to a single event in JSON format and output it to Elasticsearch. An important point is that auditd does not assure the order of the records of the same event.

An auditd log file example:


type=LOGIN msg=audit(1613845201.335:6537): pid=32161 uid=0 old-auid=4294967295 auid=0 tty=(none) old-ses=4294967295 ses=350 res=1
type=USER_START msg=audit(1613845201.349:6538): pid=32161 uid=0 auid=0 ses=350 msg='op=PAM:session_open grantors=pam_loginuid,pam_keyinit,pam_limits,pam_systemd acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=CRED_REFR msg=audit(1613845201.349:6539): pid=32161 uid=0 auid=0 ses=350 msg='op=PAM:setcred grantors=pam_env,pam_fprintd acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=CRED_DISP msg=audit(1613845201.356:6540): pid=32161 uid=0 auid=0 ses=350 msg='op=PAM:setcred grantors=pam_env,pam_fprintd acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=USER_END msg=audit(1613845201.357:6541): pid=32161 uid=0 auid=0 ses=350 msg='op=PAM:session_close grantors=pam_loginuid,pam_keyinit,pam_limits,pam_systemd acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=SYSCALL msg=audit(1613845287.701:6542): arch=c000003e syscall=2 success=yes exit=17 a0=55f360492dc0 a1=20000 a2=1e4 a3=4000 items=1 ppid=1792 pid=2435 auid=4294967295 uid=201327710 gid=0 euid=201327710 suid=0 fsuid=201327710 egid=201327708 sgid=0 fsgid=201327708 tty=(none) ses=4294967295 comm="smbd" exe="/usr/sbin/smbd" key=(null)
type=CWD msg=audit(1613845287.701:6542):  cwd="/sasdata"
type=PATH msg=audit(1613845287.701:6542): item=0 name="." inode=100916940 dev=08:04 mode=040775 ouid=0 ogid=201327708 rdev=00:00 objtype=NORMAL cap_fp=0000000000000000 cap_fi=0000000000000000 cap_fe=0 cap_fver=0
type=PROCTITLE msg=audit(1613845287.701:6542): proctitle="/usr/sbin/smbd"
type=SYSCALL msg=audit(1613845287.702:6543): arch=c000003e syscall=2 success=yes exit=17 a0=55f360492dc0 a1=30000 a2=0 a3=4000 items=1 ppid=1792 pid=2435 auid=4294967295 uid=201327710 gid=0 euid=201327710 suid=0 fsuid=201327710 egid=201327708 sgid=0 fsgid=201327708 tty=(none) ses=4294967295 comm="smbd" exe="/usr/sbin/smbd" key=(null)
type=CWD msg=audit(1613845287.702:6543):  cwd="/sasdata"
type=PATH msg=audit(1613845287.702:6543): item=0 name="." inode=100916940 dev=08:04 mode=040775 ouid=0 ogid=201327708 rdev=00:00 objtype=NORMAL cap_fp=0000000000000000 cap_fi=0000000000000000 cap_fe=0 cap_fver=0
type=PROCTITLE msg=audit(1613845287.702:6543): proctitle="/usr/sbin/smbd"
type=NETFILTER_CFG msg=audit(1613845330.926:6544): table=nat family=2 entries=0
type=NETFILTER_CFG msg=audit(1613845330.926:6544): table=filter family=2 entries=0
type=NETFILTER_CFG msg=audit(1613845330.926:6544): table=mangle family=2 entries=0

The simplest algorithm would be to group all audit events which have the same timestamp and audit event ID (SYSCALL, CWD, PATH and PROCTITLE with same timestamp 1613845287.701 and audit event ID 6542) in a single event. I have checked the multiline parser plugin (https://docs.fluentd.org/parser/multiline), but, if I understood right, it requires defining a static order of regexes and I need something more flexible to parse auditd log file. I have also looked into a custom parser plugin, but it seems I always get a single log line to parse and can't preserve state between each log line. Am I missing something and I could do this with a custom parser or would I need to change the tail input plugin to provide all the new lines to my custom parser instead of just one ?