Cannot tail linux audit logs

[Kevin Brunette]

I'm attempting to tail some linux audit logs in the directory: /var/log/audit/audit.log. I've tried this on AWS linux and also CentOS but either fluent.d simply ignores the file but I set a read permission I get:

"var/observe/audit.log unreadable. It is excluded and would be examined next time." 

Without setting any directory permissions on the folder, fluent.d will ignore the file and there is no tailing.  I then ran the command: chmod a=r /var/log/audit/ and now the fluent.d logs show that it is trying to read the log but it is unreadable. I've also tried to create a syslink to that directory and read from the file but I get the same error..

Here is my config: 

<source>

  @type tail

  tag linux_logs.raw

  path /var/log/audit/audit.log

  read_from_head true

  pos_file /etc/td-agent/test.pos

  <parse>

    @type regexp

    expression /(?<message>.+)/

  </parse>

</source>

####

## Filter descriptions:

##

<filter **>

  @type record_transformer

  <record>

    hostname "${hostname}"

    timestamp "${time}"

  </record>

</filter>

####

## Output descriptions:

##

<match **>

  @type http

  endpoint https://myendpoint/

  open_timeout 2

  headers {"Authorization":"Bearer <token> <token2>"}

  <format>

    @type json

  </format>

  <buffer>

    @type memory

    flush_interval 10s

    compress gzip

  </buffer>

</match>

The logs are trying to read the file but it is "Unreadable and will be excluded and would be examined next time". 

How do I tail this file?? 

[Kevin Brunette]

This was a permissions issue. On Amazon linux I had to change the systemd service file to run as root so it would be able to parse and read the files. 

1) Edit service file to run as root:

vi /lib/systemd/system/td-agent.service

2) Change the User and Group to “root”.

[Unit]
Description=td-agent: Fluentd based data collector for Treasure Data
Documentation=https://docs.treasuredata.com/display/public/PD/About+Treasure+Data%%27s+Server-Side+Agent
After=network-online.target
Wants=network-online.target

[Service]
User=root
Group=root
                       

3) Reload the daemon

systemctl daemon-reload