Docker logging with in_tail and dynamic container log paths

[Traiano Welcome]

Hi

I'm trying to use in_tail to collect docker container json logs from the docker sandbox directory using this configuration:

---- (td-agent) ---

<source>
  @type    tail
  path     /var/lib/docker/containers/*/*.log
  pos_file /var/spool/td-agent/docker.log
  format json
  tag      docker.**
  @label   @mainstream
</source>

----

However it doesn't appear that td-agent even tries to follow the docker logs.

Using a specific file path I do get in_tail reading logs though:

----

<source>
  @type    tail
  path     /var/lib/docker/containers/3232535ewewd2232/some.log
  pos_file /var/spool/td-agent/*.log
  format json
  tag      docker.**
</source>
----

Is there something specific I need to do to get in_tail correctly processing docker container logs? Is the wildcard format correct?

Thanks in advance for any help,

Traiano

[Traiano Welcome]

Any idea ?

[Mr. Fiber]

You don't set `read_from_head true` in tail configuration,

so in_tail reads logs from the bottom.

Is the problem in_tail doesn't start to watch new files or

in_tail starts to watch new files but doesn't read all logs from new file?

Masahiro

Any idea ?

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Traiano Welcome]

Hi Masahiro

On Wed, Jun 27, 2018 at 11:20 AM, Mr. Fiber <repea...@gmail.com> wrote:

You don't set `read_from_head true` in tail configuration,

so in_tail reads logs from the bottom.

Understood. I tried read_from_head as a last resort while troubleshooting this. Without read_from_head, same problem.

 

Is the problem in_tail doesn't start to watch new files or

in_tail starts to watch new files but doesn't read all logs from new file?

in_tail doesn't start even. Nothing seems to be sent to td-agent. Testing by restarting docker containers.