Best practice for collecting syslogs from containers?

[Stephen Chan]

   What is the current practice for collecting syslogs from all the containers into the syslog of the host machine/image?

   The quickest solution that comes to mind would be to create an additional syslog listener socket in a directory that is mounted into all the containers, and within each container symlink /dev/log to the mounted syslog socket.

   Any reasons not to do this?

   Thanks,

   Steve

[Jérôme Petazzoni]

That should work.

To refine it even further, you could:

- create a volume in the host, e.g. /syslog

- start the syslogd container, binding /syslog into it, so that it creates the syslog listener socket

- start the other containers, binding /syslog as well, and symlink /dev/log to it as you said

--
You received this message because you are subscribed to the Google Groups "docker-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to docker-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Kushal Pisavadia]

Jérôme,

Are you able to provide a quick gist of a Dockerfile showing this working? I don't think I quite understand.

Ideally I'd like to point file descriptors for stdout/stderr for a container at a file of my choosing when I do `docker run id`.

--

Kushal Pisavadia

@KushalP

Government Digital Service

[Jérôme Petazzoni]

Ah, this seems to be different from the original question.

The stdout+stderr of each container is already sent to a specific file (/var/lib/docker/containers/<cid>/<cid>-json.log).

I would suggest to read from that file (which is in JSON format) to output it in your own format.

[Kushal Pisavadia]

Interesting. I'm already doing that, but it feels like an incredibly dirty hack.

One problem with doing that is docker (using 0.6.3) JSON escapes logs that are provided on stdout. This means you have to JSON decode each line and then unescape the 'log' key for every log line generated.

[Kushal Pisavadia]

I should add that my apps currently ship JSON formatted logs to stdout and that's why I'm seeing this issue.

[Jérôme Petazzoni]

You could tail the logs through "jq -r .log".

It works pretty well, with only two downsides:

- jq will add a newline after each entry, and Docker logs already include a \n, so you will get annoying blank lines

- jq will buffer the output, so if you "|grep ." to remove blank lines, output will no longer be realtime.

Sucks for realtime processing, but perfect for postprocessing/grepping logs!

[Jérôme Petazzoni]

By the way, since I felt that the Balance of the Force had to be restored, I proposed the following pull request for jq, to allow the removal of the blank lines:

https://github.com/stedolan/jq/pull/216

And I noticed that the master branch of jq has a --unbuffered flag which seems to be... very convenient as well :-)

[Kushal Pisavadia]

Thanks Jerome.

An alternative I was looking was to use 'docker run -a stdin,stdout id' and pipe the output to a file of my choosing.

I haven't had time to play around with it yet though.