Log forwarding in Nomad

[Subir Pradhanang]

Hi all,

I am using nomad's Java driver to deploy jar files, and am able to check the logs of the deployed java app through the "nomad fs cat" command. However, I'm not being able to figure out how the logs could be shipped to external boxes for centralized logging. Before trying out nomad, we used to ship the logs to Elasticsearch using Logstash as log forwarder.

Any pointers on this would be greatly appreciated.

Thanks,

Subir

[Samit Pal]

Hi Subir,

I'm new to nomad so I might be wrong. But I think what you need is a log shipper task in your task group along with the java app task. And then use the NOMAD_ALLOC_DIR env variable to share the logs among the two. https://www.nomadproject.io/docs/jobspec/environment.html#task_dir has some documentation around this. 

[Diptanu Choudhury]

Hi Subir,

The Allocation directory is shared across all the tasks in a task group, so all tasks can read the alloc/logs directory where the rotated log files are stored. So you could run a log shipper task in every task group.

We also plan to implement forwarding logs over syslog in the future, we just haven't got around to implementing it.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/nomad/issues
IRC: #nomad-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Nomad" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nomad-tool+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nomad-tool/3dad6906-4710-4434-ba8e-99501308cdf8%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Thanks,
Diptanu Choudhury

Web - www.linkedin.com/in/diptanu

Twitter - @diptanu

[Subir Pradhanang]

Hi Diptanu and Samit,

Thanks for your prompt response. The approach that you have suggested i.e. to run a log shipper task in every task group - does this task have to be run as a docker container? Or is there any other way to implement this?

What I have done at the moment is something like this: https://github.com/hashicorp/nomad/issues/688#issuecomment-208846664

Not sure if this is the right approach, but what I am doing at the moment is I have a log shipping agent (logstash in this case) setup on the host. This monitors Nomad's job logs in /opt/nomad/alloc/*/alloc/logs/* and forwards them to ElasticSearch.

Thanks,

Subir