logsene Docker agent + JSON logs problem

[Raman Gupta]

Hello, I just installed the Docker sematext agent, and I am very impressed at how easy it was to start shipping my logs to Sematext. However, I have one issue:

My java-based application is configured to produce JSON logs (using https://github.com/logstash/logstash-logback-encoder), but sematext does not read these logs as JSON -- it simply treats the JSON as a string and shows it in the logsene `message` field as such.

Is this an issue with the docker agent?

Regards,

Raman

[Otis Gospodnetic]

Hi Raman,

So you are running a Java app inside your container(s)?

If so, I suspect we could transform JSON logs and avoid the whole log event JSON ending up in the "message" field.  A colleague of mine will follow up.

Otis

--

Monitoring - Log Management - Alerting - Anomaly Detection

Solr & Elasticsearch Consulting Support Training - http://sematext.com/

tel: +1 347 480 1610   fax: +1 718 679 9190

--
You received this message because you are subscribed to the Google Groups "Sematext Docker Agent" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sematext-docker-...@googlegroups.com.
To post to this group, send email to sematext-d...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sematext-docker-agent/7b15b927-d5bd-4508-a6e7-0fd66c3f92a0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Raman Gupta]

Yes, its a java app, but I don't think that matters -- from the docker agent perspective its just container logs. The log output is pure json -- one json document per line.

Thanks, I look forward to the followup.

Regards,

Raman

On Monday, July 25, 2016 at 9:48:21 PM UTC-4, Otis Gospodnetic wrote:

Hi Raman,

So you are running a Java app inside your container(s)?

If so, I suspect we could transform JSON logs and avoid the whole log event JSON ending up in the "message" field.  A colleague of mine will follow up.

Otis

--

Monitoring - Log Management - Alerting - Anomaly Detection

Solr & Elasticsearch Consulting Support Training - http://sematext.com/

tel: +1 347 480 1610   fax: +1 718 679 9190

On Mon, Jul 25, 2016 at 4:30 PM, Raman Gupta <rocke...@gmail.com> wrote:

Hello, I just installed the Docker sematext agent, and I am very impressed at how easy it was to start shipping my logs to Sematext. However, I have one issue:

My java-based application is configured to produce JSON logs (using https://github.com/logstash/logstash-logback-encoder), but sematext does not read these logs as JSON -- it simply treats the JSON as a string and shows it in the logsene `message` field as such.

Is this an issue with the docker agent?

Regards,

Raman

--
You received this message because you are subscribed to the Google Groups "Sematext Docker Agent" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sematext-docker-agent+unsub...@googlegroups.com.

[Stefan Thies]

Hi Raman, 

SDA should parse and ship line delimited JSON. Sounds like bug, we will check. 

Thanks for reporting. I'll come back to you. 

Stefan

[nirnirle]

I have the exact same problem, is there an answer?

[Otis Gospodnetic]

Hi,

On Wednesday, June 21, 2017 at 8:55:10 AM UTC-4, nirnirle wrote:

I have the exact same problem, is there an answer?

Looks like the problem in this case was that JSON being posted was not actually JSON, but only embedded it, so not really a problem any more after you've changed that, I believe.