Integrating riemann with elasticsearch,influxdb and rabbitmq

[Mario Giammarco]

Hello,

I need to build a monitoring infrastructure.

I need to collect data from elastic or influx and send to slack and rabbitmq.

I have these problems:

- In influxdb can easily prepare triggers with  chronograf but the I can send them only with http post;

- in elastic search I have no praticaly way to prepare triggers and so I must send raw events to riemann;

- riemann should send alerts using amqp or http post to rabbitmq.

So questions are:

- is there an integration of riemann and elasticsearch that I am not aware of?

- can I use http post to produce e consume riemann events?

- have you other hints?

Thanks,

Mario

[James Turnbull]

> So questions are:
>
> - is there an integration of riemann and elasticsearch that I am not
> aware of?

There is a Logstash export to Riemann plugin you could use:
https://www.elastic.co/guide/en/logstash/current/plugins-outputs-riemann.html.

> - can I use http post to produce e consume riemann events?

I'd use a client over HTTP - http://riemann.io/clients.html.

> - have you other hints?

The Riemann HowTo docs are a good starting place
(http://riemann.io/howto.html) as are the API docs
(http://riemann.io/api.html).

Kind Regards

James Turnbull

--
* Monitoring with Prometheus (https://prometheusbook.com)
* The Packer Book (https://packerbook.com)
* The Terraform Book (https://terraformbook.com)
* The Art of Monitoring (https://artofmonitoring.com)
* The Docker Book (https://dockerbook.com)
* The Logstash Book (https://logstashbook.com)

[Sanel Zukan]

If is possible, probably better approach would be to send events to riemann
and let riemann forward them to influxdb and ES for storage. This way you'll
be able create triggers easily. Eg.

data -> riemann -> [influxdb | ES]

If that is not possible, try with this:

* Chronograf has 'exec' handler and you can use 'riemann-client'
program to send events to riemann. It should be part of
riemann-c-client package in your distro.

* If 'exec' is not good, there is 'tcp' handler and you'll need to
open socket connection on riemann side.

* As James mentioned, you can use Logstash in front of ES to split
events and pass them to ES and riemann simultaneously.

Another approach (if is suitable for your case) is to poll events from
riemann in ES and influxdb and implement triggers there. That will
require some clojure coding and can look like:

1. Run two separate threads, one for ES and one for influxdb
connections.
2. Poll for last entries or poll using some criteria every X seconds.
3. Results inject in riemann stream.
4. From there, use riemann alerting facility.

To connect to rabbitmq, you can use ampq with langohr [1] or send http
post events via clj-http (comes bundled with riemann). Also check
opentsdb.clj [2] on how to create forwarder.

[1] http://clojurerabbitmq.info/articles/getting_started.html
[2] https://github.com/riemann/riemann/blob/master/src/riemann/opentsdb.clj

Best,
Sanel

[Mario Giammarco]

Thanks all for replies, I am reading them.

Another question: can you consume graphite data from riemann?

[Sanel Zukan]

Mario Giammarco <mgiam...@gmail.com> writes:
> Thanks all for replies, I am reading them.
> Another question: can you consume graphite data from riemann?

Yes, via graphite-server [1].

[1] http://riemann.io/api/riemann.transport.graphite.html#var-graphite-server

> Il giorno venerdì 6 settembre 2019 12:42:04 UTC+2, Sanel Z. ha scritto:
>>

> --
> You received this message because you are subscribed to the Google Groups "Riemann Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to riemann-user...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/riemann-users/9a60a6e6-4039-467b-bd33-6748a1fbbe2d%40googlegroups.com.