Riemann is sending only internal events to influxdb

[Pawel Akonom]

Hi,

I have configured riemann to send events to influxdb using influxBatchSender. I have integrated icinga with riemann using oscp_command so every time icinga is executing check it is sending an event to riemann.

Here is my /etc/riemann/riemann.config:

(logging/init {:file "/var/log/riemann/riemann.log"})
 
; Listen on the local interface over TCP (5555), UDP (5555), and websockets
; (5556)
(let [host "0.0.0.0"]
  (tcp-server {:host host})
  (udp-server {:host host})
  (ws-server  {:host host}))
 
; Expire old events from the index every 5 seconds.
(periodically-expire 5)
 
(def influx {
        :version  :0.9
        :host     "localhost"
        :port     8086
        :username "riemann"
        :password "riemann"
        :db       "riemann_db"
    }
)
 
(def influxBatchSender
  (batch 100 1/10
         (async-queue! :agg {:queue-size 1000
                             :core-pool-size 1
                             :max-pool-size 4
                             :keep-alive-time 60000}
                             (influxdb influx))))
  
(let [index (index)]
  (streams
   influxBatchSender
   #(info %)
  )
)

In the riemann log file I can see all events - riemann internal events and icinga events:

INFO [2016-05-23 13:25:44,147] Thread-6 - riemann.config - #riemann.codec.Event{:host metrics-collector1, :service riemann executor agg threads active, :state ok, :description nil, :metric 0, :tags nil, :time 1464024344143/1000, :ttl 20}
INFO [2016-05-23 13:25:44,147] Thread-6 - riemann.config - #riemann.codec.Event{:host metrics-collector1, :service riemann executor agg threads alive, :state ok, :description nil, :metric 1, :tags nil, :time 1464024344143/1000, :ttl 20}
INFO [2016-05-23 13:25:51,203] defaultEventExecutorGroup-2-2 - riemann.config - #riemann.codec.Event{:host config-server1, :service log-warnings, :state OK, :description OK: No stream alerts triggered for stream: Config-Server1_Log_Warnings, :metric nil, :tags nil, :time 1.464024351203E9, :ttl 600.0, :source icinga}
INFO [2016-05-23 13:25:51,301] defaultEventExecutorGroup-2-1 - riemann.config - #riemann.codec.Event{:host model-master1, :service boot-fs-disk-usage, :state OK, :description DISK OK - free space: /boot 418 MB (92% inode=99%):, :metric nil, :tags nil, :time 1.464024351301E9, :ttl 600.0, :source icinga}

But in the influxdb only riemann internal events:

[root@metrics-collector1 tmp]# influx
Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring.
Connected to http://localhost:8086 version 0.9.6.1
InfluxDB shell 0.9.6.1
> use riemann_db
Using database riemann_db
> show measurements
name: measurements
------------------
name
riemann executor agg accepted rate
riemann executor agg completed rate
riemann executor agg queue capacity
riemann executor agg queue size
riemann executor agg queue used
riemann executor agg rejected rate
riemann executor agg threads active
riemann executor agg threads alive
riemann index size
riemann netty event-executor threads active
riemann server tcp 0.0.0.0:5555 conns
riemann server tcp 0.0.0.0:5555 in latency 0.0
riemann server tcp 0.0.0.0:5555 in latency 0.5
riemann server tcp 0.0.0.0:5555 in latency 0.95
riemann server tcp 0.0.0.0:5555 in latency 0.99
riemann server tcp 0.0.0.0:5555 in latency 0.999
riemann server tcp 0.0.0.0:5555 in rate
riemann server udp 0.0.0.0:5555 in rate
riemann server ws 0.0.0.0:5556 conns
riemann server ws 0.0.0.0:5556 in rate
riemann server ws 0.0.0.0:5556 out rate
riemann streams latency 0.0
riemann streams latency 0.5
riemann streams latency 0.95
riemann streams latency 0.99
riemann streams latency 0.999
riemann streams rate

Do you know can be the problem?

[Pradeep Chhetri]

Riemann Influxdb plugin checks for the metric field in the Riemann event. If the value is set to some value, then only it sends the event to influxdb otherwise it just discards them.

I can see that the two icinga events in the part of the riemann log you posted, have metric field as nil. Can you verify if this is the case with other icinga events as well by going through the complete log.

--
You received this message because you are subscribed to the Google Groups "Riemann Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to riemann-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Pawel Akonom]

Thanks a lot for your replay. I set metric field to 0 for icinga events and now I can see icinga events in influxdb.

It was a good hint :)

[Aphyr]

On 05/23/2016 02:14 PM, Pawel Akonom wrote:
>
> Thanks a lot for your replay. I set metric field to 0 for icinga events and now
> I can see icinga events in influxdb.
>
> It was a good hint :)

If events without metrics have meaning for influxdb, maybe we should drop this
filter from the influxdb adapter.

--Kyle

[James Turnbull]

I'd be happy to adjust it if someone can let me know whether it's a
realTM thing. :)

Just log an issue.

https://github.com/riemann/riemann/issues

Regards

James

--
* The Art of Monitoring (https://artofmonitoring.com)
* The Docker Book (https://dockerbook.com)
* The LogStash Book (https://logstashbook.com)
* Pro Puppet (http://tinyurl.com/ppuppet2 )
* Pro Linux System Administration (http://tinyurl.com/linuxadmin)
* Hardening Linux (http://tinyurl.com/hardeninglinux)