Maintenance mode and java exception

[Pawel Akonom]

Hi,

I wanted to test maintenance mode from riemann examples:
http://riemann.io/howto.html#query-the-index-from-within-a-stream

I have added function to my riemann.config

(defn maintenance-mode?
  "Is Riemann currently in maintenance mode?"
  []
  ; Take an expression representing a query for maintenance mode
  (->> '(and (= host nil)
             (= service "maintenance-mode"))
       ; Search the current Riemann core's index for any matching events
       (riemann.index/search (:index @core))
       ; Take the first match
       first
       ; Find its state
       :state
       ; Is it the string "active"?
       (= "active")))

and the code inside the streams to execute the function - just like in riemann example

(let [index (index)]
  (streams
  (changed-state {:init "ok"}
    (where (not (maintenance-mode? ))
      (email "pawel....@vnomic.com")))
  #(info %)
 )
)

After restarting riemann java exception occurs in riemann.log file

INFO [2016-06-23 10:47:59,046] defaultEventExecutorGroup-2-1 - riemann.config - #riemann.codec.Event{:host ESB_1, :service cpu-0/percent-idle-riemann, :state nil, :description nil, :metric 9
9.8998998998999, :tags [collectd], :time 1466693279, :ttl 100.0, :ds_index 0, :ds_name value, :ds_type gauge, :type_instance idle, :type percent, :plugin_instance 0, :plugin cpu}
WARN [2016-06-23 10:47:59,047] defaultEventExecutorGroup-2-1 - riemann.streams - riemann.config$eval37$new_fork__5101__auto____174$stream__175@38f4b236 threw
java.lang.RuntimeException: Unable to resolve symbol: host in this context, compiling:(NO_SOURCE_PATH:0:0)
        at clojure.lang.Compiler.analyze(Compiler.java:6464)
        at clojure.lang.Compiler.analyze(Compiler.java:6406)
        at clojure.lang.Compiler$HostExpr$Parser.parse(Compiler.java:957)
        at clojure.lang.Compiler.analyzeSeq(Compiler.java:6644)
        at clojure.lang.Compiler.analyze(Compiler.java:6445)
        at clojure.lang.Compiler.analyze(Compiler.java:6406)
        at clojure.lang.Compiler.analyzeSeq(Compiler.java:6639)
        at clojure.lang.Compiler.analyze(Compiler.java:6445)
        at clojure.lang.Compiler.access$100(Compiler.java:38)
        at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:6050)
        at clojure.lang.Compiler.analyzeSeq(Compiler.java:6644)
        at clojure.lang.Compiler.analyze(Compiler.java:6445)
        at clojure.lang.Compiler.analyzeSeq(Compiler.java:6632)
        at clojure.lang.Compiler.analyze(Compiler.java:6445)
        at clojure.lang.Compiler.analyzeSeq(Compiler.java:6632)
        at clojure.lang.Compiler.analyze(Compiler.java:6445)
        at clojure.lang.Compiler.analyze(Compiler.java:6406)
        at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5782)
        at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5217)
        at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3846)
        at clojure.lang.Compiler.analyzeSeq(Compiler.java:6642)
        at clojure.lang.Compiler.analyze(Compiler.java:6445)
        at clojure.lang.Compiler.analyze(Compiler.java:6406)
        at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5782)
        at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5217)
        at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3846)
        at clojure.lang.Compiler.analyzeSeq(Compiler.java:6642)
        at clojure.lang.Compiler.analyze(Compiler.java:6445)
        at clojure.lang.Compiler.eval(Compiler.java:6700)
        at clojure.lang.Compiler.eval(Compiler.java:6666)
        at clojure.core$eval.invoke(core.clj:2927)
        at riemann.query$fun.invoke(query.clj:212)
        at riemann.index$nbhm_index$reify__3482.search(index.clj:85)
        at riemann.core$wrap_index$reify__5826.search(core.clj:216)
        at riemann.config$maintenance_mode_QMARK_.invoke(riemann.config:49)
        at riemann.config$eval37$new_fork__5101__auto____174$stream__175.invoke(riemann.config:97)
        at riemann.streams$changed$stream__5127$fn__5139.invoke(streams.clj:1548)
        at riemann.streams$changed$stream__5127.invoke(streams.clj:1548)
        at riemann.streams$by_fn$stream__5103$fn__5108.invoke(streams.clj:1512)
        at riemann.streams$by_fn$stream__5103.invoke(streams.clj:1512)
        at riemann.core$stream_BANG_$fn__5727.invoke(core.clj:19)
        at riemann.core$stream_BANG_.invoke(core.clj:18)
        at riemann.transport$handle.invoke(transport.clj:159)
        at riemann.transport.udp$udp_handler.invoke(udp.clj:51)
        at riemann.transport.udp$gen_udp_handler$fn__6043.invoke(udp.clj:41)
        at riemann.transport.udp.proxy$io.netty.channel.ChannelInboundHandlerAdapter$ff19274a.channelRead(Unknown Source)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
        at io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
        at io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
        at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:36)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.RuntimeException: Unable to resolve symbol: host in this context
        at clojure.lang.Util.runtimeException(Util.java:221)
        at clojure.lang.Compiler.resolveIn(Compiler.java:6940)
        at clojure.lang.Compiler.resolve(Compiler.java:6884)
        at clojure.lang.Compiler.analyzeSymbol(Compiler.java:6845)
        at clojure.lang.Compiler.analyze(Compiler.java:6427)
        ... 52 more
INFO [2016-06-23 10:47:59,048] defaultEventExecutorGroup-2-1 - riemann.config - #riemann.codec.Event{:host ESB_1, :service cpu-0/percent-idle, :state nil, :description nil, :metric 99.8998998998999, :tags [collectd], :time 1466693279, :ttl 100.0, :ds_index 0, :ds_name value, :ds_type gauge, :type_instance idle, :type percent, :plugin_instance 0, :plugin cpu}

I am running riemann version 0.2.10. Do you know what is causing java exception?

Br,
Pawel

[Brian Lalor]

The   (->> '(and (= host nil) looks fishy to me (and I know it’s in the docs). Maybe try :host instead?

--
You received this message because you are subscribed to the Google Groups "Riemann Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to riemann-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

— 

Brian Lalor

bla...@bravo5.org

[Pawel Akonom]

Hi Brian,

So how the final code for maintenance-mode function should looks like? I am a beginner for riemann and clojure.

Br,
Pawel

[Pradeep Chhetri]

Hi Pawel,

Can you try this code for maintenance-mode function please: 

https://gist.github.com/pradeepchhetri/bfccdee75da665e763b7af4f3b7d18a1

Let us know how it goes.

--
You received this message because you are subscribed to the Google Groups "Riemann Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to riemann-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Regards,

Pradeep Chhetri

[Pawel Akonom]

Hi Pradeep,

Seems to work - at least I can't see any java exception in log file. I need to test maintenance mode now.

Thank you for your hint.

Br,
Pawel

W dniu czwartek, 23 czerwca 2016 16:51:38 UTC+2 użytkownik Pawel Akonom napisał:

[James Turnbull]

Pradeep Chhetri wrote:
> Hi Pawel,
>
> Can you try this code for maintenance-mode function please:
>

Thanks Pradeep! Merged your PR to fix the docs too.

Regards

James


--
* The Art of Monitoring (https://artofmonitoring.com)
* The Docker Book (https://dockerbook.com)
* The LogStash Book (https://logstashbook.com)
* Pro Puppet (http://tinyurl.com/ppuppet2 )
* Pro Linux System Administration (http://tinyurl.com/linuxadmin)
* Hardening Linux (http://tinyurl.com/hardeninglinux)

[Pawel Akonom]

Hi,

I was testing maintenacne-mode activation by sending special event with python riemann client https://github.com/borntyping/python-riemann-client

Simple test I have created it was saving all events to log file if maintenance-mode is not activated. Here is the riemann.config file:

[root@metrics-collector1 ~]# cat /etc/riemann/riemann.config
; -*- mode: clojure; -*-
; vim: filetype=clojure

(logging/init {:file "/var/log/riemann/riemann.log"})

(defn maintenance-mode?
  "Is Riemann currently in maintenance mode?"
  []
  ; Take an expression representing a query for maintenance mode
  (->> '(and (= :host nil)
             (= :service "maintenance-mode"))
       ; Search the current Riemann core's index for any matching events
       (riemann.index/search (:index @core))
       ; Take the first match
       first
       ; Find its state
       :state
       ; Is it the string "active"?
       (= "active")))

; Listen on the local interface over TCP (5555), UDP (5555), and websockets
; (5556)
(let [host "0.0.0.0"]
  (tcp-server {:host host})
  (udp-server {:host host})
  (ws-server  {:host host}))

; Expire old events from the index every 5 seconds.
(periodically-expire 5)

(let [index (index)]
  ; Inbound events will be passed to these streams:
  (streams
    (default :ttl 60
      ; Index all events immediately.
      index

      (where (not (maintenance-mode?)) #(info %) )

      ; Log expired events.
      (expired
        (fn [event] (info "expired" event))))))


After starting riemann server all events started to be saved to /var/log/riemann/riemann.log file.

I have sent maintenance-mode event with python riemann client:

[root@metrics-collector1 ~]# /usr/bin/riemann-client -H localhost -P 5555 send -h nil -s maintenance-mode -S active -l 100
{
  "host": "nil",
  "service": "maintenance-mode",
  "state": "active",
  "ttl": 100
}

Then I have checked the log file and events was still saved. I have expected events will not be saved to the log file when maintenance-mode is activated. Do you know why it didn't work?

Br,
Pawel

[Stepan Karamyshev]

I suppose you have string "nil" in event at host VS nil in maintenance-mode block

Can you recheck this?

29 июня 2016 г., в 16:25, Pawel Akonom <pawel....@gmail.com> написал(а):

[Pawel Akonom]

Hi,

I have checked rieman.config again and I have added quotes to nil:

(defn maintenance-mode?
  "Is Riemann currently in maintenance mode?"
  []
  ; Take an expression representing a query for maintenance mode
  (->> '(and (= :host "nil")
             (= :service "maintenance-mode"))
       ; Search the current Riemann core's index for any matching events
       (riemann.index/search (:index @core))
       ; Take the first match
       first
       ; Find its state
       :state
       ; Is it the string "active"?
       (= "active")))

After this change when I sent the same event via python riemann client it started to work :) In the event I sent hostname is a string "nil" not nil in the meaning of no value.

After this modification and sending maintenace-mode activation event only expired events was visible in the log file so it is working like it should.

Thank you Seva for the hint with "nil".

Br,
Pawel