Cannot get compaction metrics

[Premal Shah]

Hi,

I'm trying to get hannibal to show compaction metrics in the UI and this is what is shows "No Data recorded yet for MetricDef [object Object]"

compactions.set-loglevels-on-startup is set to true

The log file pattern is untouched compactions.logfile-path-pattern = "(?i)\"/logs/(.*regionserver.*[.].*)\""

Here is the listing for one of my regionservers when i git http://<<some-regionserver>>:60030/logs/

SecurityAuth.audit 0 bytes Jun 7, 2013 5:04:45 AM

hbase-hbase-regionserver.gc.0 85603212 bytes Jul 6, 2013 12:03:06 AM

hbase-hbase-regionserver.log 35445 bytes Jul 6, 2013 12:02:41 AM

hbase-hbase-regionserver.log.2013-06-28 90673152 bytes Jun 28, 2013 11:59:59 PM

hbase-hbase-regionserver.log.2013-06-29 67196416 bytes Jun 29, 2013 11:59:10 PM

hbase-hbase-regionserver.log.2013-06-30 24361002 bytes Jun 30, 2013 11:59:39 PM

hbase-hbase-regionserver.log.2013-07-01 18950087 bytes Jul 1, 2013 11:57:15 PM

hbase-hbase-regionserver.log.2013-07-02 11235328 bytes Jul 2, 2013 11:59:40 PM

hbase-hbase-regionserver.log.2013-07-03 15085786 bytes Jul 3, 2013 11:57:29 PM

hbase-hbase-regionserver.log.2013-07-04 44412750 bytes Jul 4, 2013 11:59:45 PM

hbase-hbase-regionserver.log.2013-07-05 91374512 bytes Jul 5, 2013 11:59:21 PM

hbase-hbase-regionserver.out 117600 bytes Jul 6, 2013 12:02:43 AM

hbase-hbase-regionserver.out.1 39430 bytes Jul 4, 2013 6:49:20 AM

hbase-hbase-regionserver.out.2 20727 bytes Jul 3, 2013 10:11:46 PM

hbase-hbase-regionserver.out.3 613 bytes Jul 3, 2013 3:14:32 PM

hbase-hbase-regionserver.out.4 57344 bytes Jul 3, 2013 7:33:29 AM

hbase-hbase-regionserver.out.5 559 bytes Jun 29, 2013 7:31:02 AM

On startup, this what I see

Play server process ID is 28809

[info] play - database [default] connected at jdbc:h2:data/metrics.h2

[info] application - Application has started in Prod-Mode, starting Update-Metrics-Actor

[info] play - Starting application default Akka system.

[info] play - Application started (Prod)

[info] application - start updating RegionMetrics... (Sat Jul 06 00:11:45 UTC 2013)

[info] play - Listening for HTTP on port 9000...

[info] application - completed updating 43 RegionMetrics, took 3442ms... (Sat Jul 06 00:11:48 UTC 2013) 

[info] application - setting Loglevels for the Regionservers

[info] application - Found path matching compactions.logfile-path-pattern: hbase-hbase-regionserver.gc.0

[info] application - Discovered log file url pattern: [http://%hostname%:%infoport%/logs/hbase-hbase-regionserver.gc.0]

[info] application - start updating CompactionMetrics... (Sat Jul 06 00:12:11 UTC 2013)

[info] application - Initializing log offset to [29583617] for log file at http://<redacted>:60030/logs/hbase-hbase-regionserver.gc.0 with content-length [30632193]

[info] application - Initializing log offset to [16038587] for log file at http://<redacted>:60030/logs/hbase-hbase-regionserver.gc.0 with content-length [17087163]

Does the log file pattern look right? 

Should it not find http://%hostname%:%infoport%/logs/hbase-hbase-regionserver.log

I tried updating the regex to 

compactions.logfile-path-pattern = "(?i)\"/logs/hbase-hbase-regionserver.log\""

and it fails on init with this error

[ERROR] [07/06/2013 00:11:25.484] [application-akka.actor.default-dispatcher-1] [akka://application/user/updateMetricsActor] No group 1

java.lang.IndexOutOfBoundsException: No group 1

at java.util.regex.Matcher.group(Matcher.java:470)

at models.LogFile$$anonfun$discoverLogFileUrlPattern$1$$anonfun$apply$mcV$sp$1.apply(LogFile.scala:129)

at models.LogFile$$anonfun$discoverLogFileUrlPattern$1$$anonfun$apply$mcV$sp$1.apply(LogFile.scala:123)

at models.hbase092.HBase092$$anonfun$eachRegionServer$1$$anonfun$apply$1.apply(HBase092.scala:15)

at models.hbase092.HBase092$$anonfun$eachRegionServer$1$$anonfun$apply$1.apply(HBase092.scala:14)

at scala.collection.Iterator$class.foreach(Iterator.scala:660)

at scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:573)

at scala.collection.IterableLike$class.foreach(IterableLike.scala:73)

at scala.collection.JavaConversions$JCollectionWrapper.foreach(JavaConversions.scala:592)

at models.hbase092.HBase092$$anonfun$eachRegionServer$1.apply(HBase092.scala:14)

at models.hbase092.HBase092$$anonfun$eachRegionServer$1.apply(HBase092.scala:11)

at models.HBase$class.withAdmin(HBase.scala:44)

at models.hbase092.HBase092.withAdmin(HBase092.scala:9)

at models.hbase092.HBase092.eachRegionServer(HBase092.scala:11)

at models.LogFile$$anonfun$discoverLogFileUrlPattern$1.apply$mcV$sp(LogFile.scala:122)

at scala.util.control.Breaks.breakable(Breaks.scala:35)

at models.LogFile$.discoverLogFileUrlPattern(LogFile.scala:121)

at models.LogFile$.init(LogFile.scala:159)

at actors.UpdateMetricsActor$$anonfun$receive$1.apply(UpdateMetricsActor.scala:41)

at actors.UpdateMetricsActor$$anonfun$receive$1.apply(UpdateMetricsActor.scala:23)

at akka.actor.Actor$class.apply(Actor.scala:318)

at actors.UpdateMetricsActor.apply(UpdateMetricsActor.scala:22)

at akka.actor.ActorCell.invoke(ActorCell.scala:626)

at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:197)

at akka.dispatch.Mailbox.run(Mailbox.scala:179)

at akka.dispatch.ForkJoinExecutorConfigurator$MailboxExecutionTask.exec(AbstractDispatcher.scala:516)

at akka.jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:259)

at akka.jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:975)

at akka.jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1479)

at akka.jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)

Any help is appreciated. Thanx in advance.

[Nils Kübler]

Hi,

Try to use braces in the pattern

compactions.logfile-path-pattern = "(?i)\"/logs/(hbase-hbase-regionserver.log)\""

this should allow the Hannibal to find the correct portion of the regex.

Cheers,
Nils

[Premal Shah]

Sweet. That works. Thanx. Hannibal is an awesome tool. Thanx for sharing it with the community.

[Jean-Pierre König]

Hi,

Would you mind writing a recommendation of Hannibal on LinkedIn? http://www.linkedin.com/company/ymc-ag/hannibal-23310691/product?goback=%2Enmp_*1_*1_*1_*1_*1_*1_*1_*1_*1_*1%2Ebzp_*1_*1_*1_*1_*1_*1_*1_*1_*1_*1_ymc*5ag&trk=biz_product

Sincerely,

Jean-Pierre Koenig