Failed to save new beam due to JsonMappingException
458 views
Skip to first unread message
Igor Yurinok
unread,
Mar 2, 2016, 1:42:38 PM3/2/16
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to Druid User
Hello,
We have storm topology which ingests data to the druid in real-time. When we submit topology to storm it works OK for a while and after that stops.
In log file we see following errors:
2016-03-02 17:56:33.463 c.m.t.b.ClusteredBeam [WARN] Emitting alert: [anomaly] Failed to create merged beam: druid:overlord/ody_live_bi_event { } java.lang.IllegalStateException: Failed to save new beam for identifier[druid:overlord/ody_live_bi_event] timestamp[2016-03-02T17:40:00.000Z] at com.metamx.tranquility.beam.ClusteredBeam$$anonfun$2.applyOrElse(ClusteredBeam.scala:286) ~[stormjar.jar:?] at com.metamx.tranquility.beam.ClusteredBeam$$anonfun$2.applyOrElse(ClusteredBeam.scala:283) ~[stormjar.jar:?] at com.twitter.util.Future$$anonfun$rescue$1.apply(Future.scala:924) ~[stormjar.jar:?] at com.twitter.util.Future$$anonfun$rescue$1.apply(Future.scala:922) ~[stormjar.jar:?] at com.twitter.util.Promise$Transformer.liftedTree1$1(Promise.scala:112) ~[stormjar.jar:?] at com.twitter.util.Promise$Transformer.k(Promise.scala:112) ~[stormjar.jar:?] at com.twitter.util.Promise$Transformer.apply(Promise.scala:122) ~[stormjar.jar:?] at com.twitter.util.Promise$Transformer.apply(Promise.scala:103) ~[stormjar.jar:?] at com.twitter.util.Promise$$anon$1.run(Promise.scala:366) [stormjar.jar:?] at com.twitter.concurrent.LocalScheduler$Activation.run(Scheduler.scala:178) [stormjar.jar:?] at com.twitter.concurrent.LocalScheduler$Activation.submit(Scheduler.scala:136) [stormjar.jar:?] at com.twitter.concurrent.LocalScheduler.submit(Scheduler.scala:207) [stormjar.jar:?] at com.twitter.concurrent.Scheduler$.submit(Scheduler.scala:92) [stormjar.jar:?] at com.twitter.util.Promise.runq(Promise.scala:350) [stormjar.jar:?] at com.twitter.util.Promise.updateIfEmpty(Promise.scala:721) [stormjar.jar:?] at com.twitter.util.ExecutorServiceFuturePool$$anon$2.run(FuturePool.scala:107) [stormjar.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_45] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_45] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_45] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_45] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_45] Caused by: com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.ArrayIndexOutOfBoundsException) (through reference chain: com.fasterxml.jackson.module.scala.deser.MapBuilderWrapper["beams"]) at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:210) ~[stormjar.jar:?] at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:177) ~[stormjar.jar:?] at com.fasterxml.jackson.databind.deser.std.ContainerDeserializerBase.wrapAndThrow(ContainerDeserializerBase.java:88) ~[stormjar.jar:?] at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringMap(MapDeserializer.java:507) ~[stormjar.jar:?] at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:341) ~[stormjar.jar:?] at com.fasterxml.jackson.module.scala.deser.UnsortedMapDeserializer.deserialize(UnsortedMapDeserializerModule.scala:76) ~[stormjar.jar:?] at com.fasterxml.jackson.module.scala.deser.UnsortedMapDeserializer.deserialize(UnsortedMapDeserializerModule.scala:39) ~[stormjar.jar:?] at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3702) ~[stormjar.jar:?] at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2807) ~[stormjar.jar:?] at com.metamx.tranquility.beam.ClusteredBeamMeta$.fromBytes(ClusteredBeam.scala:467) ~[stormjar.jar:?] at com.metamx.tranquility.beam.ClusteredBeam$$anon$1$$anonfun$modify$1.apply(ClusteredBeam.scala:153) ~[stormjar.jar:?] at com.metamx.tranquility.beam.ClusteredBeam$$anon$1$$anonfun$modify$1.apply(ClusteredBeam.scala:149) ~[stormjar.jar:?] at com.twitter.util.Try$.apply(Try.scala:13) ~[stormjar.jar:?] ... 6 more Caused by: java.lang.ArrayIndexOutOfBoundsException 2016-03-02 17:56:33.463 c.m.e.c.LoggingEmitter [INFO] Event [{"feed":"alerts","timestamp":"2016-03-02T17:56:33.463Z","service":"tranquility","host":"localhost","severity":"anomaly","description":"Failed to create merged beam: druid:overlord/ody_live_bi_event","data":{"exceptionType":"java.lang.IllegalStateException","exceptionMessage":"Failed to save new beam for identifier[druid:overlord/ody_live_bi_event] timestamp[2016-03-02T17:40:00.000Z]","exceptionStackTrace":"java.lang.IllegalStateException: Failed to save new beam for identifier[druid:overlord/ody_live_bi_event] timestamp[2016-03-02T17:40:00.000Z]\n\tat com.metamx.tranquility.beam.ClusteredBeam$$anonfun$2.applyOrElse(ClusteredBeam.scala:286)\n\tat com.metamx.tranquility.beam.ClusteredBeam$$anonfun$2.applyOrElse(ClusteredBeam.scala:283)\n\tat com.twitter.util.Future$$anonfun$rescue$1.apply(Future.scala:924)\n\tat com.twitter.util.Future$$anonfun$rescue$1.apply(Future.scala:922)\n\tat com.twitter.util.Promise$Transformer.liftedTree1$1(Promise.scala:112)\n\tat com.twitter.util.Promise$Transformer.k(Promise.scala:112)\n\tat com.twitter.util.Promise$Transformer.apply(Promise.scala:122)\n\tat com.twitter.util.Promise$Transformer.apply(Promise.scala:103)\n\tat com.twitter.util.Promise$$anon$1.run(Promise.scala:366)\n\tat com.twitter.concurrent.LocalScheduler$Activation.run(Scheduler.scala:178)\n\tat com.twitter.concurrent.LocalScheduler$Activation.submit(Scheduler.scala:136)\n\tat com.twitter.concurrent.LocalScheduler.submit(Scheduler.scala:207)\n\tat com.twitter.concurrent.Scheduler$.submit(Scheduler.scala:92)\n\tat com.twitter.util.Promise.runq(Promise.scala:350)\n\tat com.twitter.util.Promise.updateIfEmpty(Promise.scala:721)\n\tat com.twitter.util.ExecutorServiceFuturePool$$anon$2.run(FuturePool.scala:107)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\n\tat java.lang.Thread.run(Thread.java:745)\nCaused by: com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.ArrayIndexOutOfBoundsException) (through reference chain: com.fasterxml.jackson.module.scala.deser.MapBuilderWrapper[\"beams\"])\n\tat com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:210)\n\tat com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:177)\n\tat com.fasterxml.jackson.databind.deser.std.ContainerDeserializerBase.wrapAndThrow(ContainerDeserializerBase.java:88)\n\tat com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringMap(MapDeserializer.java:507)\n\tat com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:341)\n\tat com.fasterxml.jackson.module.scala.deser.UnsortedMapDeserializer.deserialize(UnsortedMapDeserializerModule.scala:76)\n\tat com.fasterxml.jackson.module.scala.deser.UnsortedMapDeserializer.deserialize(UnsortedMapDeserializerModule.scala:39)\n\tat com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3702)\n\tat com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2807)\n\tat com.metamx.tranquility.beam.ClusteredBeamMeta$.fromBytes(ClusteredBeam.scala:467)\n\tat com.metamx.tranquility.beam.ClusteredBeam$$anon$1$$anonfun$modify$1.apply(ClusteredBeam.scala:153)\n\tat com.metamx.tranquility.beam.ClusteredBeam$$anon$1$$anonfun$modify$1.apply(ClusteredBeam.scala:149)\n\tat com.twitter.util.Try$.apply(Try.scala:13)\n\t... 6 more\nCaused by: java.lang.ArrayIndexOutOfBoundsException\n"}}] 2016-03-02 17:56:33.463 c.m.t.t.Tranquilizer [WARN] Failed to send 3 messages. java.lang.IllegalStateException: Failed to save new beam for identifier[druid:overlord/ody_live_bi_event] timestamp[2016-03-02T17:40:00.000Z] at com.metamx.tranquility.beam.ClusteredBeam$$anonfun$2.applyOrElse(ClusteredBeam.scala:286) ~[stormjar.jar:?] at com.metamx.tranquility.beam.ClusteredBeam$$anonfun$2.applyOrElse(ClusteredBeam.scala:283) ~[stormjar.jar:?] at com.twitter.util.Future$$anonfun$rescue$1.apply(Future.scala:924) ~[stormjar.jar:?] at com.twitter.util.Future$$anonfun$rescue$1.apply(Future.scala:922) ~[stormjar.jar:?] at com.twitter.util.Promise$Transformer.liftedTree1$1(Promise.scala:112) ~[stormjar.jar:?] at com.twitter.util.Promise$Transformer.k(Promise.scala:112) ~[stormjar.jar:?] at com.twitter.util.Promise$Transformer.apply(Promise.scala:122) ~[stormjar.jar:?] at com.twitter.util.Promise$Transformer.apply(Promise.scala:103) ~[stormjar.jar:?] at com.twitter.util.Promise$$anon$1.run(Promise.scala:366) [stormjar.jar:?] at com.twitter.concurrent.LocalScheduler$Activation.run(Scheduler.scala:178) [stormjar.jar:?] at com.twitter.concurrent.LocalScheduler$Activation.submit(Scheduler.scala:136) [stormjar.jar:?] at com.twitter.concurrent.LocalScheduler.submit(Scheduler.scala:207) [stormjar.jar:?] at com.twitter.concurrent.Scheduler$.submit(Scheduler.scala:92) [stormjar.jar:?] at com.twitter.util.Promise.runq(Promise.scala:350) [stormjar.jar:?] at com.twitter.util.Promise.updateIfEmpty(Promise.scala:721) [stormjar.jar:?] at com.twitter.util.ExecutorServiceFuturePool$$anon$2.run(FuturePool.scala:107) [stormjar.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_45] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_45] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_45] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_45] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_45] Caused by: com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.ArrayIndexOutOfBoundsException) (through reference chain: com.fasterxml.jackson.module.scala.deser.MapBuilderWrapper["beams"]) at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:210) ~[stormjar.jar:?] at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:177) ~[stormjar.jar:?] at com.fasterxml.jackson.databind.deser.std.ContainerDeserializerBase.wrapAndThrow(ContainerDeserializerBase.java:88) ~[stormjar.jar:?] at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringMap(MapDeserializer.java:507) ~[stormjar.jar:?] at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:341) ~[stormjar.jar:?] at com.fasterxml.jackson.module.scala.deser.UnsortedMapDeserializer.deserialize(UnsortedMapDeserializerModule.scala:76) ~[stormjar.jar:?] at com.fasterxml.jackson.module.scala.deser.UnsortedMapDeserializer.deserialize(UnsortedMapDeserializerModule.scala:39) ~[stormjar.jar:?] at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3702) ~[stormjar.jar:?] at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2807) ~[stormjar.jar:?] at com.metamx.tranquility.beam.ClusteredBeamMeta$.fromBytes(ClusteredBeam.scala:467) ~[stormjar.jar:?] at com.metamx.tranquility.beam.ClusteredBeam$$anon$1$$anonfun$modify$1.apply(ClusteredBeam.scala:153) ~[stormjar.jar:?] at com.metamx.tranquility.beam.ClusteredBeam$$anon$1$$anonfun$modify$1.apply(ClusteredBeam.scala:149) ~[stormjar.jar:?] at com.twitter.util.Try$.apply(Try.scala:13) ~[stormjar.jar:?] ... 6 more Caused by: java.lang.ArrayIndexOutOfBoundsException
Igor Yurinok
unread,
Mar 4, 2016, 3:29:37 PM3/4/16
Reply to author
Sign in to reply to author
Forward
Sign in to forward
Delete
You do not have permission to delete messages in this group
Copy link
Report message
Show original message
Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message
to Druid User
We solved this issue by updating all Jackson libraries from version 2.6.0 to version 2.6.3