problem loading segments into historical
Pablo Nebrera
2014-01-18 10:18:28,932 INFO [ZkCoordinator-0] io.druid.server.coordination.ZkCoordinator - New node[/druid/loadQueue/rb003:8083/rb_event_2014-01-11T17:00:00.000Z_2014-01-11T18:00:00.000Z_2014-01-11T17:00:00.000Z] with segmentClass[class io.druid.server.coordination.SegmentChangeRequestLoad]
2014-01-18 10:18:28,932 INFO [ZkCoordinator-0] io.druid.server.coordination.ZkCoordinator - Loading segment rb_event_2014-01-11T17:00:00.000Z_2014-01-11T18:00:00.000Z_2014-01-11T17:00:00.000Z
2014-01-18 10:18:28,932 INFO [ZkCoordinator-0] io.druid.segment.loading.OmniSegmentLoader - Deleting directory[/opt/rb/var/druid/coordinator/indexCache/rb_event/2014-01-11T17:00:00.000Z_2014-01-11T18:00:00.000Z/2014-01-11T17:00:00.000Z/0]
2014-01-18 10:18:28,932 INFO [ZkCoordinator-0] io.druid.server.coordination.ServerManager - Told to delete a queryable for a dataSource[rb_event] that doesn't exist.
2014-01-18 10:18:28,932 WARN [ZkCoordinator-0] io.druid.server.coordination.ZkCoordinator - Unable to delete segmentInfoCacheFile[/opt/rb/var/druid/coordinator/indexCache/info_dir/rb_event_2014-01-11T17:00:00.000Z_2014-01-11T18:00:00.000Z_2014-01-11T17:00:00.000Z]
2014-01-18 10:18:28,932 INFO [ZkCoordinator-0] io.druid.server.coordination.SingleDataSegmentAnnouncer - Unannouncing segment[rb_event_2014-01-11T17:00:00.000Z_2014-01-11T18:00:00.000Z_2014-01-11T17:00:00.000Z] at path[/druid/servedSegments/rb003:8083/rb_event_2014-01-11T17:00:00.000Z_2014-01-11T18:00:00.000Z_2014-01-11T17:00:00.000Z]
2014-01-18 10:18:28,932 INFO [ZkCoordinator-0] io.druid.curator.announcement.Announcer - unannouncing [/druid/servedSegments/rb003:8083/rb_event_2014-01-11T17:00:00.000Z_2014-01-11T18:00:00.000Z_2014-01-11T17:00:00.000Z]
2014-01-18 10:18:28,932 ERROR [ZkCoordinator-0] io.druid.curator.announcement.Announcer - Path[/druid/servedSegments/rb003:8083/rb_event_2014-01-11T17:00:00.000Z_2014-01-11T18:00:00.000Z_2014-01-11T17:00:00.000Z] not announced, cannot unannounce.
2014-01-18 10:18:28,932 ERROR [ZkCoordinator-0] io.druid.server.coordination.ZkCoordinator - Failed to load segment for dataSource: {class=io.druid.server.coordination.ZkCoordinator, exceptionType=class io.druid.segment.loading.SegmentLoadingException, exceptionMessage=Exception loading segment[rb_event_2014-01-11T17:00:00.000Z_2014-01-11T18:00:00.000Z_2014-01-11T17:00:00.000Z], segment=DataSegment{size=153929, shardSpec=LinearShardSpec{partitionNum=0}, metrics=[events], dimensions=[action, classification, domain_name, dst, dst_as, dst_as_name, dst_country, dst_country_code, dst_net, dst_net_name, dst_port, ethdst, ethdst_vendor, ethlength_range, ethsrc, ethsrc_vendor, group_id, group_name, icmpcode, icmptype, iplen, iplen_range, l4_proto, msg, priority, rev, sensor_id, sensor_ip, sensor_name, sig_generator, sig_id, src, src_as, src_as_name, src_country, src_country_code, src_net, src_net_name, src_port, tos, ttl, type], version='2014-01-11T17:00:00.000Z', loadSpec={type=s3_zip, bucket=redborder, key=rb/rb_event/2014-01-11T17:00:00.000Z_2014-01-11T18:00:00.000Z/2014-01-11T17:00:00.000Z/0/index.zip}, interval=2014-01-11T17:00:00.000Z/2014-01-11T18:00:00.000Z, dataSource='rb_event', binaryVersion='9'}}
io.druid.segment.loading.SegmentLoadingException: Exception loading segment[rb_event_2014-01-11T17:00:00.000Z_2014-01-11T18:00:00.000Z_2014-01-11T17:00:00.000Z]
at io.druid.server.coordination.ZkCoordinator.addSegment(ZkCoordinator.java:239)
at io.druid.server.coordination.SegmentChangeRequestLoad.go(SegmentChangeRequestLoad.java:44)
at io.druid.server.coordination.ZkCoordinator$1.childEvent(ZkCoordinator.java:131)
at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:494)
at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:488)
at org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92)
at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:293)
at org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:83)
at org.apache.curator.framework.recipes.cache.PathChildrenCache.callListeners(PathChildrenCache.java:485)
at org.apache.curator.framework.recipes.cache.EventOperation.invoke(EventOperation.java:35)
at org.apache.curator.framework.recipes.cache.PathChildrenCache$11.run(PathChildrenCache.java:755)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: io.druid.segment.loading.SegmentLoadingException: Unknown loader type[s3_zip]. Known types are [local]
at io.druid.segment.loading.OmniSegmentLoader.getPuller(OmniSegmentLoader.java:170)
at io.druid.segment.loading.OmniSegmentLoader.getSegmentFiles(OmniSegmentLoader.java:125)
at io.druid.segment.loading.OmniSegmentLoader.getSegment(OmniSegmentLoader.java:93)
at io.druid.server.coordination.ServerManager.loadSegment(ServerManager.java:129)
at io.druid.server.coordination.ZkCoordinator.addSegment(ZkCoordinator.java:235)
... 19 more
2014-01-18 10:18:28,932 INFO [ZkCoordinator-0] com.metamx.emitter.core.LoggingEmitter - Event [{"feed":"alerts","timestamp":"2014-01-18T10:18:28.932Z","service":"historical","host":"rb003:8083","severity":"component-failure","description":"Failed to load segment for dataSource","data":{"class":"io.druid.server.coordination.ZkCoordinator","exceptionType":"io.druid.segment.loading.SegmentLoadingException","exceptionMessage":"Exception loading segment[rb_event_2014-01-11T17:00:00.000Z_2014-01-11T18:00:00.000Z_2014-01-11T17:00:00.000Z]","exceptionStackTrace":"io.druid.segment.loading.SegmentLoadingException: Exception loading segment[rb_event_2014-01-11T17:00:00.000Z_2014-01-11T18:00:00.000Z_2014-01-11T17:00:00.000Z]\n\tat io.druid.server.coordination.ZkCoordinator.addSegment(ZkCoordinator.java:239)\n\tat io.druid.server.coordination.SegmentChangeRequestLoad.go(SegmentChangeRequestLoad.java:44)\n\tat io.druid.server.coordination.ZkCoordinator$1.childEvent(ZkCoordinator.java:131)\n\tat org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:494)\n\tat org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:488)\n\tat org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92)\n\tat com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:293)\n\tat org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:83)\n\tat org.apache.curator.framework.recipes.cache.PathChildrenCache.callListeners(PathChildrenCache.java:485)\n\tat org.apache.curator.framework.recipes.cache.EventOperation.invoke(EventOperation.java:35)\n\tat org.apache.curator.framework.recipes.cache.PathChildrenCache$11.run(PathChildrenCache.java:755)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)\n\tat java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)\n\tat java.util.concurrent.FutureTask.run(Unknown Source)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)\n\tat java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)\n\tat java.util.concurrent.FutureTask.run(Unknown Source)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)\n\tat java.lang.Thread.run(Unknown Source)\nCaused by: io.druid.segment.loading.SegmentLoadingException: Unknown loader type[s3_zip]. Known types are [local]\n\tat io.druid.segment.loading.OmniSegmentLoader.getPuller(OmniSegmentLoader.java:170)\n\tat io.druid.segment.loading.OmniSegmentLoader.getSegmentFiles(OmniSegmentLoader.java:125)\n\tat io.druid.segment.loading.OmniSegmentLoader.getSegment(OmniSegmentLoader.java:93)\n\tat io.druid.server.coordination.ServerManager.loadSegment(ServerManager.java:129)\n\tat io.druid.server.coordination.ZkCoordinator.addSegment(ZkCoordinator.java:235)\n\t... 19 more\n","segment":{"dataSource":"rb_event","interval":"2014-01-11T17:00:00.000Z/2014-01-11T18:00:00.000Z","version":"2014-01-11T17:00:00.000Z","loadSpec":{"type":"s3_zip","bucket":"redborder","key":"rb/rb_event/2014-01-11T17:00:00.000Z_2014-01-11T18:00:00.000Z/2014-01-11T17:00:00.000Z/0/index.zip"},"dimensions":"action,classification,domain_name,dst,dst_as,dst_as_name,dst_country,dst_country_code,dst_net,dst_net_name,dst_port,ethdst,ethdst_vendor,ethlength_range,ethsrc,ethsrc_vendor,group_id,group_name,icmpcode,icmptype,iplen,iplen_range,l4_proto,msg,priority,rev,sensor_id,sensor_ip,sensor_name,sig_generator,sig_id,src,src_as,src_as_name,src_country,src_country_code,src_net,src_net_name,src_port,tos,ttl,type","metrics":"events","shardSpec":{"type":"linear","partitionNum":0},"binaryVersion":9,"size":153929,"identifier":"rb_event_2014-01-11T17:00:00.000Z_2014-01-11T18:00:00.000Z_2014-01-11T17:00:00.000Z"}}}]
2014-01-18 10:18:28,934 INFO [ZkCoordinator-0] io.druid.server.coordination.ZkCoordinator - Completed processing for node[/druid/loadQueue/rb003:8083/rb_event_2014-01-11T17:00:00.000Z_2014-01-11T18:00:00.000Z_2014-01-11T17:00:00.000Z]
2014-01-18 10:18:28,934 INFO [ZkCoordinator-0] io.druid.server.coordination.ZkCoordinator - /druid/loadQueue/rb003:8083/rb_event_2014-01-11T17:00:00.000Z_2014-01-11T18:00:00.000Z_2014-01-11T17:00:00.000Z was removed
What could it be??
Thanks
Pablo
Pablo Nebrera
druid.extensions.coordinates=["io.druid.extensions:druid-s3-extensions:0.6.39"]
to the historical node configuration too
thanks for all your time
Pablo