mysql> select count(*) from prod_segments where used=1;
+----------+
| count(*) |
+----------+
| 485 |
+----------+
1 row in set (0.00 sec)
2014-12-12 04:46:39,841 INFO [DatabaseSegmentManager-Exec--0] io.druid.db.DatabaseSegmentManager - Polled and found 485 segments in the database
962 entries
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/b9b98d54-0fa8-4319-859e-48bd8421c25c%40googlegroups.com.--
You received this message because you are subscribed to a topic in the Google Groups "Druid Development" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/druid-development/RymYllsIo8k/unsubscribe.
To unsubscribe from this group and all its topics, send an email to druid-developm...@googlegroups.com.
To post to this group, send email to druid-de...@googlegroups.com.
2014-12-11 07:19:54,188 ERROR [ZkCoordinator-0] io.druid.server.coordination.ZkCoordinator - Failed to load segment for dataSource: {class=io.druid.server.coordination.ZkCoordinator
, exceptionType=class io.druid.segment.loading.SegmentLoadingException, exceptionMessage=Exception loading segment[batch_test3_2014-12-10T09:13:00.000Z_2014-12-10T09:14:00.000Z_2014
-12-11T07:17:55.478Z], segment=DataSegment{size=643, shardSpec=NoneShardSpec, metrics=[count], dimensions=[source_client, offerid, parentid, schemaversion, icmpid, eventid], version
='2014-12-11T07:17:55.478Z', loadSpec={type=hdfs, path=/grid/lambda/druid/batch_test3/batch_test3/20141210T091300.000Z_20141210T091400.000Z/2014-12-11T07_17_55.478Z/0/index.zip}, in
terval=2014-12-10T09:13:00.000Z/2014-12-10T09:14:00.000Z, dataSource='batch_test3', binaryVersion='9'}}
io.druid.segment.loading.SegmentLoadingException: Exception loading segment[batch_test3_2014-12-10T09:13:00.000Z_2014-12-10T09:14:00.000Z_2014-12-11T07:17:55.478Z]
at io.druid.server.coordination.ZkCoordinator.addSegment(ZkCoordinator.java:136)
at io.druid.server.coordination.SegmentChangeRequestLoad.go(SegmentChangeRequestLoad.java:44)
at io.druid.server.coordination.BaseZkCoordinator$1.childEvent(BaseZkCoordinator.java:113)
at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:509)
at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:503)
at org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92)
at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
at org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:83)
at org.apache.curator.framework.recipes.cache.PathChildrenCache.callListeners(PathChildrenCache.java:500)
at org.apache.curator.framework.recipes.cache.EventOperation.invoke(EventOperation.java:35)
{"dataSource":"batch_test3","interval":"2014-12-10T09:13:00.000Z/2014-12-10T09:14:00.000Z","version":"2014-12-11T07:17:55.478Z","loadSpec":{"type":"hdfs","path":"/grid/lambda/druid/batch_test3/batch_test3/20141210T091300.000Z_20141210T091400.000Z/2014-12-11T07_17_55.478Z/0/index.zip"},"dimensions":"parentid","metrics":"count","shardSpec":{"type":"none"},"binaryVersion":9,"size":643,"identifier":"batch_test3_2014-12-10T09:13:00.000Z_2014-12-10T09:14:00.000Z_2014-12-11T07:17:55.478Z"}
--
You received this message because you are subscribed to the Google Groups "Druid Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/CAOARWOBYmeAg%3DtnK0eA3zeGAw6gukUVdtbdtVOE-6iy1Va_kfw%40mail.gmail.com.
-rw-r--r-- |
druid |
hadoop |
729 B |
3 |
128 MB |
index.zip |
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/CACW6nteJ4hMxGNTkRoEm2eAR0QsX%2BVkL5KZP6rzDi%3D1bQ86n6w%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/CAOARWODYyO0gSYk6cHLHY0sM_5OA6X%3DkJCiWtSMqLUd8Z0PohQ%40mail.gmail.com.
2014-12-11 07:17:54,109 ERROR [ZkCoordinator-0] io.druid.server.coordination.ZkCoordinator - Failed to load segment for dataSource: {class=io.druid.server.coordination.ZkCoordinator, exceptionType=class io.druid.segment.loading.SegmentLoadingException, exceptionMessage=Exception loading segment[batch_test3_2014-12-10T09:13:00.000Z_2014-12-10T09:14:00.000Z_2014-12-11T07:17:00.380Z], segment=DataSegment{size=643, shardSpec=NoneShardSpec, metrics=[count], dimensions=[source_client, offerid, parentid, schemaversion, icmpid, eventid], version='2014-12-11T07:17:00.380Z', loadSpec={type=hdfs, path=/grid/lambda/druid/batch_test3/batch_test3/20141210T091300.000Z_20141210T091400.000Z/2014-12-11T07_17_00.380Z/0/index.zip}, interval=2014-12-10T09:13:00.000Z/2014-12-10T09:14:00.000Z, dataSource='batch_test3', binaryVersion='9'}}
io.druid.segment.loading.SegmentLoadingException: Exception loading segment[batch_test3_2014-12-10T09:13:00.000Z_2014-12-10T09:14:00.000Z_2014-12-11T07:17:00.380Z]
at io.druid.server.coordination.ZkCoordinator.addSegment(ZkCoordinator.java:136)
at io.druid.server.coordination.SegmentChangeRequestLoad.go(SegmentChangeRequestLoad.java:44)
at io.druid.server.coordination.BaseZkCoordinator$1.childEvent(BaseZkCoordinator.java:113)
at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:509)
at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:503)
at org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92)
at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
at org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:83)
at org.apache.curator.framework.recipes.cache.PathChildrenCache.callListeners(PathChildrenCache.java:500)
at org.apache.curator.framework.recipes.cache.EventOperation.invoke(EventOperation.java:35)
at org.apache.curator.framework.recipes.cache.PathChildrenCache$10.run(PathChildrenCache.java:762)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: io.druid.segment.loading.SegmentLoadingException: Path[/grid/lambda/druid/batch_test3/batch_test3/20141210T091300.000Z_20141210T091400.000Z/2014-12-11T07_17_00.380Z/0/index.zip] doesn't exist.
at io.druid.storage.hdfs.HdfsDataSegmentPuller.checkPathAndGetFilesystem(HdfsDataSegmentPuller.java:94)
at io.druid.storage.hdfs.HdfsDataSegmentPuller.getSegmentFiles(HdfsDataSegmentPuller.java:52)
at io.druid.segment.loading.OmniSegmentLoader.getSegmentFiles(OmniSegmentLoader.java:125)
at io.druid.segment.loading.OmniSegmentLoader.getSegment(OmniSegmentLoader.java:93)
at io.druid.server.coordination.ServerManager.loadSegment(ServerManager.java:145)
at io.druid.server.coordination.ZkCoordinator.addSegment(ZkCoordinator.java:132) ... 17 more
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/CACW6ntepHhOG2B5Orh_ORXFz%2B8VL6jJftEQ_Qkn58gd0LGVpBw%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/CAOARWOD8NWMPUhuDmhuWmP38uBNmeyxakO3hspF8TiDh8EQtKA%40mail.gmail.com.
...
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/2ddcc867-8d61-44fa-a06e-d9d0b34e8d91%40googlegroups.com.
To unsubscribe from this group and all its topics, send an email to druid-development+unsubscribe@googlegroups.com.
To post to this group, send email to druid-development@googlegroups.com.
2014-12-22 06:55:15,215 ERROR [ZkCoordinator-0] io.druid.server.coordination.ZkCoordinator - Failed to load segment for dataSource: {class=io.druid.server.coordination.ZkCoordinator, exceptionType=class io.druid.segment.loading.SegmentLoadingException, exceptionMessage=Exception loading segment[batch_test_2014-12-09T06:39:00.000Z_2014-12-09T06:40:00.000Z_2014-12-22T06:47:01.999Z], segment=DataSegment{size=643, shardSpec=NoneShardSpec, metrics=[count], dimensions=[source_client, offerid, parentid, schemaversion, icmpid, eventid], version='2014-12-22T06:47:01.999Z', loadSpec={type=hdfs, path=hdfs://b1nn1.ch.flipkart.com:8020/grid/lambda/druid/batch_test/batch_test/20141209T063900.000Z_20141209T064000.000Z/2014-12-22T06_47_01.999Z/0/index.zip}, interval=2014-12-09T06:39:00.000Z/2014-12-09T06:40:00.000Z, dataSource='batch_test', binaryVersion='9'}}
io.druid.segment.loading.SegmentLoadingException: Exception loading segment[batch_test_2014-12-09T06:39:00.000Z_2014-12-09T06:40:00.000Z_2014-12-22T06:47:01.999Z]
at io.druid.server.coordination.ZkCoordinator.loadSegment(ZkCoordinator.java:140)
at io.druid.server.coordination.ZkCoordinator.addSegment(ZkCoordinator.java:165)
at io.druid.server.coordination.SegmentChangeRequestLoad.go(SegmentChangeRequestLoad.java:44)
at io.druid.server.coordination.BaseZkCoordinator$1.childEvent(BaseZkCoordinator.java:127)
at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:509)
at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:503)
at org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92)
at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
at org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:83)
at org.apache.curator.framework.recipes.cache.PathChildrenCache.callListeners(PathChildrenCache.java:500)
at org.apache.curator.framework.recipes.cache.EventOperation.invoke(EventOperation.java:35)
at org.apache.curator.framework.recipes.cache.PathChildrenCache$10.run(PathChildrenCache.java:762)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: io.druid.segment.loading.SegmentLoadingException: Some IOException
at io.druid.storage.hdfs.HdfsDataSegmentPuller.getSegmentFiles(HdfsDataSegmentPuller.java:61)
at io.druid.segment.loading.OmniSegmentLoader.getSegmentFiles(OmniSegmentLoader.java:125)
at io.druid.segment.loading.OmniSegmentLoader.getSegment(OmniSegmentLoader.java:93)
at io.druid.server.coordination.ServerManager.loadSegment(ServerManager.java:145)
at io.druid.server.coordination.ZkCoordinator.loadSegment(ZkCoordinator.java:136)
... 18 more
Caused by: java.io.FileNotFoundException: /grid/2/druid/historical/persistent/zk_druid/batch_test/2014-12-09T06:39:00.000Z_2014-12-09T06:40:00.000Z/2014-12-22T06:47:01.999Z/0/00000.smoosh (No such file or directory)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
at io.druid.utils.CompressionUtils.unzip(CompressionUtils.java:94)
at io.druid.storage.hdfs.HdfsDataSegmentPuller.getSegmentFiles(HdfsDataSegmentPuller.java:57)
... 22 more
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/4ee9da80-a723-4576-a3e1-205aa72303dc%40googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/CAOARWODjRv0WZDWS0XW9t0QQ54rhKKNsXTUY-LC9oA%3Dt8X36EQ%40mail.gmail.com.
...