The loaded data can't be queried.

tao...@yahoo-inc.com

unread,

Apr 16, 2014, 1:07:59 PM4/16/14

to druid-de...@googlegroups.com

Hi, All:

I follow the this doc "http://druid.io/docs/0.6.73/Tutorial:-Loading-Your-Data-Part-1.html" to load the data.

But I can't query these data.

Here is my json query file:

{

"queryType": "groupBy",

"dataSource": "wikipedia",

"granularity": "all",

"dimensions": [ "page" ],

"orderBy": {

"type": "default",

"columns": [ { "dimension": "edit_count", "direction": "DESCENDING" } ],

"limit": 10

},

"aggregations": [

{"type": "longSum", "fieldName": "count", "name": "edit_count"}

],

"filter": { "type": "selector", "dimension": "country", "value": "United States" },

"intervals": ["2013-08-31T00:00/2020-01-01T00"]

}

When I send it to broker, it return the null result. And the Historical had exception:

2014-04-16 17:01:48,644 ERROR [ZkCoordinator-0] io.druid.server.coordination.ZkCoordinator - Failed to load segment for dataSource: {class=io.druid.server.coordination.ZkCoordinator, exceptionType=class io.druid.segment.loading.SegmentLoadingException, exceptionMessage=Exception loading segment[wikipedia_2013-08-31T00:00:00.000Z_2013-09-01T00:00:00.000Z_2014-04-16T17:00:40.325Z], segment=DataSegment{size=4462, shardSpec=NoneShardSpec, metrics=[count, added, deleted, delta], dimensions=[anonymous, city, continent, country, language, namespace, newpage, page, region, robot, unpatrolled, user], version='2014-04-16T17:00:40.325Z', loadSpec={type=local, path=/tmp/druid/localStorage/wikipedia/2013-08-31T00:00:00.000Z_2013-09-01T00:00:00.000Z/2014-04-16T17:00:40.325Z/0/index.zip}, interval=2013-08-31T00:00:00.000Z/2013-09-01T00:00:00.000Z, dataSource='wikipedia', binaryVersion='9'}}

io.druid.segment.loading.SegmentLoadingException: Exception loading segment[wikipedia_2013-08-31T00:00:00.000Z_2013-09-01T00:00:00.000Z_2014-04-16T17:00:40.325Z]

at io.druid.server.coordination.ZkCoordinator.addSegment(ZkCoordinator.java:239)

at io.druid.server.coordination.SegmentChangeRequestLoad.go(SegmentChangeRequestLoad.java:44)

at io.druid.server.coordination.ZkCoordinator$1.childEvent(ZkCoordinator.java:131)

at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:494)

at org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:488)

at org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92)

at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:293)

at org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:83)

at org.apache.curator.framework.recipes.cache.PathChildrenCache.callListeners(PathChildrenCache.java:485)

at org.apache.curator.framework.recipes.cache.EventOperation.invoke(EventOperation.java:35)

at org.apache.curator.framework.recipes.cache.PathChildrenCache$11.run(PathChildrenCache.java:755)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

at java.util.concurrent.FutureTask.run(FutureTask.java:262)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

at java.util.concurrent.FutureTask.run(FutureTask.java:262)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:744)

Caused by: io.druid.segment.loading.SegmentLoadingException: Asked to load path[/tmp/druid/localStorage/wikipedia/2013-08-31T00:00:00.000Z_2013-09-01T00:00:00.000Z/2014-04-16T17:00:40.325Z/0/index.zip], but it doesn't exist.

at io.druid.segment.loading.LocalDataSegmentPuller.getFile(LocalDataSegmentPuller.java:100)

at io.druid.segment.loading.LocalDataSegmentPuller.getSegmentFiles(LocalDataSegmentPuller.java:41)

at io.druid.segment.loading.OmniSegmentLoader.getSegmentFiles(OmniSegmentLoader.java:125)

at io.druid.segment.loading.OmniSegmentLoader.getSegment(OmniSegmentLoader.java:93)

at io.druid.server.coordination.ServerManager.loadSegment(ServerManager.java:129)

at io.druid.server.coordination.ZkCoordinator.addSegment(ZkCoordinator.java:235)

... 17 more

Can somebody help me to have a look at it?

Thanks.

Fangjin Yang

unread,

Apr 16, 2014, 11:47:56 PM4/16/14

to druid-de...@googlegroups.com

Hi Tao, the exception ": Asked to load path[/tmp/druid/localStorage/wikipedia/2013-08-31T00:00:00.000Z_2013-09-01T00:00:00.000Z/2014-04-16T17:00:40.325Z/0/index.zip], but it doesn't exist." means that the files you indexed no longer exists in your system. Did you happen to restart the computer or clear out the tmp drive? For local testing, you may want to these files in a more permanent folder.

- FJ

tao...@yahoo-inc.com

unread,

Apr 17, 2014, 12:17:49 AM4/17/14

to druid-de...@googlegroups.com

Hi, Fangjin:

I stop all the nodes, then drop the tables in MySQL, and clean the dir "consumers" and "druid" in zookeeper, at last remove the dir "rm -rf /tmp/*" in every node.

Finish these steps, I restart all the nodes, and start to load the data through indexing, but still have this exception on historical node....

Are there some info I ignore to clean up?

2014-04-17 04:16:59,087 ERROR [ZkCoordinator-0] io.druid.server.coordination.ZkCoordinator - Failed to load segment for dataSource: {class=io.druid.server.coordination.ZkCoordinator, exceptionType=class io.druid.segment.loading.SegmentLoadingException, exceptionMessage=Exception loading segment[wikipedia_2013-08-31T00:00:00.000Z_2013-09-01T00:00:00.000Z_2014-04-17T04:10:53.010Z], segment=DataSegment{size=4462, shardSpec=NoneShardSpec, metrics=[count, added, deleted, delta], dimensions=[anonymous, city, continent, country, language, namespace, newpage, page, region, robot, unpatrolled, user], version='2014-04-17T04:10:53.010Z', loadSpec={type=local, path=/tmp/druid/localStorage/wikipedia/2013-08-31T00:00:00.000Z_2013-09-01T00:00:00.000Z/2014-04-17T04:10:53.010Z/0/index.zip}, interval=2013-08-31T00:00:00.000Z/2013-09-01T00:00:00.000Z, dataSource='wikipedia', binaryVersion='9'}}

io.druid.segment.loading.SegmentLoadingException: Exception loading segment[wikipedia_2013-08-31T00:00:00.000Z_2013-09-01T00:00:00.000Z_2014-04-17T04:10:53.010Z]