Hi,Please help me out, how o setup druid with HDFS. Instead of Storing data at /tmp/druid/localStorage, Store at HDFS path.
One more thing how can i change the path of local storage also like as instead of Store at /tmp/druid/localStorage Store as some other path may be /tmp/druid/localStorageNew etc.I have tried put this entry at Broker,Reltime and Historical config but it doesn't work.druid.pusher.local=truedruid.pusher.local.storageDirectory=/tmp/druid/localStorage1druid.storage.local.storageDirectory=[{"path": "/tmp/druid/indexCache1", "maxSize"\: 10000000000}]
I have tried different way for path writing as druid.pusher.local.storageDirectory=/tmp/druid/localStorage1 and druid.storage.local.storageDirectory=[{"path": "/tmp/druid/indexCache1", "maxSize"\: 10000000000}]. But it doesn't change. All data store at /tmp/druid/localStorage.
--
You received this message because you are subscribed to the Google Groups "Druid Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-developm...@googlegroups.com.
To post to this group, send email to druid-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/dee4ac79-0cde-48dd-8dc1-7b4150db0047%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
|
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/9c506ba9-af61-413e-b841-2370bec77ca4%40googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/8af5af12-3623-4f04-98ac-6596bc205f17%40googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/0fffa1f5-35d5-42dc-a661-f9a22fd338cf%40googlegroups.com.
...To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/druid-development/0fffa1f5-35d5-42dc-a661-f9a22fd338cf%40googlegroups.com?utm_medium=email&utm_source=footer" target="_blank" onmousedown="this.href='https://groups.google.com/d/msgid/druid-development/0fffa1f5-35d5-42dc-a661-f9a22fd338cf%40googlegroups.com?utm_medium\75email\46utm_source\75footer';return true;"
...
...
...
Nishant <table cellspacing="0" cellpadding="0" border="0" style="border-collapse:collapse;font-size:11px;fon
Software Engineer | METAMARKETS
/job_1406914384450_0001/job_1406914384450_0001_1.jhist to file:/tmp/hadoop-yarn/staging/history/done_intermediate/tarun/job_1406914384450_0001-1406914786322-tarun-wikipedia%2Ddetermine_partitions_groupby%2DOptional.of-1406914826342-0-0-FAILED-default-1406914798528.jhist_tmp 2014-08-01 23:10:26,539 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: file:/tmp/hadoop-yarn/staging/history/done_intermediate/tarun/job_1406914384450_0001-1406914786322-tarun-wikipedia%2Ddetermine_partitions_groupby%2DOptional.of-1406914826342-0-0-FAILED-default-1406914798528.jhist_tmp 2014-08-01 23:10:26,542 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying file:/tmp/hadoop-yarn/staging/tarun/.staging/job_1406914384450_0001/job_1406914384450_0001_1_conf.xml to file:/tmp/hadoop-yarn/staging/history/done_intermediate/tarun/job_1406914384450_0001_conf.xml_tmp 2014-08-01 23:10:26,558 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: file:/tmp/hadoop-yarn/staging/history/done_intermediate/tarun/job_1406914384450_0001_conf.xml_tmp 2014-08-01 23:10:26,561 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: file:/tmp/hadoop-yarn/staging/history/done_intermediate/tarun/job_1406914384450_0001.summary_tmp to file:/tmp/hadoop-yarn/staging/history/done_intermediate/tarun/job_1406914384450_0001.summary 2014-08-01 23:10:26,561 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: file:/tmp/hadoop-yarn/staging/history/done_intermediate/tarun/job_1406914384450_0001_conf.xml_tmp to file:/tmp/hadoop-yarn/staging/history/done_intermediate/tarun/job_1406914384450_0001_conf.xml 2014-08-01 23:10:26,562 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: file:/tmp/hadoop-yarn/staging/history/done_intermediate/tarun/job_1406914384450_0001-1406914786322-tarun-wikipedia%2Ddetermine_partitions_groupby%2DOptional.of-1406914826342-0-0-FAILED-default-1406914798528.jhist_tmp to file:/tmp/hadoop-yarn/staging/history/done_intermediate/tarun/job_1406914384450_0001-1406914786322-tarun-wikipedia%2Ddetermine_partitions_groupby%2DOptional.of-1406914826342-0-0-FAILED-default-1406914798528.jhist 2014-08-01 23:10:26,584 INFO [Thread-58] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped JobHistoryEventHandler. super.stop() 2014-08-01 23:10:26,588 INFO [Thread-58] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Setting job diagnostics to Task failed task_1406914384450_0001_m_000000 Job failed as tasks failed. failedMaps:1 failedReduces:0
...To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/druid-development/9c506ba9-af61-413e-b841-2370bec77ca4%40googlegroups.com?utm_medium=email&utm_source=footer" target="_blank" onmousedown="this.href='https://groups.google.com/d/msgid/druid-development/9c506ba9-af61-413e-b841-2370bec77ca4%40googlegroups.