How to set custom JAVA_HOME for druid hadoop job on yarn?

[zhao weinan]

Hi, 

Recently i'm moving druid hadoop batch jobs to new yarn cluster which running on java6 which we can't upgrade, so i need customize the java_home for yarn tasks.

After googling a while, nothing found... Any suggestions?

Thanks!

[Gian Merlino]

Looking through the hadoop docs, maybe mapreduce.map.env and mapreduce.reduce.env would make that possible? I'd try setting them both to "JAVA_HOME=/path/to/java7" and seeing what happens.

[Zhao Weinan]

HI Gian,

Thanks a lot, I've tried it out, mapreduce.map.env and mapreduce.reduce.env dont work, but mapred.child.env and yarn.app.mapreduce.am.env work.

So if anyone need to run Druid on Hadoop Cluster over java6, you can ask ops to install a jdk7 on every machine, then adding two properties to your mapred-site.xml as Gian says: mapred.child.env and yarn.app.mapreduce.am.env, then setting them both to "JAVA_HOME=/path/to/java7"

--
You received this message because you are subscribed to a topic in the Google Groups "Druid Development" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/druid-development/ldXVYxJ5Hlo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to druid-developm...@googlegroups.com.
To post to this group, send email to druid-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-development/1c49787b-55dd-49ea-9e08-3de910403262%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Gian Merlino]

Cool, that's good to know. Thanks for reporting back!

[Charles Allen]

There are lots of java7 features used in Druid. maven-compiler-plugin is set to use 1.7/1.7 as source and target. I would be surprised if Druid components work correctly in 1.6

[Eric Tschetter]

Just as a clarification, Zhao has offered a method of updating the
version of java used for Druid indexing tasks on a Hadoop cluster that
is still primarily operating off of Java6.

As long as Java 7 is also installed, you can set the property he
suggested and it will run your tasks with Java 7 instead of Java 6,
thus allowing the Druid indexing code to work *and* allowing Ops to
still have all of their default stuff operating with Java 6.

--Eric

> You received this message because you are subscribed to the Google Groups
> "Druid Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to druid-developm...@googlegroups.com.
> To post to this group, send email to druid-de...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/druid-development/c8e4b008-9d86-4cea-b2b6-71b50e3d28bc%40googlegroups.com.

[Charles Allen]

that makes more sense. Thanks for the clarification