Druid and Hadoop 3.1 (Hadoop indexing).

[alexandr....@apm-consult.com]

Hello,

We are looking into Metatron application, and want to use HDFS as an ingestion source.

Our investigations show's us that this way is no working now.

Druid builded with 2.7.* HDP dependicies, and there is no way to launch correct application on cluster (YARN), because of HDP 3.1 used in our case.

(https://druid.apache.org/docs/latest/operations/other-hadoop.html - this recipes did not help.)

We build Druid 15 from github, and build it with HDP 3.1.1 dependicies, and it is a solution. Everything working as expected in hadoop_indexing area, but metatron.app is useless without customized Druid...

Is there any way to get Druid sources, to build it with HDP 3.1. dependicies? 

This is very important for us, due to existing cluster...

[Alchan Lee]

Thank you for your interest. 

Also, my apologies for the slow reply; it was national holidays in Korea.

Our druid source code is still managed internally, so I should make a proper binary and provide it.

Please download the druid from this link for Hadoop 3.1.1

FYI, we plan to open our customized druid source code soon.

And if you have any further question, please feel free to ask.

Thanks,

Alchan.

[alexandr....@apm-consult.com]

Hello Alchan Lee!

Many thanks!

[alexandr....@apm-consult.com]

Hello Alchan Lee!

Please, help.

Where can i find the latest druid build(druid-metatron-3.3.3.tar.gz) for Hadoop 3.1.1?

And can i ask you to build it with dependicies from Hortonworks repository:

https://mvnrepository.com/repos/hortonworks-releases

Hadoop.version - 3.1.1.3.1.4.0-315

Due to new Guava used there, we afraid about Hadoop indexing problems...

[alex.sid...@gmail.com]

Alchan Lee hello!

Please help, with 3.1.1 customized Druid build...