Druid 0.8.3 - How to download druid dependencies using proxy

Jvalant Patel

unread,

Sep 21, 2016, 2:14:20 AM9/21/16

to Druid User

Hello,

I am using druid 0.8.3 with the core extensions ( kafka-eight and mysql-metadata-storage ). I have deployed druid on the machine which doesn't have direct connection to remote repositories ( upstream proxy). Sometimes, druid tries to install maven dependencies and fails to start.

From the different blog posts and druid docs, it seems that druid tar contains all the required jars that druid and its core extensions need. But, in this case it tries to download jars and fails.

My question is regarding the possibility of using proxy to download jars. Is there any possibility that applying proxy setting to start command would let druid download required jars?

I tried using java -Dhttp.proxyHost=<host> -Dhttp.proxyPort=<post> -classpath lib/*:config/_common:config/broker io.druid.cli.Main server broker command but druid is still not able to download the missing artifacts.

If this is not the case, then what are the workarounds ? use of pull-deps to pull the dependencies and re-bundle druid will solve the issue ?

Any suggestions on this would be the great help.

Thanks,
Jvalant

sample exception:

INFO: HV000001: Hibernate Validator 5.1.3.Final

2016-09-17T03:33:39,994 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, coordinates=[io.druid.extensions:druid-kafka-eight, io.druid.exten

sions:mysql-metadata-storage], defaultVersion='0.8.3', localRepository='extensions-repo', remoteRepositories=[]}]

2016-09-17T03:33:40,770 INFO [main] io.druid.initialization.Initialization - Loading extension[io.druid.extensions:druid-kafka-eight] for class[io.druid.cli.CliCommandCreator]

2016-09-17T03:33:58,644 ERROR [main] io.druid.initialization.Initialization - Unable to resolve artifacts for [io.druid.extensions:druid-kafka-eight:jar:0.8.3 (runtime) -> [] < []].

org.eclipse.aether.resolution.DependencyResolutionException: Failed to collect dependencies at io.druid.extensions:druid-kafka-eight:jar:0.8.3 -> org.apache.kafka:kafka_2.10:jar:0.8.2.1 -> org.apache.zookeeper:zookeeper:jar:3.4.6 -> jline:jline:jar:0.9.94 -> junit:junit

:jar:4.11 -> org.hamcrest:hamcrest-core:jar:1.3

at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:380) ~[aether-impl-0.9.0.M2.jar:?]

at io.tesla.aether.internal.DefaultTeslaAether.resolveArtifacts(DefaultTeslaAether.java:289) ~[tesla-aether-0.0.5.jar:0.0.5]

at io.druid.initialization.Initialization.getClassLoaderForCoordinates(Initialization.java:254) [druid-server-0.8.3.jar:0.8.3]

at io.druid.initialization.Initialization.getFromExtensions(Initialization.java:154) [druid-server-0.8.3.jar:0.8.3]

at io.druid.cli.Main.main(Main.java:84) [druid-services-0.8.3.jar:0.8.3]

Caused by: org.eclipse.aether.collection.DependencyCollectionException: Failed to collect dependencies at io.druid.extensions:druid-kafka-eight:jar:0.8.3 -> org.apache.kafka:kafka_2.10:jar:0.8.2.1 -> org.apache.zookeeper:zookeeper:jar:3.4.6 -> jline:jline:jar:0.9.94 ->

junit:junit:jar:4.11 -> org.hamcrest:hamcrest-core:jar:1.3

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:292) ~[aether-impl-0.9.0.M2.jar:?]

at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:342) ~[aether-impl-0.9.0.M2.jar:?]

... 4 more

Caused by: org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.hamcrest:hamcrest-core:jar:1.3

at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:335) ~[maven-aether-provider-3.1.1.jar:3.1.1]

at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:217) ~[maven-aether-provider-3.1.1.jar:3.1.1]

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:461) ~[aether-impl-0.9.0.M2.jar:?]

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:573) ~[aether-impl-0.9.0.M2.jar:?]

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:261) ~[aether-impl-0.9.0.M2.jar:?]

at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:342) ~[aether-impl-0.9.0.M2.jar:?]

... 4 more

Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact org.hamcrest:hamcrest-core:pom:1.3 from/to central (http://repo.maven.apache.org/maven2): /usr/local/druid/extensions-repo/org/hamcrest/hamcrest-core/1.3/aether-64c62218-8c

31-4b93-9fa5-da92c1dc099b-hamcrest-core-1.3.pom-in-progress (No such file or directory)

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:459) ~[aether-impl-0.9.0.M2.jar:?]

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:262) ~[aether-impl-0.9.0.M2.jar:?]

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:239) ~[aether-impl-0.9.0.M2.jar:?]

at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:320) ~[maven-aether-provider-3.1.1.jar:3.1.1]

at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:217) ~[maven-aether-provider-3.1.1.jar:3.1.1]

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:461) ~[aether-impl-0.9.0.M2.jar:?]

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:573) ~[aether-impl-0.9.0.M2.jar:?]

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:261) ~[aether-impl-0.9.0.M2.jar:?]

at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:342) ~[aether-impl-0.9.0.M2.jar:?]

... 4 more

Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact org.hamcrest:hamcrest-core:pom:1.3 from/to central (http://repo.maven.apache.org/maven2): /usr/local/druid/extensions-repo/org/hamcrest/hamcrest-core/1.3/aether-64c62218-8c31-4

b93-9fa5-da92c1dc099b-hamcrest-core-1.3.pom-in-progress (No such file or directory)

at io.tesla.aether.connector.AetherRepositoryConnector$2.wrap(AetherRepositoryConnector.java:830) ~[aether-connector-okhttp-0.0.9.jar:0.0.9]

at io.tesla.aether.connector.AetherRepositoryConnector$2.wrap(AetherRepositoryConnector.java:824) ~[aether-connector-okhttp-0.0.9.jar:0.0.9]

at io.tesla.aether.connector.AetherRepositoryConnector$GetTask.flush(AetherRepositoryConnector.java:619) ~[aether-connector-okhttp-0.0.9.jar:0.0.9]

at io.tesla.aether.connector.AetherRepositoryConnector.get(AetherRepositoryConnector.java:238) ~[aether-connector-okhttp-0.0.9.jar:0.0.9]

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:535) ~[aether-impl-0.9.0.M2.jar:?]

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:436) ~[aether-impl-0.9.0.M2.jar:?]

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:262) ~[aether-impl-0.9.0.M2.jar:?]

at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:239) ~[aether-impl-0.9.0.M2.jar:?]

at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:320) ~[maven-aether-provider-3.1.1.jar:3.1.1]

at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:217) ~[maven-aether-provider-3.1.1.jar:3.1.1]

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:461) ~[aether-impl-0.9.0.M2.jar:?]

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:573) ~[aether-impl-0.9.0.M2.jar:?]

at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:573)

Jvalant Patel

unread,

Sep 21, 2016, 6:01:34 PM9/21/16

to Druid User

Hi All,

Any pointers on this would be greatly appreciated. Addition to above question, the other question in my mind was about the druid tar. So, does druid 0.8.3 tar comes with all the necessary jars that it needs for druid and core extensions ?

Thanks,
Jvalant

Parag Jain

unread,

Sep 21, 2016, 11:11:07 PM9/21/16

to druid...@googlegroups.com

Druid bundles all the core extensions in the tar ball except from mysql, they are in the "extensions" directory. Read this - http://druid.io/docs/latest/operations/including-extensions.html for how to download and include extensions. Use pull-deps to download the extensions and place them in the extensions folder. I would not recommend downloading extensions at runtime.

On Wednesday, September 21, 2016 5:01 PM, Jvalant Patel <jvalan...@menlosecurity.com> wrote:

Hi All,

Any pointers on this would be greatly appreciated. Addition to above question, the other question in my mind was about the druid tar. So, does druid 0.8.3 tar comes with all the necessary jars that it needs for druid and core extensions ?

Thanks,
Jvalant

On Tuesday, September 20, 2016 at 11:14:20 PM UTC-7, Jvalant Patel wrote:

Hello,

I am using druid 0.8.3 with the core extensions ( kafka-eight and mysql-metadata-storage ). I have deployed druid on the machine which doesn't have direct connection to remote repositories ( upstream proxy). Sometimes, druid tries to install maven dependencies and fails to start.

From the different blog posts and druid docs, it seems that druid tar contains all the required jars that druid and its core extensions need. But, in this case it tries to download jars and fails.

My question is regarding the possibility of using proxy to download jars. Is there any possibility that applying proxy setting to start command would let druid download required jars?

I tried using java -Dhttp.proxyHost=<host> -Dhttp .proxyPort=<post> -classpath lib/*:config/_common:config/ broker io.druid.cli.Main server broker command but druid is still not able to download the missing artifacts.

If this is not the case, then what are the workarounds ? use of pull-deps to pull the dependencies and re-bundle druid will solve the issue ?

Any suggestions on this would be the great help.

Thanks,
Jvalant

sample exception:

INFO: HV000001: Hibernate Validator 5.1.3.Final

2016-09-17T03:33:39,994 INFO [main] io.druid.guice. JsonConfigurator - Loaded class[class io.druid.guice. ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{ searchCurrentClassloader=true, coordinates=[io.druid. extensions:druid-kafka-eight, io.druid.exten
sions:mysql-metadata-storage], defaultVersion='0.8.3', localRepository='extensions- repo', remoteRepositories=[]}]
2016-09-17T03:33:40,770 INFO [main] io.druid.initialization. Initialization - Loading extension[io.druid.extensions: druid-kafka-eight] for class[io.druid.cli. CliCommandCreator]
2016-09-17T03:33:58,644 ERROR [main] io.druid.initialization. Initialization - Unable to resolve artifacts for [io.druid.extensions:druid- kafka-eight:jar:0.8.3 (runtime) -> [] < []].
org.eclipse.aether.resolution. DependencyResolutionException: Failed to collect dependencies at io.druid.extensions:druid- kafka-eight:jar:0.8.3 -> org.apache.kafka:kafka_2.10: jar:0.8.2.1 -> org.apache.zookeeper: zookeeper:jar:3.4.6 -> jline:jline:jar:0.9.94 -> junit:junit

:jar:4.11 -> org.hamcrest:hamcrest-core: jar:1.3

at org.eclipse.aether.internal. impl.DefaultRepositorySystem. resolveDependencies( DefaultRepositorySystem.java: 380) ~[aether-impl-0.9.0.M2.jar:?]

at io.tesla.aether.internal. DefaultTeslaAether. resolveArtifacts( DefaultTeslaAether.java:289) ~[tesla-aether-0.0.5.jar:0.0. 5]
at io.druid.initialization. Initialization. getClassLoaderForCoordinates( Initialization.java:254) [druid-server-0.8.3.jar:0.8.3]
at io.druid.initialization. Initialization. getFromExtensions( Initialization.java:154) [druid-server-0.8.3.jar:0.8.3]
at io.druid.cli.Main.main(Main. java:84) [druid-services-0.8.3.jar:0.8. 3]

Caused by: org.eclipse.aether.collection. DependencyCollectionException: Failed to collect dependencies at io.druid.extensions:druid- kafka-eight:jar:0.8.3 -> org.apache.kafka:kafka_2.10: jar:0.8.2.1 -> org.apache.zookeeper: zookeeper:jar:3.4.6 -> jline:jline:jar:0.9.94 ->

junit:junit:jar:4.11 -> org.hamcrest:hamcrest-core: jar:1.3

at org.eclipse.aether.internal. impl. DefaultDependencyCollector. collectDependencies( DefaultDependencyCollector. java:292) ~[aether-impl-0.9.0.M2.jar:?]
at org.eclipse.aether.internal. impl.DefaultRepositorySystem. resolveDependencies( DefaultRepositorySystem.java: 342) ~[aether-impl-0.9.0.M2.jar:?]
... 4 more
Caused by: org.eclipse.aether.resolution. ArtifactDescriptorException: Failed to read artifact descriptor for org.hamcrest:hamcrest-core: jar:1.3

at org.apache.maven.repository. internal. DefaultArtifactDescriptorReade r.loadPom( DefaultArtifactDescriptorReade r.java:335) ~[maven-aether-provider-3.1.1. jar:3.1.1]
at org.apache.maven.repository. internal. DefaultArtifactDescriptorReade r.readArtifactDescriptor( DefaultArtifactDescriptorReade r.java:217) ~[maven-aether-provider-3.1.1. jar:3.1.1]

at org.eclipse.aether.internal. impl. DefaultDependencyCollector. process( DefaultDependencyCollector. java:461) ~[aether-impl-0.9.0.M2.jar:?]
at org.eclipse.aether.internal. impl. DefaultDependencyCollector. process( DefaultDependencyCollector. java:573) ~[aether-impl-0.9.0.M2.jar:?]
at org.eclipse.aether.internal. impl. DefaultDependencyCollector. process( DefaultDependencyCollector. java:573) ~[aether-impl-0.9.0.M2.jar:?]
at org.eclipse.aether.internal. impl. DefaultDependencyCollector. process( DefaultDependencyCollector. java:573) ~[aether-impl-0.9.0.M2.jar:?]
at org.eclipse.aether.internal. impl. DefaultDependencyCollector. process( DefaultDependencyCollector. java:573) ~[aether-impl-0.9.0.M2.jar:?]
at org.eclipse.aether.internal. impl. DefaultDependencyCollector. collectDependencies( DefaultDependencyCollector. java:261) ~[aether-impl-0.9.0.M2.jar:?]
at org.eclipse.aether.internal. impl.DefaultRepositorySystem. resolveDependencies( DefaultRepositorySystem.java: 342) ~[aether-impl-0.9.0.M2.jar:?]
... 4 more
Caused by: org.eclipse.aether.resolution. ArtifactResolutionException: Could not transfer artifact org.hamcrest:hamcrest-core: pom:1.3 from/to central (http://repo.maven.apache.org/ maven2): /usr/local/druid/extensions- repo/org/hamcrest/hamcrest- core/1.3/aether-64c62218-8c
31-4b93-9fa5-da92c1dc099b- hamcrest-core-1.3.pom-in- progress (No such file or directory)
at org.eclipse.aether.internal. impl.DefaultArtifactResolver. resolve( DefaultArtifactResolver.java: 459) ~[aether-impl-0.9.0.M2.jar:?]
at org.eclipse.aether.internal. impl.DefaultArtifactResolver. resolveArtifacts( DefaultArtifactResolver.java: 262) ~[aether-impl-0.9.0.M2.jar:?]
at org.eclipse.aether.internal. impl.DefaultArtifactResolver. resolveArtifact( DefaultArtifactResolver.java: 239) ~[aether-impl-0.9.0.M2.jar:?]

at org.apache.maven.repository. internal. DefaultArtifactDescriptorReade r.loadPom( DefaultArtifactDescriptorReade r.java:320) ~[maven-aether-provider-3.1.1. jar:3.1.1]
at org.apache.maven.repository. internal. DefaultArtifactDescriptorReade r.readArtifactDescriptor( DefaultArtifactDescriptorReade r.java:217) ~[maven-aether-provider-3.1.1. jar:3.1.1]

at org.eclipse.aether.internal. impl. DefaultDependencyCollector. process( DefaultDependencyCollector. java:461) ~[aether-impl-0.9.0.M2.jar:?]
at org.eclipse.aether.internal. impl. DefaultDependencyCollector. process( DefaultDependencyCollector. java:573) ~[aether-impl-0.9.0.M2.jar:?]
at org.eclipse.aether.internal. impl. DefaultDependencyCollector. process( DefaultDependencyCollector. java:573) ~[aether-impl-0.9.0.M2.jar:?]
at org.eclipse.aether.internal. impl. DefaultDependencyCollector. process( DefaultDependencyCollector. java:573) ~[aether-impl-0.9.0.M2.jar:?]
at org.eclipse.aether.internal. impl. DefaultDependencyCollector. process( DefaultDependencyCollector. java:573) ~[aether-impl-0.9.0.M2.jar:?]
at org.eclipse.aether.internal. impl. DefaultDependencyCollector. collectDependencies( DefaultDependencyCollector. java:261) ~[aether-impl-0.9.0.M2.jar:?]
at org.eclipse.aether.internal. impl.DefaultRepositorySystem. resolveDependencies( DefaultRepositorySystem.java: 342) ~[aether-impl-0.9.0.M2.jar:?]
... 4 more
Caused by: org.eclipse.aether.transfer. ArtifactTransferException: Could not transfer artifact org.hamcrest:hamcrest-core: pom:1.3 from/to central (http://repo.maven.apache.org/ maven2): /usr/local/druid/extensions- repo/org/hamcrest/hamcrest- core/1.3/aether-64c62218-8c31- 4
b93-9fa5-da92c1dc099b- hamcrest-core-1.3.pom-in- progress (No such file or directory)
at io.tesla.aether.connector. AetherRepositoryConnector$2. wrap( AetherRepositoryConnector. java:830) ~[aether-connector-okhttp-0.0. 9.jar:0.0.9]
at io.tesla.aether.connector. AetherRepositoryConnector$2. wrap( AetherRepositoryConnector. java:824) ~[aether-connector-okhttp-0.0. 9.jar:0.0.9]
at io.tesla.aether.connector. AetherRepositoryConnector$ GetTask.flush( AetherRepositoryConnector. java:619) ~[aether-connector-okhttp-0.0. 9.jar:0.0.9]
at io.tesla.aether.connector. AetherRepositoryConnector.get( AetherRepositoryConnector. java:238) ~[aether-connector-okhttp-0.0. 9.jar:0.0.9]
at org.eclipse.aether.internal. impl.DefaultArtifactResolver. performDownloads( DefaultArtifactResolver.java: 535) ~[aether-impl-0.9.0.M2.jar:?]
at org.eclipse.aether.internal. impl.DefaultArtifactResolver. resolve( DefaultArtifactResolver.java: 436) ~[aether-impl-0.9.0.M2.jar:?]
at org.eclipse.aether.internal. impl.DefaultArtifactResolver. resolveArtifacts( DefaultArtifactResolver.java: 262) ~[aether-impl-0.9.0.M2.jar:?]
at org.eclipse.aether.internal. impl.DefaultArtifactResolver. resolveArtifact( DefaultArtifactResolver.java: 239) ~[aether-impl-0.9.0.M2.jar:?]

at org.apache.maven.repository. internal. DefaultArtifactDescriptorReade r.loadPom( DefaultArtifactDescriptorReade r.java:320) ~[maven-aether-provider-3.1.1. jar:3.1.1]
at org.apache.maven.repository. internal. DefaultArtifactDescriptorReade r.readArtifactDescriptor( DefaultArtifactDescriptorReade r.java:217) ~[maven-aether-provider-3.1.1. jar:3.1.1]

at org.eclipse.aether.internal. impl. DefaultDependencyCollector. process( DefaultDependencyCollector. java:461) ~[aether-impl-0.9.0.M2.jar:?]
at org.eclipse.aether.internal. impl. DefaultDependencyCollector. process( DefaultDependencyCollector. java:573) ~[aether-impl-0.9.0.M2.jar:?]

at org.eclipse.aether.internal. impl. DefaultDependencyCollector. process( DefaultDependencyCollector. java:573)

--
You received this message because you are subscribed to the Google Groups "Druid User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+...@googlegroups.com.
To post to this group, send email to druid...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/e295c3d6-cdc2-4596-ace6-b10caa85b1a7%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Parag Jain

unread,

Sep 21, 2016, 11:16:45 PM9/21/16

to druid...@googlegroups.com

Just realized that you are using Druid 0.8.3, please ignore what I said earlier that applies only to 0.9 versions. For 0.8.3 you can follow this - http://druid.io/docs/0.8.3/operations/including-extensions.html

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/1998738333.2729684.1474513851397%40mail.yahoo.com.

Jvalant Patel

unread,

Sep 21, 2016, 11:27:05 PM9/21/16

to Druid User, pja...@yahoo-inc.com

Hi Parag,

Really appreciate you recommendation. So, if I use pull-deps tool, does it downloads all the required jar? I mean is it guaranteed that it won't fail to start if my machine won't have internet connectivity?

Can I start my druid processes providing proxy settings to download jars ?

Hoping for your response.

Thanks,
Jvalant

Gian Merlino

unread,

Sep 21, 2016, 11:50:29 PM9/21/16

to druid...@googlegroups.com

Hey Jvalant,

This is all much simpler in 0.9.x, so I recommend upgrading to that if possible. But if you are stuck on 0.8.x, you can download extension dependencies with a command like:

java \

-cp "lib/*" \

-Ddruid.extensions.coordinates='["io.druid.extensions:druid-hdfs-storage","io.druid.extensions:postgresql-metadata-storage"]' \

-Ddruid.extensions.localRepository="extensions-repo" \

io.druid.cli.Main tools pull-deps

And then set druid.extensions.localRepository to your extensions-repo in common.runtime.properties.

Gian

To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+unsubscribe@googlegroups.com.

To post to this group, send email to druid...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/f5832981-9a7f-4797-9484-00cf4b6ea0e6%40googlegroups.com.

Jvalant Patel

unread,

Sep 22, 2016, 2:36:44 AM9/22/16

to Druid User

Hi Gian,

Thanks for the suggestion. Is it possible that I provide proxy settings while starting druid processes?

Ex.

java -Dhttp.proxyHost=<host> -Dhttp.proxyPort=<post> -classpath lib/*:config/_common:config/broker io.druid.cli.Main server broker

Please let me know if there is any possibility to pass proxy settings to druid processes to download required dependencies at run time.

Thanks,
Jvalant

On Wednesday, September 21, 2016 at 8:50:29 PM UTC-7, Gian Merlino wrote:

Hey Jvalant,

This is all much simpler in 0.9.x, so I recommend upgrading to that if possible. But if you are stuck on 0.8.x, you can download extension dependencies with a command like:

java \
-cp "lib/*" \
-Ddruid.extensions.coordinates='["io.druid.extensions:druid-hdfs-storage","io.druid.extensions:postgresql-metadata-storage"]' \
-Ddruid.extensions.localRepository="extensions-repo" \
io.druid.cli.Main tools pull-deps

And then set druid.extensions.localRepository to your extensions-repo in common.runtime.properties.

Gian

Gian Merlino

unread,

Sep 22, 2016, 12:48:26 PM9/22/16

to druid...@googlegroups.com

I'm not sure if that'll work; I haven't ever tried using a proxy to download dependencies with 0.8.x. Usually I stuck with running pull-deps in advance.

Gian

To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+unsubscribe@googlegroups.com.

To post to this group, send email to druid...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/1352da9b-4456-4c2d-9802-bf7f8195fb0b%40googlegroups.com.

Reply all

Reply to author

Forward