Loading data tutorial fails with Google cloud storage saying "No FileSystem for scheme: gs"

1,486 views
Skip to first unread message

abhishek....@gslab.com

unread,
May 4, 2018, 2:40:03 AM5/4/18
to Druid User
I have setup a druid cluster with google cloud storage as deep storage. 
I followed http://druid.io/docs/latest/tutorials/cluster.html documentation for this.

Used Load batch data from following page to load the data without making any changes: 

This task fails with following exceptions:
java.lang.Exception: java.io.IOException: No FileSystem for scheme: gs
	at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462) ~[hadoop-mapreduce-client-common-2.7.3.jar:?]
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:529) [hadoop-mapreduce-client-common-2.7.3.jar:?]
Caused by: java.io.IOException: No FileSystem for scheme: gs
	at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2660) ~[hadoop-common-2.7.3.jar:?]


I am using following config for google:
druid.storage.type=google
druid.google.bucket= <bucketname>
druid.google.prefix=druid/segments

druid.indexer.logs.type=google
druid.indexer.logs.bucket=<bucketname>
druid.indexer.logs.prefix=druid/indexing-logs


I am getting indexer-logs in the bucket but nothing in the segments.

Erik Dubbelboer

unread,
May 5, 2018, 5:17:04 AM5/5/18
to Druid User
Did you load the google extension?:

druid.extensions.loadList=["druid-google-extensions"]
Reply all
Reply to author
Forward
0 new messages