Ok so I managed to get elephantdb-server running against my local HDFS. I have a scalding job that I'm running against local HDFS, writing ElephantDB. From there I was able to start the elephantdb-server using:
00:05:38.788 [main] DEBUG org.apache.hadoop.conf.Configuration - java.io.IOException: config()
at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:211)
at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:198)
at hadoop_util.core$configuration.invoke(core.clj:53)
at hadoop_util.core$filesystem.invoke(core.clj:65)
at elephantdb.common.config$read_global_config.invoke(config.clj:76)
at elephantdb.keyval.core$_main.invoke(core.clj:246)
at clojure.lang.Var.invoke(Var.java:419)
at user$eval5.invoke(form-init7480547441345866642.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:6619)
at clojure.lang.Compiler.eval(Compiler.java:6609)
at clojure.lang.Compiler.load(Compiler.java:7064)
at clojure.lang.Compiler.loadFile(Compiler.java:7020)
at clojure.main$load_script.invoke(main.clj:294)
at clojure.main$init_opt.invoke(main.clj:299)
at clojure.main$initialize.invoke(main.clj:327)
at clojure.main$null_opt.invoke(main.clj:362)
at clojure.main$main.doInvoke(main.clj:440)
at clojure.lang.RestFn.invoke(RestFn.java:421)
at clojure.lang.Var.invoke(Var.java:419)
at clojure.lang.AFn.applyToHelper(AFn.java:163)
at clojure.lang.Var.applyTo(Var.java:532)
at clojure.main.main(main.java:37)
Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS: s3n://some/bucket/elephantdb/global-conf.clj,
expected: file:///
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:310)
at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:47)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:357)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:648)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93)
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28)
at elephantdb.common.config$read_clj_config.invoke(config.clj:40)
at elephantdb.common.config$read_global_config.invoke(config.clj:76)
at elephantdb.keyval.core$_main.invoke(core.clj:246)
at clojure.lang.Var.invoke(Var.java:419)
at user$eval5.invoke(form-init7480547441345866642.clj:1)
at clojure.lang.Compiler.eval(Compiler.java:6619)
at clojure.lang.Compiler.eval(Compiler.java:6609)
at clojure.lang.Compiler.load(Compiler.java:7064)
at clojure.lang.Compiler.loadFile(Compiler.java:7020)
at clojure.main$load_script.invoke(main.clj:294)
at clojure.main$init_opt.invoke(main.clj:299)
at clojure.main$initialize.invoke(main.clj:327)
at clojure.main$null_opt.invoke(main.clj:362)
at clojure.main$main.doInvoke(main.clj:440)
at clojure.lang.RestFn.invoke(RestFn.java:421)
at clojure.lang.Var.invoke(Var.java:419)
at clojure.lang.AFn.applyToHelper(AFn.java:163)
at clojure.lang.Var.applyTo(Var.java:532)
at clojure.main.main(main.java:37)
Clearly I have the understanding of a potato, and I have naively misconfigured this somehow. I suspect the issue is with the entry
but I have no idea what the correct entry for that would be... What is the correct way to configure and start a cluster of elephantdb-server processes that are pointing to S3 for their HDFS storage?