Unknown repository URI: Hive

[Nathan Neff]

Hello all,

I've been able to run the kite-dataset CLI example on a Virtual
Machine running Hive client (no HiveServer2)

When running kite-dataset on a cluster with HiveServer2, I get "Unknown repository URI: hive"

I'm able to connect to HiveServer2 using both hive and beeline.

Below are my commands and settings [1].  The exact commands that I ran
are here:  https://github.com/NathanNeff/hadoop-examples/tree/master/kite-sdk/simple-cli

Any ideas?

Thanks,
--Nate

[1]

debug=true kite-dataset -v create sandwiches -s sandwich.avsc
(Kite version "0.15.0-cdh5.3.1")

/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/bin/../lib/kite/bin/kite-dataset debug: Using HADOOP_COMMON_HOME=/opt/cloudera/parcels/CDH/lib/hadoop
/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/bin/../lib/kite/bin/kite-dataset debug: Using HADOOP_MAPRED_HOME=/opt/cloudera/parcels/CDH/lib/hadoop/../hadoop-mapreduce
/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/bin/../lib/kite/bin/kite-dataset debug: Using HBASE_HOME=/opt/cloudera/parcels/CDH/lib/hadoop/../hbase
/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/bin/../lib/kite/bin/kite-dataset debug: Using HIVE_HOME=/usr/lib/hive
/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/bin/../lib/kite/bin/kite-dataset debug: Using HIVE_CONF_DIR=/etc/hive/conf
/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/bin/../lib/kite/bin/kite-dataset debug: Using HADOOP_CLASSPATH=/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/bin/../lib/kite/bin/kite-dataset:/etc/zookeeper:/etc/hive/conf <snipped many more classpath settings>

Argument error
java.lang.IllegalArgumentException: Unknown repository URI: hive
    at org.kitesdk.data.spi.Registration.lookupRepoUri(Registration.java:93)
    at org.kitesdk.data.spi.DatasetRepositories.repositoryFor(DatasetRepositories.java:46)
    at org.kitesdk.data.spi.DatasetRepositories.repositoryFor(DatasetRepositories.java:67)
    at org.kitesdk.cli.commands.BaseDatasetCommand.getDatasetRepository(BaseDatasetCommand.java:74)
    at org.kitesdk.cli.commands.CreateDatasetCommand.run(CreateDatasetCommand.java:104)
    at org.kitesdk.cli.Main.run(Main.java:161)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.kitesdk.cli.Main.main(Main.java:239)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

[Ryan Blue]

This happens when Kite can't load its Hive dependencies, thinks it can't
talk to Hive, and doesn't register any Hive URI patterns. What is in
/usr/lib/hive? If that directory exists, Kite assumes it contains Hive
libraries, but it clearly doesn't contain all of the ones Kite expects.
Do you know where Hive keeps them on a HiveServer2 install?

rb


--
Ryan Blue
Software Engineer
Cloudera, Inc.

[Nathan Neff]

Hi Ryan,

Thanks for your help.  I set HIVE_HOME similar to HBASE_HOME (above) and it worked.

(e.g.) HBASE_HOME=/opt/cloudera/parcels/CDH/lib/hadoop/../hbase

export HIVE_HOME=/opt/cloudera/parcels/CDH/lib/hadoop/../hive

debug=true kite-dataset create sandwich -s sandwiches.avsc

The machine I was running the kite-dataset command from is a Hive

"gateway", so I looked at the hive command's shell script, and

it is using the same HIVE_HOME as the one above (without symlinks

of course, but it resolves to the same dir as HIVE_HOME above)

Thanks again!