Creating dataset backed by Hive changing database

[Buntu Dev]

Currently when I create the dataset using Kite CLI, its visible under the 'default' database. Is there a way to specify/change the default database the dataset is created under? If so, how should I update the Kite dataset sink configs 'kite.repo.uri' and 'dataset.name' in Flume config to specify the new '<database>/dataset' ?

Thanks!

[Joey Echeverria]

Yes, you can change the Hive database by setting a namespace in the
URI. Refer to the full Hive URI documentation[1] if you need to, but
the gist is you need a URI like the following:

dataset:hive:db/table

where db is the name of the database/namespace and table is the name
of the table.

For you Flume config, the easiest way is to switch to the dataset.uri
config parameter and pass in the same URI you use for create.

Either way, you have to update Flume to the version in CDH 5.2 or
later as support for namespaces wasn't added until 5.2.

-Joey

> --
> You received this message because you are subscribed to the Google Groups
> "CDK Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cdk-dev+u...@cloudera.org.
> For more options, visit https://groups.google.com/a/cloudera.org/d/optout.



--
Joey Echeverria
Senior Infrastructure Engineer

[Buntu Dev]

Thanks Joey, I was able to create the dataset this way 'dataset:hive:/user/hive/warehouse/db/<table>'.

I do see the dataset.uri in the FlumeConfigCommand but do not see the config parameter for the Kite dataset sink in the Flume doc:

  https://flume.apache.org/FlumeUserGuide.html#kite-dataset-sink-experimental

Thanks!

[Ryan Blue]

On 03/11/2015 10:32 AM, Buntu Dev wrote:
> Thanks Joey, I was able to create the dataset this way
> 'dataset:hive:/user/hive/warehouse/db/<table>'.
>
> I do see the dataset.uri in the FlumeConfigCommand

> <https://github.com/kite-sdk/kite/blob/master/kite-tools-parent/kite-tools/src/main/java/org/kitesdk/cli/commands/FlumeConfigCommand.java#L174> but

> do not see the config parameter for the Kite dataset sink in the Flume doc:
> https://flume.apache.org/FlumeUserGuide.html#kite-dataset-sink-experimental

The doc is based on the Flume 1.5.0 release, which had a much older
version of the sink. The new version is included in CDH and possibly
other distros, but hasn't made it in an upstream release yet.

rb


--
Ryan Blue
Software Engineer
Cloudera, Inc.

[Buntu Dev]

We are using CDH 5.3.0, so should be good for now.. thanks again!

[Joey Echeverria]

You can always check the CDH-version of the user guide for the latest
docs of CDH releases:

http://archive.cloudera.com/cdh5/cdh/5/flume-ng-1.5.0-cdh5.3.0/FlumeUserGuide.html#kite-dataset-sink

-Joey

[Buntu Dev]

Nice thanks!