"GSS initiate failed" when making jdbc:hive2 connection to a Kerberized hadoop cluster
Marek K.
Hi,
When trying to run the Sample Feed as per the Kylo documentation, it is failing at the “Register Tables/RegusterFeedTables” processor step with the error:
Caused by: java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://ukx02738:10000/default;principal=hive/ukx02738.uk....@INTL.INTLROOT.FID-INTL.COM: GSS initiate failed
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:215)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:163)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
at org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataSource.java:1556)
at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1545)
... 24 common frames omitted
Caused by: org.apache.thrift.transport.TTransportException: GSS initiate failed
at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:221)
However, the same connection string works OK via beeline provided that I’ve got a valid ticket for the nifi user on my edge node (ukx10678) ; nifi being authorized via Sentry to have access to hive on our Kerberized Hadoop cluster i.e.,
[DEV] nifi@ukx10678 /app/nifi/home $ kinit -kt $PWD/$(id -un).keytab $(id -un)/$(hostname -f)
[DEV] nifi@ukx10678 /app/nifi/home $ klist
Ticket cache: FILE:/tmp/krb5cc_21380
Default principal: nifi/ukx10678.uk....@INTL.INTLROOT.FID-INTL.COM
Valid starting Expires Service principal
05/07/18 16:58:12 05/07/18 20:58:12 krbtgt/INTL.INTLROOT...@INTL.INTLROOT.FID-INTL.COM
[DEV] nifi@ukx10678 /app/nifi/home $
[DEV] nifi@ukx10678 /app/nifi/home $ beeline
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
Beeline version 1.1.0-cdh5.13.0 by Apache Hive
beeline> !connect jdbc:hive2://ukx02738:10000/default;principal=hive/ukx02738.uk....@INTL.INTLROOT.FID-INTL.COM
scan complete in 1ms
Connecting to jdbc:hive2://ukx02738:10000/default;principal=hive/ukx02738.uk....@INTL.INTLROOT.FID-INTL.COM
Connected to: Apache Hive (version 1.1.0-cdh5.13.0)
Driver: Hive JDBC (version 1.1.0-cdh5.13.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://ukx02738:10000/default> show databases;
+------------------+--+
| database_name |
+------------------+--+
| a585374_test |
| a604363_test |
| apdacd |
| dc |
| default |
| model_implement |
| poc |
| rocana |
| test |
| users |
+------------------+--+
10 rows selected (0.665 seconds)
This would therefore suggest that somehow my Nifi config is simply not requesting a Kerberos ticket correctly. Could you advise on what I may be missing in my configuration ( it there a jaas.conf file needed somewhere?); or what I should be checking.
That said, I can see that authentication must be working in other processors as I can see that nifi/kylo related directories have been created on my hadoop cluster.
[DEV] nifi@ukx10678 /app/nifi/home $ hdfs dfs -ls /app/nifi/model.db/users1/mak_20180703_003
Found 4 items
drwxr-xr-x - nifi nifi 0 2018-07-04 11:50 /app/nifi/model.db/users1/mak_20180703_003/feed
drwxr-xr-x - nifi nifi 0 2018-07-04 11:50 /app/nifi/model.db/users1/mak_20180703_003/invalid
drwxr-xr-x - nifi nifi 0 2018-07-04 11:50 /app/nifi/model.db/users1/mak_20180703_003/profile
drwxr-xr-x - nifi nifi 0 2018-07-04 11:50 /app/nifi/model.db/users1/mak_20180703_003/valid
Any guidance or advice would be most welcome.
Regards,
Marek
