Cascading flow with Hive Partition Tap getting failed with Kerberos Issue
175 views
Skip to first unread message
Ganesh Auti
Sep 30, 2015, 2:08:01 PM9/30/15
to cascading-user
Hi,
I am getting an exception while running Cascading flow with Hive Partition Tap. I am able to run Cascading flow with Hive Tap without any trouble but facing issues while running flow with Hive Partition Tap. Does anyone have any idea about the root cause of this issue.
I am using Cascading version 3.0.1
Flow execution Stack-trace:
2015-09-30 11:18:50,313 INFO [main] cascading.tap.hadoop.util.Hadoop18TapUtil: setting up task: 'attempt_1443047889969_9065_m_000001_1' - hdfs://nameservice1/user/hive/warehouse/test_db.db/test_patition_table/_temporary/_attempt_1443047889969_9065_m_000001_1
2015-09-30 11:18:50,313 INFO [main] cascading.tap.hadoop.io.TapOutputCollector: creating path: partition_field1=20//part-00001-00008
2015-09-30 11:18:50,548 INFO [main] hive.metastore: Trying to connect to metastore with URI thrift://<metastore_URI>:9083
2015-09-30 11:18:50,588 ERROR [main] org.apache.thrift.transport.TSaslTransport: SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
at org.apache.thrift.transport.TSaslClientTransport.handleSaslStartMessage(TSaslClientTransport.java:94)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:271)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:409)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:230)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1483)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:64)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:74)
at cascading.tap.hive.MetaStoreClientFactory.newInstance(MetaStoreClientFactory.java:57)
at cascading.tap.hive.HiveTap.resourceExists(HiveTap.java:202)
at cascading.tap.hive.HiveTap.registerPartition(HiveTap.java:346)
at cascading.tap.hive.HivePartitionTap$HivePartitionCollector.closeCollector(HivePartitionTap.java:152)
at cascading.tap.partition.BasePartitionTap$PartitionCollector.close(BasePartitionTap.java:186)
at cascading.flow.stream.element.SinkStage.cleanup(SinkStage.java:128)
at cascading.flow.stream.graph.StreamGraph.cleanup(StreamGraph.java:190)
at cascading.flow.hadoop.FlowMapper.run(FlowMapper.java:167)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192)
... 33 more
2015-09-30 11:18:50,592 WARN [main] hive.metastore: Failed to connect to the MetaStore Server...
2015-09-30 11:18:50,592 INFO [main] hive.metastore: Waiting 1 seconds before next connection attempt.
2015-09-30 11:18:51,592 INFO [main] hive.metastore: Trying to connect to metastore with URI thrift://<metastore_uri>:9083
2015-09-30 11:18:51,595 ERROR [main] org.apache.thrift.transport.TSaslTransport: SASL negotiation failure
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
Thanks!
Andre Kelpe
Oct 1, 2015, 4:26:19 AM10/1/15
to cascading-user
Do I understand that right, that your metastore server uses kerberos authentication? I have not seen that kind if set-up before and we simply might not support it right now. We are creating the metastore client here: https://github.com/Cascading/cascading-hive/blob/2.0/src/main/java/cascading/tap/hive/MetaStoreClientFactory.java
Let me know, if you have any idea, what we are doing wrong.--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascading-use...@googlegroups.com.
To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at http://groups.google.com/group/cascading-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/7d1864a1-c870-48ea-8c5a-e568ff57099c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Message has been deleted
Ganesh Auti
Oct 5, 2015, 6:14:15 PM10/5/15
to cascading-user
Hi André,
Yes, metastore server is using Kerberos authentication.
Reply all
Reply to author
Forward
0 new messages