Cascading hive metastore with Kerberos authentication

586 views

Skip to first unread message

Kunal Ghosh

unread,

Mar 28, 2018, 11:46:04 AM3/28/18

to cascading-user

Hi,

I have created a sample program that read data from hdfs text file and sink it to hive partition tap(Kerberos authentication). But I am getting following exception.

Code :

Configuration conf = new Configuration();

conf.addResource(new Path(configurationFilePath));

conf.set("hive.metastore.uris", "thrift://hivehost:9083");

conf.set("hadoop.security.authentication", "kerberos");

conf.set("debug", "true");

UserGroupInformation.setConfiguration(conf);

try {

UserGroupInformation.loginUserFromKeytab("us...@EIO.COM", "/home/user1/user1.keytab");

} catch (IOException e1) {

// TODO Auto-generated catch block

e1.printStackTrace();

}

System.setProperty("hive.metastore.sasl.enabled", "true");

System.setProperty("hive.security.authorization.enabled", "false");

System.setProperty("hive.metastore.kerberos.principal", "hive/hive...@EIO.COM");

System.setProperty("hive.metastore.execute.setugi", "true");

HiveTableDescriptor partitionedDescriptor = new HiveTableDescriptor(databaseName, tableName, columnNames,columnTypes,new String[]{"groupkey"},"\t" ,HiveTableDescriptor.HIVE_DEFAULT_SERIALIZATION_LIB_NAME,new Path("/hive_data/"+databaseName+".db/"+tableName+"/"));

System.out.println(">>>> PartitionedDescriptor >>> "+partitionedDescriptor.toScheme());

HiveTap hiveTap = new HiveTap(partitionedDescriptor , partitionedDescriptor.toScheme(),SinkMode.REPLACE,false);

try{

hiveTap.commitResource(conf);

}catch(Exception e){

e.printStackTrace();

}

Tap partitionTap = new HivePartitionTap( hiveTap,SinkMode.UPDATE );

Error ::

018-03-28 11:32:20,581 INFO [main] org.apache.hadoop.security.SecurityUtil: Updating Configuration

2018-03-28 11:32:20,624 WARN [main] org.apache.hadoop.metrics2.impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-maptask.properties,hadoop-metrics2.properties

2018-03-28 11:32:20,685 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).

2018-03-28 11:32:20,686 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started

2018-03-28 11:32:20,700 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens:

2018-03-28 11:32:20,700 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1522156796285_0036, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@376a0d86)

2018-03-28 11:32:20,863 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: TIMELINE_DELEGATION_TOKEN, Service: 192.168.1.69:8188, Ident: (owner=user1, renewer=yarn, realUser=, issueDate=1522251126835, maxDate=1522855926835, sequenceNumber=41, masterKeyId=70)

2018-03-28 11:32:20,868 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: HDFS_DELEGATION_TOKEN, Service: 192.168.1.113:8020, Ident: (HDFS_DELEGATION_TOKEN token 40 for user1)

2018-03-28 11:32:20,985 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.

2018-03-28 11:32:21,169 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /data/hadoop/yarn/local/usercache/user1/appcache/application_1522156796285_0036

2018-03-28 11:32:21,434 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id

2018-03-28 11:32:21,843 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1

2018-03-28 11:32:21,843 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false

2018-03-28 11:32:21,851 INFO [main] org.apache.hadoop.mapred.Task: Using ResourceCalculatorProcessTree : [ ]

2018-03-28 11:32:22,039 INFO [main] cascading.tap.hadoop.io.MultiInputSplit: current split input path: hdfs://eiotwo:8020/hdfsdata/input/Reg_Employee1.txt

2018-03-28 11:32:22,040 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: cascading.tap.hadoop.io.MultiInputSplit@41de5768

2018-03-28 11:32:22,061 INFO [main] org.apache.hadoop.mapred.MapTask: numReduceTasks: 0

2018-03-28 11:32:22,068 INFO [main] cascading.flow.hadoop.FlowMapper: cascading version: 4.0.0-wip-38

2018-03-28 11:32:22,068 INFO [main] cascading.flow.hadoop.FlowMapper: child jvm opts: -Xms1024M -Xmx2048M

2018-03-28 11:32:22,249 INFO [main] cascading.flow.hadoop.FlowMapper: flow node id: FAB896D7DE674C9CAE058A570E3A1A41, ordinal: 0

2018-03-28 11:32:22,295 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition

2018-03-28 11:32:22,315 INFO [main] cascading.flow.hadoop.FlowMapper: sourcing from: Hfs["TextDelimited[['empno', 'empcode', 'empfirstname', 'emplastname', 'job', 'hiredate', 'empaddr1', 'empaddr2', 'phone', 'salary', 'comm', 'deptno' | Integer, String, String, String, String, DateType{dateFormatString='dd-MM-yy',canonicalType='long'}, String, String, String, Integer, Double, Integer]]"]["hdfs://eiotwo:8020/hdfsdata/input/Reg_Employee1.txt"]

2018-03-28 11:32:22,316 INFO [main] cascading.flow.hadoop.FlowMapper: sinking to: HivePartitionTap["HiveTap["TextDelimited[['s_empno', 's_empcode', 's_empfirstname', 's_emplastname', 's_job', 's_hiredate', 's_empaddr1', 's_empaddr2', 's_phone', 's_salary', 's_comm', 's_deptno', 'expr_0', 'expr_1', 'errrow', 'rowid' | String, String, String, String, String, String, String, String, String, String, String, String, String, String, String, Integer]]"]["hdfs://eiotwo:8020/hive_data/electric_io.db/R_92145_4081"]"]["HivePartition{delimiter='/', partitionFields=''groupkey' | String'}"]["[]"]

2018-03-28 11:32:22,316 INFO [main] cascading.flow.hadoop.FlowMapper: flow node id: FAB896D7DE674C9CAE058A570E3A1A41, mem on start (mb), free: 393, total: 424, max: 8085

2018-03-28 11:32:22,475 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir

2018-03-28 11:32:22,475 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir

2018-03-28 11:32:22,475 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress

2018-03-28 11:32:22,475 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class

2018-03-28 11:32:22,476 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class

2018-03-28 11:32:22,477 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id

2018-03-28 11:32:22,485 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.work.output.dir is deprecated. Instead, use mapreduce.task.output.dir

2018-03-28 11:32:22,486 INFO [main] cascading.tap.hadoop.util.Hadoop18TapUtil: setting up task: 'attempt_1522156796285_0036_m_000000_0' - hdfs://eiotwo:8020/electric_io/expr_92145_4081/_temporary/_attempt_1522156796285_0036_m_000000_0

2018-03-28 11:32:22,486 INFO [main] cascading.tap.hadoop.io.TapOutputCollector: creating path: part-00000

2018-03-28 11:32:22,659 INFO [main] cascading.tap.hadoop.util.Hadoop18TapUtil: setting up task: 'attempt_1522156796285_0036_m_000000_0' - hdfs://eiotwo:8020/electric_io/expr_logs_92145_4081/_temporary/_attempt_1522156796285_0036_m_000000_0

2018-03-28 11:32:22,659 INFO [main] cascading.tap.hadoop.io.TapOutputCollector: creating path: part-00000

2018-03-28 11:32:22,779 INFO [main] cascading.tap.hadoop.util.Hadoop18TapUtil: setting up task: 'attempt_1522156796285_0036_m_000000_0' - hdfs://eiotwo:8020/hive_data/electric_io.db/R_92145_4081/_temporary/_attempt_1522156796285_0036_m_000000_0

2018-03-28 11:32:22,779 INFO [main] cascading.tap.hadoop.io.TapOutputCollector: creating path: groupkey=A1//part-00000-00000

2018-03-28 11:32:22,794 INFO [main] cascading.tap.hadoop.io.TapOutputCollector: closing tap collector for: hdfs://eiotwo:8020/electric_io/expr_92145_4081/part-00000

2018-03-28 11:32:22,930 INFO [main] cascading.tap.hadoop.util.Hadoop18TapUtil: committing task: 'attempt_1522156796285_0036_m_000000_0' - hdfs://eiotwo:8020/electric_io/expr_92145_4081/_temporary/_attempt_1522156796285_0036_m_000000_0

2018-03-28 11:32:22,985 INFO [main] cascading.tap.hadoop.util.Hadoop18TapUtil: saved output of task 'attempt_1522156796285_0036_m_000000_0' to hdfs://eiotwo:8020/electric_io/expr_92145_4081

2018-03-28 11:32:22,985 INFO [main] cascading.tap.hadoop.io.TapOutputCollector: closing tap collector for: hdfs://eiotwo:8020/electric_io/expr_logs_92145_4081/part-00000

2018-03-28 11:32:23,019 INFO [main] cascading.tap.hadoop.util.Hadoop18TapUtil: committing task: 'attempt_1522156796285_0036_m_000000_0' - hdfs://eiotwo:8020/electric_io/expr_logs_92145_4081/_temporary/_attempt_1522156796285_0036_m_000000_0

2018-03-28 11:32:23,068 INFO [main] cascading.tap.hadoop.util.Hadoop18TapUtil: saved output of task 'attempt_1522156796285_0036_m_000000_0' to hdfs://eiotwo:8020/electric_io/expr_logs_92145_4081

2018-03-28 11:32:23,290 INFO [main] hive.metastore: Trying to connect to metastore with URI thrift://hivehost:9083

2018-03-28 11:32:23,343 WARN [main] hive.metastore: set_ugi() not successful, Likely cause: new client talking to old server. Continuing without it.

org.apache.thrift.transport.TTransportException

at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)

at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)

at org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:380)

at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:230)

at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)

at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_set_ugi(ThriftHiveMetastore.java:3604)

at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.set_ugi(ThriftHiveMetastore.java:3590)

at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:425)

at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:230)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1483)

at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:64)

at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:74)

at cascading.tap.hive.MetaStoreClientFactory.newInstance(MetaStoreClientFactory.java:57)

at cascading.tap.hive.HiveTap.resourceExists(HiveTap.java:202)

at cascading.tap.hive.HiveTap.registerPartition(HiveTap.java:346)

at cascading.tap.hive.HivePartitionTap$HivePartitionCollector.closeCollector(HivePartitionTap.java:152)

at cascading.tap.partition.BasePartitionTap$PartitionCollector.close(BasePartitionTap.java:189)

at cascading.flow.stream.element.SinkStage.cleanup(SinkStage.java:129)

at cascading.flow.stream.graph.StreamGraph.cleanup(StreamGraph.java:187)

at cascading.flow.hadoop.FlowMapper.run(FlowMapper.java:167)

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)

at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:422)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)

at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)

2018-03-28 11:32:23,352 INFO [main] hive.metastore: Connected to metastore.

2018-03-28 11:32:23,420 WARN [main] org.apache.hadoop.hive.metastore.RetryingMetaStoreClient: MetaStoreClient lost connection. Attempting to reconnect.

org.apache.thrift.transport.TTransportException

at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)

at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)

at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)

at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)

at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)

at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)

at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table(ThriftHiveMetastore.java:1134)

at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table(ThriftHiveMetastore.java:1120)

at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1146)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:91)

at com.sun.proxy.$Proxy16.getTable(Unknown Source)

at cascading.tap.hive.HiveTap.resourceExists(HiveTap.java:203)