Hive always loses some permissions created in Ranger

24 views
Skip to first unread message

Carol Chapman

unread,
Jul 25, 2022, 10:26:05 PM7/25/22
to MR3
At present, I find that online mr3 hive always loses the permissions of some tables given by Ranger after running for a period of time.

Here is the error log:

2022-07-13T14:59:45,745  WARN [HiveServer2-Handler-Pool: Thread-49] thrift.ThriftCLIService: Error executing statement:
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.NullPointerException
        at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:89) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) ~[hive-service-3.1.3.jar:3.1.3]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_201]
        at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_201]
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) ~[hadoop-common-3.1.2.jar:?]
        at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) ~[hive-service-3.1.3.jar:3.1.3]
        at com.sun.proxy.$Proxy72.executeStatementAsync(Unknown Source) ~[?:?]
        at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:312) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:567) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557) ~[hive-exec-3.1.3.jar:3.1.3]
        at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542) ~[hive-exec-3.1.3.jar:3.1.3]
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) ~[hive-exec-3.1.3.jar:3.1.3]
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) ~[hive-exec-3.1.3.jar:3.1.3]
        at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) ~[hive-exec-3.1.3.jar:3.1.3]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_201]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_201]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201]
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
        at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:89) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) ~[hive-service-3.1.3.jar:3.1.3]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_201]
        at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_201]
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) ~[hadoop-common-3.1.2.jar:?]
        at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) ~[hive-service-3.1.3.jar:3.1.3]
        at com.sun.proxy.$Proxy72.submitBackgroundOperation(Unknown Source) ~[?:?]
        at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:280) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.operation.Operation.run(Operation.java:260) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:568) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:554) ~[hive-service-3.1.3.jar:3.1.3]
        at sun.reflect.GeneratedMethodAccessor62.invoke(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_201]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_201]
        at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) ~[hive-service-3.1.3.jar:3.1.3]
        ... 18 more
Caused by: java.lang.NullPointerException
        at org.apache.hive.service.cli.session.SessionManager.submitBackgroundOperation(SessionManager.java:665) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.session.HiveSessionImpl.submitBackgroundOperation(HiveSessionImpl.java:590) ~[hive-service-3.1.3.jar:3.1.3]
        at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_201]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_201]
        at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) ~[hive-service-3.1.3.jar:3.1.3]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_201]
        at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_201]
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) ~[hadoop-common-3.1.2.jar:?]
        at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) ~[hive-service-3.1.3.jar:3.1.3]
        at com.sun.proxy.$Proxy72.submitBackgroundOperation(Unknown Source) ~[?:?]
        at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:280) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.operation.Operation.run(Operation.java:260) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:568) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:554) ~[hive-service-3.1.3.jar:3.1.3]
        at sun.reflect.GeneratedMethodAccessor62.invoke(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_201]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_201]
        at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) ~[hive-service-3.1.3.jar:3.1.3]
        ... 18 more
2022-07-13T14:59:45,746  INFO [HiveServer2-Handler-Pool: Thread-49] thrift.ThriftCLIService: Session disconnected without closing properly.
2022-07-13T14:59:45,746  INFO [HiveServer2-Handler-Pool: Thread-49] thrift.ThriftCLIService: Closing the session: SessionHandle [f386c73e-8995-4638-9b68-7165aead878e]
2022-07-13T14:59:45,746  INFO [f386c73e-8995-4638-9b68-7165aead878e HiveServer2-Handler-Pool: Thread-49] operation.OperationManager: Closing operation: OperationHandle [opType=EXECUTE_STATEMENT, getHandleIdentifier()=433eb954-118d-41ae-86eb-952bae21f09b]
2022-07-13T14:59:45,746  INFO [f386c73e-8995-4638-9b68-7165aead878e HiveServer2-Handler-Pool: Thread-49] operation.OperationManager: Removed queryId: hue_20220713145945_31000b4b-09ff-455f-b79f-45ef4eb1cf14 corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT, getHandleIdentifier()=433eb954-118d-41ae-86eb-952bae21f09b] with tag: null
2022-07-13T14:59:45,758  INFO [f386c73e-8995-4638-9b68-7165aead878e HiveServer2-Handler-Pool: Thread-49] exec.TableScanOperator: RECORDS_OUT_INTERMEDIATE:0, RECORDS_OUT_OPERATOR_TS_0:0,
2022-07-13T14:59:45,759  INFO [f386c73e-8995-4638-9b68-7165aead878e HiveServer2-Handler-Pool: Thread-49] exec.SelectOperator: RECORDS_OUT_INTERMEDIATE:0, RECORDS_OUT_OPERATOR_SEL_1:0,
2022-07-13T14:59:45,759  INFO [f386c73e-8995-4638-9b68-7165aead878e HiveServer2-Handler-Pool: Thread-49] exec.ListSinkOperator: RECORDS_OUT_INTERMEDIATE:0, RECORDS_OUT_OPERATOR_LIST_SINK_3:0,
2022-07-13T14:59:45,774  INFO [f386c73e-8995-4638-9b68-7165aead878e HiveServer2-Handler-Pool: Thread-49] operation.Operation: Closing operation log /tmp/hue/operation_logs/f386c73e-8995-4638-9b68-7165aead878e/hue_20220713145945_31000b4b-09ff-455f-b79f-45ef4eb1cf14 without delay
2022-07-13T14:59:45,774  INFO [f386c73e-8995-4638-9b68-7165aead878e HiveServer2-Handler-Pool: Thread-49] conf.HiveConf: Using the default value passed in for log id: f386c73e-8995-4638-9b68-7165aead878e
2022-07-13T14:59:45,774 ERROR [f386c73e-8995-4638-9b68-7165aead878e HiveServer2-Handler-Pool: Thread-49] session.HiveSessionImpl: Failed to cleanup session log dir: SessionHandle [f386c73e-8995-4638-9b68-7165aead878e]
java.io.FileNotFoundException: File does not exist: /tmp/hue/operation_logs/f386c73e-8995-4638-9b68-7165aead878e
        at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2396) ~[commons-io-2.6.jar:2.6]
        at org.apache.hive.service.cli.session.HiveSessionImpl.cleanupSessionLogDir(HiveSessionImpl.java:828) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.session.HiveSessionImpl.close(HiveSessionImpl.java:781) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.session.HiveSessionImplwithUGI.close(HiveSessionImplwithUGI.java:93) ~[hive-service-3.1.3.jar:3.1.3]
        at sun.reflect.GeneratedMethodAccessor205.invoke(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_201]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_201]
        at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) ~[hive-service-3.1.3.jar:3.1.3]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_201]
        at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_201]
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) ~[hadoop-common-3.1.2.jar:?]
        at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) ~[hive-service-3.1.3.jar:3.1.3]
        at com.sun.proxy.$Proxy72.close(Unknown Source) ~[?:?]
        at org.apache.hive.service.cli.session.SessionManager.closeSession(SessionManager.java:552) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.CLIService.closeSession(CLIService.java:241) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.thrift.ThriftBinaryCLIService$1.deleteContext(ThriftBinaryCLIService.java:147) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:300) ~[hive-exec-3.1.3.jar:3.1.3]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_201]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_201]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201]
2022-07-13T14:59:45,774  INFO [HiveServer2-Handler-Pool: Thread-49] lockmgr.DbTxnManager: Stopped heartbeat for query: null
2022-07-13T14:59:45,814  INFO [HiveServer2-Handler-Pool: Thread-49] session.SessionState: Deleted directory: /tmp/hue/databank01/f386c73e-8995-4638-9b68-7165aead878e on fs with scheme hdfs
2022-07-13T14:59:45,819  INFO [HiveServer2-Handler-Pool: Thread-49] session.SessionState: Deleted directory: /tmp/hue/f386c73e-8995-4638-9b68-7165aead878e on fs with scheme file
2022-07-13T14:59:45,819  INFO [HiveServer2-Handler-Pool: Thread-49] metastore.HiveMetaStoreClient: Closed a connection to metastore, current connections: 7
2022-07-13T15:00:01,684  INFO [HiveServer2-Handler-Pool: Thread-772] service.CompositeService: Session closed, SessionHandle [f8c2652a-bd18-4b89-bf22-8e88b967bc10], current sessions:0
2022-07-13T15:00:01,684  INFO [HiveServer2-Handler-Pool: Thread-772] conf.HiveConf: Using the default value passed in for log id: f8c2652a-bd18-4b89-bf22-8e88b967bc10
2022-07-13T15:00:01,684  INFO [f8c2652a-bd18-4b89-bf22-8e88b967bc10 HiveServer2-Handler-Pool: Thread-772] conf.HiveConf: Using the default value passed in for log id: f8c2652a-bd18-4b89-bf22-8e88b967bc10
2022-07-13T15:00:01,684  INFO [HiveServer2-Handler-Pool: Thread-772] conf.HiveConf: Using the default value passed in for log id: f8c2652a-bd18-4b89-bf22-8e88b967bc10
2022-07-13T15:00:01,684  INFO [HiveServer2-Handler-Pool: Thread-772] thrift.ThriftCLIService: Session disconnected without closing properly.
2022-07-13T15:00:01,684  INFO [HiveServer2-Handler-Pool: Thread-772] thrift.ThriftCLIService: Closing the session: SessionHandle [f8c2652a-bd18-4b89-bf22-8e88b967bc10]
2022-07-13T15:00:01,684  INFO [f8c2652a-bd18-4b89-bf22-8e88b967bc10 HiveServer2-Handler-Pool: Thread-772] operation.OperationManager: Closing operation: OperationHandle [opType=EXECUTE_STATEMENT, getHandleIdentifier()=6b724d80-d308-415a-a1cf-c6fdd10ef4ed]
2022-07-13T15:00:01,684  INFO [f8c2652a-bd18-4b89-bf22-8e88b967bc10 HiveServer2-Handler-Pool: Thread-772] operation.OperationManager: Removed queryId: hue_20220713110546_73a5ee6c-b01c-4ed0-96ed-dff58ed77fe4 corresponding to operation: OperationHandle [opType=EXECUTE_STATEMENT, getHandleIdentifier()=6b724d80-d308-415a-a1cf-c6fdd10ef4ed] with tag: null
2022-07-13T15:00:01,713  INFO [f8c2652a-bd18-4b89-bf22-8e88b967bc10 HiveServer2-Handler-Pool: Thread-772] operation.Operation: Closing operation log /tmp/hue/operation_logs/f8c2652a-bd18-4b89-bf22-8e88b967bc10/hue_20220713110546_73a5ee6c-b01c-4ed0-96ed-dff58ed77fe4 without delay
2022-07-13T15:00:01,713  INFO [f8c2652a-bd18-4b89-bf22-8e88b967bc10 HiveServer2-Handler-Pool: Thread-772] conf.HiveConf: Using the default value passed in for log id: f8c2652a-bd18-4b89-bf22-8e88b967bc10
2022-07-13T15:00:01,713 ERROR [f8c2652a-bd18-4b89-bf22-8e88b967bc10 HiveServer2-Handler-Pool: Thread-772] session.HiveSessionImpl: Failed to cleanup session log dir: SessionHandle [f8c2652a-bd18-4b89-bf22-8e88b967bc10]
java.io.FileNotFoundException: File does not exist: /tmp/hue/operation_logs/f8c2652a-bd18-4b89-bf22-8e88b967bc10
        at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2396) ~[commons-io-2.6.jar:2.6]
        at org.apache.hive.service.cli.session.HiveSessionImpl.cleanupSessionLogDir(HiveSessionImpl.java:828) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.session.HiveSessionImpl.close(HiveSessionImpl.java:781) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.session.HiveSessionImplwithUGI.close(HiveSessionImplwithUGI.java:93) ~[hive-service-3.1.3.jar:3.1.3]
        at sun.reflect.GeneratedMethodAccessor205.invoke(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_201]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_201]
        at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) ~[hive-service-3.1.3.jar:3.1.3]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_201]
        at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_201]
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) ~[hadoop-common-3.1.2.jar:?]
        at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) ~[hive-service-3.1.3.jar:3.1.3]
        at com.sun.proxy.$Proxy72.close(Unknown Source) ~[?:?]
        at org.apache.hive.service.cli.session.SessionManager.closeSession(SessionManager.java:552) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.CLIService.closeSession(CLIService.java:241) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.thrift.ThriftBinaryCLIService$1.deleteContext(ThriftBinaryCLIService.java:147) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:300) ~[hive-exec-3.1.3.jar:3.1.3]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_201]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_201]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201]

Carol Chapman

unread,
Jul 25, 2022, 10:26:50 PM7/25/22
to MR3
If I need to provide any additional information, I will provide it at the first time

Sungwoo Park

unread,
Jul 26, 2022, 12:27:33 AM7/26/22
to MR3
Do you see the same problem when using Beeline to connect to HiveServer2?

Or, could you share how to reproduce the error? This is probably an issue with Hive because Hive-MR3 replaces only the backend of Hive.

Cheers,

--- Sungwoo

Message has been deleted

Carol Chapman

unread,
Jul 26, 2022, 3:22:23 AM7/26/22
to MR3
This problem is not easy to recur. Because every time you restart it, it is always OK, but it will always go wrong with a long running time.
I have assigned multiple permission accounts to hive in Ranger, and the accounts with low permission often cause problems.
But I found this error when I started hive:
2022-07-19T14:24:46,255 ERROR [main] util.PolicyRefresher: failed to save policies to cache file '/etc/ranger/xxxx/policycache/hiveServer2_xxxxx_hive.json'
java.io.FileNotFoundException: /etc/ranger/xxxx/policycache/hiveServer2_xxxxx_hive.json (Permission denied)
        at java.io.FileOutputStream.open0(Native Method) ~[?:1.8.0_201]
        at java.io.FileOutputStream.open(FileOutputStream.java:270) ~[?:1.8.0_201]
        at java.io.FileOutputStream.<init>(FileOutputStream.java:213) ~[?:1.8.0_201]
        at java.io.FileOutputStream.<init>(FileOutputStream.java:162) ~[?:1.8.0_201]
        at java.io.FileWriter.<init>(FileWriter.java:90) ~[?:1.8.0_201]
        at org.apache.ranger.plugin.util.PolicyRefresher.saveToCache(PolicyRefresher.java:392) ~[ranger-plugins-common-1.2.0.3.1.0.0-78.jar:1.2.0.3.1.0.0-78]
        at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:210) ~[ranger-plugins-common-1.2.0.3.1.0.0-78.jar:1.2.0.3.1.0.0-78]
        at org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:149) ~[ranger-plugins-common-1.2.0.3.1.0.0-78.jar:1.2.0.3.1.0.0-78]
        at org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:222) ~[ranger-plugins-common-1.2.0.3.1.0.0-78.jar:1.2.0.3.1.0.0-78]
        at org.apache.ranger.authorization.hive.authorizer.RangerHivePlugin.init(RangerHiveAuthorizer.java:1933) ~[ranger-hive-plugin-1.2.0.3.1.0.0-78.jar:1.2.0.3.1.0.0-78]
        at org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer.<init>(RangerHiveAuthorizer.java:124) ~[ranger-hive-plugin-1.2.0.3.1.0.0-78.jar:1.2.0.3.1.0.0-78]
        at org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizerFactory.createHiveAuthorizer(RangerHiveAuthorizerFactory.java:37) ~[ranger-hive-plugin-1.2.0.3.1.0.0-78.jar:1.2.0.3.1.0.0-78]
        at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:893) ~[hive-exec-3.1.3.jar:3.1.3]
        at org.apache.hadoop.hive.ql.session.SessionState.applyAuthorizationPolicy(SessionState.java:1843) ~[hive-exec-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:131) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.cli.CLIService.init(CLIService.java:115) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.CompositeService.init(CompositeService.java:59) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:240) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:990) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.server.HiveServer2.access$1400(HiveServer2.java:137) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1256) ~[hive-service-3.1.3.jar:3.1.3]
        at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1100) ~[hive-service-3.1.3.jar:3.1.3]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_201]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_201]

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_201]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_201]
        at org.apache.hadoop.util.RunJar.run(RunJar.java:318) ~[hadoop-common-3.1.2.jar:?]
        at org.apache.hadoop.util.RunJar.main(RunJar.java:232) ~[hadoop-common-3.1.2.jar:?]


Is this the reason? (I didn't have time to read the code in detail. I just guessed that the cache expired and I couldn't read the file.)

Sungwoo Park

unread,
Jul 26, 2022, 4:32:11 AM7/26/22
to MR3
Not sure why you get the error. Normally you should find these files under Ranger policy cache directory:

hive@hiveserver2-59bd84cf58-cg6dl:/opt/mr3-run/hiveserver2-ranger-policycache$ ls
hiveServer2_BLUE_hive.json  hiveServer2_BLUE_hive_roles.json  hive_audit_solr_spool

The policy cache directory is specified with ranger.plugin.hive.policy.cache.dir in ranger-hive-security.xml, e.g.:

      <property>
        <name>ranger.plugin.hive.policy.cache.dir</name>
        <value>/opt/mr3-run/hiveserver2-ranger-policycache</value>
      </property>

Configuring Ranger can be a tough task because there is not much documentation about it. For this reason, we maintain Ranger configurations only for Hive-MR3 on Kubernetes.

Cheers,

--- Sungwoo

Carol Chapman

unread,
Jul 27, 2022, 1:20:44 AM7/27/22
to MR3
Okay, I'll first solve the error I encountered in my startup, and then observe it for a period of time
Reply all
Reply to author
Forward
0 new messages