Livy impersonation when executing OS command from PySpark

[Raimondas Berniunas‏]

Dear All,

Has someone faced with such scenario.

a) Connection to the cluster is through Livy with impersonation.

b) When running Pyspark code (e.g. selecting from hive table) all the permissions are ok, i.e. based on user which livy uses when imperonating.

c) when i run a code executing OS command user is livy, but not that which was used to login the Hue.

import subprocess


subprocess.call(["hdfs", "dfs", "-copyFromLocal", "localFile", "hdfsFile"])

I.e. owner of hdfsFile will be livy.

My main worry is if livy user has more rights then the user can access all the livy user's resources.

My question if someone has faced with similar situation. So if there are some solutions I would appreciate for sharing it.

Thank you in advance!

Regards,

Raimondas