HDFS username

[Simon U]

Hi,

Where can I change the username the HDFS is accessed with?

> hdfs.defaults("fs") 
[1] "Java-Object{DFS[DFSClient[clientName=DFSClient_NONMAPREDUCE_1313042347_1, ugi=rstudio (auth:SIMPLE)]]}"

It is using HDFS with username "rstudio".

As a result I obviously get an error (admin is the owner here):

> hdfs.mkdir("/home/rstudio/rstudiocsv")
Error in .jcall("RJavaTools", "Ljava/lang/Object;", "invokeMethod", cl,  : 
  org.apache.hadoop.security.AccessControlException: Permission denied: user=rstudio, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x

The HDFS superuser is "hdfs".

How can I change this setting or have ability to switch between different accounts?

I would like to run hdfs.* commands with admin or hdfs username.

Cheers

S

[Antonio Piccolboni]

Can you try

Sys.setenv(HADOOP_USER_NAME  = "hdfs")

and then repeat your command?  I don't have a running cluster with authentication on, so I can't test it myself in the very short term.

--
post: rha...@googlegroups.com ||
unsubscribe: rhadoop+u...@googlegroups.com ||
web: https://groups.google.com/d/forum/rhadoop?hl=en-US
---
You received this message because you are subscribed to the Google Groups "RHadoop" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rhadoop+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Simon U]

Hi,

This particular does not work.

Running export HADOOP_USER_NAME=hdfs on the cluster machine does change the username from whoami.

Running this command in Rstudio shell does not work (Rstudio is on a different machine).

This might be a good lead, but still no luck...

Cheers,

S

[Saar Golde]

Are you logging into rstudio with the username rstudio? Could you log into rstudio with the username of the user you want to be working with?

The way I usually handle it is let users log into rstudio server installed on one of the data nodes ("edge node") using individual user names, and add them to the hdfs permissions group. This way you handle access and permissions the way they are supposed to be handled, not via people using superuser accounts. 

It does require us to define the required system variables in the R script (usinf Sys.setenv). 

--