java.io.IOException: Failed to login: No Alluxio User is found. Current login principals are...

[wjli...@gmail.com]

Hi

    When I used Alluxio in Spark-Shell, I encountered the following problem (Spark version 2.1.0, Alluxio version 1.3.0 and I did not use HADOOP)：

     

17/03/16 10:07:49 INFO type: Alluxio client (version 1.3.0) is trying to connect with FileSystemMasterClient master @ /xxx.xxx.xxx.xxx:19998

java.io.IOException: Failed to login: No Alluxio User is found. Current login principals are [UnixPrincipal: xxx, UnixNumericUserPrincipal: 1001, UnixNumericGroupPrincipal [Primary Group]: 1001, xxx].

 If you are running hadoop with Alluxio, please double check whether you have alluxio client jar included in HADOOP_CLASSPATH. Otherwise, you are expected to see this error.

  at alluxio.security.LoginUser.login(LoginUser.java:98)

  at alluxio.security.LoginUser.get(LoginUser.java:57)

  at alluxio.security.authentication.PlainSaslTransportProvider.getClientTransport(PlainSaslTransportProvider.java:53)

  at alluxio.AbstractClient.connect(AbstractClient.java:173)

  at alluxio.AbstractClient.retryRPC(AbstractClient.java:346)

  at alluxio.client.file.FileSystemMasterClient.getStatus(FileSystemMasterClient.java:183)

  at alluxio.client.file.BaseFileSystem.getStatus(BaseFileSystem.java:176)

  at alluxio.client.file.BaseFileSystem.getStatus(BaseFileSystem.java:168)

  at alluxio.hadoop.AbstractFileSystem.getFileStatus(AbstractFileSystem.java:293)

  at alluxio.hadoop.FileSystem.getFileStatus(FileSystem.java:25)

  at org.apache.hadoop.fs.Globber.getFileStatus(Globber.java:57)

  at org.apache.hadoop.fs.Globber.glob(Globber.java:252)

  at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1676)

  at org.apache.hadoop.mapred.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:259)

  at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:229)

  at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)

  at org.apache.spark.rdd.HadoopRDD.getPartitions(HadoopRDD.scala:202)

  at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:252)

  at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:250)

  at scala.Option.getOrElse(Option.scala:121)

  at org.apache.spark.rdd.RDD.partitions(RDD.scala:250)

  at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)

  at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:252)

  at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:250)

  at scala.Option.getOrElse(Option.scala:121)

  at org.apache.spark.rdd.RDD.partitions(RDD.scala:250)

  at org.apache.spark.SparkContext.runJob(SparkContext.scala:1958)

  at org.apache.spark.rdd.RDD.count(RDD.scala:1157)

  ... 48 elided

     But I use the Spark 2.0.1 is normal， I also tried using Alluxio 1.4.0 in the Spark 2.1.0, but still have the above problem.

     Finally, after a constant attempt, I found that the following configuration of Alluxio can make the normal use in the Spark 2.1.0 version (Alluxio server and the client should use the configuration)：

alluxio.security.authorization.permission.enabled=false

alluxio.security.authentication.type=NOSASL

     I do not know what kind of impact this will bring to Alluxio, but only this configuration can be used normally.

[Bin Fan]

Hi 

even you don't use Hadoop, could you still try the following two options and see if that works for you:

- set $HADOOP_CLASSPATH as suggested to include your alluxio client jar.

Even you don't use Hadoop, Spark somehow respects this environment variable.

- set $HADOOP_CONF_DIR env-var pointing to $ALLUXIO_HOME/conf

- Bin

[Gene Pang]

Hi,

I think this is related to this bug: https://alluxio.atlassian.net/browse/ALLUXIO-2828 which was fixed with this PR: https://github.com/Alluxio/alluxio/pull/5302

You can try out the latest Alluxio code to see if that works for you.

Thanks,

Gene

[Gene Pang]

Hi,

Some documentation was added about this issue: http://www.alluxio.org/docs/master/en/Running-Spark-on-Alluxio.html#failed-to-login-issues-with-spark-shell

Thanks,

Gene