Cannot locate default realm when launch alluxio to yarn

[kong....@gmail.com]

Hi,

I'm trying to launch alluxio in yarn by following this page.

And since we have secured HDFS, I set my kerberos settings by following this page.

The only difference is, instead of providing realm and kdc, I have a krb5.conf file on my launch server(client server).

So, I added the following to the conf/alluxio-env.sh:

ALLUXIO_JAVA_OPTS+=" -Djava.security.krb5.conf=/home/..../krb5.conf"

After launching the alluxio by running ./integration/yarn/bin/alluxio-yarn.sh,

the application failed very soon.

Got an error in the log:

Launching Application Master
2018-08-16 09:27:51,156 INFO  ApplicationMaster - Starting Application Master with args [-num_workers, 3, -master_address, <master-host>, -resource_path, hdfs:///user/<path-to-the-resource>]
2018-08-16 09:27:51,492 ERROR ApplicationMaster - Error running Application Master
java.lang.IllegalArgumentException: Can't get Kerberos realm
        at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:65)
        at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:306)
        at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:352)
        at alluxio.yarn.ApplicationMaster.main(ApplicationMaster.java:195)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.security.authentication.util.KerberosUtil.getDefaultRealm(KerberosUtil.java:88)
        at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:63)
        ... 3 more
Caused by: KrbException: Cannot locate default realm
        at sun.security.krb5.Config.getDefaultRealm(Config.java:1029)
        ... 9 more

Then I realized that in the container, the krb5.conf will be definitely not there because the alluxio is trying to find that file in the container.

I wonder is there a way that I can use a krb5.conf file in the yarn integration?

[Andrew Audibert]

Hi,

Alluxio will copy all contents from it's conf/ directory into the container, so you can add your krb5.conf there. Then in alluxio-env.sh you can point to ${ALLUXIO_HOME}/conf/krb5.conf.

Hope that helps,

Andrew

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to alluxio-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Andrew Audibert

Alluxio, Inc. | Alluxio Open Source | Alluxio Community Site

[Mu Kong]

Hi Andrew,

Thanks for your reply!

I followed your suggestion, it seems to work.

But since I need to put the krb5.conf into conf/ directory, does that mean I need to put my keytab there too and modify the alluxio.master.keytab.file and alluxio.worker.keytab.file in conf/alluxio-site.properties file as ${ALLUXIO_HOME}/conf/xxxx.keytab ?

Thanks in advance!

Mu

[Andrew Audibert]

That sounds right. Is it working?

[kong....@gmail.com]

Yes, it seems working.

I launched a cluster with 3 workers on yarn and one master outside of yarn.

I think I need to write a client application to test the setup.

[kong....@gmail.com]

Actually, I found issues related to this.

After I launch the cluster, the workers on the Yarn cluster are complaining about org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]

Not sure if I should put them here. Maybe I will start another thread.

[and...@alluxio.com]

Link to new thread: https://groups.google.com/forum/#!topic/alluxio-users/kj0WqZNpN5U