User [hue] is not authorized to view the logs for attempt_1427717947268_0003_r_000000_0

[kawa...@gmail.com]

Hi,

I want to browse logs generated by my MapReduce job using HUE. While it works fine when I browse the Resource Manager Web UI, it doesn't work when I use the jobbrowser from HUE due to:

User [hue] is not authorized to view the logs for attempt_1427717947268_0003_r_000000_0

I guess this is because the hue user runs the HUE server. Is there any trick to make sure that the hue user impersonate the real user when browsing the logs? It should be working fine if the logs are read directly from the HDFS directories, but it seems that the logs are fetched from the Resource Manager Web UI, but the hue user doesn't impersonate anyone when talking to RM.

Thanks!

Adam

[Romain Rigaux]

Yarn does not have any impersonation or even a GetLog() API. So Hue does not have any other choice than fetching the logs itself. A current workaround is to allow Hue to fetch them with below (Hue is already a proxy user for HDFS, Hive, Oozie... anyway).

<property>
<name>mapreduce.cluster.acls.enabled</name>
<value>true</value>
</property>
<property>
<name>mapreduce.job.acl-view-job</name>
<!-- Allow NO users except owner and admins, and Hue sessions -->
<value>mapred,hue</value>
</property>
<property>
<name>mapreduce.job.acl-modify-job</name>
<!-- Allow NO users except owner and admins, and Hue sessions -->
<value>mapred,hue</value>
</property>

To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+u...@cloudera.org.

[Adam Kawa]

Hi Romain,

Thanks a lot for a quick reply!

2015-03-31 18:24 GMT+02:00 Romain Rigaux <rom...@cloudera.com>:

Yarn does not have any impersonation or even a GetLog() API. So Hue does not have any other choice than fetching the logs itself.

Couldn't we read them directly from HDFS assuming that YARN log aggregation is enabled?

 

A current workaround is to allow Hue to fetch them with below (Hue is already a proxy user for HDFS, Hive, Oozie... anyway).

<property>
<name>mapreduce.cluster.acls.enabled</name>
<value>true</value>
</property>
<property>
<name>mapreduce.job.acl-view-job</name>
<!-- Allow NO users except owner and admins, and Hue sessions -->
<value>mapred,hue</value>
</property>
<property>
<name>mapreduce.job.acl-modify-job</name>
<!-- Allow NO users except owner and admins, and Hue sessions -->
<value>mapred,hue</value>
</property>

Yes, but I would be a partial solution only, because I guess that this will work only for MapReduce jobs... In my case I would like to run and browse the logs from other types of apps e.g Spark.

Cheers!

Adam

[Romain Rigaux]

So in practice as Hue is just curling the Web UI. If reading directly from HDFS we should face the same permissions problem plus would need to build up the HDFS paths and fetch those AFAIK.

In the case of Spark, right now Hue just opens the standard Spark App UI.

We should be able to display a mini summary of the metrics:
https://issues.apache.org/jira/browse/SPARK-3454

Regarding the Spark logs, are you just looking for the ones in the 'Executor' tab?

We could pull those when we have SPARK-3454 (bit cleaner than parsing the Spark App UI).

Romain

[John Ross Tasipit]

 I have tried the mapreduce parameters to add on mapred-site.xml using Ambari but it seems that Hue is not using the parameters, unless I specified the mapreduce param on Hadoop Properties of Pig Editor. It seems that Hue is not communicating on Mapred. I have tried to put the plugin on /usr/hdp/current/hadoop-mapreduce/lib/ but still no luck. Please help.