Hue fails to connect to ResourceManager on first run
35 views
Skip to first unread message
Ivan Dzikovsky
May 23, 2017, 11:46:29 AM5/23/17
to Hue-Users
Hi guys!
Sometimes, after restarting Hue, on "Check Configuration" page, I see the following warning:
> Failed to contact an active Resource Manager: YARN RM returned a failed response: Unable to authenticate
And after reloading this page everything works fine.
(I have Kerberos configuration)
So I've made a little investigation on this, and found that this warning appears because Hue tries to perform mutual authentication two times at once.
One in connection that performs configuration check, and second in connection that tries to get number of active jobs to show on JobBrowser indicator.
Workflow of whats inside looks like this (inside requests-kerberos package):
1. It tries to connect to RM, and get 401 response
2a. It generates header to authenticate user, and at the same time? generates krb_context that would be used to authenticate server.
2b. It performs user authentication.
Sometimes, after restarting Hue, on "Check Configuration" page, I see the following warning:
> Failed to contact an active Resource Manager: YARN RM returned a failed response: Unable to authenticate
And after reloading this page everything works fine.
(I have Kerberos configuration)
So I've made a little investigation on this, and found that this warning appears because Hue tries to perform mutual authentication two times at once.
One in connection that performs configuration check, and second in connection that tries to get number of active jobs to show on JobBrowser indicator.
Workflow of whats inside looks like this (inside requests-kerberos package):
1. It tries to connect to RM, and get 401 response
2a. It generates header to authenticate user, and at the same time? generates krb_context that would be used to authenticate server.
2b. It performs user authentication.
3. It performs server authentication.
So in performs step 2 in first thread, than in second.
Than it performs step 3 in first thread and fails, because it cant authenticate server in first thread using krb_context that was regenerated in second.
The only good solution that I see, is to wrap steps 2 and 3 in threading.Lock().
Patch with this changes attached.
Does anyone have reproduce of this issue?
Should I open JIRA on this, or I'm doing something wrong?
So in performs step 2 in first thread, than in second.
Than it performs step 3 in first thread and fails, because it cant authenticate server in first thread using krb_context that was regenerated in second.
The only good solution that I see, is to wrap steps 2 and 3 in threading.Lock().
Patch with this changes attached.
Does anyone have reproduce of this issue?
Should I open JIRA on this, or I'm doing something wrong?
Romain Rigaux
May 23, 2017, 9:59:07 PM5/23/17
to Hue-Users, Ivan Dzikovsky, Prakash Ranade
On which Hue are you? (Might be fixed on master)
--
You received this message because you are subscribed to the Google Groups "Hue-Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+u...@cloudera.org.
Ivan Dzikovsky
May 24, 2017, 2:12:12 AM5/24/17
to Hue-Users, vanya...@gmail.com, ran...@cloudera.com
I'm on 3.12. Should I doublecheck on master, or can you please help me to find a commit/jira-ticket with potential fix?
Ivan Dzikovsky
May 24, 2017, 9:27:18 AM5/24/17
to Hue-Users
Maybe I have something wrong with my configuration.
I'll do some more investigations, and write up updates if this problem is really on Hue side.
Sorry for disturbing you.
I'll do some more investigations, and write up updates if this problem is really on Hue side.
Sorry for disturbing you.
Reply all
Reply to author
Forward
0 new messages