LLAP Editor does not work when max_number_of_sessions > 0

21 views
Skip to first unread message

Theyaa Matti

unread,
May 4, 2020, 10:07:17 AM5/4/20
to Hue-Users
The LLAP editor does not work when you assign a value of over 0 to the max_number_of_sessions. There is a bug in hiveserver2.py file that it find an incorrect session id that belongs to a regular HS2 session and tries to use that for LLAP. Then it throws the exception I mentioned in my previous post.


Romain Rigaux

unread,
May 4, 2020, 10:02:01 PM5/4/20
to Theyaa Matti, Hue-Users
You should not need to change this property. Which issues do you have if you keep it to 1?

On Mon, May 4, 2020 at 7:07 AM Theyaa Matti <theya...@gmail.com> wrote:
The LLAP editor does not work when you assign a value of over 0 to the max_number_of_sessions. There is a bug in hiveserver2.py file that it find an incorrect session id that belongs to a regular HS2 session and tries to use that for LLAP. Then it throws the exception I mentioned in my previous post.


--
To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+u...@cloudera.org.

Theyaa Matti

unread,
May 4, 2020, 11:55:29 PM5/4/20
to Hue-Users, theya...@gmail.com
I debugging the code a bit more, I found the issue related to how the LLAP editor check for existing session to use for running its query.

The file having issue is:/usr/hdp/current/hue/desktop/libs/notebook/src/notebook/connectors/hiveserver2.py

And specifically this function:
def _get_session_by_id(self, notebook, type='hive'):
session = self._get_session(notebook, type)
if session:
session_id = session.get('id')
if session_id:
filters = {'id': session_id, 'application': 'beeswax' if type == 'hive' else type}
if not is_admin(self.user):
filters['owner'] = self.user
return Session.objects.get(**filters)

The line session = self._get_session(notebook, type), returns an active session id. 
But when the execution hits line return Session.objects.get(**filters), it can't find an active session with that session_id and type llap.

There seem to be a bug in Hue where all hive sessions, HS2 and LLAP, are stored in Session object under type beeswax. That is why when looking for that session_id and type llap it can't find it.

And it throws the exception below.

Traceback (most recent call last):

  File "/usr/hdp/current/hue/desktop/libs/notebook/src/notebook/decorators.py", line 114, in wrapper

    return f(*args, **kwargs)

  File "/usr/hdp/current/hue/desktop/libs/notebook/src/notebook/api.py", line 222, in execute

    response = _execute_notebook(request, notebook, snippet)

  File "/usr/hdp/current/hue/desktop/libs/notebook/src/notebook/api.py", line 153, in _execute_notebook

    response['handle'] = interpreter.execute(notebook, snippet)

  File "/usr/hdp/current/hue/desktop/libs/notebook/src/notebook/connectors/hiveserver2.py", line 98, in decorator

    return func(*args, **kwargs)

  File "/usr/hdp/current/hue/desktop/libs/notebook/src/notebook/connectors/hiveserver2.py", line 294, in execute

    _session = self._get_session_by_id(notebook, snippet['type'])

  File "/usr/hdp/current/hue/desktop/libs/notebook/src/notebook/connectors/hiveserver2.py", line 675, in _get_session_by_id

    ss = Session.objects.get(**filters)

  File "/usr/hdp/current/hue/build/env/lib/python2.7/site-packages/Django-1.11.29-py2.7.egg/django/db/models/manager.py", line 85, in manager_method

    return getattr(self.get_queryset(), name)(*args, **kwargs)

  File "/usr/hdp/current/hue/build/env/lib/python2.7/site-packages/Django-1.11.29-py2.7.egg/django/db/models/query.py", line 380, in get

    self.model._meta.object_name

DoesNotExist: Session matching query does not exist.

To unsubscribe from this group and stop receiving emails from it, send an email to hue-...@cloudera.org.

Theyaa Matti

unread,
May 4, 2020, 11:57:20 PM5/4/20
to Hue-Users, theya...@gmail.com
In order to make it work, I had to change the line filters = {'id': session_id, 'application': 'beeswax' if type == 'hive' else type} to filters = {'id': session_id, 'application': 'beeswax' if type == 'hive' or type == 'llap' else type}

jeremy montgomery

unread,
May 5, 2020, 6:36:26 AM5/5/20
to Theyaa Matti, Hue-Users
can you put in a PR for Romain?

To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+u...@cloudera.org.

Romain Rigaux

unread,
May 5, 2020, 9:33:42 AM5/5/20
to jeremy montgomery, Theyaa Matti, Hue-Users
Thanks for all the details!

This is something that will get cleaned-up with https://issues.cloudera.org/browse/HUE-8758, will add a note about LLAP there.

Indeed, feel free to copy them in a new issue and send a pull request https://github.com/cloudera/hue

Theyaa Matti

unread,
May 5, 2020, 9:52:48 AM5/5/20
to Hue-Users, aleric...@gmail.com, theya...@gmail.com
Thank you Romain. I am happy to contribute of you agree with the fix I have proposed?

Best.
To unsubscribe from this group and stop receiving emails from it, send an email to hue-...@cloudera.org.

Romain Rigaux

unread,
May 5, 2020, 10:37:26 AM5/5/20
to Theyaa Matti, Hue-Users, jeremy montgomery
It is easier to see in a PR but the logic is good, go ahead!

Romain

To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+u...@cloudera.org.

Theyaa Matti

unread,
May 5, 2020, 12:35:50 PM5/5/20
to Hue-Users, theya...@gmail.com, aleric...@gmail.com
I have no problem putting it into a PR. Since this is my first time doing that, would you please provide me with instructions on how to proceed with a PR?
To unsubscribe from this group and stop receiving emails from it, send an email to hue-...@cloudera.org.

Romain Rigaux

unread,
May 5, 2020, 4:18:48 PM5/5/20
to Theyaa Matti, Hue-Users, jeremy montgomery

To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+u...@cloudera.org.
Reply all
Reply to author
Forward
0 new messages