LLAP Editor does not work when max_number_of_sessions > 0

[Theyaa Matti]

The LLAP editor does not work when you assign a value of over 0 to the max_number_of_sessions. There is a bug in hiveserver2.py file that it find an incorrect session id that belongs to a regular HS2 session and tries to use that for LLAP. Then it throws the exception I mentioned in my previous post.

[Romain Rigaux]

You should not need to change this property. Which issues do you have if you keep it to 1?

On Mon, May 4, 2020 at 7:07 AM Theyaa Matti <theya...@gmail.com> wrote:

The LLAP editor does not work when you assign a value of over 0 to the max_number_of_sessions. There is a bug in hiveserver2.py file that it find an incorrect session id that belongs to a regular HS2 session and tries to use that for LLAP. Then it throws the exception I mentioned in my previous post.

--
To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+u...@cloudera.org.

[Theyaa Matti]

I debugging the code a bit more, I found the issue related to how the LLAP editor check for existing session to use for running its query.

The file having issue is:/usr/hdp/current/hue/desktop/libs/notebook/src/notebook/connectors/hiveserver2.py

And specifically this function:

def _get_session_by_id(self, notebook, type='hive'):
  session = self._get_session(notebook, type)
  if session:
    session_id = session.get('id')
    if session_id:
      filters = {'id': session_id, 'application': 'beeswax' if type == 'hive' else type}
      if not is_admin(self.user):
        filters['owner'] = self.user
      return Session.objects.get(**filters)

The line session = self._get_session(notebook, type), returns an active session id. 

But when the execution hits line return Session.objects.get(**filters), it can't find an active session with that session_id and type llap.

There seem to be a bug in Hue where all hive sessions, HS2 and LLAP, are stored in Session object under type beeswax. That is why when looking for that session_id and type llap it can't find it.

And it throws the exception below.

Traceback (most recent call last):

  File "/usr/hdp/current/hue/desktop/libs/notebook/src/notebook/decorators.py", line 114, in wrapper

    return f(*args, **kwargs)

  File "/usr/hdp/current/hue/desktop/libs/notebook/src/notebook/api.py", line 222, in execute

    response = _execute_notebook(request, notebook, snippet)

  File "/usr/hdp/current/hue/desktop/libs/notebook/src/notebook/api.py", line 153, in _execute_notebook

    response['handle'] = interpreter.execute(notebook, snippet)

  File "/usr/hdp/current/hue/desktop/libs/notebook/src/notebook/connectors/hiveserver2.py", line 98, in decorator

    return func(*args, **kwargs)

  File "/usr/hdp/current/hue/desktop/libs/notebook/src/notebook/connectors/hiveserver2.py", line 294, in execute

    _session = self._get_session_by_id(notebook, snippet['type'])

  File "/usr/hdp/current/hue/desktop/libs/notebook/src/notebook/connectors/hiveserver2.py", line 675, in _get_session_by_id

    ss = Session.objects.get(**filters)

  File "/usr/hdp/current/hue/build/env/lib/python2.7/site-packages/Django-1.11.29-py2.7.egg/django/db/models/manager.py", line 85, in manager_method

    return getattr(self.get_queryset(), name)(*args, **kwargs)

  File "/usr/hdp/current/hue/build/env/lib/python2.7/site-packages/Django-1.11.29-py2.7.egg/django/db/models/query.py", line 380, in get

    self.model._meta.object_name

DoesNotExist: Session matching query does not exist.

To unsubscribe from this group and stop receiving emails from it, send an email to hue-...@cloudera.org.

[Theyaa Matti]

In order to make it work, I had to change the line filters = {'id': session_id, 'application': 'beeswax' if type == 'hive' else type} to filters = {'id': session_id, 'application': 'beeswax' if type == 'hive' or type == 'llap' else type}

[jeremy montgomery]

can you put in a PR for Romain?

To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+u...@cloudera.org.

[Romain Rigaux]

Thanks for all the details!

This is something that will get cleaned-up with https://issues.cloudera.org/browse/HUE-8758, will add a note about LLAP there.

Indeed, feel free to copy them in a new issue and send a pull request https://github.com/cloudera/hue

[Theyaa Matti]

Thank you Romain. I am happy to contribute of you agree with the fix I have proposed?

Best.

To unsubscribe from this group and stop receiving emails from it, send an email to hue-...@cloudera.org.

[Romain Rigaux]

It is easier to see in a PR but the logic is good, go ahead!

Romain

To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+u...@cloudera.org.

[Theyaa Matti]

I have no problem putting it into a PR. Since this is my first time doing that, would you please provide me with instructions on how to proceed with a PR?

To unsubscribe from this group and stop receiving emails from it, send an email to hue-...@cloudera.org.

[Romain Rigaux]

Feel free to let us know if questions!

https://github.com/cloudera/hue

https://opensource.com/article/19/7/create-pull-request-github

To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+u...@cloudera.org.