Hue cache locations? Database migration? 1/3 Servers consistently generates an error

[Jordan Moore]

We've been running Hue 3.11.0 for awhile, and it's been pretty solid, but users are wanting more advanced SQL features and it seems 3.12.0 really targets those. 

 

I was playing around with the release-3.12.0 and release-4.0.0 branches and I have feel like I may have inadvertently caused a forward database migration during my testing. 

We have Hue's static resources being served by Nginx, and each Hue Server configured to share a remote MySQL Server. 

While I was testing on this one host, something caused this particular one to go out of sync. One server is CentOS 6 and there are two other running CentOS 7. As mentioned, the latter 2 are on Hue 3.11.0, and the first I was configuring and kicking the tires on between both 4.0.0 and 3.12.0.

Long story short, I realized I should not be testing the new server while the load balancer was still pointing at it, so I reverted back to 3.11.0. Now, in the Hive editor I get

• 'NoneType' object has no attribute 'update_data'

Which I see HUE-4546 addresses that fix for Hue 3.12.0, but what confuses me is why only this host has the problem after downgrading to 3.11.0? I can submit the exact same query from the other two 3.11.0 servers just fine. 

Question: It's a Python error, and I think my rpmbuild package cleans up the relevant installation directories, so nothing should be cached other than maybe the nginx static assets. Are the desktop_* tables in the hue MySQL database, something I should I be clearing? I couldn't quite determine if a "syncdb" operation needed to happen or how to do so without CM. If I do clear something from that database, what side-effects does that have for other users with saved queries, notebooks, workflows, etc?

Follow-up: What is the recommended upgrade path? Certainly starting with a fresh database is ideal and the database migrations should work, but it seems this is not the case.

 

[Romain Rigaux]

Would you have the trace on the /logs page of Hue after getting this error? (would give the line numbers)

Correct commands to upgrade should be just:

hue syncdb

hue migrate

(and AFAIK there is no DB changes between 3.12 and 4.0)

--
You received this message because you are subscribed to the Google Groups "Hue-Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+unsubscribe@cloudera.org.

[Jordan Moore]

Seems I spoke too soon. The files are not cleaned up when using yum erase. 

I'm not seeing the error anymore after removing the 3.12 package, cleaning the install directory, then installing the 3.11 package. 

I'll need to rebuild the RPM to figure out the cleanup operation, but I can manage that. 

Jordan Moore 
Staff Consultant | Avalon Consulting, LLC 
M:  217 622-5993 
LinkedIn | Google+ | Twitter

-------------------------------------------------------------------------------------------------------------
This message (including any attachments) contains confidential information 
intended for a specific individual and purpose, and is protected by law. If 
you are not the intended recipient, you should delete this message. Any 
disclosure, copying, or distribution of this message, or the taking of any 
action based on it, is strictly prohibited.

[Jordan Moore]

Okay, was able to reproduce.

[24/Jun/2017 22:32:20 -0500] access       INFO     172.28.43.11 jomoore - "POST /notebook/api/close_statement HTTP/1.0"
[24/Jun/2017 22:32:20 -0500] access       INFO     172.28.43.11 jomoore - "POST /notebook/api/execute HTTP/1.0"
[24/Jun/2017 22:32:20 -0500] decorators   ERROR    Error running <function execute at 0x7f5b45922140>
Traceback (most recent call last):
, kwargs={})
[24/Jun/2017 22:32:19 -0500] thrift_util  DEBUG    Thrift call <class 'TCLIService.TCLIService.Client'>.FetchResults returned in 1ms: TFetchResultsResp(status=TStatus(errorCode=None, errorMessage=None, sq
lState=None, infoMessages=None, statusCode=0), results=TRowSet(rows=[], columns=[TColumn(i32Val=None, byteVal=None, i16Val=None, i64Val=None, stringVal=TStringColumn(nulls='\x00', values=[]), boolVal=None
, doubleVal=None, binaryVal=None)], startRowOffset=0), hasMoreRows=False)
[24/Jun/2017 22:32:19 -0500] thrift_util  DEBUG    Thrift call: <class 'TCLIService.TCLIService.Client'>.GetResultSetMetadata(args=(TGetResultSetMetadataReq(operationHandle=TOperationHandle(hasResultSet=T
rue, modifiedRowCount=None, operationType=0, operationId=THandleIdentifier(secret='\xe6L\xc0\x0e\xb7mM\xc3\xa9\n\xcd\x82b{x\x93', guid='\xd6\xd8\x17\xfeMkA\x8f\x86H\xc6B)`\\x'))),), kwargs={})
[24/Jun/2017 22:32:19 -0500] thrift_util  DEBUG    Thrift call <class 'TCLIService.TCLIService.Client'>.GetResultSetMetadata returned in 1ms: TGetResultSetMetadataResp(status=TStatus(errorCode=None, error
Message=None, sqlState=None, infoMessages=None, statusCode=0), schema=TTableSchema(columns=[TColumnDesc(comment=None, columnName='_c0', typeDesc=TTypeDesc(types=[TTypeEntry(mapEntry=None, unionEntry=None,
 arrayEntry=None, userDefinedTypeEntry=None, structEntry=None, primitiveEntry=TPrimitiveTypeEntry(typeQualifiers=None, type=7))]), position=1)]))
[24/Jun/2017 22:32:20 -0500] access       INFO     172.28.43.11 jomoore - "POST /notebook/api/close_statement HTTP/1.0"
[24/Jun/2017 22:32:20 -0500] access       INFO     172.28.43.11 jomoore - "POST /notebook/api/execute HTTP/1.0"
[24/Jun/2017 22:32:20 -0500] decorators   ERROR    Error running <function execute at 0x7f5b45922140>
Traceback (most recent call last):
  File "/opt/hue/desktop/libs/notebook/src/notebook/decorators.py", line 81, in decorator
    return func(*args, **kwargs)
  File "/opt/hue/desktop/libs/notebook/src/notebook/api.py", line 160, in execute
    response = _execute_notebook(request, notebook, snippet)
  File "/opt/hue/desktop/libs/notebook/src/notebook/api.py", line 128, in _execute_notebook
    history.update_data(notebook)
AttributeError: 'NoneType' object has no attribute 'update_data'
[24/Jun/2017 22:32:42 -0500] access       INFO     172.28.43.11 jomoore - "POST /notebook/api/execute HTTP/1.0"
[24/Jun/2017 22:32:42 -0500] decorators   ERROR    Error running <function execute at 0x7f5b45922140>
Traceback (most recent call last):
  File "/opt/hue/desktop/libs/notebook/src/notebook/decorators.py", line 81, in decorator
    return func(*args, **kwargs)
  File "/opt/hue/desktop/libs/notebook/src/notebook/api.py", line 160, in execute
    response = _execute_notebook(request, notebook, snippet)
  File "/opt/hue/desktop/libs/notebook/src/notebook/api.py", line 128, in _execute_notebook
    history.update_data(notebook)
AttributeError: 'NoneType' object has no attribute 'update_data'

​

Jordan Moore 
Staff Consultant | Avalon Consulting, LLC 
M:  217 622-5993 
LinkedIn | Google+ | Twitter

-------------------------------------------------------------------------------------------------------------
This message (including any attachments) contains confidential information 
intended for a specific individual and purpose, and is protected by law. If 
you are not the intended recipient, you should delete this message. Any 
disclosure, copying, or distribution of this message, or the taking of any 
action based on it, is strictly prohibited.

[Romain Rigaux]

Does it still happen on a new query?

[Jordan Moore]

Not anymore, no.

Very strange...

I did get some intermittent errors in Beeswax upon query submission that says something along the lines of "document has no default value 'is_trashed', which upon research, is a field in the database but yet a syncdb plus migrate didn't immediately fix it. Same queries worked in the Notebook snippets, so that was my workaround for some time, but I haven't seen either mentioned error in Beeswax recently.

[Romain Rigaux]

This issue has several occurrences e.g. https://issues.cloudera.org/browse/HUE-4546 but we are not aware of new ones.

If you have a trace of the other issue, I could quick search it too.

On Sat, Jun 24, 2017 at 3:18 PM, Jordan Moore <moo...@avalonconsult.com> wrote: