Hue failing & needs to be restarted to submit hive query.

[ambuj...@infochimps.com]

Sometimes, Hive queries submitted through Hue are failing, and causing Hue to become unresponsive or crash. When we restart the Hue server, then only we are once again able to submit the HIVE queries. Why is this happening? 

We are using CDH4.6.0. The Error logs are below:

Please suggest me if any one have got the resolution on this issue. 

/var/log/hue/error.log

[22/Jul/2014 01:04:20 +0000] wsgiserver ERROR WSGI () error: [('SSL routines', 'SSL23_WRITE', 'ssl handshake failure')]
Traceback (most recent call last):
File "/usr/share/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1296, in run
conn.communicate()
File "/usr/share/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1206, in communicate
req.simple_response("408 Request Timeout")
File "/usr/share/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 618, in simple_response
self.wfile.sendall("".join(buf))
File "/usr/share/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1131, in sendall
return self._safe_call(False, super(SSL_fileobject, self).sendall, *args, **kwargs)
File "/usr/share/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1083, in _safe_call
return call(*args, **kwargs)
File "/usr/share/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 753, in sendall
bytes_sent = self.send(data)
File "/usr/share/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1134, in send
return self._safe_call(False, super(SSL_fileobject, self).send, *args, **kwargs)
File "/usr/share/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1113, in _safe_call
raise FatalSSLAlert(*e.args)

FatalSSLAlert: [('SSL routines', 'SSL23_WRITE', 'ssl handshake failure')]

/var/log/hue/beeswax_server.out

14/08/01 11:54:06 WARN avro.AvroSerdeUtils: Encountered AvroSerdeException determining schema. Returning signal schema to indicate problem
org.apache.hadoop.hive.serde2.avro.AvroSerdeException: Neither avro.schema.literal nor avro.schema.url specified, can't determine table schema
at org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.determineSchemaOrThrowException(AvroSerdeUtils.java:66)
at org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils.determineSchemaOrReturnErrorSchema(AvroSerdeUtils.java:87)
at org.apache.hadoop.hive.serde2.avro.AvroSerDe.initialize(AvroSerDe.java:59)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:250)
at org.apache.hadoop.hive.ql.metadata.Partition.getDeserializer(Partition.java:251)
at org.apache.hadoop.hive.ql.metadata.Partition.initialize(Partition.java:217)
at org.apache.hadoop.hive.ql.metadata.Partition.(Partition.java:107)
at org.apache.hadoop.hive.ql.metadata.Hive.getPartitionsByNames(Hive.java:1720)
at org.apache.hadoop.hive.ql.optimizer.ppr.PartitionPruner.pruneBySequentialScan(PartitionPruner.java:364)
at org.apache.hadoop.hive.ql.optimizer.ppr.PartitionPruner.prune(PartitionPruner.java:219)
at org.apache.hadoop.hive.ql.optimizer.pcr.PcrOpProcFactory$FilterPCR.process(PcrOpProcFactory.java:112)
at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:89)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:87)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:124)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:101)
at org.apache.hadoop.hive.ql.optimizer.pcr.PartitionConditionRemover.transform(PartitionConditionRemover.java:86)
at org.apache.hadoop.hive.ql.optimizer.Optimizer.optimize(Optimizer.java:102)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:8261)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:459)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:349)
at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.checkedCompile(BeeswaxServiceImpl.java:247)
at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.compile(BeeswaxServiceImpl.java:200)
at com.cloudera.beeswax.BeeswaxServiceImpl$2.run(BeeswaxServiceImpl.java:830)
at com.cloudera.beeswax.BeeswaxServiceImpl$2.run(BeeswaxServiceImpl.java:823)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1438)
at com.cloudera.beeswax.BeeswaxServiceImpl.doWithState(BeeswaxServiceImpl.java:772)
at com.cloudera.beeswax.BeeswaxServiceImpl.query(BeeswaxServiceImpl.java:822)
at com.cloudera.beeswax.api.BeeswaxService$Processor$query.getResult(BeeswaxService.java:915)
at com.cloudera.beeswax.api.BeeswaxService$Processor$query.getResult(BeeswaxService.java:899)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:244)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)

at java.lang.Thread.run(Thread.java:662)

/var/log/hue/runcpserver.log

[22/Jul/2014 01:04:20 +0000] wsgiserver ERROR WSGI () error: [('SSL routines', 'SSL23_WRITE', 'ssl handshake failure')]
Traceback (most recent call last):
File "/usr/share/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1296, in run
conn.communicate()
File "/usr/share/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1206, in communicate
req.simple_response("408 Request Timeout")
File "/usr/share/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 618, in simple_response
self.wfile.sendall("".join(buf))
File "/usr/share/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1131, in sendall
return self._safe_call(False, super(SSL_fileobject, self).sendall, *args, **kwargs)
File "/usr/share/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1083, in _safe_call
return call(*args, **kwargs)
File "/usr/share/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 753, in sendall
bytes_sent = self.send(data)
File "/usr/share/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1134, in send
return self._safe_call(False, super(SSL_fileobject, self).send, *args, **kwargs)
File "/usr/share/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1113, in _safe_call
raise FatalSSLAlert(*e.args)
FatalSSLAlert: [('SSL routines', 'SSL23_WRITE', 'ssl handshake failure')]

 

[Romain Rigaux]

What is your version of OpenSSL?

It could be:
https://bitbucket.org/cherrypy/cherrypy/issue/589/ssl-errors

Romain

To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+u...@cloudera.org.

[ambuj...@infochimps.com]

Hi Romain,

The openssl version is - OpenSSL 1.0.1g 7 Apr 2014

Thanks & regards,

Ambuj

[Mohit Pannalal]

I know this is old thread but looking for some clue on this error. I am facing this error with HUE 3.9 version.

And have openssl-1.0.1e-42.el6_7.4.x86_64 installed in the HUE hosts

Regards,

Mohit

[Mohit Pannalal]

Okay. I was able to overcome this problem by using hue close_sessions command. I use 0 days and it closed all the hung sessions which helped a lot.

Putting this command on crontab so that it keeps closing old sessions and allows new sessions.

Reference :  http://gethue.com/hadoop-tutorial-hive-and-impala-queries-life-cycle/

On Wednesday, 13 August 2014 13:31:58 UTC+5:30, ambuj...@infochimps.com wrote: