Hue Webserver is getting down frequently

[Patrick S.]

Hey,

I am Patrick and I am new to CDP and any of it's components. Since yesterday I have trouble with my Hue Webserver. It is going down frequently. When i restart it, it is oke for a few seconds and then it dies again.

The Cloudera Manager says:

"The health test result for HUE_SERVER_WEB_METRIC_COLLECTION has become bad: The Cloudera Manager Agent is not able to communicate with this role's web server."

I dont know exactly where to look for the troublemaker, but the Hue Logs tell me this:

"7:25:37 AM
ERROR
exception
Internal Server Error: /desktop/debug/is_alive Traceback (most recent call last): File "/opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/hue/build/env/lib/python2.7/site-packages/Django-1.11.29-py2.7.egg/django/core/handlers/exception.py", line 41, in inner response = get_response(request) File "/opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/hue/build/env/lib/python2.7/site-packages/Django-1.11.29-py2.7.egg/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/hue/build/env/lib/python2.7/site-packages/Django-1.11.29-py2.7.egg/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/hue/build/env/lib/python2.7/site-packages/Django-1.11.29-py2.7.egg/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/hue/build/env/lib/python2.7/site-packages/Django-1.11.29-py2.7.egg/django/utils/decorators.py", line 184, in inner with self: File "/opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/hue/build/env/lib/python2.7/site-packages/Django-1.11.29-py2.7.egg/django/db/transaction.py", line 158, in __enter__ if not connection.get_autocommit(): File "/opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/hue/build/env/lib/python2.7/site-packages/Django-1.11.29-py2.7.egg/django/db/backends/base/base.py", line 385, in get_autocommit self.ensure_connection() File "/opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/hue/build/env/lib/python2.7/site-packages/Django-1.11.29-py2.7.egg/django/db/backends/base/base.py", line 213, in ensure_connection self.connect() File "/opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/hue/build/env/lib/python2.7/site-packages/Django-1.11.29-py2.7.egg/django/db/utils.py", line 94, in __exit__ six.reraise(dj_exc_type, dj_exc_value, traceback) File "/opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/hue/build/env/lib/python2.7/site-packages/Django-1.11.29-py2.7.egg/django/db/backends/base/base.py", line 213, in ensure_connection self.connect() File "/opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/hue/build/env/lib/python2.7/site-packages/Django-1.11.29-py2.7.egg/django/db/backends/base/base.py", line 189, in connect self.connection = self.get_new_connection(conn_params) File "/opt/cloudera/parcels/CDH-7.1.6-1.cdh7.1.6.p0.10506313/lib/hue/build/env/lib/python2.7/site-packages/Django-1.11.29-py2.7.egg/django/db/backends/postgresql/base.py", line 176, in get_new_connection connection = Database.connect(**conn_params) File "/opt/cloudera/cm-agent/service/hue/psycopg2/__init__.py", line 130, in connect conn = _connect(dsn, connection_factory=connection_factory, **kwasync) OperationalError: FATAL: remaining connection slots are reserved for non-replication superuser connections"

We didint made changes to our postgres db backend, so I dont think that this is the error i'm looking for. Can anyone help me to find out whats going on?

Best Patrick

[Ramanathan V]

check the memory  on nol level ?

--
To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+u...@cloudera.org.

[Patrick S.]

Hey Ramanathan,

what do you mean with nol level?

I think the Error i posted in my first response with the "perationalError: FATAL: remaining connection slots are reserved for non-replication superuser connections" isnt that misleading as i expected.

Yesterday Airflow was going live in our cluster and its db is located on the same host as the hive db. I think we could have a max_connections problem on postgres db server. I shut down airflow now and i'm looking whats happening in the next hour with the hue webserver.

Thanks !

[Sathish kumar Venkatesan]

Patrick,

seem like Hue is getting exception while opening database connection. and below connection slots exception is aslo  related to database connection pool. try to increase the db connection pool and try.

conn = _connect(dsn, connection_factory=connection_factory, **kwasync) OperationalError: FATAL: remaining connection slots are reserved for no

[Patrick S.]

hey guys, thanks for your help. it was the postgres db backend. we had only 100 connections allowed. -.-*