REQUEST: I have a request that would be very helpful to a number of us. Could you post onto your blog some idealized hue.ini files and state the conditions they are configured for. It certainly would be useful to have a reference for full HA with Kerberos and full SSL. Thanks in advanced.
Versions: CentOS 6.7, Java 1.7, Â CDH 5.7, Hue came with the yum install.Â
Install: I have installed HA for HDFS and YARN, Kerberos is working for all services, I installed SSL for the cluster using a Java Keystore/Truststore - I am using a self-signed cert in the truststore. This works for Hue. I am currently running a 12 node cluster on AWS. All of this is manually installed. There is no Cloudera Manager.
Situation: HDFS, YARN and MapReduce are working well from the command line. Beeline and Oozie are running from the command line. Hue comes up and I can log in, all of the services work but the Job Browser.Â
Error: Hue does not recognize the Resource Manager. It is just not communicating.
Diag: My working theory is everything should now be communicating in HTTPS. I made a mapping of all of the secure ports I used in the configuration files and applied them to hue.ini. I have now written over 20 different combinations of variables and read a number of tickets. This is now as clear as a brick wall.
Configurations: These are the current set of configs for Hadoop. Your corrections are welcomed.
[hadoop]
    # Configuration for HDFS NameNode
    # ------------------------------------------------------------------------
    [[hdfs_clusters]]
        # HA support by using HttpFs
        [[[default]]]
            # Enter the filesystem uri
            #fs_defaultfs=hdfs://localhost:8020
            fs_defaultfs=hdfs://mycluser
            hdfs_port=8020
            http_port=5007
            # NameNode logical name.
            logical_name=hdfs-ha
            # WKD Use WebHdfs/HttpFs as the communication mechanism.
            # Domain should be the NameNode or HttpFs host.
            # Default port is 14000 for HttpFs. Required for HA
            # WKD Change this if your HDFS cluster is Kerberos-secured
            ## security_enabled=false
            security_enabled=true
            # WKD In secure mode (HTTPS), if SSL certificates from YARN Rest APIs
            # have to be verified against certificate authority
            ## ssl_cert_ca_verify=True
            ssl_cert_ca_verify=False
            # Directory of the Hadoop configuration
            ## hadoop_conf_dir=$HADOOP_CONF_DIR when set or '/etc/hadoop/conf'
    # Configuration for YARN (MR2)
    # ------------------------------------------------------------------------
    [[yarn_clusters]]
        [[[default]]]
            # WKD Enter the host on which you are running the ResourceManager
            ## resourcemanager_host=localhost
            resourcemanager_host=master03.invalid
            # The port where the ResourceManager IPC listens on
            resourcemanager_port=8032
            # Whether to submit jobs to this cluster
            submit_to=True
            # Resource Manager logical name (required for HA)
            logical_name=rm1
            # WKD Change this if your YARN cluster is Kerberos-secured
            ## security_enabled=false
            security_enabled=true
            # WKD URL of the ResourceManager API
            # WKD URL of the ProxyServer API
            # WKD URL of the HistoryServer API
            # WKD In secure mode (HTTPS), if SSL certificates from YARN Rest APIs
            # have to be verified against certificate authority
            ## ssl_cert_ca_verify=True
            ssl_cert_ca_verify=False
        # HA support by specifying multiple clusters.
        # Redefine different properties there.
        # e.g.
        [[[ha]]]
            # WKD Enter the host on which you are running the ResourceManager
            ## resourcemanager_host=localhost
            resourcemanager_host=master04.invalid
            # WKD The port where the ResourceManager IPC listens on
            resourcemanager_port=8032
            # Un-comment to enable
            submit_to=True
            # WKD Resource Manager logical name (required for HA)
            logical_name=rm2
            # WKD Change this if your YARN cluster is Kerberos-secured
            # security_enabled=false
            security_enabled=true
            # WKD URL of the ResourceManager API
            # WKD URL of the ProxyServer API
            # WKD URL of the HistoryServer API
            # WKD In secure mode (HTTPS), if SSL certificates from YARN Rest APIs
            # have to be verified against certificate authority
            ## ssl_cert_ca_verify=True
            ssl_cert_ca_verify=False
Error messages from error.log
[27/May/2016 00:02:18 -0700] cluster    ERROR   RM default is not available, skipping it: ('Connection aborted.', error(111, 'Connection refused'))
Traceback (most recent call last):
 File "/usr/lib/hue/desktop/libs/hadoop/src/hadoop/cluster.py", line 223, in get_next_ha_yarncluster
  cluster_info = rm.cluster()
 File "/usr/lib/hue/desktop/libs/hadoop/src/hadoop/yarn/resource_manager_api.py", line 118, in cluster
  return self._execute(self._root.get, 'cluster/info', params=params, headers={'Accept': _JSON_CONTENT_TYPE})
 File "/usr/lib/hue/desktop/libs/hadoop/src/hadoop/yarn/resource_manager_api.py", line 160, in _execute
  response = function(*args, **kwargs)
 File "/usr/lib/hue/desktop/core/src/desktop/lib/rest/resource.py", line 98, in get
  return self.invoke("GET", relpath, params, headers=headers, allow_redirects=True)
 File "/usr/lib/hue/desktop/core/src/desktop/lib/rest/resource.py", line 79, in invoke
  urlencode=self._urlencode)
 File "/usr/lib/hue/desktop/core/src/desktop/lib/rest/http_client.py", line 163, in execute
  raise self._exc_class(ex)
RestException: ('Connection aborted.', error(111, 'Connection refused'))
[27/May/2016 00:02:18 -0700] cluster    ERROR   RM ha is not available, skipping it: ('Connection aborted.', error(111, 'Connection refused'))
"error.log" 674L, 65818C