yarn_cluster.SECURITY_ENABLED.get() ... 'NoneType' object has no attribute 'SECURITY_ENABLED'

[Jordan Moore]

[13/Jul/2017 18:22:22 -0500] views        ERROR    Error in config validation by Sqoop: 'NoneType' object has no attribute 'SECURITY_ENABLED'
Traceback (most recent call last):
  File "/opt/hue/desktop/core/src/desktop/views.py", line 547, in _get_config_errors
    for confvar, error in validator(request.user):
  File "/opt/hue/apps/sqoop/src/sqoop/conf.py", line 43, in config_validator
    if yarn_cluster.SECURITY_ENABLED.get() and not os.path.exists(SQOOP_CONF_DIR.get()):
AttributeError: 'NoneType' object has no attribute 'SECURITY_ENABLED'

As a result of this error, I am unable to go to the "desktop/dump_config" page, but the Check Config is telling me that the ResourceManager can't be contacted. 

Basically, I tried to enable YARN HA, and can't quite determine where I went wrong since I have the same config file in a test and prod cluster, and the test cluster works fine... 

The "rm1" labeled ResourceManager of YARN is in standby mode in prod, so I tried making it the [default] configured RM in Hue, but that didn't seem to help 

My hue.ini is Puppet managed, and it looks like this. All the variables are making it into the generated file, so unless I have a typo in the prod hostnames, I am confused where to go from here.  

  [[yarn_clusters]]

    [[[default]]]
      # Enter the host on which you are running the ResourceManager
      resourcemanager_host=<%= @yarn_rm1_host %>

      # The port where the ResourceManager IPC listens on
      resourcemanager_port=<%= @yarn_rm1_port %>

      # Whether to submit jobs to this cluster
      submit_to=<%= @submit_to_yarn %>

      # Resource Manager logical name (required for HA)
      logical_name=<%= @yarn_rm1_logical_name %>

      # Change this if your YARN cluster is Kerberos-secured
      security_enabled=<%= @yarn_kerberos_enabled %>

      # URL of the ResourceManager API
      resourcemanager_api_url=<%= @yarn_rm1_api_url %>

      # URL of the ProxyServer API
      proxy_api_url=<%= @yarn_proxy_api_url %>

      # URL of the HistoryServer API
      history_server_api_url=<%= @yarn_history_server_api_url %>

      # URL of the Spark History Server
      spark_history_server_url=<%= @spark_history_server_url %>

      # In secure mode (HTTPS), if SSL certificates from YARN Rest APIs
      # have to be verified against certificate authority
      ## ssl_cert_ca_verify=True

    # HA support by specifying multiple clusters.
    # Redefine different properties there.
    # e.g.

    [[[ha]]]
      # Resource Manager logical name (required for HA)
      logical_name=<%= @yarn_rm2_logical_name %>

      resourcemanager_host=<%= @yarn_rm2_host %>
      resourcemanager_port=<%= @yarn_rm2_port %>

      # Un-comment to enable
      submit_to=<%= @submit_to_yarn %>

      # URL of the ResourceManager API
      resourcemanager_api_url=<%= @yarn_rm2_api_url %>

      history_server_api_url=<%= @yarn_history_server_api_url %>
      proxy_api_url=<%= @yarn_rm2_proxy_api_url %>
      spark_history_server_url=<%= @spark_history_server_url %>
      security_enabled=<%= @yarn_kerberos_enabled %>

[Romain Rigaux]

Is

submit_to=True

?

If yes, it would be because https://github.com/cloudera/hue/blob/master/desktop/libs/hadoop/src/hadoop/cluster.py#L213 does not pick any RM for some reason

--
You received this message because you are subscribed to the Google Groups "Hue-Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+unsubscribe@cloudera.org.

[Jordan Moore]

Yes, it is. Sorry, was rushed when writing that. Here's the fully (sanitized) templated values.

Does capitalization matter? Shouldn't since bool('true') == True

[[yarn_clusters]]

    [[[default]]]

      # Enter the host on which you are running the ResourceManager

      resourcemanager_host=node-rm2

      # The port where the ResourceManager IPC listens on

      resourcemanager_port=8032

      # Whether to submit jobs to this cluster

      submit_to=true

      # Resource Manager logical name (required for HA)

      logical_name=rm2

      # Change this if your YARN cluster is Kerberos-secured

      security_enabled=false

      # URL of the ResourceManager API

      resourcemanager_api_url=http://node-rm2:8088

      # URL of the ProxyServer API

      proxy_api_url=http://node-rm2:8088

      # URL of the HistoryServer API

      history_server_api_url=http://hist-server:19888

      # URL of the Spark History Server

      spark_history_server_url=http://node-rm2:18080

      # In secure mode (HTTPS), if SSL certificates from YARN Rest APIs

      # have to be verified against certificate authority

      ## ssl_cert_ca_verify=True

    # HA support by specifying multiple clusters.

    # Redefine different properties there.

    # e.g.

    [[[ha]]]

      # Resource Manager logical name (required for HA)

      logical_name=rm1

      resourcemanager_host=node-rm1

      resourcemanager_port=8032

      # Un-comment to enable

      submit_to=true

      # URL of the ResourceManager API

      resourcemanager_api_url=http://node-rm1:8088

      history_server_api_url=http://hist-server:19888

      proxy_api_url=http://node-rm1:8088

      spark_history_server_url=http://node-rm2:18080

      security_enabled=false

To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+u...@cloudera.org.

[Jordan Moore]

I hate to be that guy, but it seems to have magically fixed itself when

1. changing the case to True (which gets overwritten every few minutes by Puppet)

2. I noticed I had an old psuedo-distributed.ini file in the conf folder. I deleted this in favor of the hue.ini that is being managed. 

Thanks for the assist, regardless!