Hue on EMR

[Oved]

Hi,

I am launching EMR cluster release 5.3.1 with Hive 2.1.1 and Hue 3.11.

I tried to follow the instruction in this page http://gethue.com/introducing-s3-support-in-hue/.

I am launching the cluster through python script using boto3 with the following configuration Json:

[

     {

      "Classification": "core-site",

      "Properties": {

        "fs.s3a.awsAccessKeyId":"<aws key>",

        "fs.s3a.awsSecretAccessKey": "<aws secret key>"

        }

    },

    {

      "Classification": "hive-site",

      "Properties": {

        "hive.metastore.warehouse.dir":"s3://<bucket_name>/<hive-folder>",

        "javax.jdo.option.ConnectionURL": "jdbc:mysql://<rds-url>:3306/hivedb?createDatabaseIfNotExist=true",

        "javax.jdo.option.ConnectionDriverName": "org.mariadb.jdbc.Driver",

        "javax.jdo.option.ConnectionUserName": "<db_user>",

        "javax.jdo.option.ConnectionPassword": "db_pass",

        "hive.exec.scratchdir":"/hive_temp/",

        "hive.exec.stagingdir" : "${hive.exec.scratchdir}/${user.name}/.staging",

        "hive.exec.dynamic.partition.mode":"nonstrict",

        "hive.exec.parallel":"true",

        "hive.exec.compress.intermediate":"true",

        "hive.optimize.index.filter":"true",

        "hive.optimize.index.groupby":"true",

        "hive.cluster.delegation.key.update-interval":"31536000000",

        "hive.cluster.delegation.token.renew-interval":"31536000000",

        "hive.cluster.delegation.token.max-lifetime":"31536000000"

        }

    },

    {

  "Classification": "hue-ini",

  "Properties": {},

  "Configurations": [

    {

      "Classification": "desktop",

      "Properties": {"user_access_history_size":"50",

                     "time_zone":"Europe/Berlin"

                     },

      "Configurations": [

        {

          "Classification": "database",

          "Properties": {

            "name": "hue_db",

            "user": "hue_user",

            "password": "hue_pass",

            "host": "<rds_host>",

            "port": "3306",

            "engine": "mysql"

          },

          "Configurations": []

        }

      ]

    },

    ## HUE AWS

    {

      "Classification": "aws",

      "Properties": {},

      "Configurations": [

      {

          "Classification": "aws_accounts",

          "Properties": {},

         "Configurations": [

      {     "Classification": "default",

             "Properties": {"allow_environment_credentials": "False",

                            "region": "eu-central-1"}

           }

          ]

          }]

        }

      ]

    }

  ]

This gives me back an error:

botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the RunJobFlow operation: Classification 'aws_accounts' is not valid for parent classification 'aws'. Did I nested the son incorrectly?

In addition if I remove the Hue AWS part, the cluster is launched without errors but when logging into Hue there is an error for misconfiguration:

Hive  - Failed to access Hive warehouse: s3://<my_bucket>/<hive_directory>

Also when going into Query editor, there is an error for "Could not connect to <master-node-ip>:1000".

Thanks in advance for your help.

[John Cheng]

I have the same problem. Hue with s3 works fine in EMR 4.X without extra configurations.

However, when it comes to EMR 5.X, the function seems to be unsupported.

Oved於 2017年3月7日星期二 UTC+8上午3時16分14秒寫道：

[Romain Rigaux]

It might be that http://gethue.com/introducing-s3-support-in-hue/ is not being configured properly by EMR 5

--
You received this message because you are subscribed to the Google Groups "Hue-Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+unsubscribe@cloudera.org.