Installing Genie on Amazon AWS+EMR

[jose.r...@mercadolibre.com]

Hello,

We are willing to try Genie as a tool to deliver our queries to our EMR clusters running Hive, Spark and Presto. 

We found instructions at the wiki (https://github.com/Netflix/genie/wiki/Setup), followed most of them and now have a Genie instance running at an EC2 instance. But is not clear to us how to register EMR clusters to a Genie instance and get it to work. We are also not sure if we should install any additional services in order to have a full PROD deploy.

If someone can provide some insight / additional information on this procedure, it would be of great help to us.

Thanks!

[Tom Gianos]

Hi Jose,

I don't think you need any other services if you have a Genie 2 node and a database. Genie 3 will have the ability to include many other services for cluster robustness like leadership election, session persistence and security.

In terms of registering an EMR cluster did you get a chance to look at the Data Model wiki page? https://github.com/Netflix/genie/wiki/Data-Model It walks through an end to end example.

Also you can take a look at the docker container: https://github.com/Netflix-Skunkworks/zerotodocker/wiki/Genie-2.2.1 and the docker file to see how that was setup.

If you have any more specific questions we'll help as much as we can.

Thanks,

Tom

[Jose Rozanec]

Hi Tom!

We did not check the Data Model REST API before. We are checking it now and reviewing configurations.

Thanks!

José.

--
You received this message because you are subscribed to a topic in the Google Groups "genie" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/genieoss/Kk0OgobDtxQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to genieoss+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.