Genie and hdfs

[rob mancuso]

Thx for contributing this!

Has anyone tried using genie with yarn computer clusters configured to point to remote hdfs clusters (instead of s3)... for internal (non-cloud deployments)?

From what I've read, hadoop is supposed to be compatible within major versions (ie yarn cluster on 2.7 talking to hdfs on 2.6), but curious if anyone has tried this configuration or has any thought on the idea.

Thx!

[Tom Gianos]

Hey Rob,

You can definitely use Genie with HDFS in a non-S3/cloud configuration. We actually do this for our Docker example which you can look at here: https://github.com/Netflix-Skunkworks/zerotodocker/wiki/Genie .

After you look over the example if you still have questions let us know.

Thanks,

Tom

[rob mancuso]

Thx Tom, much appreciated