Job Request 'dependencies' from different stores

[Yogesh]

Hi,
In https://netflix.github.io/genie/docs/3.0.4/rest/#_job_request 'dependencies', the examples for the 'dependencies' attribute are S3 locations. Is the data store pluggable, so if I like to use something like Azure Blob Store, I can provide the keys with the 'job request'.
Thanks,
Yogesh.

[Tom Gianos]

Hi Yogesh,

This is indeed pluggable. What you need to do is implement the FileTransfer interface. You can see existing implementations for local, http and s3. Once the implementation is on the classpath it is discovered at runtime. We assume a resource can be identified by a unique URI scheme like s3:// or file:// in order to determine which file transfer implementation to use when downloading or uploading a file. Not familiar with the Azure blob store but I assume it has something similar to s3:// for URI's?

If you do implement something for Azure PR's are always welcome!

Tom