Nomad Integration with AWS ALB?

[Rusty Ross]

I am wondering if there are conversations happening in regards to augmenting Nomad to integrate with Amazon's new ALB (Application Load Balancer). Specifically, I am talking about the ability of Nomad to natively register jobs into ALB Target Groups, in  very much the same way it currently is able to register/register jobs as Consul services.

Ideally, an ALB Target Group could be declared in a Nomad job definition and on run, Nomad would register the port (and optionally a health check) for each job instance it ran, by calling the AWS API to do so. On the flip side, when Nomad terminated jobs, it would be great if it could deregister them from the Target Group.

Is there currently any motion on this? If not, would a PR in this area be a welcome contribution?

Best,

Rusty

[Diptanu Choudhury]

Hi Rusty,

We are going to introduce client plugins in Nomad which would allow plugins to receive callbacks when a task starts/stops, etc. This would allow us to make load balancer plugins like the one you are suggesting, once that framework is in place it would be great to get a PR in to support AWS ALB.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/nomad/issues
IRC: #nomad-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Nomad" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nomad-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nomad-tool/67fa681c-a02d-4012-b2a7-bfe037dcc8e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Thanks,
Diptanu Choudhury

Web - www.linkedin.com/in/diptanu

Twitter - @diptanu

[ja...@fpcomplete.com]

On Monday, August 22, 2016 at 8:59:02 AM UTC-4, Diptanu Choudhury wrote:

Hi Rusty,

We are going to introduce client plugins in Nomad which would allow plugins to receive callbacks when a task starts/stops, etc. This would allow us to make load balancer plugins like the one you are suggesting, once that framework is in place it would be great to get a PR in to support AWS ALB.

That is really exciting to hear!

[Martin Atkins]

Hi Rusty,

This is a bit different than what you were asking about, but at work I've started to build a tool to sync healthy service endpoints from Consul into ALB target groups:

    https://github.com/saymedia/consul-ec2-alb

The warning in the README about it being very experimental is worth repeating (I literally just wrote it today and have only tested it on a toy consul/nomad deployment) but with a little more work it could get the end result you were (presumably) looking for here, automatically registering and deregistering targets in the load balancer. Just a little more indirect than what you were asking about, learning about the service state via the checks that Nomad registers in Consul, rather than Nomad directly integrating with the ALB itself.

[Justin Walz]

Hi Diptanu,

Which release is this feature expected to make it into? Looks like a good opportunity to solve some problems we have been having.

Thanks,

Justin

On Monday, August 22, 2016 at 5:59:02 AM UTC-7, Diptanu Choudhury wrote:

Hi Rusty,

We are going to introduce client plugins in Nomad which would allow plugins to receive callbacks when a task starts/stops, etc. This would allow us to make load balancer plugins like the one you are suggesting, once that framework is in place it would be great to get a PR in to support AWS ALB.

On Sun, Aug 21, 2016 at 7:35 PM, Rusty Ross <rusty...@full360.com> wrote:

I am wondering if there are conversations happening in regards to augmenting Nomad to integrate with Amazon's new ALB (Application Load Balancer). Specifically, I am talking about the ability of Nomad to natively register jobs into ALB Target Groups, in  very much the same way it currently is able to register/register jobs as Consul services.

Ideally, an ALB Target Group could be declared in a Nomad job definition and on run, Nomad would register the port (and optionally a health check) for each job instance it ran, by calling the AWS API to do so. On the flip side, when Nomad terminated jobs, it would be great if it could deregister them from the Target Group.

Is there currently any motion on this? If not, would a PR in this area be a welcome contribution?

Best,

Rusty

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/nomad/issues
IRC: #nomad-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Nomad" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nomad-tool+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nomad-tool/67fa681c-a02d-4012-b2a7-bfe037dcc8e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alex Dadgar]

Hey Justin,

We are targeting the next major release, 0.6.

Thanks,

Alex

To unsubscribe from this group and stop receiving emails from it, send an email to nomad-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nomad-tool/b33441a7-f0e6-4d2a-9a53-0c0af25cf9a2%40googlegroups.com.

[Justin Walz]

Hi Alex,

Thanks for the response. Does that mean 0.5 will be released soon? :) 

[Timothy Perrett]

This would actually be really useful for implementing auto-scalling engines, and lots of other funky stuff. 

For the OP: we solve this outside of the scheduler. Typically routing traffic into services on your cluster requires quite a bit of jumping around with the networking. For example, if you are doing dynamic port remapping, and a container moves its backend host, how do you dynamically (without loosing connections on the ELB) migrate to the new backend? We elected to handle this outside of the scheduler, as a sort of "environmental integration" inside our orchestration engine, so even with the plugins (FWIW, i think they are going to be very helpful) its not a complete solution; you the user will still have to handle a range of potentially awkward cases. 

Cheers

-- T

On Monday, 22 August 2016 05:59:02 UTC-7, Diptanu Choudhury wrote:

Hi Rusty,

We are going to introduce client plugins in Nomad which would allow plugins to receive callbacks when a task starts/stops, etc. This would allow us to make load balancer plugins like the one you are suggesting, once that framework is in place it would be great to get a PR in to support AWS ALB.

On Sun, Aug 21, 2016 at 7:35 PM, Rusty Ross <rusty...@full360.com> wrote:

I am wondering if there are conversations happening in regards to augmenting Nomad to integrate with Amazon's new ALB (Application Load Balancer). Specifically, I am talking about the ability of Nomad to natively register jobs into ALB Target Groups, in  very much the same way it currently is able to register/register jobs as Consul services.

Ideally, an ALB Target Group could be declared in a Nomad job definition and on run, Nomad would register the port (and optionally a health check) for each job instance it ran, by calling the AWS API to do so. On the flip side, when Nomad terminated jobs, it would be great if it could deregister them from the Target Group.

Is there currently any motion on this? If not, would a PR in this area be a welcome contribution?

Best,

Rusty

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/nomad/issues
IRC: #nomad-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Nomad" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nomad-tool+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nomad-tool/67fa681c-a02d-4012-b2a7-bfe037dcc8e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.