Deregister a service on health check fail

532 views
Skip to first unread message

Andy Gayton

unread,
Jul 14, 2015, 6:11:53 PM7/14/15
to consu...@googlegroups.com
1/ I'm pretty certain this isn't currently possible?
2/ Are there any plans to add this ability?

Thanks!

David Birdsong

unread,
Jul 14, 2015, 6:16:08 PM7/14/15
to consu...@googlegroups.com
+1 

I find myself writing a lot of service shutdown code where it's entire purpose is so easily subverted by a bug in my program or OOM killer or any and all ways that a process goes down ungracefully.

I'd love it if services could specify a bad-health-expiry or something.

Scott MacLellan

unread,
Jul 15, 2015, 12:29:52 PM7/15/15
to consu...@googlegroups.com
This is definitely possible right now using watches/events.  I have this working in a small prototype I am building.

Setup an event so that when the check fails you can deregister the service. If you want to deregister locally on the node running the service you can then fire an event on the target node which would then deregister locally.

If you go down this path watch out for false failures that would deregister the service too early (i.e. a process needs time to start up and may have bad healthchecks at the start). To mitigate rapid failures I am using a kv to track how many times a failure has been reported and then deregistering after multiple failures. For my usecase I can also switch to a TTL based healthcheck and avoid these rapid failures.

Armon Dadgar

unread,
Jul 15, 2015, 9:36:57 PM7/15/15
to consu...@googlegroups.com, Scott MacLellan
I believe we have a ticket open for this as well. I think the idea is to allow a TTL to be provided
with a service, such that if its failing for a long enough period of time, it is automatically reaped out.

Best Regards,
Armon Dadgar
--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/consul/issues
IRC: #consul on Freenode
---
You received this message because you are subscribed to the Google Groups "Consul" group.
To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/consul-tool/1266bae7-371f-47da-b5e8-eb5c5e99d26e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Nicholas Capo

unread,
Aug 6, 2015, 3:20:42 PM8/6/15
to consu...@googlegroups.com, Scott MacLellan
Catching up on old mail.

@Armon Do you have a link to the ticket?



Cameron Davison

unread,
Aug 6, 2015, 4:04:53 PM8/6/15
to consu...@googlegroups.com, Scott MacLellan
Reply all
Reply to author
Forward
0 new messages