Fwd: How much time in the "left" state

[Armon Dadgar]

Forgot to CC the list.

Armon Dadgar

Sent from my iPhone

Begin forwarded message:

From: Armon Dadgar <armon....@gmail.com>
Date: September 4, 2014 at 11:57:18 AM PDT
To: Laurent Raufaste <anal...@glop.org>
Subject: Re: How much time in the "left" state

Not quite. A mode in either the failed or left state will stick around for 72h before getting reaped. We try to reconnect to nodes in the failed state but not those in the left state. Using force-leave can be a way to stop reconnecting to known dead nodes.

Armon Dadgar

Sent from my iPhone

On Sep 4, 2014, at 11:49 AM, Laurent Raufaste <anal...@glop.org> wrote:

Thanks for you answer.

So if I read this correctly:

"Consul periodically tries to reconnect to "failed" nodes in case it is a network partition. After some configured amount of time (by default 72 hours), Consul will reap "failed" nodes and stop trying to reconnect. The force-leave command can be used to transition the "failed" nodes to "left" nodes more quickly."

And your addition:
"but the short of it is Consul keeps them around for ~72h for book keeping, and then automatically removes them"

This means that nodes stay as "failed" for 72h and transition to "left"

Then nodes stay as "left" for 72h and disappear.

Is that right?

On Thursday, September 4, 2014 2:25:46 PM UTC-4, Armon Dadgar wrote:

Hey Laurent,

Nope not doing anything wrong! I’ve updated the docs for a node lifecycle with 0.4, so when

that ships the documentation will be there as well, but the short of it is Consul keeps them

around for ~72h for book keeping, and then automatically removes them.

I wouldn’t worry about them! We will be adding a tunable to change that time interval at some

point, I know there is an open ticket for it.

Best Regards,

Armon Dadgar

From: Laurent Raufaste <ana...@glop.org>
Reply: Laurent Raufaste <ana...@glop.org>>
Date: September 3, 2014 at 10:33:17 PM
To: consu...@googlegroups.com <consu...@googlegroups.com>>
Subject:  How much time in the "left" state

Hi,

How long do consul stays in the "left" state before really leaving the cluster memory? (i.e. not being listed anymore by the members command)

Our story:

We are using AWS EC2 spot instances a lot, and we are constantly losing instances, that's not a problem for us, they just get automatically replaced later.

I started deploying consul on a bunch of them, and when an instance dies, consul is marked as failed.

If I force-leave them when legit (instance is dead), they are in the "left" state.

At this rate, if there is no way to remove them, I'll have thousands in the "left" state.

Am I doing something wrong here?

Thanks!

--
You received this message because you are subscribed to the Google Groups "Consul" group.
To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Consul" group.
To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.