Re: [disco-dev] Blacklisting and reblancing

[Prashanth Mundkur]

On 00:39 Fri 03 May, guil...@lomig.net wrote:
> Hello,
>
> We have a 15 (old) nodes cluster, to which we added 15 (new) nodes, the
> idea being to switch the 15 old nodes off. We thus currently have a
> 30-nodes cluster.
>
> As a first try, I only blacklisted 3 nodes. For what I understood,
> blacklist for ddfs + disco followed by a few runs of the GC should migrate
> all data away from the old nodes.
>
> After about 50 GC runs, the nodes still do not show as green in the web UI.
> I did not find anything relevant in the logs.

Hmm, you should see messages that log the number of blobs pending
replication during each run. I'm assuming the lager log settings are
the default ones. Which version of Disco are you running?

> My questions are thus:
> - is the the right way of doing it? (i already did that in the past with
> success)

Yes, this is the right way.

> - is there a better way? I am happy enough to get dirty to get it working
> if needed.

The replication could be made a little aggressive. Currently, it just
tries to create one new replica per blob per GC run. The code is
written such so that we could easily create more, but I'd like to be
sure that everything else is working well first.

--prashanth