gerrit migration to new company, advice needed

[ze_paredes]

Dear Gerrit experts,

Our system (master and four mirrors, version 2.7) should migrate to another company (changes to new LDAP, IPs, domain, email, users_ID, groups, etc).
TMK, out of the obvious configuration changes in the ~/review_site/etc files and mysql, the following components should be updated:

• Update new user accounts and emails in tables: accounts, account_external_ids
• Update the meta/config branch with the new domain groups in all git (~2000) repositories
• Update all repo manifests with new URL
  

For the first point I'm planing to ask the users to use the "Register new email" feature, from there the email can be used to find out the new account_id and update the reviewdb tables

For the second point the plan is to create a new "pre-migration" branch from refs/meta/config and run a script to substitute the old groups names with the new ones, and then the very migration day just merge the pre-migration branch to refs/meta/config

I'm a bit worried on the open code changes, but guessing the internal account_id will perfectly do the trick.

Finally I'm assuming the replication plugin will automatically sort out the update of the mirrors, both git repos and database

Can you please share some advice? Do you think we're missing something or the action plan are wrong?

Many thanks in advance!

Jose

[Shawn Pearce]

On Fri, Dec 13, 2013 at 8:17 PM, ze_paredes <jose.par...@gmail.com> wrote:
> Our system (master and four mirrors, version 2.7) should migrate to another
> company (changes to new LDAP, IPs, domain, email, users_ID, groups, etc).
> TMK, out of the obvious configuration changes in the ~/review_site/etc files
> and mysql, the following components should be updated:
>
> Update new user accounts and emails in tables: accounts,
> account_external_ids
> Update the meta/config branch with the new domain groups in all git (~2000)
> repositories
> Update all repo manifests with new URL

Seems like a well thought out plan.

> For the first point I'm planing to ask the users to use the "Register new
> email" feature, from there the email can be used to find out the new
> account_id and update the reviewdb tables

Nice trick. :-)

It does require the user to be able to get the link at the new address
and still login to Gerrit with the old account.

> For the second point the plan is to create a new "pre-migration" branch from
> refs/meta/config and run a script to substitute the old groups names with
> the new ones, and then the very migration day just merge the pre-migration
> branch to refs/meta/config

Yes, good plan.

> I'm a bit worried on the open code changes, but guessing the internal
> account_id will perfectly do the trick.

Correct these link on account_id. Keep that stable and the changes will be fine.

> Finally I'm assuming the replication plugin will automatically sort out the
> update of the mirrors, both git repos and database

It only does Git and not the database. You may need to make sure all
repositories exist on the mirrors. If I remember it does creation by
watching for a trigger in the server that is fired when the server
makes a new repository. After the server has the repository existing,
it assumes that exists on all mirrors and won't create if missing
later.

But you are correct that it will copy the new refs/meta/config.

> Can you please share some advice? Do you think we're missing something or
> the action plan are wrong?

Seems sane to me. We have done a few migrations between servers and
shifted from one group system to another, and one account system to
another, and this is roughly what we did. Match users using email
address from account_external_ids table, relying in some cases on the
"mailto:" records created by "Register Email Address". Write new
refs/meta/config branches ahead of time and merge them in on migration
day. It does work.

I can't share my programs for either step because they are heavily
tied to internal things I can't share. By the time I extract something
useful that I can share, it would be this emai. :-(

The the refs/meta/config update we actually wrote a Makefile. I
generated a list of targets for each project and wrote a rule to
convert "*.new_config: *.old_config" or something. The rule made a new
Git repository locally, fetched refs/meta/config, checked out
FETCH_HEAD, updated the groups file, committed it, then pushed it back
to refs/meta/new-config for flag day.

The reason I used Makefile was then I ran it with `make -j32` and it
only took a few minutes on my workstation, instead of several hours. A
different target was used to do the final merge and push on migration
day, and again we ran that `make -j32`. Each time a project was pushed
successfully it created the file make was expecting; projects that
failed did not create that file, allowing the rule to run again the
next time we fired up make. I think we had like 2 failures so these
were easy to find on the second run as it was the only two targets
make ran, and we fixed the issue, and re-ran to finish. :-)

[Nicholas Mucci]

At $OLD_DAYJOB we acquired a company and brought their users into our main Gerrit system with their old company email addresses and inconsistently formatted usernames.  Eventually we had to change email addresses and standardize usernames, and went with processes much like those you've described.  It was a fairly painless procedure for us.

-Nick

[ze_paredes]

Hi Shawn and Nicholas,

First of all BIG thanks for your kind advice!

I'm just waiting for our internal expert to review/detail the plan, we may return with some extra question :-)

Happy New Year!

Jose

[ze_paredes]

Hi again,

Just realized the migration downtime can be reduced close to cero!

What if we already now push the new access rules to the "meta/config" pointing to the new ldap groups?

Since the rules are "invalid" gerrit will just skip them and write a WARN message in the logs, am I wrong?

In general, the very migration day we just stop gerrit, update the LDAP authentication parameters and start gerrit

(... the database update should have it's own section, indeed)

Thanks,

Jose

Den lördagen den 14:e december 2013 kl. 18:34:10 UTC+1 skrev Shawn Pearce: