Replication Plugin

[Lekan Swansons]

Hi all,

I am trying to configure gerrit replication with github. I am able to log in to gerrit using oauth. I created a test repo in github which i import into gerrit using the replication plugin.

*replication.config file

ubuntu@ubuntu:~/gerrit_home$ cat etc/replication.config
[remote "Lekanswanson"]
    url = https://github.com/${name}.git
    push = refs/*:refs/*
    projects = Lekanswanson/FinalTestRepo

I am able to pull from gerrit and github from the cli and ssh keys are setup

I also tried to follow this tutorial to no avail 

https://gerrit.googlesource.com/plugins/replication/+doc/master/src/main/resources/Documentation/config.md

I simply want anything i push into gerrit master to also get pushed to github. \

regards

LA

[Antoine Musso]

Hello,

Our Gerrit https://gerrit.wikimedia.org/ does replicate to https://github.com/wikimedia/ The configuration we have for GitHub:

[remote "github"]
  authGroup = mediawiki-replication
  createMissingRepositories = false
  maxRetries = 50
  mirror = false
  projects = ^(?:(?!apps\\/android\\/).)*$
  projects = ^(?:(?!apps\\/ios\\/).)*$
  push = +refs/heads/*:refs/heads/*
  push = +refs/tags/*:refs/tags/*
  remoteNameStyle = dash
  replicatePermissions = false
  rescheduleDelay = 15
  url = g...@github.com:wikimedia/${name}

The authGroup is a Gerrit group you have to create and does not need to have any member. We have made it not visible to register user. The sole purpose of this group is to be added to the repository you want to replicate by granting the group the Read permission on refs/tags/* and refs/heads/*.  The default is to replicate everything to all remotes so I guess you are set.

Secondly we have disabled createMissingRepositories cause GitHub doesn't have the ssh Gerrit command to create a repository. If I understand it properly that is only possible when the destination is a Gerrit instance.

There is also adminUrl but it is also to create a repository on a Gerrit target.

Thus for Github replication, you have to manually create the repository. Since Gerrit keeps retrying the replication, it will eventually catch up and be able to push the references.

There should be a log file in $GERRIT_SITE/logs/replication_log which is handy to find out what is happening.

-- 
Antoine "hashar" Musso
Release Engineering

[Lekan Swansons]

Hi

I followed the configuration you provided and to test it, i added my github repo to gerrit. \

I then cloned the repo into a workspace using git clone. Created a simple script, did a push and review and merged to gerrit master.

The script however still doesnt show up on github.

Here are the modified settings to the replication.config

[remote "GITHUB_USER"]
    authGroup = Lekanswanson

    createMissingRepositories = false
    maxRetries = 50

    push = refs/*:refs/*
    url = https://g...@github.com:GITHUB_USER/${name}.git

    remoteNameStyle = dash
    replicatePermissions = false
    rescheduleDelay = 15

    projects = ^FinalTestRepo

I also created a group for Lekanswanson 

The repositories already exist in git and github so im not sure what step im missing.

regards,

LA

[Lekan Swansons]

Hi 

From replication log i was getting this error

ubuntu@ubuntu:~/gerrit_workspace/FinalTestRepo$ cat ~/gerrit_home/logs/replication_log
[2022-05-10 02:34:59,382] Group "Lekanswanson" not recognized, removing from authGroup [CONTEXT request="SSH" ]
[2022-05-10 02:56:21,720] Group "Lekanswanson" not recognized, removing from authGroup [CONTEXT request="SSH" ]
[2022-05-10 02:56:38,080] Group "Lekanswanson" not recognized, removing from authGroup [CONTEXT request="SSH" ]
[2022-05-10 02:57:46,696] Group "Lekanswanson" not recognized, removing from authGroup [CONTEXT request="SSH" ]
[2022-05-10 02:58:40,057] Group "ubuntu" not recognized, removing from authGroup [CONTEXT request="SSH" ]

I have created the right group and when i reload the plugin i dont get any errors.

Replication is still not observed in github

here is the replication config file updated

[remote "Lekanswanson"]

    authGroup = Lekanswanson
    createMissingRepositories = false

    mirror = false
    maxRetries = 50

    push = +refs/heads/*:refs/heads/*
    push = +refs/tags/*:refs/tags/*

    url = g...@github.com:Lekanswanson/${name}

    remoteNameStyle = dash
    replicatePermissions = false
    rescheduleDelay = 15
    projects = ^FinalTestRepo

regards

LA

[Lekan Swansons]

Hi, 

Do you maybe have any idea what I could be missing, this is the final step i need to complete and any help would be appreciated.

On Monday, May 9, 2022 at 9:02:06 PM UTC+1 Antoine Musso wrote:

[Antoine Musso]

Le 10/05/2022 à 05:04, Lekan Swansons a écrit :
> Hi
>

You can probably drop AuthGroup entirely, from the doc:

> By default, replicates without group control, i.e. replicates
everything to all remotes.

Then that would replicate everything.

Maybe the replication plugin reached the maximum number of retries (you
have set maxRetries = 50) and thus no more attempt to replicate? You can
manually trigger a replication over the ssh admin console:

ssh -p 29418 <us...@your.gerrit.hostname> replication start --now --wait
FinalTestRepo

Which will start replicating FinalTestRepo repository immediately
(--now) and give you some interactive feedback (--wait).

Example:

ssh -p 29418 gerrit.wikimedia.org replication start --now --wait
mediawiki/extensions/examples
Replicate mediawiki/extensions/examples ref ..all.. to github.com,
Succeeded! (OK)
Replicate mediawiki/extensions/examples ref ..all.. to
gerrit2001.wikimedia.org, Succeeded! (OK)
Replication of mediawiki/extensions/examples ref ..all.. completed to 2
nodes,
----------------------------------------------
Replication completed successfully!

[Lekan Swansons]

Hi Antoine

You are an absolute legend. I managed to get it to work thank you so much. I had to tweak one or two things but once i was able to get the error messages it was easy to figure out the issue and fix it.

regards

LA

[Antoine Musso]

Le 11/05/2022 à 13:23, Lekan Swansons a écrit :
> Hi Antoine
>
> You are an absolute legend. I managed to get it to work thank you so
> much. I had to tweak one or two things but once i was able to get the
> error messages it was easy to figure out the issue and fix it.

Amazing thank you and congratulations!

Out of interest for others that might have issue with replication: what
was the issue and what did you change to resolve it?

[Lekan Swansons]

Hi 

So for my setup I already had a repo from github which i imported and attempted to replicate. 

I got a basic setting but when I reload the plugin in ran start --all or start --all url I was not getting any issues

It wasn't until I ran it with --wait that i could see what was wrong. 

First issue the url wasn't setup correctly. It was duplicating the repo name ie git@github:user/user-project

To fix this I had to remove the $user for the url so url was g...@github.com:${name}. and projects was user/project for specific project

I also had to change the remoteNameStyle from dash to forward slash as i noticed the /got replaced with a - for me

Finally when i added the authGroup it didnt work but this could be because i dont have the group added to the project as reviewer. It wasn't the default gerrit group which is added to all projects.

Removing authGroup made it work.

Also im not sure if in the [remote "github"] it makes a difference if its github or your repo name but for me i set it to github and it works.

Most important thing is to run the ssh -p port host host@ip start --now with the --wait to be able to see the output of the command. This can help debugging.

regards

Lekan

[Lekan Swansons]

Here is the configuration im using. 

[remote "github"]

    createMissingRepositories = false
    mirror = false
    maxRetries = 50
    push = +refs/heads/*:refs/heads/*
    push = +refs/tags/*:refs/tags/*

    url = g...@github.com:${name}
    remoteNameStyle = forwardslash

    replicatePermissions = false
    rescheduleDelay = 15

    projects = Lekanswanson/FinalTestRepo

[Antoine Musso]

Hello Lekan,

Thank you very much for the detailed debugging report! I am very happy to see --wait has helped diagnose the issue.

Antoine Musso

[Lekan Swansons]

For this command

Most important thing is to run the ssh -p port host host@ip start --now with the --wait to be able to see the output of the command.

I made a slight mistake

run --> ssh -p "port" "username@ip" replication start --now --wait