Two way sync between Gerrit and Github

[Jonathan "Duke" Leto]

Howdy,

Preface: I am new to Gerrit (but not Git), so go easy on me :)

I have a situation where I have a set of git repos (about 20) which
have a canonical location in Gerrit, but which are also mirrored to
Github.

All development happens in Gerrit and stuff is synced to Github weekly
or monthly.

As it turns out, there are a huge number of forks and pull requests on
Github that currently need to manually be migrated over via patches
(icky).

So my question is: Is anybody automatically syncing (via a bot or
cron) Github pull requests to Gerrit by making appropriate patchsets
for them?

Does Gerrit expose a web API where I can create new patchsets via code?

Thanks for any and all help!

Duke

--
Jonathan "Duke" Leto <jona...@leto.net>
209.691.DUKE // http://leto.net
NOTE: Personal email is only checked twice a day at 10am/2pm PST,
please call/text for time-sensitive matters.

[Shawn Pearce]

On Fri, Jan 27, 2012 at 12:52, Jonathan "Duke" Leto <jal...@gmail.com> wrote:
> Preface: I am new to Gerrit (but not Git), so go easy on me :)
>
> I have a situation where I have a set of git repos (about 20) which
> have a canonical location in Gerrit, but which are also mirrored to
> Github.
>
> All development happens in Gerrit and stuff is synced to Github weekly
> or monthly.
>
> As it turns out, there are a huge number of forks and pull requests on
> Github that currently need to manually be migrated over via patches
> (icky).

Not that I know of, but a way to import GitHub pull requests as Gerrit
changes would be very cool.

> So my question is: Is anybody automatically syncing (via a bot or
> cron) Github pull requests to Gerrit by making appropriate patchsets
> for them?
>
> Does Gerrit expose a web API where I can create new patchsets via code?

Web API? This is Git. We don't have web APIs. :-)

Patches can only be created by a Git push, either over SSH or HTTP...
but it has to be a send-pack/receive-pack stream going into the
git-receive-pack "command" or URL handler in the Gerrit server to
create new changes or attach patch sets to existing changes.

[lucamilanesio]

Just to track the status on this old 2012 thread on GitHub to Gerrit synch.

On Thursday, February 2, 2012 3:18:08 PM UTC, Shawn Pearce wrote:

On Fri, Jan 27, 2012 at 12:52, Jonathan "Duke" Leto <jal...@gmail.com> wrote:
> Preface: I am new to Gerrit (but not Git), so go easy on me :)
>
> I have a situation where I have a set of git repos (about 20) which
> have a canonical location in Gerrit, but which are also mirrored to
> Github.
>
> All development happens in Gerrit and stuff is synced to Github weekly
> or monthly.
>
> As it turns out, there are a huge number of forks and pull requests on
> Github that currently need to manually be migrated over via patches
> (icky).

Not that I know of, but a way to import GitHub pull requests as Gerrit
changes would be very cool.

That's my next step on the forthcoming GitHub plugin for Gerrit

https://gerrit-review.googlesource.com/#/admin/projects/plugins/github

There will be either manual or automatic pull-requests import into Gerrit Changes.

I would recommend the manual approach anyway (you list the pull-requests pending on the branches)

where you can decide which one you want to start to review or not.

Potentially there could be quite a lot of pull-requests (as anybody is free to fork and request) and you

may not necessarily want to create so much noise on your repo.

However I am introducing an automatic robot as well, that will blindly import everything that pops-up in GitHub.

 

> So my question is: Is anybody automatically syncing (via a bot or
> cron) Github pull requests to Gerrit by making appropriate patchsets
> for them?
>
> Does Gerrit expose a web API where I can create new patchsets via code?

Web API? This is Git. We don't have web APIs. :-)

Patches can only be created by a Git push, either over SSH or HTTP...
but it has to be a send-pack/receive-pack stream going into the
git-receive-pack "command" or URL handler in the Gerrit server to
create new changes or attach patch sets to existing changes.

The way I am planning to implement the Patch-Set creation would be:

a) Get the pull-request branch and squash all the commits together (otherwise one pull-request may result in multiple dependant changes)

b) Create a Change with the initial patch-set of the squashed commits

c) For every new commit / update on the pull-request, resquash everything together and submit a new patch-set

All the comments from GitHub to Gerrit and the other way around will be mirrored.

What do you think ? 

Luca. 

[Matthias Sohn]

I would prefer not to squash the commits coming from GitHub since I prefer to

review a series of dependent small changes over one big change and also this

is easier to understand for everybody since each commit in GitHub translate to

a change in Gerrit.

--

Matthias

[Luca Milanesio]

Hey Matthias,

thank you for your feedback, see below my comments.

The problem I see is the synchronisation of the Change status vs. Pull Request: if we allow a Pull Request to be split into multiple changes,

we cannot synch back to GitHub the overall status of merged / abandoned Changes to their associated Pull Requests.

To be honest with you I think that the Pull Request model is inherently flawed as communicate the wrong message to an Open Source project contributor:

"I've done a bunch of changes, if you like them pull and merge them" (GitHub approach) ... rather than "this is I want to propose to be contributed to the project: open for discussion" (Gerrit approach).

A "big Pull Request" is inherently a mistake and non-cooperative approach IMHO. Moreover we do not have guarantees that the individual commits in a pull-request make any sense at all :-O

I would rather then leave it to the choice of the person in Gerrit that is making the interactive conversion:

a. Squash everything into a Change OR

b. Create one Change per Commit in a Pull Request

It is unclear for me however how to track the status synch in case b. ... any idea ?

Luca.

[Fredrik Luthander]

Hi there Luca.

Is this plugin specific to github, or can it fetch "changes" from any git-URL too? Perhaps that's another plugin entirely.. Not sure which aim you have with this plugin yet. :)

Thanks!

--

Med vänlig hälsning / Best regards,

   Fredrik Luthander

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en
 
---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Luca Milanesio]

At the moment no, it will be tailored to GitHub as we will synch the pull request status as well, bi-directionally to GitHub.

Potentially a "pull style" replicator would be cool ;-)

... but would more likely to be an extension of the current replication plugin :-)

Luca.

[Matthias Sohn]

On Mon, Jun 24, 2013 at 12:23 PM, Luca Milanesio <luca.mi...@gmail.com> wrote:

having this choice would be useful 

 

It is unclear for me however how to track the status synch in case b. ... any idea ?

it seems GitHub pull requests only can track status on pull request level. I think this would mean

GitHub status update can only happen when all corresponding changes in Gerrit were accepted

(or rejected). If you only want to accept a few of the changes in the pull request this can't be

reflected in the GitHub fork without rewriting the corresponding branch in GitHub (or create a new one).

Maybe the decision should be left to the one reviewing the changes in Gerrit, so if a pull request contains

a patch series (multiple commits) ask the reviewer with every change he accepts if the pull request should

be considered done. When he accepted all patches in the patch series the pull request could be closed

automatically.

--

Matthias

[gjdev]

This would be a very welcome plugin!

Personally I wouldn't bother with (b), and just implement your initial suggestion (a). Imho a Github pull-request should implement a single feature/unit, and multiple commits in the request are (at least for how we use them) conceptually equal to amending gerrit change requests. On most projects I have worked with github pull requests more or less had the same scope as gerrit review requests. Adding commits to a pull request is usually done based on feedback/review of the request. It's also easy enough to close the request and ask the committer for something that has a smaller scope. I think things will quickly become very complex with (b), both in implementation of the plugin, and in workflow of managing gerrit-reviews and github-pulls.

[lucamilanesio]

GitHub plugin is getting closer, the following parts have been completed:

a) OAuth sign-in (either as primary Gerrit authentication OR as additional sign-in on the GitHub plugin)

b) Auto-create Gerrit projects and security

c) Clone and configure replication from Gerrit to GitHub

The plugin does not include CSS branding, *BUT* the UX is designed for being branded and shaped as desired.

I am tackling now the PULL request part, in theory the most complicated but in reality NOT: once you have set-up the GitHub authentication infrastructure and replication, it is really straightforward afterwards :-)

I would like to make it both interactive *AND* batch.

1. Interactive: you see the list of pull requests pending and you can select the ones to import and the strategy (1 pull request = 1 change; 1 pull request = N dependant changes)

2. Batch: a new GitHub replication plugin, branched from the current replication in Gerrit, to automatically work in "pull mode" with GitHub. Current replication plugin works *ONLY* in push mode and that wouldn't work with GitHub (changes can be made on GitHub as pull requests: you need to poll and pull rather than just pushing)

Any suggestions, ideas on the above ?

We should be around 1-2 weeks away from the FULL plugin to be released then :-)

(for the joy of the WikiMedia folks ;-) )

Luca.

[Thomas Broyer]

On Wednesday, September 18, 2013 11:13:51 AM UTC+2, lucamilanesio wrote:

GitHub plugin is getting closer, the following parts have been completed:

a) OAuth sign-in (either as primary Gerrit authentication OR as additional sign-in on the GitHub plugin)

b) Auto-create Gerrit projects and security

c) Clone and configure replication from Gerrit to GitHub

The plugin does not include CSS branding, *BUT* the UX is designed for being branded and shaped as desired.

I am tackling now the PULL request part, in theory the most complicated but in reality NOT: once you have set-up the GitHub authentication infrastructure and replication, it is really straightforward afterwards :-)

I would like to make it both interactive *AND* batch.

1. Interactive: you see the list of pull requests pending and you can select the ones to import and the strategy (1 pull request = 1 change; 1 pull request = N dependant changes)

So what would happen if you chose the 1PR=1change strategy upon merging? I suppose you'd end up with one commit in in the repo corresponding with the change and the PR is closed, but would it also be updated (force-push) and thus could have the "merged" badge (rather than just being "closed"); or would a comment be instead posted with the SHA1 of the merged commit? (or merge commit? depending on Gerrit's merge strategy)

 

2. Batch: a new GitHub replication plugin, branched from the current replication in Gerrit, to automatically work in "pull mode" with GitHub. Current replication plugin works *ONLY* in push mode and that wouldn't work with GitHub (changes can be made on GitHub as pull requests: you need to poll and pull rather than just pushing)

Any suggestions, ideas on the above ?

We should be around 1-2 weeks away from the FULL plugin to be released then :-)

(for the joy of the WikiMedia folks ;-) )

I'm looking forward to seeing it in action :)

[Luca Milanesio]

Hi Thomas,

my answers in-line:

On 18 Sep 2013, at 14:03, Thomas Broyer <t.br...@gmail.com> wrote:

On Wednesday, September 18, 2013 11:13:51 AM UTC+2, lucamilanesio wrote:

GitHub plugin is getting closer, the following parts have been completed:

a) OAuth sign-in (either as primary Gerrit authentication OR as additional sign-in on the GitHub plugin)

b) Auto-create Gerrit projects and security

c) Clone and configure replication from Gerrit to GitHub

The plugin does not include CSS branding, *BUT* the UX is designed for being branded and shaped as desired.

I am tackling now the PULL request part, in theory the most complicated but in reality NOT: once you have set-up the GitHub authentication infrastructure and replication, it is really straightforward afterwards :-)

I would like to make it both interactive *AND* batch.

1. Interactive: you see the list of pull requests pending and you can select the ones to import and the strategy (1 pull request = 1 change; 1 pull request = N dependant changes)

So what would happen if you chose the 1PR=1change strategy upon merging? I suppose you'd end up with one commit in in the repo corresponding with the change and the PR is closed, but would it also be updated (force-push) and thus could have the "merged" badge (rather than just being "closed"); or would a comment be instead posted with the SHA1 of the merged commit? (or merge commit? depending on Gerrit's merge strategy)

1PR = 1Change means that all the commits in the Pull Request will be squashed together and merged into Gerrit, according to the Gerrit merge strategy (merge, rebase, cherry-pick).

With regards to the Pull Request status on GitHub ... need to work it out the best approach there.

Suggestions are welcomed :-)

According to GitHub, just the merge and cherry-pick are supported:

https://help.github.com/articles/merging-a-pull-request

If another strategy is chosen on Gerrit which does not match the ones supported in GitHub, I guess a commit on GitHub with the submitted and merged commit on Gerrit would suffice.

Possibly then just trigger the close of the Pull request (https://help.github.com/articles/closing-a-pull-request)

What do you think ?

2. Batch: a new GitHub replication plugin, branched from the current replication in Gerrit, to automatically work in "pull mode" with GitHub. Current replication plugin works *ONLY* in push mode and that wouldn't work with GitHub (changes can be made on GitHub as pull requests: you need to poll and pull rather than just pushing)

Any suggestions, ideas on the above ?

We should be around 1-2 weeks away from the FULL plugin to be released then :-)

(for the joy of the WikiMedia folks ;-) )

I'm looking forward to seeing it in action :)

Actually you can even now :-)

As a "rule of development life" we always experiment what we are doing, eating our own dog-food: the "latest and greatest" version of the Gerrit with GitHub plugin can be used at:

http://gerrithub.io

We called it GerritHub because it is really Gerrit process on top of GitHub.

Luca.

[Shuang Wu]

Hi, Luca,

  Any updates? I am really interested in this plugin. I set up Gerrit in my previous job and really liked it, especially the clean git history without all the commits that address reviewer's comments. Github is used at my current job, (I tried to push for Gerrit, but didn't succeed). So if this bi-directional syncing of commits and comments can be done, I will return to Gerrit happily. Thanks for your work.

  Btw, I prefer the one github PR <=> one Gerrit Change mapping. It's a cleaner mental picture, at least for me.

Shuang

[Luca Milanesio]

Hi Shuang,

I will present the news at the forthcoming Gerrit User Summit 2014 @Mountain View, during my talk "Gerrit or GitHub? Take both!"

See: https://sites.google.com/site/gerritsummit2014/schedule

Are you joining the Summit ?

Luca.

-- 
-- 
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

--- 
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.