New plugin: Visible Review Commits

121 views
Skip to first unread message

Olivier Croquette

unread,
Jan 14, 2018, 2:02:22 PM1/14/18
to Repo and Gerrit Discussion
Hi everyone,

a common issue for new and even older Gerrit users is that active code reviews are not available automatically in their local repositories. The standard way to fetch them is cumbersome. Some tools give more comfort and even a GUI, but not everyone uses them. And some users even want to see automatically all active code reviews without manual action.

That's the reason why I wrote a small plugin that maintains the references to the last patch set of active changes in a normal namespace (refs/headsreview/).

https://github.com/ocroquette/gerrit-visible-review-commits

It's my first plugin, so feedback on the implementation is more than welcome.

Best regards
Olivier


Matthias Sohn

unread,
Jan 14, 2018, 4:59:01 PM1/14/18
to Olivier Croquette, Repo and Gerrit Discussion
On Sun, Jan 14, 2018 at 8:02 PM, Olivier Croquette <ocroq...@free.fr> wrote:
Hi everyone,

a common issue for new and even older Gerrit users is that active code reviews are not available automatically in their local repositories. The standard way to fetch them is cumbersome. Some tools give more comfort and even a GUI, but not everyone uses them. And some users even want to see automatically all active code reviews without manual action.

That's the reason why I wrote a small plugin that maintains the references to the last patch set of active changes in a normal namespace (refs/headsreview/).


so you get all open changes downloaded as result of clone or fetch ?
 
https://github.com/ocroquette/gerrit-visible-review-commits

It's my first plugin, so feedback on the implementation is more than welcome.

getting all refs in [1] can yield a huge result, we have some repositories with several 100k refs.
I'd try to start from querying Gerrit's list of open changes to reduce the number of refs which
need to be processed.


-Matthias

Olivier Croquette

unread,
Jan 14, 2018, 7:38:27 PM1/14/18
to Matthias Sohn, Repo and Gerrit Discussion
On 2018-1-14 22:58, Matthias Sohn wrote:
so you get all open changes downloaded as result of clone or fetch ?
Correct, that's the idea.

I forgot to mention that the integration with CI servers is also easier since they can see the active patch sets and trigger builds on them.


getting all refs in [1] can yield a huge result, we have some repositories with several 100k refs.
I'd try to start from querying Gerrit's list of open changes to reduce the number of refs which
need to be processed.
I see, thanks for the feedback. Unfortunately, I need the list of refs in reviews/... to find out which ones must be deleted.

JGit seems to provide a way to get only specific refs in RefDatabase.java:    public abstract Map<String, Ref> getRefs(String prefix)

I will try it out.

Luca Milanesio

unread,
Jan 15, 2018, 3:12:48 AM1/15/18
to Olivier Croquette, Matthias Sohn, Repo and Gerrit Discussion, Dave Borowitz

On 15 Jan 2018, at 00:38, Olivier Croquette <ocroq...@free.fr> wrote:

On 2018-1-14 22:58, Matthias Sohn wrote:
so you get all open changes downloaded as result of clone or fetch ?
Correct, that's the idea.

Yes, it's a good idea indeed.
The time for CI systems to run fetch for changes on Gerrit is typically very high due to the high number of advertised refs.

A different approach to the same problem is:

Instead of generating extra refs, which would make the advertised refs even larger, it filters out the "non-interesting" refs for a CI system.

The current implementation relies on Gerrit permission backend; however, I am planning to use a new ref filter extension exposed by Dave Borowitz exactly for this purpose.
You may want to take a similar path for your plugin, without having to crunch and generate a lot of extra refs.


I forgot to mention that the integration with CI servers is also easier since they can see the active patch sets and trigger builds on them.

getting all refs in [1] can yield a huge result, we have some repositories with several 100k refs.
I'd try to start from querying Gerrit's list of open changes to reduce the number of refs which
need to be processed.
I see, thanks for the feedback. Unfortunately, I need the list of refs in reviews/... to find out which ones must be deleted.

JGit seems to provide a way to get only specific refs in RefDatabase.java:    public abstract Map<String, Ref> getRefs(String prefix)

I will try it out.


--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

David Ostrovsky

unread,
Jan 15, 2018, 3:18:44 AM1/15/18
to Repo and Gerrit Discussion

Am Montag, 15. Januar 2018 09:12:48 UTC+1 schrieb lucamilanesio:


On 15 Jan 2018, at 00:38, Olivier Croquette <ocroq...@free.fr> wrote:

On 2018-1-14 22:58, Matthias Sohn wrote:
so you get all open changes downloaded as result of clone or fetch ?
Correct, that's the idea.

Yes, it's a good idea indeed.
The time for CI systems to run fetch for changes on Gerrit is typically very high due to the high number of advertised refs.

A different approach to the same problem is:

Instead of generating extra refs, which would make the advertised refs even larger, it filters out the "non-interesting" refs for a CI system.

The current implementation relies on Gerrit permission backend; however, I am planning to use a new ref filter extension exposed by Dave Borowitz exactly for this purpose.

Are you talking about this change that is still pending for review?

* [1] https://gerrit-review.googlesource.com/#/c/gerrit/+/133352

Luca Milanesio

unread,
Jan 15, 2018, 5:58:12 AM1/15/18
to David Ostrovsky, Repo and Gerrit Discussion
Sadly not merged yet ... let me put the review into my backlog.

lucamilanesio

unread,
Oct 24, 2018, 2:53:41 AM10/24/18
to Repo and Gerrit Discussion
The change for refs filtering isn't merged yet on Gerrit :-(
However, there will be a lot of value if I could share the gerrit-refsfilter plugin, so that Jenkins Slaves could have a big improvement on the clone phase :-)

What if I create the repository now on gerrit-review?

Name: modules/git-refs-filter
Description: Gerrit lib module to allow filtering out refs in the Git advertizing protocol phase.

Thanks for the feedback :-)

Luca.

David Ostrovsky

unread,
Oct 24, 2018, 3:58:14 AM10/24/18
to Repo and Gerrit Discussion

Am Mittwoch, 24. Oktober 2018 08:53:41 UTC+2 schrieb lucamilanesio:
The change for refs filtering isn't merged yet on Gerrit :-(
However, there will be a lot of value if I could share the gerrit-refsfilter plugin, so that Jenkins Slaves could have a big improvement on the clone phase :-)

What if I create the repository now on gerrit-review?

Name: modules/git-refs-filter
Description: Gerrit lib module to allow filtering out refs in the Git advertizing protocol phase.

+1.

Luca Milanesio

unread,
Oct 24, 2018, 5:14:21 AM10/24/18
to David Ostrovsky, Luca Milanesio, Repo and Gerrit Discussion

On 24 Oct 2018, at 09:58, David Ostrovsky <david.o...@gmail.com> wrote:


Am Mittwoch, 24. Oktober 2018 08:53:41 UTC+2 schrieb lucamilanesio:
The change for refs filtering isn't merged yet on Gerrit :-(
However, there will be a lot of value if I could share the gerrit-refsfilter plugin, so that Jenkins Slaves could have a big improvement on the clone phase :-)

What if I create the repository now on gerrit-review?

Name: modules/git-refs-filter
Description: Gerrit lib module to allow filtering out refs in the Git advertizing protocol phase.

+1.

Actually I saw now the the ref filtering has been moved into the PermissionBackend and thus ... *it is* the right time to ship the module :-)
It should now be super-fast because instead of going through each change and check permissions, I can simply filter the list ... and there you go !

Luca.



Thanks for the feedback :-)

Luca.

On Sunday, January 14, 2018 at 10:59:01 PM UTC+1, Matthias Sohn wrote:
On Sun, Jan 14, 2018 at 8:02 PM, Olivier Croquette <ocroq...@free.fr> wrote:
Hi everyone,

a common issue for new and even older Gerrit users is that active code reviews are not available automatically in their local repositories. The standard way to fetch them is cumbersome. Some tools give more comfort and even a GUI, but not everyone uses them. And some users even want to see automatically all active code reviews without manual action.

That's the reason why I wrote a small plugin that maintains the references to the last patch set of active changes in a normal namespace (refs/headsreview/).


so you get all open changes downloaded as result of clone or fetch ?
 
https://github.com/ocroquette/gerrit-visible-review-commits

It's my first plugin, so feedback on the implementation is more than welcome.

getting all refs in [1] can yield a huge result, we have some repositories with several 100k refs.
I'd try to start from querying Gerrit's list of open changes to reduce the number of refs which
need to be processed.


-Matthias

lucamilanesio

unread,
Nov 7, 2018, 3:28:39 AM11/7/18
to Repo and Gerrit Discussion


On Wednesday, October 24, 2018 at 10:14:21 AM UTC+1, lucamilanesio wrote:


On 24 Oct 2018, at 09:58, David Ostrovsky <david.o...@gmail.com> wrote:


Am Mittwoch, 24. Oktober 2018 08:53:41 UTC+2 schrieb lucamilanesio:
The change for refs filtering isn't merged yet on Gerrit :-(
However, there will be a lot of value if I could share the gerrit-refsfilter plugin, so that Jenkins Slaves could have a big improvement on the clone phase :-)

What if I create the repository now on gerrit-review?

Name: modules/git-refs-filter
Description: Gerrit lib module to allow filtering out refs in the Git advertizing protocol phase.

+1.

So far I got no objections in creating the repo ... doing it as we speak :-)

P.S. The module has been mentioned as well in the JenkinsWorld 2018 video about integrating Gerrit Code Review and Jenkins (see [1]) and thus I have to publish it somewhere, otherwise people will start complain they can't replicate what I've shown in the Demo in Nice last month :-(


Luca.
 

Actually I saw now the the ref filtering has been moved into the PermissionBackend and thus ... *it is* the right time to ship the module :-)
It should now be super-fast because instead of going through each change and check permissions, I can simply filter the list ... and there you go !

Luca.



Thanks for the feedback :-)

Luca.

On Sunday, January 14, 2018 at 10:59:01 PM UTC+1, Matthias Sohn wrote:
On Sun, Jan 14, 2018 at 8:02 PM, Olivier Croquette <ocroq...@free.fr> wrote:
Hi everyone,

a common issue for new and even older Gerrit users is that active code reviews are not available automatically in their local repositories. The standard way to fetch them is cumbersome. Some tools give more comfort and even a GUI, but not everyone uses them. And some users even want to see automatically all active code reviews without manual action.

That's the reason why I wrote a small plugin that maintains the references to the last patch set of active changes in a normal namespace (refs/headsreview/).


so you get all open changes downloaded as result of clone or fetch ?
 
https://github.com/ocroquette/gerrit-visible-review-commits

It's my first plugin, so feedback on the implementation is more than welcome.

getting all refs in [1] can yield a huge result, we have some repositories with several 100k refs.
I'd try to start from querying Gerrit's list of open changes to reduce the number of refs which
need to be processed.


-Matthias

--
--

More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss+unsubscribe@googlegroups.com.

lucamilanesio

unread,
Nov 7, 2018, 3:32:39 AM11/7/18
to Repo and Gerrit Discussion


On Wednesday, November 7, 2018 at 8:28:39 AM UTC, lucamilanesio wrote:


On Wednesday, October 24, 2018 at 10:14:21 AM UTC+1, lucamilanesio wrote:


On 24 Oct 2018, at 09:58, David Ostrovsky <david.o...@gmail.com> wrote:


Am Mittwoch, 24. Oktober 2018 08:53:41 UTC+2 schrieb lucamilanesio:
The change for refs filtering isn't merged yet on Gerrit :-(
However, there will be a lot of value if I could share the gerrit-refsfilter plugin, so that Jenkins Slaves could have a big improvement on the clone phase :-)

What if I create the repository now on gerrit-review?

Name: modules/git-refs-filter
Description: Gerrit lib module to allow filtering out refs in the Git advertizing protocol phase.

+1.

So far I got no objections in creating the repo ... doing it as we speak :-)

P.S. The module has been mentioned as well in the JenkinsWorld 2018 video about integrating Gerrit Code Review and Jenkins (see [1]) and thus I have to publish it somewhere, otherwise people will start complain they can't replicate what I've shown in the Demo in Nice last month :-(


Hitting a solid rock: when I try to create a repository on gerrit-review.googlesource.com I got:
Error 429: Too many requests from this client. Try again later.

Is there anyone at Google to help me?

Luca.

lucamilanesio

unread,
Dec 11, 2018, 11:29:12 AM12/11/18
to Repo and Gerrit Discussion


On Wednesday, November 7, 2018 at 8:32:39 AM UTC, lucamilanesio wrote:


On Wednesday, November 7, 2018 at 8:28:39 AM UTC, lucamilanesio wrote:


On Wednesday, October 24, 2018 at 10:14:21 AM UTC+1, lucamilanesio wrote:


On 24 Oct 2018, at 09:58, David Ostrovsky <david.o...@gmail.com> wrote:


Am Mittwoch, 24. Oktober 2018 08:53:41 UTC+2 schrieb lucamilanesio:
The change for refs filtering isn't merged yet on Gerrit :-(
However, there will be a lot of value if I could share the gerrit-refsfilter plugin, so that Jenkins Slaves could have a big improvement on the clone phase :-)

What if I create the repository now on gerrit-review?

Name: modules/git-refs-filter
Description: Gerrit lib module to allow filtering out refs in the Git advertizing protocol phase.

+1.

So far I got no objections in creating the repo ... doing it as we speak :-)

P.S. The module has been mentioned as well in the JenkinsWorld 2018 video about integrating Gerrit Code Review and Jenkins (see [1]) and thus I have to publish it somewhere, otherwise people will start complain they can't replicate what I've shown in the Demo in Nice last month :-(


Hitting a solid rock: when I try to create a repository on gerrit-review.googlesource.com I got:
Error 429: Too many requests from this client. Try again later.

Is there anyone at Google to help me?

Luca.


I finally managed to create the repository and pushed the first change for review at:

Comments and feedback are welcome :-)
Reply all
Reply to author
Forward
0 new messages