The Gerrit Code Review ESC meeting minutes have been published at:
Luca.
On Tue, Mar 16, 2021 at 9:35 AM Sven Selberg <sven.s...@axis.com> wrote:
>
>
>
> On Monday, March 15, 2021 at 10:32:56 PM UTC+1 lucamilanesio wrote:
>>
>> The Gerrit Code Review ESC meeting minutes have been published at:
>> https://www.gerritcodereview.com/2021-03-09-esc-minutes.html
>
>
> Regarding "Gerrit events rewrite and GCloud pub/sub notifications".
> Where can I find a design document for this overhaul of the events system?
The phrasing " beginning of the initiative of having Gerrit notes
exported as events" makes it sound as if there is a big initiative
starting, but the change linked as well as
https://gerrit-review.googlesource.com/c/gerrit/+/298962 is all there
is to it.
googlesource.com is a deployment that is very light on
backend-plugins, and we always ask our customers to go through the
REST API instead, which this mechanism provides. There is no further
overhaul planned, at least not from our side.
To me at least, hints that there is some sort of rewrite of the events that is planned or in progress.And if there exists no such initiative I'm a bit confused about these statements.
BRSven
--
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--
Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en
---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/repo-discuss/243f7e17-cedb-42ea-a4f9-c9d43d2c75dfn%40googlegroups.com.
On Tue, Mar 16, 2021 at 12:05 PM Sven Selberg <sven.s...@axis.com> wrote:
>> >> The Gerrit Code Review ESC meeting minutes have been published at:
>> >> https://www.gerritcodereview.com/2021-03-09-esc-minutes.html
>> >
>> >
>> > Regarding "Gerrit events rewrite and GCloud pub/sub notifications".
>> > Where can I find a design document for this overhaul of the events system?
>>
>> The phrasing " beginning of the initiative of having Gerrit notes
>> exported as events" makes it sound as if there is a big initiative
>> starting, but the change linked as well as
>> https://gerrit-review.googlesource.com/c/gerrit/+/298962 is all there
>> is to it.
>>
>> googlesource.com is a deployment that is very light on
>> backend-plugins, and we always ask our customers to go through the
>> REST API instead, which this mechanism provides. There is no further
>> overhaul planned, at least not from our side.
With regards to the design document, there was one written by Alice which is now largely obsolete anyway.That’s the reason why it was abandoned.
With regards to the design document, there was one written by Alice which is now largely obsolete anyway.That’s the reason why it was abandoned.Why I'm really asking is that:* I would be most interested in taking part in design-discussions and implementation of a new event system.
* I like Han-wen's idea of a "something happened" trigger, at least for Change-events.Potentially this could simplify the event pipeline and make all current Change-events triggers superfluous.At the end of the "potential" future event pipe there would be a ChangeEvent-generator that maps"something-happened" events into Change-Events.
To view this discussion on the web visit https://groups.google.com/d/msgid/repo-discuss/b4f6efbe-9c32-4419-b9f0-be40dada893dn%40googlegroups.com.
And that's not counting the extra computational effort from the event-streamconsumers that previously could filter events based on a local string comparison.You would off course negate many of these downsides if you populated the"something-happened" event with some cheap properties like [type, project, branch],__which, now that I think about it, you haven't claimed that you wont. __/Sven
if the internal event system were also based on ref update
notifications + meta_diff, that might be an improvement.
I'm adding Patrick in case he has further thoughts.
Maybe this discussion is better suited to the next ESC meeting?
--
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--
Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en
---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/repo-discuss/a9b780e9-9892-4fbc-916c-c9dc2cf4896en%40googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/repo-discuss/CAM7sg%3D3Ac-Jfcp-ZH3cL8-n8%2BvMm7ta1_g3SyJ1%2BjSbRSYxMkg%40mail.gmail.com.
>
> In extreme cases, their may be hundreds or thousands of consumers of events,
> and having basic information such as a change's project and branch can be
> essential to creating a scalable system. Since adding that info into the
> original event is cheap and it allows for greater scalablility, that is likely
> why that info was there in the first place. Removing that information seems
> like a potential regression and a move towards a less scalable system than the
> current one.
>
> A third point which may also be overlooked, is latency. If a consumer has to
> turn around and make another request to the server, that could add unwanted
> latency for consumers which are across the globe. I'm not sure if that matters
> to anyone?
Your last point is very important for us, because we are multi-site and the current architecture is an AP system, which is missing the ‘C’ part of the global consistency.
That means that when the CI system receives the event, it may NOT know from which instance it was generated and therefore would have no idea *which* Gerrit’s REST-API to call.
If you just call *any Gerrit* you may actually get a 404, because the change hasn’t been propagated yet.
This is actually happening *also* on *.gs.com and we can clearly see that from the Gerrit-CI builds: at times, even if the Checks API are reporting that a change needs verification, the associated REST-API to Gerrit are failing because the change hasn’t been replicated yet to that node.
Having instead the payload *inside* the notification message, or at least the most important parts of It that are needed for the CI to do his job, it would allow an AP system to work properly.
Anyway, we do need a design document where we put use-cases in writing and discuss organically on them :-)
Having instead the payload *inside* the notification message, or at least the most important parts of It that are needed for the CI to do his job, it would allow an AP system to work properly.
Anyway, we do need a design document where we put use-cases in writing and discuss organically on them :-)Wouldn't an online (3 hour or so) conference to vent thoughts and ideas for the event rewrite be a good thing?
I have some ideas that are extremely simplified and that I haven't thought through:* Write some sort of framework that can parse "something happened" events that can be used by event publishing plugins.
* Rip out the current "event-structure" (com.google.gerrit.server.events, com.google.gerrit.server.data, ?) from core into a lib (legacy-events)
* Remove stream-events ssh command from core Gerrit
* Write a plugin ("legacy-events") that uses the "something happened" framework and maps this to the the legacy-events lib to construct legacy events and implement a stream-events command equivalent.
* The legacy-events lib could also be used by other event-publishing plugins that are interested in keeping backwards compatibility.
Luca.
>
> -Martin
>
> --
> The Qualcomm Innovation Center, Inc. is a member of Code
> Aurora Forum, hosted by The Linux Foundation
>
> --
> --
> To unsubscribe, email repo-discuss...@googlegroups.com
> More info at http://groups.google.com/group/repo-discuss?hl=en
>
> ---
> You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/repo-discuss/1667594.b6kNENyT7j%40mfick-lnx.
--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en
---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/repo-discuss/87ec183f-16cd-47e6-8426-502de6740cfcn%40googlegroups.com.
On Thu, Mar 18, 2021 at 10:29 AM Sven Selberg <sven.s...@axis.com> wrote:
>> Your last point is very important for us, because we are multi-site and the current architecture is an AP system, which is missing the ‘C’ part of the global consistency.
>> That means that when the CI system receives the event, it may NOT know from which instance it was generated and therefore would have no idea *which* Gerrit’s REST-API to call.
>>
>> If you just call *any Gerrit* you may actually get a 404, because the change hasn’t been propagated yet.
>>
>> This is actually happening *also* on *.gs.com and we can clearly see that from the Gerrit-CI builds: at times, even if the Checks API are reporting that a change needs verification, the associated REST-API to Gerrit are failing because the change hasn’t been replicated yet to that node.
>
>
> We have managed to trigger similar behavior with a single-primary setup.
> 1. CI system reacts on event from RabbitMQ
> 2. a micro-service (with limited data) tries to GET the change with a revision SHA1 of latest patchset.
> This round-trip was fast enough so that the index wasn't updated yet so Gerrit couldn't identify the change with the commit sha1 and returned 404.
>
> So the CI resiliency problem is not unique to the push-poll approach of "something happened" and/or multi-primary systems.
If you GET the change using the project~changenum as identifier, the
index is not involved.