New project proposal: aws-kinesis-events

syntonyze

unread,

Mar 24, 2021, 6:49:19 AM3/24/21

to Repo and Gerrit Discussion

Hi all,

As part of our efforts to make Gerrit more cloud-native[1], GerritForge recently developed an implementation of the events-broker[2] that allows ingestion and consumption of events over AWS kinesis streams[3].

There are many different domains in which this plugin is useful.

A couple worth mentioning:

Maintain the consistency of Gerrit nodes in a multi-site deployment.
Provide reliable events integration to CI/CD systems by removing the need for permanent SSH connections and allowing Gerrit and the CI/CD systems to live in different locations / AZs / sites.

The kinesis-events plugin is currently hosted by gerrithub[4] and if you think this will be useful to the community, we would like to move the project to gerrit-review.googlesource.com.

Proposal:

Project name: plugins/aws-kinesis-events

Project description: Events Broker API for sending Gerrit Code Review events over AWS kinesis

What do you think?

Thanks,

Tony

[1]https://gitenterprise.me/2021/01/04/2021-whats-cooking-in-gerritforge/

[2]https://gerrit.googlesource.com/modules/events-broker/

[3]https://aws.amazon.com/kinesis/

[4]https://review.gerrithub.io/plugins/gitiles/GerritForge/kinesis-events/+/refs/heads/master

Luca Milanesio

unread,

Mar 24, 2021, 6:56:43 AM3/24/21

to Repo and Gerrit Discussion, Luca Milanesio

Hi Tony,

Thanks for sharing this, it looks a very interesting plugin indeed.

On 24 Mar 2021, at 10:49, syntonyze <synt...@gmail.com> wrote:

Hi all,
As part of our efforts to make Gerrit more cloud-native[1], GerritForge recently developed an implementation of the events-broker[2] that allows ingestion and consumption of events over AWS kinesis streams[3].

There are many different domains in which this plugin is useful.
A couple worth mentioning:

Maintain the consistency of Gerrit nodes in a multi-site deployment.
Provide reliable events integration to CI/CD systems by removing the need for permanent SSH connections and allowing Gerrit and the CI/CD systems to live in different locations / AZs / sites.

The latter is very useful indeed when you have, for instance, Gerrit on-premises and Jenkins CI/CD on the Cloud in AWS.

You *could* of course use also Kafka or RabbitMQ, but the issue si that you would have *yet another thing to manage* in addition to Gerrit. Supporting a cloud-native PubSub is a great idea.

Any plans for GCloud also?

The kinesis-events plugin is currently hosted by gerrithub[4] and if you think this will be useful to the community, we would like to move the project to gerrit-review.googlesource.com.

Proposal:
Project name: plugins/aws-kinesis-events
Project description: Events Broker API for sending Gerrit Code Review events over AWS kinesis

What do you think?

+1

Any plans to amend AWS-Gerrit to use Kinesis instead of Kafka?

Luca.

Thanks,

Tony

[1]https://gitenterprise.me/2021/01/04/2021-whats-cooking-in-gerritforge/
[2]https://gerrit.googlesource.com/modules/events-broker/
[3]https://aws.amazon.com/kinesis/
[4]https://review.gerrithub.io/plugins/gitiles/GerritForge/kinesis-events/+/refs/heads/master

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/repo-discuss/CACv40UwhJXSRjFapRzjPrDm5evcCL7Vw0qje%2BD7FpBmaAz2p-w%40mail.gmail.com.

Marcin Czech

unread,

Mar 24, 2021, 8:59:07 AM3/24/21

to Luca Milanesio, Tony, Repo and Gerrit Discussion

Hi Luca,

You *could* of course use also Kafka or RabbitMQ, but the issue si that you would have *yet another thing to manage* in addition to Gerrit. Supporting a cloud-native PubSub is a great idea.
Any plans for GCloud also?

Yes, there is ongoing work on GCloud PubSub events plugin. I'm in the middle of proposing new repository similar to the one proposed by @Tony

Marcin

To view this discussion on the web visit https://groups.google.com/d/msgid/repo-discuss/97F9056F-7E07-4F7D-B348-A0E2F8F61528%40gmail.com.

syntonyze

unread,

Mar 24, 2021, 9:22:35 AM3/24/21

to Marcin Czech, Luca Milanesio, Repo and Gerrit Discussion

Yep, having aws-gerrit to use Kinesis instead of (or as well as) Kafka is on the radar too.
Kafka is notoriously fiddly to set up, tune and maintain and KMS is quite pricy compared to Kinesis.

For Gerrit installations deployed in AWS, it could make a lot of sense to stream over Kinesis and it is likely to integrate with infrastructure management solutions that are already in place.

To view this discussion on the web visit https://groups.google.com/d/msgid/repo-discuss/CAL3eZx_TOEBPh1fMa2W7kDakSZO5SpqFEb76w-tqyxXnAJ2P%3DA%40mail.gmail.com.

Luca Milanesio

unread,

Mar 24, 2021, 9:27:23 AM3/24/21

to Tony, Luca Milanesio, Marcin Czech, Repo and Gerrit Discussion

+1

Also, most of the times the Gerrit admins have *lots of stuff* on their plate, having yet another PubSub system like Kafka to manage may be a problem if a company doesn’t have it in-house already.