Gerrit multi site configuration

[Jigar R]

Hello Gerrit community, 

Thank you for guiding me for building a proof-of-concept for gerrit multi-site. We are thinking to use it for production. I have following questions in regards to production deployment:

Setup:

We are aiming for case#7 as mentioned at https://gerrit.googlesource.com/plugins/multi-site/+/master/DESIGN.md 

2 gerrit v3.2.6 hosts in separate data centers ( one with read-write permission and another with read-only)

As of now, we use following plugins

• websession-broker
• zookeeper-refdb
• Kafka-events
• multi-site
• replication 

Questions:

1. How to configure one gerrit instance as read-only? As of now, in my proof-of-concept both gerrit instances are in read-write mode.

    Should I use read-only plugin https://gerrit.googlesource.com/plugins/readonly/ ? Or should I configure notedb.config with write=false?

2. Currently, I look at dashboard and count number of projects on each instances to determine whether they are in-sync or not. Is there a better way to do this? 

3. I have following config on both Gerrit instances. There is no plugin configuration page for kafka-events. Not sure about, how will plugin deal with kafka unavailability.

$ cat gerrit.config

[gerrit]

.....

.....

[plugin "kafka-events"]

    sendAsync = true

    bootstrapServers = kafka:9092

    groupId = instance-1

    numberOfSubscribers = 6

    pollingIntervalMs = 1000

    enableAutoCommit = true

    autoCommitIntervalMs = 1000

    autoOffsetReset = latest

--

Thanks,

Jigar R

[Luca Milanesio]

On 17 Feb 2021, at 19:29, Jigar R <jigarra...@gmail.com> wrote:

Hello Gerrit community, 

Thank you for guiding me for building a proof-of-concept for gerrit multi-site. We are thinking to use it for production.

Thanks for sharing this, that looks amazing !

I have following questions in regards to production deployment:

Setup:

We are aiming for case#7 as mentioned at https://gerrit.googlesource.com/plugins/multi-site/+/master/DESIGN.md 

2 gerrit v3.2.6 hosts in separate data centers ( one with read-write permission and another with read-only)

That’s a good start I would say.

Are you then planning to go upwards to more advanced multi-site models?

As of now, we use following plugins

• websession-broker

This isn’t mandatory for a stage #7

• zookeeper-refdb

This isn’t mandatory for a stage #7

• Kafka-events
• multi-site
• replication 

All those three are required.

Questions:

1. How to configure one gerrit instance as read-only? As of now, in my proof-of-concept both gerrit instances are in read-write mode.

    Should I use read-only plugin https://gerrit.googlesource.com/plugins/readonly/ ? Or should I configure notedb.config with write=false?

You define that in your HAProxy policies.

If you want to enforce the RO operations, you can also add the readonly plugin as you mentioned.

2. Currently, I look at dashboard and count number of projects on each instances to determine whether they are in-sync or not. Is there a better way to do this? 

The multi-site plugin publishes a metric for that:

plugins_multi_site_multi_site_subscriber_subscriber_replication_status_sec_behind

If the above metric is zero, means the instance is up-to-date, if that number is positive, the instance is behind.

3. I have following config on both Gerrit instances. There is no plugin configuration page for kafka-events.

The configuration doc is:

https://gerrit.googlesource.com/plugins/kafka-events/+/refs/heads/master/src/main/resources/Documentation/config.md

Not sure about, how will plugin deal with kafka unavailability.

$ cat gerrit.config

[gerrit]

.....

.....

[plugin "kafka-events"]

    sendAsync = true

    bootstrapServers = kafka:9092

^^^^ you need to specify ALL the bootstrap servers there, to manage Kafka availability.

    groupId = instance-1

    numberOfSubscribers = 6

    pollingIntervalMs = 1000

    enableAutoCommit = true

    autoCommitIntervalMs = 1000

    autoOffsetReset = latest

Luca.

--

Thanks,

Jigar R

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/repo-discuss/64477faa-2f3f-4d86-b1a5-ebafad68be66n%40googlegroups.com.

[Jigar R]

Hi Luca,

How can I access this metric? Is it though some command line or REST endpoint?

[Luca Milanesio]

Gerrit metrics can be collected using one of the reporters plugins:

- Prometheus

- JMX

- Cloudwatch

- Graphite

See the list at:

https://gerrit-review.googlesource.com/admin/repos/q/filter:metrics-reporter

HTH

Luca.

To view this discussion on the web visit https://groups.google.com/d/msgid/repo-discuss/d4457d70-49d6-42e9-93bb-396754d2e44fn%40googlegroups.com.

[Jigar R]

This is great information. It takes some time to get plugins in my company. Is it possible to see this information without plugin? 

[Luca Milanesio]

Nope, I believe you should get your company to improve their change-management process, as exposing metrics is a primary feature of any production system and they shouldn’t block or slowdown the adoption of such plugins.

Luca.

To view this discussion on the web visit https://groups.google.com/d/msgid/repo-discuss/8a879926-5d9c-4a5a-a5c8-a2c0be224130n%40googlegroups.com.