Request to create plugins/sync-index project on gerrit-review

[Hugo Arès]

Hi

We have a couple of internal plugins that we would like to open source. So far, we kept them to ourselves because we were not sure if there would be interest in such plugins but I guess that open sourcing them is the only way to know.

Sync-index is the first one we would like to open source. It allows to synchronize secondary indexes between two Gerrit instances sharing the same repo and database.

That plugin is part of our active/passive master cluster solution. Long term, we want to switch to an external secondary index (i.e. Elasticsearch) but in the meantime, sync-index allows us to keep both master indexes up to date so we can failover between servers anytime without having to worry about an out-of-date index.

Thank you for considering this project creation request.

Hugo

[Luca Milanesio]

Hi Hugo, thanks for sharing it :-)

Edwin is about to create a project right now (he is actually sit next to me :-) )

Luca.

-- 
-- 
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

--- 
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Luca Milanesio]

Edwin has just done it:

https://gerrit-review.googlesource.com/#/admin/projects/plugins/sync-index

Thanks.

Luca.

[Bassem Rabil]

Submitting the initial version of this plugin is pending on a change in Gerrit core [1]. There has a been a discussion on this change about the impact of the proposed extension point on the online reindexer, and we probably need one of the maintainers to have a look and review this change.

[1] https://gerrit-review.googlesource.com/#/c/72607/

Thanks

Bassem

On Wednesday, November 11, 2015 at 12:34:08 PM UTC-5, lucamilanesio wrote:

Edwin has just done it:

https://gerrit-review.googlesource.com/#/admin/projects/plugins/sync-index

Thanks.

Luca.

On 11 Nov 2015, at 09:31, Luca Milanesio <Luca.Mi...@gmail.com> wrote:

Hi Hugo, thanks for sharing it :-)

Edwin is about to create a project right now (he is actually sit next to me :-) )

Luca.

On 11 Nov 2015, at 08:46, Hugo Arès <hugo...@ericsson.com> wrote:

Hi

We have a couple of internal plugins that we would like to open source. So far, we kept them to ourselves because we were not sure if there would be interest in such plugins but I guess that open sourcing them is the only way to know.

Sync-index is the first one we would like to open source. It allows to synchronize secondary indexes between two Gerrit instances sharing the same repo and database.

That plugin is part of our active/passive master cluster solution. Long term, we want to switch to an external secondary index (i.e. Elasticsearch) but in the meantime, sync-index allows us to keep both master indexes up to date so we can failover between servers anytime without having to worry about an out-of-date index.

Thank you for considering this project creation request.

Hugo

-- 
-- 

To unsubscribe, email repo-discuss+unsub...@googlegroups.com

[Luca Milanesio]

Hi Bassem,

there is value anyway in sharing the plugin regardless of the change's approval.

Just mention that the plugin requires a patch on Gerrit ... and we can start experimenting it :-)

It can be actually a *BIG* push for the change to be merged.

Luca.

To unsubscribe, email repo-discuss...@googlegroups.com

[Hector Oswaldo Caballero]

The initial commit of sync-index plugin was just pushed for review:

https://gerrit-review.googlesource.com/#/c/73215/

Thanks,

Hector

To unsubscribe, email repo-discuss+unsubscribe@googlegroups.com

More info at http://groups.google.com/group/repo-discuss?hl=en

--- 
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bassem Rabil]

Now the initial version of the plugin [1] has been pushed, can we go back to the discussion on the extension change [2] ? This plugin has been running for us in production environment for around 4 months now, and the CPU cost on core is minimal to broadcast an event for the change being reindexed compared to the CPU required for reindexing a change and checking its mergeability against the new branch tip. 

In conclusion, the extension point brings no significant load on the Gerrit instance, even during online reindexing upon upgrading to a newer release using newer secondary index version. 

Since elastic search did not make it in Gerrit 2.12, we believe that this plugin can be efficient in synchronizing local seondary index between 2 master instances sharing the same Gerrit DB and git repositories.

[1] https://gerrit-review.googlesource.com/#/c/73215/

[2] https://gerrit-review.googlesource.com/#/c/72607/

Regards

Bassem

[Bassem Rabil Guendy]

Hi Jeremy

How are you building the plugin ? i.e. Are you building it from within Gerrit tree ? or Stand-alone ?
The sync-index plugin was tested against stable-2.11 and it requires the Gerrit core change [1]. So my proposal to you to experiment with the plugin is to cherry pick this core change on top of stable-2.11 branch of Gerrit core.

[1] https://gerrit-review.googlesource.com/#/c/72607/

Thanks and Regards
Bassem Guendy

On 01/20/2016 09:12 AM, Jeremy Yang wrote:

Hi Bassem,

To slove the inconsistence of "Index" between the two Gerrit Instance,in my opinoin,this “sync-index” plugin will be a good idea!

But I have encountered some problems when I was building the plugin:

(1) the buck build system cannot read the “GERRIT_TESTS” in buck file:

NameError: name 'GERRIT_TESTS' is not defined

(2)Which version of Gerrit did the plugin support?

Regards

Jeremy Yang

在 2015年12月17日星期四 UTC+8上午3:46:56，Bassem Rabil写道：

[Bassem Rabil Guendy]

Hi Jeremy

You need to use Gerrit 2.12 tree where the buck build was fixed for this error, however if you are building against 2.10 then building using maven would workaround this.  We would be interested in your test results when experimenting with the plugin on Gerrit 2.10. Can you please reply all to include Gerrit community when reporting your experimentation results to keep the community posted.

Regards
Bassem

On 01/21/2016 08:08 PM, Jeremy Yang wrote:

1、 Building it  within Gerrit tree 2.11/2.12 ，and cannot pass，because：

NameError: name 'GERRIT_TESTS' is not defined

Did you have encountered this problem?

2、At the end，I have to write a pom.xml file and use the maven tool to build a plugin.Now I have been trying to test it.

3、 Because of some special requirement I made a little change of the Gerrit core change [1] and the plugin in order to apply this plugin within the Gerrit 2.10.It looks available so far.

Thanks and Regards
Jeremy Yang

[Jeremy Yang]

Hi,Bassem

I have already made some changes of this plugin and fixed some bugs, now it can be running in  Gerrit 2.10.I looks well now.

But as far as I am concerned , it should be better if  it support synchronizing local seondary index among 3 or more master instances sharing the same Gerrit DB and git repositories. And I'm trying to realize this feature.

Thanks and Regards
Jeremy Yang

在 2015年12月17日星期四 UTC+8上午3:46:56，Bassem Rabil写道：

Now the initial version of the plugin [1] has been pushed, can we go back to the discussion on the extension change [2] ? This plugin has been running for us in production environment for around 4 months now, and the CPU cost on core is minimal to broadcast an event for the change being reindexed compared to the CPU required for reindexing a change and checking its mergeability against the new branch tip.