Repository proposal: plugins/utils-scripts

[lucamilanesio]

We are starting to create quite a number of small Gerrit plugins implemented as simple Groovy or Scala scripts.

They do not need to be compiled, just drop them into your $GERRIT_SITE/plugins directory and they'll work out-of-the-box :-)

Can we create a repository called "plugins/utils-scripts" to push all the community-contributed scripting plugins?

The first one I woudl like to push is a SSH CLI for triggering the Gerrit on-line re-indexer: small, focused and really effective :-)

import com.google.gerrit.sshd.*

import com.google.gerrit.extensions.annotations.*

import com.google.gerrit.lucene.*

import com.google.inject.*

import org.kohsuke.args4j.*

@Export("start")

@CommandMetaData(name = "start", description = "Start a new on-line re-indexing for a target Lucene index version")

class StartReindex extends SshCommand {

  @Inject OnlineReindexer.Factory reindexerFactory

  @Argument(index = 0, usage = "Index version", metaVar = "VERSION")

  int indexVersion

  public void run() {

    def indexer = reindexerFactory.create(indexVersion)

    indexer.start()

    stdout.println "On-line reindexing scheduled for version " + indexVersion

  }

}

[Edwin Kempin]

Sure, I can create this repository.

Just let's wait a moment if anyone wants to suggest a better name :-)

Maybe just "plugins/scripts"?

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Luca Milanesio]

Yes, better name indeed.

"utils" doesn't mean anything after all :-)

Any other suggestion?

Luca.

[Bassem Rabil]

I prefer scripts as well. Thanks for sharing these plugins/scripts.

We are using for this reindexer functionality via  ssh command line [1] to restart the online reindexer if it is aborted for any reason when reindexing for example a non consistent change.

[1] https://gerrit-review.googlesource.com/Documentation/cmd-index-start.html

[Luca Milanesio]

Hi Bassem, 

the problem with the current command is that you cannot force the rebuilding of the index on-line.

This is because of this part of the IndexStartCommand code:

    try {

      if (luceneVersionManager.startReindexer()) {

        stdout.println("Reindexer started");

      } else {

        stdout.println("Nothing to reindex, index is already the latest version");

      }

    } catch (ReindexerAlreadyRunningException e) {

      throw new UnloggedFailure("Failed to start reindexer: " + e.getMessage());

    }

If you want to rebuild the index it will just say: "Nothing to reindex, index is already the latest version".

If you want to force the rebuilding of Index Version N, the Groovy script I provided will do the job :-)

Luca.

[Bassem Rabil]

I agree that having this in a simple plugin as you did can tweak the behavior without having to change the Gerrit core. That is why I will be interested to check these plugins/scripts once you share them.

You received this message because you are subscribed to a topic in the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/repo-discuss/pUn8fHVh58w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to repo-discuss...@googlegroups.com.

[Edwin Kempin]

Hi Luca, 

I created the plugin project:

  https://gerrit-review.googlesource.com/#/admin/projects/plugins/scripts

Edwin

[Luca Milanesio]

Thanks Edwin,

I've already posted the first change:

https://gerrit-review.googlesource.com/#/c/72898/

Luca.

[Khai Do]

Hi Luca, I wanted to force an online reindex so tried your reindexer script (reindexer-1.0.groovy) but I can't seem to get it to work.  I have installed the groovy-provider-plugin[1] and the plugin scripts[2] on Gerrit core ver 2.11.4.  There doesn't seem to be any docs on how to run the command but from the code it looks like it's just 'ssh -p 29418 localhost gerrit reindex start' which doesn't work for me.  It doesn't seem to recognize the command.  I've noticed that the groovy provider plugin docs says it requires some core changes to work when I look at the topic[3] for those core changes it looks like some of them were abandoned.  Now I'm wondering if I'm doing something wrong or whether this plugin even works at all?  

[1] https://gerrit.gerritforge.com/view/Plugins-stable-2.11/job/plugin-scripting-groovy-provider-stable-2.11/
[2] https://gerrit.googlesource.com/plugins/scripts

[3] https://gerrit-review.googlesource.com/#/q/topic:plugin-dependencies

[Luca Milanesio]

Hi Khai,

I just tried now and it works for me.

Let me quote in-line what I've done:

On 10 May 2016, at 20:05, Khai Do <zaro...@gmail.com> wrote:

Hi Luca, I wanted to force an online reindex so tried your reindexer script (reindexer-1.0.groovy) but I can't seem to get it to work.  I have installed the groovy-provider-plugin[1]

Yes, done the same and my list of plugins is:

$ ssh -p 29418 lucamilanesio@localhost gerrit plugin ls
Name                           Version    Status   File
-------------------------------------------------------------------------------
groovy-provider                v2.11.4-23-gcf75006 ENABLED  groovy-provider.jar

and the plugin scripts[2] on Gerrit core ver 2.11.4.  

The scripting plugins are just files copied to the /plugins directory. I have then downloaded the one the reindexer (see [4]) and copied as /plugins/reindexer-1.0.groovy.

After a few minutes it appears to be loaded successfully by Gerrit, as demonstrated by the list of plugins:

$ ssh -p 29418 lucamilanesio@localhost gerrit plugin ls
Name                           Version    Status   File
-------------------------------------------------------------------------------
groovy-provider                v2.11.4-23-gcf75006 ENABLED  groovy-provider.jar
reindexer                      1.0        ENABLED  reindexer-1.0.groovy

There doesn't seem to be any docs on how to run the command but from the code it looks like it's just 'ssh -p 29418 localhost gerrit reindex start' which doesn't work for me.

Yes I know, this was already noticed during the User Summit, it seems that the documentation annotations aren't picked up.

I did not have time to investigate why :-(

However the on-line documentation is quite "reduced" but exists :-)

$ ssh -p 29418 lucamilanesio@localhost reindexer start --help
reindexer start [VERSION] [--] [--help (-h)]

 VERSION     : Index version
 --          : end of options
 --help (-h) : display this help text

 It doesn't seem to recognize the command.  I've noticed that the groovy provider plugin docs says it requires some core changes to work when I look at the topic[3] for those core changes it looks like some of them were abandoned.  

They are not needed anymore, it works out-of-the-box on a plain vanilla Gerrit :-)

[Khai Do]

Thanks Luca, finally able to try this again, looks like I just mistyped the command as 'reindex' instead of 'reindexer'.  It works for me now.

I noticed that these admin scripts don't respect gerrit access rules because it looks like any user is able to run these commands.  Is there a way to enforce access rules in the admin scripts? or would this need to be written as a proper gerrit (java) plugin?  Maybe another thought would be to enforce access rules for these scripts in the groovy-provider plugin? 

-Khai

[Luca Milanesio]

On 20 May 2016, at 16:45, Khai Do <zaro...@gmail.com> wrote:

Thanks Luca, finally able to try this again, looks like I just mistyped the command as 'reindex' instead of 'reindexer'.  It works for me now.

Coool :-)

I noticed that these admin scripts don't respect gerrit access rules because it looks like any user is able to run these commands.  

True, but you can easily submit a change to restrict them using annotations, like:

@RequiresCapability(GlobalCapability.ADMINISTRATE_SERVER)

I haven't test them personally, but there is no reason why it shouldn't work :-)

Is there a way to enforce access rules in the admin scripts? or would this need to be written as a proper gerrit (java) plugin?

Not really, see above.

 Maybe another thought would be to enforce access rules for these scripts in the groovy-provider plugin? 

Don't think it would be the right place: you could write scripting plugins that could be potentially opened to other non-admins users. It really depends on the script code itself.

Luca.

[Khai Do]

I just tested and the annotation seems to work as expected.  I'll propose that annotation to these scripts because they are 'admin'.  Also I'll find some time to fix up the docs so others won't need to bother you further :)  

I still have one lingering questions though.  Why even have a plugin to force reindex, this seem appropriate as a core change.  Maybe as 'gerrit reindex --force'?

thanks again.  -Khai

[Luca Milanesio]

On 20 May 2016, at 17:40, Khai Do <zaro...@gmail.com> wrote:

I just tested and the annotation seems to work as expected.  I'll propose that annotation to these scripts because they are 'admin'.  Also I'll find some time to fix up the docs so others won't need to bother you further :)  

Why don't you submit a patch for that? Will be happy to review it straight away :-)

I still have one lingering questions though.  Why even have a plugin to force reindex, this seem appropriate as a core change.  Maybe as 'gerrit reindex --force'?

True ... the scripting plugin was for "retrofitting" the feature to older versions, but for Gerrit master it would make sense to have it in the core.

Luca.

[Sebastian Schuberth]

Hi Luca,

on a slightly related note, are there any plans to enable storing scripts in a project's refs/meta/ namespace, e.g. refs/meta/scripts? Doing so would enable project owners to install their wanted plugins without bothering site admins with shell access, and even allow editing them online in the Gerrit web UI.

Regards,
Sebastian

[Luca Milanesio]

Hi Sebastian,

it is one thing that was mentioned as well when I presented the scripting plugins at the Gerrit User Summit.

Yes, it is a good idea IMHO because it would allow to have project-speicfic behaviour and to enable Teams to be more independent.

The problem is: Gerrit plugin infrastructure isn't ready yet for it.

The things that are missing:

- Project level isolation for Plugin loaders (now a plugin is always working globally)

- Project specific injectors

- Project specific plugin control (load, unload, enable, disable)

I would say that putting them under All-Projects/refs/meta/scripts would be the only option for now, and leaving the ability to manage them *only* to the Gerrit administrator.

Luca.

[Sebastian Schuberth]

On Mon, May 23, 2016 at 2:43 PM, Luca Milanesio
<luca.mi...@gmail.com> wrote:

> I would say that putting them under All-Projects/refs/meta/scripts would be
> the only option for now, and leaving the ability to manage them *only* to
> the Gerrit administrator.

Which still would be nice, so at least "regular" Gerrit admins (those
who are members of the "Administrators" group but do not have shell
access) could install / edit the script. So do you think having this
enabled is a worthwhile intermediate step?

--
Sebastian Schuberth

[Luca Milanesio]

Yes, I do think it will be useful and worthwhile :-)

Luca.