Proposal: open the flood gates for all Plugins' Pipelines on ci.jenkins.io

[R. Tyler Croy]

Last year we started allowing plugin developers use of new shiney elastic
compute capacity on ci.jenkins.io provided by our friends at Microsoft. Today I
would like to proposal that we open the flood gates, and add "*-plugin" to our
GitHub Organization Folder here: https://ci.jenkins.io/job/Plugins/


What's changed:

* In the realm of Pipeline administrative manageability, practically nothing
unfortunately. We are still unable to globally prevent runaway Pipelines.
We are still unable to limit capabilities of Pipelines, disallowing
specific steps, or label/node assignments.
* The shared library (https://github.com/jenkins-infra/pipeline-library), and
its `buildPlugin()` method has been working fantastically. Practically
every Jenkinsfile I see is simply: buildPlugin()
* The "file an INFRA ticket" approach has been working, but is still manual
work, and as everybody knows: automated is better than manual.
* We haven't been spending enough money on Azure. I'm only half-joking here,
Microsoft has been tremendously gracious, and we're no where close to using
our annual budget :)
* Speaking of budget, stephenc's done a lot of great work to help Jenkins
(more specifically the scm-api and github-branch-source plugins) handle the
GitHub rate limits. While we sometimes exhaust our hourly limit, the
behavior of the github-branch-source now gives me great confidence that
requires will not be disrupted/discarded, but merely delayed.


I think the risk is low, Pipeline has matured quite a bit since I first
discussed this topic with the dev community. I also think that the more
contributors who are using Pipeline as a user (editing a Jenkinsfile), the more
likely plugin contributors are going to want to have a stellar experience in
Pipeline for their plugin, so win-win :)


If this seems reasonable to folks, I'll discuss a couple last-minute items with
olblak and we can light up ci.jenkins.io with 1300+ plugins :D



Cheers
- R. Tyler Croy

------------------------------------------------------
Code: <https://github.com/rtyler>
Chatter: <https://twitter.com/agentdero>
xmpp: rty...@jabber.org

% gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

[Arnaud Héritier]

Go go go

Eat your own ...

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20170616185430.ggedoq27zymb6kg2%40blackberry.coupleofllamas.com.
For more options, visit https://groups.google.com/d/optout.

--

-----

Arnaud Héritier

http://aheritier.net

Mail/GTalk: aheritier AT gmail DOT com

Twitter/Skype : aheritier

[Jesse Glick]

On Fri, Jun 16, 2017 at 2:54 PM, R. Tyler Croy <ty...@monkeypox.org> wrote:
> add "*-plugin" to our GitHub Organization Folder here: https://ci.jenkins.io/job/Plugins/

+1 naturally! There are a lot of miscellaneous little plugins I have
been meaning to add a `Jenkinsfile` to and this would reduce the
friction to doing so.

[Mark Waite]

+1 from me.  I love buildPlugin() and I've been very grateful for the results from Windows and Linux builds, and from pliugin compatibility tester, all with a few arguments to buildPlugin().

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3o6tJv%2BKbnE8v%3DRxLHa66mcxAc9rEe0MzeAc522E1cYw%40mail.gmail.com.

[Arnaud Héritier]

Don't forget to disable all plugins jobs on DEV@cloud to avoid a mess in GitHub pull-requests status

On Fri, Jun 16, 2017 at 9:35 PM, Mark Waite <mark.ea...@gmail.com> wrote:

+1 from me.  I love buildPlugin() and I've been very grateful for the results from Windows and Linux builds, and from pliugin compatibility tester, all with a few arguments to buildPlugin().

On Fri, Jun 16, 2017 at 1:13 PM Jesse Glick <jgl...@cloudbees.com> wrote:

On Fri, Jun 16, 2017 at 2:54 PM, R. Tyler Croy <ty...@monkeypox.org> wrote:
> add "*-plugin" to our GitHub Organization Folder here: https://ci.jenkins.io/job/Plugins/

+1 naturally! There are a lot of miscellaneous little plugins I have
been meaning to add a `Jenkinsfile` to and this would reduce the
friction to doing so.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3o6tJv%2BKbnE8v%3DRxLHa66mcxAc9rEe0MzeAc522E1cYw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAO49JtEk928arEXExs%2B2T8hcigL7tF4gCMmHJxM7PWwv7xJdsA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[R. Tyler Croy]

(replies inline)

On Fri, 16 Jun 2017, Arnaud H?ritier wrote:

> Don't forget to disable all plugins jobs on DEV@cloud to avoid a mess in
> GitHub pull-requests status

That sounds like a good aheritier task :D

[Arnaud Héritier]

:-P

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20170616210947.gu2iccx7xt4h3ynd%40blackberry.coupleofllamas.com.

For more options, visit https://groups.google.com/d/optout.

[Jesse Glick]

So far I have been disabling those jobs as `Jenkinsfile`s come online. I am not sure if we want to do a batch close—it would be disruptive. Maybe we could configure the old jobs to send mail to SCM contributors of each build reminding them to consider using `Jenkinsfile` instead? No strong opinion.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAFNCU-_TAjiWwVpReFJ%2BLPYeHCdcOXatLm5cPaorGE9yRzy7bQ%40mail.gmail.com.

[Arnaud Héritier]

no strong opinion too. 

We are using a template thus it shouldn't be too hard

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1vMBCr_9tksJoix%2ByojARzdTxDgqhydq8DtU-Bubve5Q%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[R. Tyler Croy]

(replies inline)

On Fri, 16 Jun 2017, R. Tyler Croy wrote:

>
> Last year we started allowing plugin developers use of new shiney elastic
> compute capacity on ci.jenkins.io provided by our friends at Microsoft. Today I
> would like to proposal that we open the flood gates, and add "*-plugin" to our
> GitHub Organization Folder here: https://ci.jenkins.io/job/Plugins/

I have opened the flood gates with the regex: ((!blueocean-plugin)|(.*-plugin))

[Arnaud Héritier]

Hi everyone

  As discussed with Tyler I now deactivated all jobs under https://jenkins.ci.cloudbees.com/job/plugins/

  It is now important to create a Jenkinsfile with a simple call: buildPlugin()

  You can review the discovery of plugins here : https://ci.jenkins.io/job/Plugins/computation/console

  If you are owner/admin of the github repository of the plugin it might be good to edit its webhooks to remove the useless ones like jenkins.ci.cloudbees.com ones

Cheers

  

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20170627155232.b6hvdade5yczsnfx%40blackberry.coupleofllamas.com.

For more options, visit https://groups.google.com/d/optout.

[Michael Neale]

If it hasn't been done already (chatted to tyler yesterday) - can we have the "blueocean-plugin" excluded from the pattern/regex? 

On Wednesday, June 28, 2017 at 6:08:30 AM UTC+10, Arnaud Héritier wrote:

Hi everyone

  As discussed with Tyler I now deactivated all jobs under https://jenkins.ci.cloudbees.com/job/plugins/

  It is now important to create a Jenkinsfile with a simple call: buildPlugin()

  You can review the discovery of plugins here : https://ci.jenkins.io/job/Plugins/computation/console

  If you are owner/admin of the github repository of the plugin it might be good to edit its webhooks to remove the useless ones like jenkins.ci.cloudbees.com ones

Cheers

  

On Tue, Jun 27, 2017 at 5:52 PM, R. Tyler Croy <ty...@monkeypox.org> wrote:

(replies inline)

On Fri, 16 Jun 2017, R. Tyler Croy wrote:

>
> Last year we started allowing plugin developers use of new shiney elastic
> compute capacity on ci.jenkins.io provided by our friends at Microsoft. Today I
> would like to proposal that we open the flood gates, and add "*-plugin" to our
> GitHub Organization Folder here: https://ci.jenkins.io/job/Plugins/


I have opened the flood gates with the regex: ((!blueocean-plugin)|(.*-plugin))

- R. Tyler Croy

------------------------------------------------------
     Code: <https://github.com/rtyler>
  Chatter: <https://twitter.com/agentdero>
     xmpp: rty...@jabber.org

  % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20170627155232.b6hvdade5yczsnfx%40blackberry.coupleofllamas.com.

For more options, visit https://groups.google.com/d/optout.

[Michael Neale]

(ignore) - sorry I read out of order again :(

Hopefully that regex looks ok. Thanks. 

[Daniel Beck]

> On 27. Jun 2017, at 17:52, R. Tyler Croy <ty...@monkeypox.org> wrote:
>
> I have opened the flood gates with the regex: ((!blueocean-plugin)|(.*-plugin))

Does this actually work? I'd have expected something more like:

.*-plugin(?<!blueocean-plugin)

[Richard Bywater]

Just taking a look at the plugin discovery console and it seems to have fallen over. I'm guessing that means it hasn't found all the plugins?

Richard.

On Wed, 28 Jun 2017 at 08:08 Arnaud Héritier <aher...@gmail.com> wrote:

Hi everyone

  As discussed with Tyler I now deactivated all jobs under https://jenkins.ci.cloudbees.com/job/plugins/

  It is now important to create a Jenkinsfile with a simple call: buildPlugin()

  You can review the discovery of plugins here : https://ci.jenkins.io/job/Plugins/computation/console

  If you are owner/admin of the github repository of the plugin it might be good to edit its webhooks to remove the useless ones like jenkins.ci.cloudbees.com ones

Cheers

  

On Tue, Jun 27, 2017 at 5:52 PM, R. Tyler Croy <ty...@monkeypox.org> wrote:

(replies inline)

On Fri, 16 Jun 2017, R. Tyler Croy wrote:

>
> Last year we started allowing plugin developers use of new shiney elastic
> compute capacity on ci.jenkins.io provided by our friends at Microsoft. Today I
> would like to proposal that we open the flood gates, and add "*-plugin" to our
> GitHub Organization Folder here: https://ci.jenkins.io/job/Plugins/


I have opened the flood gates with the regex: ((!blueocean-plugin)|(.*-plugin))

- R. Tyler Croy

------------------------------------------------------
     Code: <https://github.com/rtyler>
  Chatter: <https://twitter.com/agentdero>
     xmpp: rty...@jabber.org

  % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F
------------------------------------------------------

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20170627155232.b6hvdade5yczsnfx%40blackberry.coupleofllamas.com.

For more options, visit https://groups.google.com/d/optout.

--

-----

Arnaud Héritier

http://aheritier.net

Mail/GTalk: aheritier AT gmail DOT com

Twitter/Skype : aheritier

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAFNCU--vAhfQ1k_qSAz7qnj0fqXG28TeS3s7UdyhB30D58dDgQ%40mail.gmail.com.

[Michael Neale]

It doesn't work in as much as blue ocean - it is still triggering builds for it. 

[R. Tyler Croy]

(replies inline)

On Tue, 27 Jun 2017, Richard Bywater wrote:

> Just taking a look at the plugin discovery console and it seems to have
> fallen over. I'm guessing that means it hasn't found all the plugins?

https://issues.jenkins-ci.org/browse/JENKINS-45142

:-/

[Richard Bywater]

On the plus side I created a PR to add a Jenkinsfile to the HTML publisher plugin and the new infrastructure automatically picked it up and built it so at least that bit is all good :)

Richard.

[Jesse Glick]

On Tue, Jun 27, 2017 at 6:59 PM, Daniel Beck <m...@beckweb.net> wrote:
>> I have opened the flood gates with the regex: ((!blueocean-plugin)|(.*-plugin))
>
> Does this actually work? I'd have expected something more like:
>
> .*-plugin(?<!blueocean-plugin)

Or

(?!blueocean-plugin).+-plugin

[Michael Neale]

Any ETA on when the pattern will be applied? still seeing failures on github with every new PR (at least as of a few hours ago)

I think there was some work/bug to be overcome to get it to rescan with such a prodigious number of repositories in a github org (it is a great pressure test of things!)

[Jesse Glick]

On Tue, Jul 4, 2017 at 5:40 AM, Michael Neale <mne...@cloudbees.com> wrote:
> Any ETA on when the pattern will be applied?

Best to file a ticket in INFRA so your issue can be tracked.

[Michael Neale]

Have done: https://issues.jenkins-ci.org/browse/INFRA-1262 - good idea 

Tyler was working on it - there is a problem with scanning repos which may require a change/patch. Could we possibly turn off the org folder (delete it) until that is the case? as this is getting very hard to work with (some times it is like we don't have CI, which is delicious irony).