[PROPOSAL] Clarify the source code canonical place is required to be under the GH org (was:[DISCUSS] Hide from the update center plugins without source code under the Jenkinsci github organization?)

[Baptiste Mathus]

Hi all,

Re-starting this thread because there are still hosted plugins that we have sometimes difficulties to find sources for (actual personal experience recently on IRC after a user asked where it was), or jenkins_release tweets pointing to nothing, and so on.

And what the project has always been striving for is "making it easier for the community".

So we'd be changing our core principle of low entry barrier?

No. Having been doing some archeology with Hosting Request wiki page history, IMO it's actually always, or at least for many a year, been considered the standard way.

Never required per-se, but always assumed (as confirmed by infra team member present since the beginning).

Though indeed, this has been more or less ambiguous or implicit in some places I guess. 

Hence this discussion about clarification.

The low barrier entry has always been and still is one of the fundamentals of the project. 

And IMO has always been provided through very easy (commit) access & hosting.

• For example, if we look at the 2008 version [1] : this is basically a HOWTO for the Hudson common svn access.
  
• Here in 2010 [2], the code of hosted plugins was assumed to be under the Jenkins community infra.
  
• In mid 2012 [3], again only the jenkinsci org is listed (svn starting to be deprecated). 

Historically, IIRC, releasing from 'elsewhere' was technically only possible for committers who actually reused their permissions to release binaries from elsewhere.

That would need double-check, but I don't think any (?) new committer was ever granted permissions to publish binaries for source code that was not planned to be ever hosted under the hudson/jenkins infra.

So What?

After having discussed with some here and there, we're considering adding that subject to be discussed at the next gov meeting.

The immediate goal is just to enrich the wiki page(s) so that no more ambiguousness/implicitness is left about it.

WDYT?

[1] https://wiki.jenkins-ci.org/pages/viewpage.action?pageId=18808859

[2] https://jenkins-ci.org/blog/2010/07/28/hosting-your-hudson-plugin-at-github/

[3] https://wiki.jenkins-ci.org/pages/viewpage.action?pageId=62063272

2015-12-01 22:21 GMT+01:00 Christopher Orr <ch...@orr.me.uk>:

On 01/12/15 16:52, 'Jesse Glick' via Jenkins Developers wrote:
> On Tue, Dec 1, 2015 at 5:32 AM, Baptiste Mathus <m...@batmat.net> wrote:
>> plugins forked under the Jenkinsci org, BUT still maintained elsewhere
>
> There are a number of such cases where the @jenkinsci fork is out of
> date. There are even cases where both the original and fork repository
> are configured to accept pull requests, which is quite confusing.
>
> Also in some cases the GitHub issue tracker is enabled (inside or
> outside @jenkinsci), and there are some issues filed in JIRA too,
> which is again confusing.

I guess being able to make plugin releases (exclusively?) via
Jenkins-on-Jenkins would solve this problem? :)

In any case, it would definitely be good if clicking on "Source code" on
the wiki led to the place where the *most recent release* was made,
rather than pointing to some abandoned fork.  Whether that means source
code needs to be exclusively hosted under @jenkinsci, I'm not 100% sure,
but doing so would certainly have a lot of benefits.

Regards,
Chris

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/565E0F62.605%40orr.me.uk.

For more options, visit https://groups.google.com/d/optout.

[Nigel Magnay]

I am strongly -1 in the idea of forcing plugins to be hosted with jenkinsci.

You will already have seen instances where those "granted power" hold an effective veto over forking plugins into that organisation. That is raising a barrier (and one that does not exist if I want to, for example, publish a maven plugin or a node package). It's contrary to the notion of an open community, is a retrograde step from the ability to fork, and inevitably sets us down the "primrose path of open source governance". Before long you'll be into "all jenkinsci plugins must format their code thusly", or all contributors must sign a CLA, or do CA, or whatever. 

Maven plugin POMs already contain the SCM location within them. If you're concerned about end-users finding the source repo, then simply make the plugin toolchain extract that URL and make it available within the plugins page. Indeed, since JenkinsCI now has 75 pages of repos - I can no longer easily find things that I know are there. 

I fail to see what benefit it would bring that could not be achieved in other ways.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANWgJS7Q1a8OP2N%2BGtNeN6MRhxNSEJPG4osw-0XO8wkf3b2Mnw%40mail.gmail.com.

[Stephen Connolly]

Well I think there is merit in saying "if you want to have your plugin listed in the OSS update centre, it must have a groupId belonging to the org.jenkins-ci.plugins tree and it must be hosted on the jenkins-ci GitHub account... Because we will want to start publishing to central at some point (hopefully in the near future) and sync to central from our infra will require a retroaction on the groupId to a defined set... So the central sync requirements will foist that on us...

That will not stop others from publishing their own plugins to central under their own groupIds...

So the question then becomes: should the OSS update centre scan all of central or just the part of central we publish to?

I personally would be in favour of making it easier for people to manage multiple update centres and let people host their own update centres, leaving the default OSS centre for the plugins the project provides and hosts...

My personal opinion without reference to my employers

-Stephen

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPYP83QrjKfGBf9hD2VU%3D2SA6c1LUCYOZLMzNPG_ma%2BK8y_wmg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
Sent from my phone

[Stephen Connolly]

On Monday 25 January 2016, Nigel Magnay <nigel....@gmail.com> wrote:

I am strongly -1 in the idea of forcing plugins to be hosted with jenkinsci.

You will already have seen instances where those "granted power" hold an effective veto over forking plugins into that organisation. 

Well this is a separate issue. I think part of the problem here is that there are 1000+ plugins. More way to do things is not helping users. Some consolidation would be a good thing... But then having said that, if we stop experimentation we cannot let better ways.

If the 2.0 update centre has ratings or otherwise provides more of an AppStore like UX then the risk of 20 plugins to have a "sleep N" build step becomes less important... There will be the top rated fart app of Jenkins and others can try to shoot for that crown, but users will pick the best (from the other users PoV) and that makes life better for users... Without that... Well you have name squatting and too many plugins

 

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPYP83QrjKfGBf9hD2VU%3D2SA6c1LUCYOZLMzNPG_ma%2BK8y_wmg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
Sent from my phone

[R. Tyler Croy]

(replies inline)

On Mon, 25 Jan 2016, Nigel Magnay wrote:

> I am strongly -1 in the idea of forcing plugins to be hosted with jenkinsci.

I think it is important to be clear what we're talking about here. We're not
forcing people to use github.com/jenkinsci for plugins, Baptiste is suggesting
that if a plugin author expects their plugin to be distributed by the Jenkins
project, that the project has access to the source code (in essence). This in
my mind is no different than Debian having rules around what kind of packages
do or do not go into the debian stable/unstable repositories which get included
by default in an OS installation.

Addtionally, I think the "Adopt a Plugin" effort is a great demonstration of
why this is important for the long-term success of plugins in the Jenkins
ecosystem.

> You will already have seen instances where those "granted power" hold an

> effective veto over forking plugins into that organisation. That *is* raising

> a barrier (and one that does not exist if I want to, for example, publish a
> maven plugin or a node package). It's contrary to the notion of an open
> community, is a retrograde step from the ability to fork, and inevitably
> sets us down the "primrose path of open source governance". Before long

> you'll be into "all jenkinsci plugins must format their code *thusly*", or

> all contributors must sign a CLA, or do CA, or whatever.

I understand the concern about gate keepers with regards plugins which may be
viewed as redundant. I think that's a good point we should try to address with
good rules about what gets hosted.


If the Jenkins project doesn't have access to the source code that *we're*
publishing, on project infrastructure like repo.jenkins-ci.org and
updates.jenkins-ci.org, how is it *that* not in conflict with the notion of an
open community? I do not believe we should spend infrastructure time and
resources hosting non-free plugins. IMO requiring plugins to be hosted in the
jenkinsci organization in order to be distributed on behalf of the project is a
reasonable trade-off to ask of authors.

> Maven plugin POMs already contain the SCM location within them. If you're
> concerned about end-users finding the source repo, then simply make the
> plugin toolchain extract that URL and make it available within the plugins
> page. Indeed, since JenkinsCI now has 75 pages of repos - I can no longer
> easily find things that I know are there.

There's nothing preventing me, based on what I've seen, from entering garbage
into that property or pointing to some URL which is completely irrelevant to
the plugin that is being distributed.

> I fail to see what benefit it would bring that could not be achieved in
> other ways.

If you do not see this as a viable solution to ensure that source code for
plugins distributed by the Jenkins projects are open source and easily
locatable to end users, please provide some alternate proposals.



Cheers
- R. Tyler Croy

------------------------------------------------------
Code: <https://github.com/rtyler>
Chatter: <https://twitter.com/agentdero>

% gpg --keyserver keys.gnupg.net --recv-key 3F51E16F
------------------------------------------------------

[Daniel Beck]

On 26.01.2016, at 00:05, Nigel Magnay <nigel....@gmail.com> wrote:

> You will already have seen instances where those "granted power" hold an effective veto over forking plugins into that organisation.

Actually not forking a plugin is done _extremely_ sparingly, and even then we're typically trying to get the author to publish their code, e.g. as an update to an abandoned plugin that does the exact same thing, just a little differently. A singular focus on developers who want to publish their own creation at the detriment of users doesn't help anyone. Users are also part of our community, and finding out the plugin you based your build system setup around is a dead end isn't a great experience.

That said, if more people volunteered to handle hosting requests, maybe none of this would happen anymore. After all, anyone with +voice on IRC can do it (modulo some JIRA permissions, which I intend to hand out freely, and only Baptiste has requested since we moved to JIRA).

Regarding unanswered hosting requests, this has completely stopped happening since we moved to JIRA. All requests older than a few days have been handled.

> That is raising a barrier (and one that does not exist if I want to, for example, publish a maven plugin or a node package). It's contrary to the notion of an open community, is a retrograde step from the ability to fork, and inevitably sets us down the "primrose path of open source governance".

Having a low barrier to entry (admitting people without requiring them to go through a lengthy process of increasing access/power) does not preclude having standards.

And we're not just running the repo: Jenkins presents a nice categorized list of plugins for you to install with two clicks right there in the tool. This is a lot more than you get from repo1.maven.org, so having different requirements seems sensible.

***

FWIW I see nothing wrong with actually requiring that plugins published to our Maven repo, for distribution using our update sites, have a copy of their source code hosted on our infrastructure. This is something we should be able to ask of everyone. Some actually only publish the finished releases and do their development elsewhere. I'm fine with this. Having the sources anywhere else means we can lose them really easily.

[Nigel Magnay]

I think it is important to be clear what we're talking about here. We're not
forcing people to use github.com/jenkinsci for plugins,

​Ok then - I'm confused. 

The title of this email thread is : "​[PROPOSAL] Clarify the source code canonical place is required to be under the GH org ", which was "Hide from the update center plugins without source code under the Jenkinsci github organization"

​Isn't that forcing people to use github.com/jenkinsci ?​

Baptiste is suggesting
that if a plugin author expects their plugin to be distributed by the Jenkins
project, that the project has access to the source code (in essence). This in
my mind is no different than Debian having rules around what kind of packages
do or do not go into the debian stable/unstable repositories which get included
by default in an OS installation.

​That I don't have a problem with, as everything I do in this sandpit is OSS. (incidentally, I also don't have a problem with closed-source, proprietary or other non-free plugins being available in the default ​plugin manager index (so long as there's a swith, e.g: similar to free / paid-for in the appstore, to make it very clear). 

 

Addtionally, I think the "Adopt a Plugin" effort is a great demonstration of
why this is important for the long-term success of plugins in the Jenkins
ecosystem.


> You will already have seen instances where those "granted power" hold an
> effective veto over forking plugins into that organisation. That *is* raising
> a barrier (and one that does not exist if I want to, for example, publish a
> maven plugin or a node package). It's contrary to the notion of an open
> community, is a retrograde step from the ability to fork, and inevitably
> sets us down the "primrose path of open source governance". Before long
> you'll be into "all jenkinsci plugins must format their code *thusly*", or
> all contributors must sign a CLA, or do CA, or whatever.


I understand the concern about gate keepers with regards plugins which may be
viewed as redundant. I think that's a good point we should try to address with
good rules about what gets hosted.


If the Jenkins project doesn't have access to the source code that *we're*
publishing, on project infrastructure like repo.jenkins-ci.org and
updates.jenkins-ci.org, how is it *that* not in conflict with the notion of an
open community? I do not believe we should spend infrastructure time and
resources hosting non-free plugins. IMO requiring plugins to be hosted in the
jenkinsci organization in order to be distributed on behalf of the project is a
reasonable trade-off to ask of authors.

So I agree with the idea that 'if you're giving free hosting, then we don't want to host non-free artifacts'. Isn't that the rule with maven hosting, and docker image hosting (neither of which require pre-approval)? I think it's also fine to raise the bar to verify that the SCM url does, indeed, exist. That said, I don't see why free artifacts can't get delivered by the standard maven-central repositories. It's the *index* of artifacts that's important, not the artifacts themselves. 

But then I worry about "good rules about what gets hosted". This makes me concerned that you're using weasel words - you're not forced to use github.com/jenkinsci.... but if you don't, none of your plugins will appear without user intervention. They'll effectively be dead without the 'blessing' of whoever is granted rights to 'approve' your plugin. And as recent "hosting request denied" events have shown - there's something of a 'first past the post' system in effect, which IMO is a terrible thing.

Hey presto - you're back to "who has commit bit" rights that we all ran away from with centralised systems like SVN, as you're completely curtailed the ability to fork.

Just about every plugin I have started (git, docker, maven-repository-server) started as a "I wonder if <X> will be useful. Let's hack on it and see what happens". Sometimes they work out as useful, sometimes not. The ones that do tend to reach an inflection point where "this is probaby useful and ready to take outside contributions" and/or "I will not have time to maintain this, it would be better to fork it into jenkinsci as it's more likely to get input there". Without early adopter feedback gathered from being in the update-centre, I doubt many would have graduated in that way. 

If you do not see this as a viable solution to ensure that source code for
plugins distributed by the Jenkins projects are open source and easily
locatable to end users, please provide some alternate proposals.

Are there any plugins for which source code is *not* distributed? On purpose?

​You could validate the SCM url at update centre manifest generation time.

Instead of plugins being released by a user doing "mvn:release", only let a jenkinsci infrastructure machine do that (similar to debian building from source). 

​The update centre data could include plugins hosted in maven central.

There''s loads of scope for the plugin manager to be able to filter (e.g: 'experimental' vs 'released', 'free' vs 'commercial' which I am forever forgettings the URLs for). Namespace plugins (a la docker images). Make jenkinsci/* mean something (potentially boot out plugins that aren't being maintained). When searching for plugins, show the information about number of installs (to identify the off-the-wall experiments from the 'everyday production' stuff). Generate groups of "these plugins commonly used together" "packs" for one-click installation from data of what plugins people are actually using. Show "people using this plugin also commonly use this plugin". Show "days since last update" information when picking plugins. Show which version of jenkins the plugin was built against.

​

Github could be searched for *every* jenkins plugin, including forks. Build all of them. Index all of them. Don't present plugin installation as a gigantic, terrifying list. When selecting a plugin display "other forks of this plugin". Include links to their sourcecode. Allow the descriptive text to come from a README.md lodged in the repository, rather than a sometimes-helpful 1-liner. 

 

Cheers
- R. Tyler Croy

------------------------------------------------------
     Code: <https://github.com/rtyler>
  Chatter: <https://twitter.com/agentdero>

  % gpg --keyserver keys.gnupg.net --recv-key 3F51E16F
------------------------------------------------------

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20160125235131.GC2891%40blackberry.coupleofllamas.com.

[Kanstantsin Shautsou]

-1 For enforcements because infra was bad and inconvenient in the last years. 

Atm there are enough places for enhancements for goal 'easier search of sources' i.e. enhance tooling, provide image badges for sourcing into GitHub documentations, make configurable badge, etc. Bot sometimes provides admin access to forked repo, sometimes - not. It impossible to work with repository because of constant delays.

Atm jenkinsci even has no any security. I.e. kohsuke for single plugin access requests providing `everyone` write permissions. Last discussion about security enhancements end with his veto on not removing `everyone` write. 

I don't think that any company (except CloudBees that has full access everywhere) would like to host their FOSS plugin under jenkinsci while they have their own tracker/documentation/release cycles and development models.

Btw, some projects can download binaries from GH releases. In theory better repository management in Jenkins may even allow using plugins hosted under any account (with url based deps?). (Would be cool to have some SAT solver based dependency manager and be closer to what linux distros allows with multiple repository sources nowadays).

Thinking that some plugin is redundant is totally subjective. People sharing their work, some other people may be interested in such work.

[Baptiste Mathus]

Added for discussion to the next governance meeting planning (i.e. tomorrow).

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/0871dd96-d197-4d1d-9407-1a884454fbb1%40googlegroups.com.

[Baptiste Mathus]

Hi all,

This subject was discussed at last gov meeting: http://meetings.jenkins-ci.org/jenkins-meeting/2016/jenkins-meeting.2016-02-03-19.00.log.html starting at 19:47:02

As expected, the discussion was a bit /multidirectional/, so the issue isn't very clear. Some seem to agree, some saying this could be a requirement for a 2.0 Update Center, and so on.

So, as advised, the subject was put on the agenda again for today's meeting (starting in ~6h) to try and have a bit more time to find a consensus (or at least some actions to head over /some/ clarification about this). 

We'll see how it goes.

Don't hesitate to attend (IRC). It's open to anyone: https://wiki.jenkins-ci.org/display/JENKINS/IRC+Channel

-- Baptiste