GitHub and Bitbucket branch source UI refactoring

191 views
Skip to first unread message

Stephen Connolly

unread,
Jun 16, 2017, 2:19:12 PM6/16/17
to jenkin...@googlegroups.com, jenkins...@googlegroups.com
Just a quick status update.

In final stages of this work now. Bobby is being a superstar and reviewing my 13k LoC change on the Bitbucket branch source - brings lots of feature parity with GitHub and adds the configuration ability of the pure Git branch source

I am finalising the GitHub Branch Source changes... likely to be another big PR

Then there's a 5k LoC change in the Git plugin

Plan is to try and get all merged next week and cut a beta

I'll be looking for people to help test at that stage.

Please respond if you think you can help (lots of bugs fixed as a side effect of the refactoring - it makes things more easy to test => I found and fixed bugs)
--
Sent from my phone

Mark Waite

unread,
Jun 16, 2017, 2:35:47 PM6/16/17
to jenkin...@googlegroups.com, jenkins...@googlegroups.com
I'd like to be part of the beta test.

Mark Waite

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BnPnMxfYrZphgYDXFD3i%2Bo_7eDn7mn2qVrzJz6wFaoVkNmc%2Bw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Stephen Connolly

unread,
Jun 16, 2017, 2:39:02 PM6/16/17
to jenkin...@googlegroups.com
Well I kind of though you were required- given the git plugin is part of the changes ;-)


For more options, visit https://groups.google.com/d/optout.

Dan Tran

unread,
Jun 16, 2017, 6:28:52 PM6/16/17
to Jenkins Users, jenkin...@googlegroups.com
I will give it a spin too.

Thanks

-Dan

On Friday, June 16, 2017 at 11:57:26 AM UTC-7, Kevin Burnett wrote:
we'd be down to try that, yes. thanks for making these changes in a way that will benefit the product long-term!

fingers are crossed that there's already a built-in way to pretend like pull requests don't exist! you're already building the branches; why also build the pull requests, eh? :)

thanks!
kb

Joseph P

unread,
Jun 16, 2017, 6:32:01 PM6/16/17
to Jenkins Developers, jenkins...@googlegroups.com
I'd love to get the bitbucket going, I really want merged PRs TODAY :D

Stephen Connolly

unread,
Jun 16, 2017, 6:52:07 PM6/16/17
to jenkin...@googlegroups.com
On Fri 16 Jun 2017 at 23:32, Joseph P <jose...@gmail.com> wrote:
I'd love to get the bitbucket going, I really want merged PRs TODAY :D

If you are on BB server you should be fine

If you are on BB cloud, unless you set permissions up when forking, only PRs from forks in the team account and PRs from the origin repo will be built.

You can choose merge, head or both

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Owen Mehegan

unread,
Jun 16, 2017, 6:57:41 PM6/16/17
to jenkin...@googlegroups.com
Does this mean we can move forward with working on GitLab branch source work after these changes are final?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BnPnMxfYrZphgYDXFD3i%2Bo_7eDn7mn2qVrzJz6wFaoVkNmc%2Bw%40mail.gmail.com.

Stephen Connolly

unread,
Jun 17, 2017, 2:30:38 AM6/17/17
to jenkin...@googlegroups.com
On Fri 16 Jun 2017 at 23:57, Owen Mehegan <omeh...@gmail.com> wrote:
Does this mean we can move forward with working on GitLab branch source work after these changes are final?

Yes

(I think the changes are final now, but let's see what it takes to get the GitHub changes polishing finished)

On Fri, Jun 16, 2017 at 11:18 AM, Stephen Connolly <stephen.al...@gmail.com> wrote:
Just a quick status update.

In final stages of this work now. Bobby is being a superstar and reviewing my 13k LoC change on the Bitbucket branch source - brings lots of feature parity with GitHub and adds the configuration ability of the pure Git branch source

I am finalising the GitHub Branch Source changes... likely to be another big PR

Then there's a 5k LoC change in the Git plugin

Plan is to try and get all merged next week and cut a beta

I'll be looking for people to help test at that stage.

Please respond if you think you can help (lots of bugs fixed as a side effect of the refactoring - it makes things more easy to test => I found and fixed bugs)
--
Sent from my phone

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAHtcACF7FpgZ%3D_i1596JMuWRipfKhxVPNT3n%3D%3DzPt%2BwUOaOb0g%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

Łukasz Zachulski

unread,
Jun 17, 2017, 2:43:29 AM6/17/17
to Jenkins Developers
Can you summarize what can be expected from those new versions
Is it really only UI refactoring or do changes introduce something more?

Stephen Connolly

unread,
Jun 17, 2017, 4:36:00 AM6/17/17
to jenkin...@googlegroups.com
So there is the UI refactoring that should make configuration easier to understand: instead of 6 checkboxes with 64 possible combinations that are hard to comprehend and identify which combinations are nonsensical... there are three "discovery" traits that control discovery of origin branches, origin PRs and fork PRs

New functionality is that we can now expose as traits configuration of the git extensions, so if you want to checkout to a local branch rather than a detached head, if you want to control submodule behaviour, etc that is all now available (we also remove the git extensions options that are incompatible with branch sources to stop people complaining when they don't work)

New functionality is that - by default - we only clone the minimal ref specs required (add the advanced clone behaviour trait and it will be back to as before)

New functionality is that we can use shared code for filtering repositories and branches by wild cards or by regex (and somebody can add Glob style wild cards via an extension plugin and *everyone* gets them!)

Then there are the subtle bugs that I fixed due to the better testability of the trait approach

Then Bitbucket gets merge commits (except mercurial repositories on Bitbucket cloud... but I hope to address that once I figure out how to get mercurial to do the merges)

I claim that the data migration is fully tested... so what I am looking for from testers is:

* any issues with data migration
* any branches / PRs that were present before and have now disappeared on first rescan after upgrade
* any branches / PRs that fall in a "rebuild" storm after upgrade / rescan

I believe there should be none of these issues but we need testing from other people's real configuration before we can push an non-beta release for a set of changes this big (40k LoC in the diffs... though a lot - at least half - of that is new tests)

On Sat 17 Jun 2017 at 07:43, Łukasz Zachulski <lzach...@gmail.com> wrote:
Can you summarize what can be expected from those new versions
Is it really only UI refactoring or do changes introduce something more?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Joseph P

unread,
Jun 17, 2017, 7:01:42 AM6/17/17
to Jenkins Developers
BB server :) so all good

Stephen Connolly

unread,
Jun 20, 2017, 9:29:33 AM6/20/17
to jenkins...@googlegroups.com, jenkin...@googlegroups.com
OK! Here we are... testing time!

These are the plugins that are being covered: (download links should be live in an hour or two)


Recommended testing procedure:

1. Set up a throw-away Jenkins running a version similar to your production environment with the pre-upgrade versions of the plugins you are using.
2. Set up ideally at least one organization folder and one standalone multibranch project building your source code - to a first order you do not care if the builds succeed or fail, only that the branches are found.
3. Trigger a scan / index of your organization folders and standalone multibranch projects.
4. Wait for the queue to complete
5. Run the script in the system script console: https://gist.github.com/stephenc/64ef58783b4438a126ad4e3f43062df1 and save the output to smoke-pre-upgrade.txt
6. Upgrade the relevant plugins, restart Jenkins.
7. Run the script in the system script console: https://gist.github.com/stephenc/64ef58783b4438a126ad4e3f43062df1 and save the output to smoke-post-upgrade.txt
8. Trigger a scan / index of your organization folders and standalone multibranch projects.
9. Wait for the queue to complete
10. Run the script in the system script console: https://gist.github.com/stephenc/64ef58783b4438a126ad4e3f43062df1 and save the output to smoke-post-rescan.txt

At this point, do a diff between smoke-pre-upgrade.txt and smoke-post-rescan.txt

You are looking for three classes of difference:

a. branch jobs that have been rebuilt for no reason (i.e. the revision is the same)
b. branch jobs that have disappeared for no good reason (i.e. the branch is still present in the backing scm)
c. branch jobs that have suddenly appeared for no good reason (i.e. the branch was there before but not found) [expecting some of these for BitBucket PRs from forks, but only after configuration updated, saved and another rescan performed]

My expectation is that nobody will have these kinds of issues.

Also try out the new UI to see what you think.

Please report back your testing results either way. Don't forget to report back your UI feedback too ;-)

After doing that test in a throw-away Jenkins, you can *optionally* repeat the test on a more production-like (emphasis on being production-like not production) instance... but this is code that has not yet completed code review (hence -alpha-1 not -beta-1) so it is at your own risk. There are additional issues to be aware when using more production-like environment:

a. You may have builds that were assuming branches were full clones, now the refspec is tightly reduced to minimize clone time. If you need a full clone you will need to add the "Advanced Clone" behaviour.
b. Mercurial repositories on Bitbucket Cloud do not support merge commits for PR building (yet)
c. Credential domains were not being correctly compared so as a result - if you are using credential domains to help sort credentials - there may be cases where the credentials are now searched for in a different domain than you had them in, so your domains may need reconfiguration to have the credentials found by the multibranch project / org folder.
d. The pipeline snippitizer is generating $class style for some of the GitHub and BitBucket specific behaviours, this is because my plan is to further consolidate the implementations and have a single shared implementation of each for these plugins, that way they can have a single @Symbol annotation... if that is too difficult then the @Symbol would need to be prefixed with gitHub / bitbucket respectively, e.g. gitHubBranches, bitbucketBranches for the discover branches behaviour.


Thanks in advance

-Stephen

On 18 June 2017 at 15:53, Michael Kobit <mko...@gmail.com> wrote:

I may be able to help with this as well.


You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/5dd15ac2-b8a2-4ebd-bb4a-3bffa4815227%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CALELY9GHbX4WuHdDKM8-bU1xR5voh-NsfHeQXNAxMjJpXkiwrw%40mail.gmail.com.

Stephen Connolly

unread,
Jun 20, 2017, 9:53:02 AM6/20/17
to jenkins...@googlegroups.com, jenkin...@googlegroups.com
If you are chomping at the bit, here are all the binaries:


SHA1 checksums:
d9c346ac8db497a35825c7dbbb934842a2bc429a  branch-api.hpi
16da429f09fb585fd1d744809ee22c8d612fb62c  cloudbees-bitbucket-branch-source.hpi
234fa8eb88dad3241d620bb0116dd12fb9decbba  git.hpi
a68be01144f3045f81a5cf3c0bc60ad12f39b643  github-branch-source.hpi
92237097815b45260bb8b272caa9be9f92eb5085  mercurial.hpi
04c321420b3752a8d8b3af89cae1bf5934607b1c  scm-api.hpi

SHA256 checksums:
858ce20992c3f179b850c512979999084b11fe7c4c173cf6d4d2e07bbfebf3e7  branch-api.hpi
8ebff7a3ec43df276d4b51d1e5bcb910bbe8eb4cd47a4be0e35f2f2ca1cd0e03  cloudbees-bitbucket-branch-source.hpi
46cbbf11395df4a085829094d5a36dee7328aeba00d33e34b44aa0dcf9898248  git.hpi
6495a60f1bf0733d807f412434c6c2e24b7bba53fd7ce348ca5319ef38571f20  github-branch-source.hpi
173d12042fe8582efdb52e740f4e939b9daa05f181c6aaff31824337d519a31c  mercurial.hpi
9b58e9e6d13ce90a91b73f38142bf0977f244df9c52b948988f9d5bdc3785481  scm-api.hpi

-Stephen

Stephen Connolly

unread,
Jun 21, 2017, 2:30:03 PM6/21/17
to jenkins...@googlegroups.com, jenkin...@googlegroups.com
How many people have been able to try this so far?

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Mark Waite

unread,
Jun 21, 2017, 2:39:30 PM6/21/17
to jenkin...@googlegroups.com, jenkins...@googlegroups.com
I haven't tried it yet.  I'm configuring the "before" state today and will capture its state, then will deploy the new code tomorrow morning and capture the after state.  I won't do anything to compare those until tomorrow evening or this weekend.

Mark Waite

Michael Neale

unread,
Jun 21, 2017, 11:29:53 PM6/21/17
to Jenkins Developers, jenkins...@googlegroups.com
I saw all jobs disappear in a github org folder that blue ocean created previously, when I triggered a scan. 

(it also doesn't seem to trigger scans if I add change the pattern, or run the wizard again). 

Stephen Connolly

unread,
Jun 22, 2017, 1:20:25 AM6/22/17
to jenkin...@googlegroups.com, jenkins...@googlegroups.com
On Thu 22 Jun 2017 at 04:29, Michael Neale <mne...@cloudbees.com> wrote:
I saw all jobs disappear in a github org folder that blue ocean created previously, when I triggered a scan. 

(it also doesn't seem to trigger scans if I add change the pattern, or run the wizard again). 

I need to investigate some of the hacks BO is using.

I suspect this is probably just one deprecated constructor that needs tweaking


For more options, visit https://groups.google.com/d/optout.

Stephen Connolly

unread,
Jun 22, 2017, 2:05:52 AM6/22/17
to jenkin...@googlegroups.com, jenkins...@googlegroups.com
I believe the issue is due to BlueOcean continually replacing the GitHubSCMNavigator and the legacy constructor not populating the discovery traits using the legacy defaults. Great that we caught this now.

This does point to an issue for anyone using BlueOcean and configuring traits on a BlueOcean-managed organization folder... namely any configuration they do in the classic UI will be blown away by the next change from the BlueOcean UI... this only applies to GitHub Org Folders

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

Joseph P

unread,
Jun 22, 2017, 2:10:41 AM6/22/17
to Jenkins Developers, jenkins...@googlegroups.com
I will try and get around to testing it today.

Michael Neale

unread,
Jun 22, 2017, 4:27:34 AM6/22/17
to Jenkins Users, jenkin...@googlegroups.com
It does look better now with the new version. I confirmed running a scan didn't make anything go away. 

Stephen Connolly

unread,
Jun 22, 2017, 6:56:39 AM6/22/17
to jenkin...@googlegroups.com, Jenkins Users
So yes, it's github-branch-source 2.2.0-alpha-3 that is required for as good as possible inter-op with BlueOcean.

Note that when using BlueOcean with GitHub Org Folders you will fall victim to JENKINS-45058 irrespective of the version of GitHub Branch Source that you are using... the migration to traits makes this issue even more evident as now it is no longer the includes / excludes / build branches / build branches that are PRs / build origin PR head / build origin PR merge / build fork PR head / build fork PR merge / pattern fields that are subject to being reset but also the new traits... so if you configure say a Git LFS checkout behaviour and then add a repository with Blue Ocean, that Git LFS behaviour will be removed by BlueOcean

https://github.com/jenkinsci/blueocean-plugin/pull/1188 or https://github.com/jenkinsci/blueocean-plugin/pull/1187 fix that bug in BlueOcean, but there are some questions that need resolving before those PRs can be finalized.

https://github.com/jenkinsci/blueocean-plugin/pull/1186 shows that the changes (as of 2.2.0-alpha-2 + ) do not introduce any regressions in the interaction with BlueOcean... though obviously as JENKINS-45058 was always present, it remains so.    

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/0a9e9a1d-ac99-4acb-a9ca-8546c40b6377%40googlegroups.com.

Mark Waite

unread,
Jun 22, 2017, 8:41:36 AM6/22/17
to jenkin...@googlegroups.com, Jenkins Users
I used the experimental update center to update to the latest beta plugin versions.  I didn't have access from my Jenkins to the CloudBees BitBucket Source Branch plugin, so I'm ignoring it.

Mark Waite

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

Stephen Connolly

unread,
Jun 22, 2017, 9:17:36 AM6/22/17
to jenkin...@googlegroups.com, Jenkins Users
It's now known as the BitBucket Branch Source plugin... but the plugin short-name is cloudbees-bitbucket-branch-source

On 22 June 2017 at 13:41, Mark Waite <mark.ea...@gmail.com> wrote:
I used the experimental update center to update to the latest beta plugin versions.  I didn't have access from my Jenkins to the CloudBees BitBucket Source Branch plugin, so I'm ignoring it.

Mark Waite

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAO49JtEgQcGACu6RgfyjbQk%2BnSO3FUvdQOGM_Q09j1nejFVOJQ%40mail.gmail.com.

Stephen Connolly

unread,
Jun 22, 2017, 9:19:43 AM6/22/17
to jenkin...@googlegroups.com, Jenkins Users
In any case we are probably more interested in other Bitbucket users as at most you (Mark) would end up using the CloudBees testing Bitbucket Server instance which is the same one I used and we are more interested in testing a variety of configurations not multiple people testing the same configuration ;-)

Mark Waite

unread,
Jun 22, 2017, 9:29:54 AM6/22/17
to jenkin...@googlegroups.com, Jenkins Users
On Thu, Jun 22, 2017 at 7:19 AM Stephen Connolly <stephen.al...@gmail.com> wrote:
In any case we are probably more interested in other Bitbucket users as at most you (Mark) would end up using the CloudBees testing Bitbucket Server instance which is the same one I used and we are more interested in testing a variety of configurations not multiple people testing the same configuration ;-)


My bitbucket testing is typically limited to the bitbucket.org account that I've used for git plugin testing.  I'm sure that type of account (and server) will be exercised by others.

I hadn't added that plugin to my standard set of installed plugins.  I'll add it now.

Mark Waite
 
On 22 June 2017 at 14:17, Stephen Connolly <stephen.al...@gmail.com> wrote:
It's now known as the BitBucket Branch Source plugin... but the plugin short-name is cloudbees-bitbucket-branch-source
On 22 June 2017 at 13:41, Mark Waite <mark.ea...@gmail.com> wrote:
I used the experimental update center to update to the latest beta plugin versions.  I didn't have access from my Jenkins to the CloudBees BitBucket Source Branch plugin, so I'm ignoring it.

Mark Waite

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

Mark Waite

unread,
Jun 22, 2017, 11:05:04 AM6/22/17
to Jenkins Users, jenkin...@googlegroups.com
My docker image that I've upgraded is showing an unexpected behavior in a GitHub (not git) 

Stephen Connolly

unread,
Jun 22, 2017, 11:09:29 AM6/22/17
to jenkins...@googlegroups.com, jenkin...@googlegroups.com
Mark, can you share details win me via hipchat?


--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/642d0696-e348-4924-85cd-49f69b6d8658%40googlegroups.com.

Mark Waite

unread,
Jun 22, 2017, 11:13:17 AM6/22/17
to Jenkins Users, jenkin...@googlegroups.com
Sorry, fat fingers.

I have a multi-branch pipeline job defined to read from https://github.com/MarkEWaite/git-client-plugin using Git with username / password credentials.  It works as expected with the alpha versions of the plugins.  When I "configure" that job, it shows the project repository, and shows the credentials as "MarkEWaite github username/password".

I have a multi-branch pipeline job defined to read from https://github.com/MarkEWaite/git-client-plugin using GitHub with username / password credentials.  It works as expected with the alpha versions of the plugins.  However, when I configure that job, it shows the GitHub credentials box and then reports HTTP Error 404 - problem accessing job/Git-Client-Folder/job/git-client-pipeline-github/null : Reason: not found.  

I assume something is causing that trailing "null" on the end of the request URL, though I don't know what would do that.  Is that likely related to the alpha versions of the plugins, or should I look elsewhere for the difference?

Those two pipeline jobs are both able to be configured in my docker instance that runs released versions of all the plugins (except the git plugin and the git client plugin, which always seem to be in some pre-release state in that docker instance).

Suggestions of where I should search?

Mark Waite

Mark Waite

unread,
Jun 22, 2017, 12:10:45 PM6/22/17
to Jenkins Developers, jenkins...@googlegroups.com
False alarm, or rather, accurate alarm that has nothing to do with the alpha releases of the plugins.

I see an NPE in the console log with the released versions of all plugins, using Jenkins 2.46.3.  It is an issue, but not an issue with the alpha releases of the plugins.

Mark Waite

Mark Waite

unread,
Jun 22, 2017, 6:40:10 PM6/22/17
to Jenkins Developers, jenkins...@googlegroups.com
JENKINS-45085 captures the problem I was seeing.  The multiple-scms plugin causes a null pointer exception in my multi-branch pipeline job when opening the configure URL of the job.  That problem is visible with Jenkins 2.46.3 and 2.60.1.

Since my environment does not need or use the multiple-scms plugin, I removed it from my Docker image and resolved the problem.

Mark Waite

Mark Waite

unread,
Jun 22, 2017, 8:02:59 PM6/22/17
to Jenkins Developers, jenkins...@googlegroups.com
Question inline for a specific plugin, but the question may be general enough to merit discussion.
The git client plugin has tests which assume its branches are full clones.  It uses that assumption to reduce the setup time for submodule tests, tests of tagging, and more.  It also uses the global pipeline library in its Jenkinsfile so that the Jenkinsfile is a single line "buildPlugin".

I very much like the single line "buildPlugin" Jenkinsfile.  I'm willing (if necessary) to rework the git client plugin tests so that it no longer assumes branches are full clones.  Unfortunately, time spent reworking those tests will be time that is not spent reviewing pull requests or testing new ideas.

Is there a way that I can keep the convenience and power of buildPlugin() without burdening other plugin developers with the weight of a full clone?

Mark Waite
 

Joseph P

unread,
Jun 23, 2017, 9:18:21 AM6/23/17
to Jenkins Developers, jenkins...@googlegroups.com

Upgrade on "prod like" went smooth :)
Could we move the question for behaviours closer to the behaviours text?

Stephen Connolly

unread,
Jun 23, 2017, 9:32:15 AM6/23/17
to jenkin...@googlegroups.com, jenkins...@googlegroups.com
On 23 June 2017 at 14:18, Joseph P <jose...@gmail.com> wrote:

Upgrade on "prod like" went smooth :)

w00t!
 
Could we move the question for behaviours closer to the behaviours text?

I do not think we can... I think it is a property of f:entry... we could perhaps get somebody to give us some CSS to make it align top


is the style that needs to change 
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/28dce0c5-4d1e-4a0c-a9a6-f5f2f703172d%40googlegroups.com.

Stephen Connolly

unread,
Jun 23, 2017, 9:32:54 AM6/23/17
to jenkin...@googlegroups.com, jenkins...@googlegroups.com
How do you find the new UI compared with the previous one?

On 23 June 2017 at 14:18, Joseph P <jose...@gmail.com> wrote:
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/28dce0c5-4d1e-4a0c-a9a6-f5f2f703172d%40googlegroups.com.

Mark Waite

unread,
Jun 23, 2017, 11:58:52 AM6/23/17
to Jenkins Developers, jenkins...@googlegroups.com
The UI experience has been great for me in the two or three places where I've used it.  I was a little surprised (and pleased) with the adaptation that the local branch setting is now a toggle.  I think that's the right approach, since (as far as I can tell) that is the 99% use case.

Earlier I reported an NPE when configuring a multi-branch pipeline that uses GitHub as source instead of Git as source.  The NPE was resolved by removing the multiple-scms plugin.  Unfortunately, the 404 is still there, along with a stack trace that starts with this:

Jun 23, 2017 9:51:38 AM hudson.ExpressionFactory2$JexlExpression evaluate
WARNING: Caught exception evaluating: descriptor.calcFillSettings(field,attrs) in /job/Git-Client-Folder/job/git-client-pipeline-github/configure. Reason: java.lang.IllegalStateException: class org.jenkinsci.plugins.github_branch_source.GitHubSCMSource$DescriptorImpl doesn't have the doFillCredentialsIdItems method for filling a drop-down list
java.lang.IllegalStateException: class org.jenkinsci.plugins.github_branch_source.GitHubSCMSource$DescriptorImpl doesn't have the doFillCredentialsIdItems method for filling a drop-down list
at hudson.model.Descriptor.calcFillSettings(Descriptor.java:412)
at sun.reflect.GeneratedMethodAccessor578.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.commons.jexl.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:258)
at org.apache.commons.jexl.parser.ASTMethod.execute(ASTMethod.java:104)
at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51)
at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:74)
at org.apache.commons.jelly.parser.EscapingExpression.evaluate(EscapingExpression.java:24)
at org.apache.commons.jelly.impl.ExpressionScript.run(ExpressionScript.java:66)
at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)

I'm not sure how to provide a repeatable condition for that bug yet, but wanted to alert you about it.  I won't investigate further on it until after the end of the working day today.

Mark Waite

Mark Waite

unread,
Jun 23, 2017, 12:24:58 PM6/23/17
to Jenkins Developers, jenkins...@googlegroups.com
I see duplicate entries in the "Add' configuration of the Bitbucket source for "Checkout over ssh".  Let me know if you need steps to see that.

I also wonder if the text "General", "Git" and "Bitbucket" should be italicized, or bold, or separated with dashes, or something, so that the user has a concept that things will be appearing under them.  They seem to be standard text currently, and it wasn't obvious to me that they were categories into which settings would be placed.

Mark Waite

Jesse Glick

unread,
Jun 23, 2017, 12:37:31 PM6/23/17
to Jenkins Dev
On Thu, Jun 22, 2017 at 8:02 PM, Mark Waite <mark.ea...@gmail.com> wrote:
> The git client plugin has tests which assume its branches are full clones.
> It uses that assumption to reduce the setup time for submodule tests, tests
> of tagging, and more.

Wait, what? It is assuming that the tests are running in a
particularly configured checkout of the `git-client-plugin`
repository? That is certainly an antipattern. Tests should not even
assume that they are running in a Git checkout at all. Could just have
been a `cp -ar` from a source tree. Definitely worth cleaning this up
if I am understanding correctly what you are saying.

Stephen Connolly

unread,
Jun 23, 2017, 12:45:33 PM6/23/17
to jenkin...@googlegroups.com, jenkins...@googlegroups.com
On 23 June 2017 at 17:24, Mark Waite <mark.ea...@gmail.com> wrote:
I see duplicate entries in the "Add' configuration of the Bitbucket source for "Checkout over ssh".  Let me know if you need steps to see that.

Shouldn't... may just be a bug in the drop down populator when you have GitHub and Bitbucket
 

I also wonder if the text "General", "Git" and "Bitbucket" should be italicized, or bold, or separated with dashes, or something, so that the user has a concept that things will be appearing under them.  They seem to be standard text currently, and it wasn't obvious to me that they were categories into which settings would be placed.

Cannot style the drop-down menu without significant JS changes that risk affecting form binding.
 

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

Mark Waite

unread,
Jun 23, 2017, 1:14:11 PM6/23/17
to jenkin...@googlegroups.com
I'm not sure I'd call it a "particularly configured checkout" since I generally consider "checkout" to be the results of a "git checkout", though I guess that is one way to describe it.  

Some of the current tests assume that the working repository is an accurate clone of the upstream repository from which it was cloned, including tags and remote branches (like tests/getSubmodules).  It was an easy way to test submodules, remote clones, and authentication, without having to construct repositories containing specific tags and submodules each time.

Since it is an anti-pattern, I'll plan to systematically replace those dependencies with similar conditions constructed from within the tests themselves.

Thanks,
Mark Waite
 
--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1hQ%2B2po6fcoM-ba72GKat6uCng0xspFDWr3n9f5RK6EQ%40mail.gmail.com.

Stephen Connolly

unread,
Jun 25, 2017, 4:20:22 AM6/25/17
to jenkin...@googlegroups.com
I have found an issue with how these plugins get their pipeline snippets generated when all are installed an the same instance at the same time.

Hopefully this is just a bug in the structs plugin and I can fix it there...

BUT if I cannot fix it in structs I WILL HAVE TO RENAME CLASSES before a beta release.

If I have to rename classes you will not be able to upgrade from -alpha to -beta or GA as I cannot have the old classes present (but I will see if I can find a way around)


For more options, visit https://groups.google.com/d/optout.

Stephen Connolly

unread,
Jun 25, 2017, 4:50:10 AM6/25/17
to jenkin...@googlegroups.com
TL;DR keep the alpha's to throw-away instances until I identify whether I need to rename classes before the beta

Stephen Connolly

unread,
Jun 26, 2017, 5:37:07 AM6/26/17
to jenkin...@googlegroups.com
Phew! I found the bug in structs... and it is an easy fix... https://issues.jenkins-ci.org/browse/JENKINS-45130 so no renaming of classes required!

On 25 June 2017 at 01:49, Stephen Connolly <stephen.al...@gmail.com> wrote:
TL;DR keep the alpha's to throw-away instances until I identify whether I need to rename classes before the beta
On Sun 25 Jun 2017 at 09:20, Stephen Connolly <stephen.alan.connolly@gmail.com> wrote:
I have found an issue with how these plugins get their pipeline snippets generated when all are installed an the same instance at the same time.

Hopefully this is just a bug in the structs plugin and I can fix it there...

BUT if I cannot fix it in structs I WILL HAVE TO RENAME CLASSES before a beta release.

If I have to rename classes you will not be able to upgrade from -alpha to -beta or GA as I cannot have the old classes present (but I will see if I can find a way around)
On Fri 23 Jun 2017 at 18:14, Mark Waite <mark.ea...@gmail.com> wrote:
On Fri, Jun 23, 2017 at 10:37 AM Jesse Glick <jgl...@cloudbees.com> wrote:
On Thu, Jun 22, 2017 at 8:02 PM, Mark Waite <mark.ea...@gmail.com> wrote:
> The git client plugin has tests which assume its branches are full clones.
> It uses that assumption to reduce the setup time for submodule tests, tests
> of tagging, and more.

Wait, what? It is assuming that the tests are running in a
particularly configured checkout of the `git-client-plugin`
repository? That is certainly an antipattern. Tests should not even
assume that they are running in a Git checkout at all. Could just have
been a `cp -ar` from a source tree. Definitely worth cleaning this up
if I am understanding correctly what you are saying.


I'm not sure I'd call it a "particularly configured checkout" since I generally consider "checkout" to be the results of a "git checkout", though I guess that is one way to describe it.  

Some of the current tests assume that the working repository is an accurate clone of the upstream repository from which it was cloned, including tags and remote branches (like tests/getSubmodules).  It was an easy way to test submodules, remote clones, and authentication, without having to construct repositories containing specific tags and submodules each time.

Since it is an anti-pattern, I'll plan to systematically replace those dependencies with similar conditions constructed from within the tests themselves.

Thanks,
Mark Waite
--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
--
Sent from my phone

Mark Waite

unread,
Jun 26, 2017, 9:38:56 AM6/26/17
to jenkins...@googlegroups.com, jenkin...@googlegroups.com
The experimental update center includes a new version of the git plugin and the mercurial plugin.

Per Kevin Burnett's comment, please don't release it this week.  I need more testing time.

Mark Waite

On Mon, Jun 26, 2017 at 7:29 AM Michael Kobit <mko...@gmail.com> wrote:

I'm going to have time to do this today. Are there newer alphas available?


On Mon, Jun 26, 2017, 08:13 Kevin Burnett <kbur...@rosettastone.com> wrote:
This is so good. :)

The pre and post diffs looked right, and the new UI and functionality gives me everything that I was hoping for.

I'm going to remove the "discover pull requests from [everywhere]" behaviors and select "Only branches that are also filed as PRs" on production as soon as possible.

Michael Neale mentioned that one issue he had seen "does look better now with the new version." I used the plugin versions that Stephen originally posted on June 20, but I take Michael's comment to mean there might be newer versions. Please make this irrelevant by issuing release versions of these plugins this week. :)

Thanks a ton!
-KB
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/2bc996a5-b2d6-4c22-884a-a7a026cc9472%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CALELY9FMzPhsq2YPKSgnMZaz14bkVM1fnahFJoZt4ytkWPj5zQ%40mail.gmail.com.

Stephen Connolly

unread,
Jun 26, 2017, 9:45:36 AM6/26/17
to jenkins...@googlegroups.com, jenkin...@googlegroups.com
Check the drop-box link...  https://www.dropbox.com/sh/47weboatdzus22w/AADNF_aBniOyEeQi9MvM82sMa?dl=0 git, mercurial, github and bitbucket are all up as far as -alpha-4

-alpha-4 is looking very near a beta release candidate (just need to confirm I have all the code reviewers happy)

I will not be releasing the GA versions this week as we have some internal quality bars that need to be met

On 26 June 2017 at 06:28, Michael Kobit <mko...@gmail.com> wrote:

I'm going to have time to do this today. Are there newer alphas available?

On Mon, Jun 26, 2017, 08:13 Kevin Burnett <kbur...@rosettastone.com> wrote:
This is so good. :)

The pre and post diffs looked right, and the new UI and functionality gives me everything that I was hoping for.

I'm going to remove the "discover pull requests from [everywhere]" behaviors and select "Only branches that are also filed as PRs" on production as soon as possible.

Michael Neale mentioned that one issue he had seen "does look better now with the new version." I used the plugin versions that Stephen originally posted on June 20, but I take Michael's comment to mean there might be newer versions. Please make this irrelevant by issuing release versions of these plugins this week. :)

Thanks a ton!
-KB

On Friday, June 23, 2017 at 12:45:44 PM UTC-4, Stephen Connolly wrote:


To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/2bc996a5-b2d6-4c22-884a-a7a026cc9472%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CALELY9FMzPhsq2YPKSgnMZaz14bkVM1fnahFJoZt4ytkWPj5zQ%40mail.gmail.com.

Stephen Connolly

unread,
Jun 26, 2017, 9:46:42 AM6/26/17
to jenkins...@googlegroups.com, jenkin...@googlegroups.com
On 26 June 2017 at 06:13, Kevin Burnett <kbur...@rosettastone.com> wrote:
This is so good. :)

Great to hear it. I love feedback (+ve or -ve beats none)
 

The pre and post diffs looked right, and the new UI and functionality gives me everything that I was hoping for.

w00t
 

I'm going to remove the "discover pull requests from [everywhere]" behaviors and select "Only branches that are also filed as PRs" on production as soon as possible.

Michael Neale mentioned that one issue he had seen "does look better now with the new version." I used the plugin versions that Stephen originally posted on June 20, but I take Michael's comment to mean there might be newer versions. Please make this irrelevant by issuing release versions of these plugins this week. :)

Thanks a ton!
-KB

On Friday, June 23, 2017 at 12:45:44 PM UTC-4, Stephen Connolly wrote:
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/2bc996a5-b2d6-4c22-884a-a7a026cc9472%40googlegroups.com.

Joseph P

unread,
Jun 26, 2017, 11:14:07 AM6/26/17
to Jenkins Developers
This UI is a lot better! Looking forward to the massive upgrade!

Just wondering with the refactoring will it be possible for PRs to rebuild when changes to target branch is notified?
When the option for Merge PR with Target branch is selected.

That would be AWESOME!

Stephen Connolly

unread,
Jun 26, 2017, 12:13:07 PM6/26/17