Help needed with Workflow support in gitlab-plugin

243 views
Skip to first unread message

Owen B. Mehegan

unread,
Nov 11, 2015, 6:22:23 PM11/11/15
to Jenkins Developers
Earlier this year I bribed an intern and another coworker to implement Workflow support in the GitLab plugin (https://github.com/jenkinsci/gitlab-plugin for reference). They made an initial pass at it, with support from jglick. Now I've realized that the support is incomplete. We've since taken over maintainership of the plugin, but we're still totally inexperienced with Jenkins plugin code. I have two issues that we need help with.

First, I can't figure out how to access the branch parameters that the plugin is setting. If you look at the 'Using it with a job' section of the readme, it has you configure Git to expect variables like gitlabSourceBranch and gitlabTargetBranch to be set when a build is triggered by Gitlab. This works fine in freestyle builds. In a Workflow build, I used the snippet generator and it gave me this output (corrected with double quotes around strings containing variables):

checkout changelog: true, poll: true, scm: [$class: 'GitSCM', branches: [[name: "origin/${gitlabSourceBranch}"]], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'PreBuildMerge', options: [fastForwardMode: 'FF', mergeRemote: 'origin', mergeTarget: "${gitlabTargetBranch}"]]], submoduleCfg: [], userRemoteConfigs: [[name: 'origin', url: 'g...@gitlab.mysite.com:test/testrepo.git']]]

But when the job is triggered, those variables are not set - I get a groovy.lang.MissingPropertyException for them. I'm pretty sure that the relevant part of the plugin code is here: https://github.com/jenkinsci/gitlab-plugin/blob/master/src/main/java/com/dabsquared/gitlabjenkins/GitLabPushTrigger.java#L235 Beyond that, I'm clueless as to what might need to change.

Second, it seems that if the most recent build of the project failed to check out the repo, the plugin fails to trigger any further builds. It logs the error, 'Could not find GitSCM for project,' and that's generated here: https://github.com/jenkinsci/gitlab-plugin/blob/master/src/main/java/com/dabsquared/gitlabjenkins/GitLabPushTrigger.java#L802 I guess this is related to the fact that the plugin ALSO can't trigger a brand new job until you run it manually one time. Jesse mentioned that this would be a problem - running it causes it to store some gitSCM state which the plugin is then able to find. I would love to find a more elegant way to handle this case. I wonder if there's a way to create an additional field in the job config, outside the Workflow script, where we could specify the repo URL and have the plugin key off of that.

Thanks in advance for any guidance anyone can offer :)

Mirko Friedenhagen

unread,
Nov 12, 2015, 1:04:22 AM11/12/15
to jenkin...@googlegroups.com

Hello Owen,

as far I know environment variables need to be prefixed with env when accessing them from Groovy, so maybe it should be "origin/${env.gitlabSourceBranch}" …?

Regards
Mirko
--
Sent from my mobile

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/710c72a2-deee-414e-9fb1-0c86128aff91%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Owen Mehegan

unread,
Nov 12, 2015, 1:07:24 AM11/12/15
to jenkin...@googlegroups.com
I tried that, but the values come back null, so I don't think they're getting set as environment variables per se.

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/CJEbh85zRx8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAK8jvqxc4xMLTXnYH2xxekJ2c-gnCXEXw6pZX2cx4%3DFcyWvrbw%40mail.gmail.com.

Robert Sandell

unread,
Nov 12, 2015, 5:51:46 AM11/12/15
to jenkin...@googlegroups.com
It seems like when the job is not an AbstractProject e.g. a WorkflowJob then the job isn't scheduled with parameters https://github.com/jenkinsci/gitlab-plugin/blob/master/src/main/java/com/dabsquared/gitlabjenkins/GitLabPushTrigger.java#L204

[1] and [2] are some examples on how to schedule a job with parameters where the job implements ParameterizedJobMixIn.ParameterizedJob which any job type that takes parameters should implement (FreeStyle, Workflow Maven etc)
There are probably more examples out there.
/B


To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAHtcACEJ9aRenR0WNz-u--LMBGEqQbq6MqBQQJ6EZgRjLEjtow%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.



--
Robert Sandell
Software Engineer
CloudBees Inc.

Kanstantsin Shautsou

unread,
Nov 12, 2015, 9:47:36 AM11/12/15
to Jenkins Developers
@Robert both examples shows how hacky workflow support. Imho API should be clear for plugin developers.

@Owen If you unsure whether variables was put into build, then check build.xml content. If unsure what SCMs code see, then place breakpoint in IDE and view available data. Usually trigger plugins designed in 2 ways: 1) place plugin specific cause with some wrapped values in it 2) place StringParameters (sometimes both duplicated and contains the same values).

AFAIR gitlab copy-pasted design from github-plugin. Github-plugin has workflow support so you can continue following changes in it. You should also define correct isApplicable(). Plus you can bump core version to latest to get normal API methods if you don't want make hacks around, but this will cut many installations then.

Jesse Glick

unread,
Nov 12, 2015, 1:37:40 PM11/12/15
to Jenkins Dev
On Wed, Nov 11, 2015 at 6:22 PM, Owen B. Mehegan <ow...@nerdnetworks.org> wrote:
> I can't figure out how to access the branch parameters that the
> plugin is setting. If you look at the 'Using it with a job' section of the
> readme, it has you configure Git to expect variables like gitlabSourceBranch
> and gitlabTargetBranch to be set when a build is triggered by Gitlab.

Two things.

First of all, the code in `onPost` is wrong. See

https://github.com/jenkinsci/workflow-plugin/blob/master/COMPATIBILITY.md

for background but basically you need to be using
`ParameterizedJobMixIn`, which allows `actions` to be passed to any
build you schedule. There is no need to check for `instanceof
AbstractProject`.

Second, the whole system the plugin is using to specify a branch per
build can be considered obsolete. Instead, depend on `scm-api`,
implement `SCMSource`, and use multibranch workflows (or other
multibranch projects). See `github-branch-source` for an example.

Owen Mehegan

unread,
Nov 12, 2015, 2:03:46 PM11/12/15
to jenkin...@googlegroups.com
Some comments inline...

On Thu, Nov 12, 2015 at 2:51 AM, Robert Sandell <rsan...@cloudbees.com> wrote:
It seems like when the job is not an AbstractProject e.g. a WorkflowJob then the job isn't scheduled with parameters https://github.com/jenkinsci/gitlab-plugin/blob/master/src/main/java/com/dabsquared/gitlabjenkins/GitLabPushTrigger.java#L204

Aha... would this explain why the plugin also doesn't seem to work with matrix jobs?
 
[1] and [2] are some examples on how to schedule a job with parameters where the job implements ParameterizedJobMixIn.ParameterizedJob which any job type that takes parameters should implement (FreeStyle, Workflow Maven etc)
There are probably more examples out there.
/B

Thanks, I might make a pass at solving the immediate problem this way, just to unblock myself.
 

Owen Mehegan

unread,
Nov 12, 2015, 2:56:19 PM11/12/15
to jenkin...@googlegroups.com
Thanks Kanstantin, checking the build.xml helps. In the non-workflow projects where the plugin works, I see the parameters set. In the Workflow project there are no parameters set at all.

I don't think I understand your comments about how trigger plugins usually handle the SCM issue. Just due to my very limited experience I think. I think your #2 suggestion is sort of what I imagined: the plugin would create another field that unfortunately duplicates something like the git repo URL, so that you have to enter that twice. I'll look through the github-plugin code and see what I can figure out. 

Owen Mehegan

unread,
Nov 12, 2015, 3:01:43 PM11/12/15
to jenkin...@googlegroups.com
Some comments inline...

On Thu, Nov 12, 2015 at 10:37 AM, Jesse Glick <jgl...@cloudbees.com> wrote:
On Wed, Nov 11, 2015 at 6:22 PM, Owen B. Mehegan <ow...@nerdnetworks.org> wrote:
> I can't figure out how to access the branch parameters that the
> plugin is setting. If you look at the 'Using it with a job' section of the
> readme, it has you configure Git to expect variables like gitlabSourceBranch
> and gitlabTargetBranch to be set when a build is triggered by Gitlab.

Two things.

First of all, the code in `onPost` is wrong. See

https://github.com/jenkinsci/workflow-plugin/blob/master/COMPATIBILITY.md

for background but basically you need to be using
`ParameterizedJobMixIn`, which allows `actions` to be passed to any
build you schedule. There is no need to check for `instanceof
AbstractProject`.

OK, I'll look at refactoring this. 

Second, the whole system the plugin is using to specify a branch per
build can be considered obsolete. Instead, depend on `scm-api`,
implement `SCMSource`, and use multibranch workflows (or other
multibranch projects). See `github-branch-source` for an example.

I think I get what you're saying: that the plugin should move toward the multibranch project model, rather than one job which takes a parameter of branch (and whose current "status" is therefore usually meaningless). I'm open to that idea but I suspect it will be more work to implement. I'll consider it part of the roadmap for this plugin. 

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/CJEbh85zRx8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

Jesse Glick

unread,
Nov 12, 2015, 5:32:41 PM11/12/15
to Jenkins Dev
On Thu, Nov 12, 2015 at 2:03 PM, Owen Mehegan <omeh...@gmail.com> wrote:
> would this explain why the plugin also doesn't seem to work with matrix jobs?

No, matrix jobs and their configuration jobs are both
`AbstractProject`. But all bets are off when it comes to
`matrix-project` compatibility. Plugins often need to have `if`-`then`
blocks to make it work.

Jesse Glick

unread,
Nov 12, 2015, 5:34:05 PM11/12/15
to Jenkins Dev
On Thu, Nov 12, 2015 at 3:01 PM, Owen Mehegan <omeh...@gmail.com> wrote:
> that the plugin should move toward the
> multibranch project model, rather than one job which takes a parameter of
> branch (and whose current "status" is therefore usually meaningless).

Exactly.

> I suspect it will be more work to implement.

Yes, it is definitely a bigger change. You can of course support both
models concurrently.
Reply all
Reply to author
Forward
0 new messages