Combining Maven Repository Server Plugin and CloudBees Folders Plugin

[Nick Stolwijk]

I'm trying to use both the Maven Repository Server Plugin[1] and the CloudBees Folders Plugin[2], but I have some trouble with it.

I have 2 projects, A and B, and I want the build of B to use the artifacts of a build of A. A is being triggered by a successful build of A. I also use the CFP to group my builds into folders.

So I have:

- Folder C

  * Build of Project A

- Folder D

  * Build of Project B

When build B is triggered, MRSP gives it the URL of build A, but the URL of the repository given to the build is malformed:

http://localhost:8080/plugin/repository/project/folder_c/build_of_project_A/Build/4/repository

When I look into the repository offered by the MRSP I see:

http://localhost:8080/plugin/repository/project/build_of_project_A/Build/4/repository

It seems that the folder is prepended to the path the plugin gives to the triggered builds.

Did I configure something wrong or is this an issue with one of the plugins?

Hope someone can help me.

With regards,

Nick Stolwijk

ps. I use the latest versions of Jenkins (1.631), MRSP (1.2) and CFP (4.10).

[1] http://wiki.jenkins-ci.org/display/JENKINS/Jenkins+Maven+Repository+Server

[2] https://wiki.jenkins-ci.org/display/JENKINS/CloudBees+Folders+Plugin

~~~ Try to leave this world a little better than you found it and, when your turn comes to die, you can die happy in feeling that at any rate you have not wasted your time but have done your best ~~~

Lord Baden-Powell

[Stephen Connolly]

Looks like the repository plugin has invalid assumptions about all jobs being at the root of Jenkins. It's probably using Item.getName in place of Item.getFullName and the corresponding reverse lookups

I'd check jira and if there isn't an issue already then create one.

(FYI this is not an issue with the folders plugin. The newer plugins that do multi-branch or repository scanning plus multi-branch will trigger the same issue in the repository plugin)

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAA9S6i74f8-bEzrLi%3D%2BFhq3nvo5TQVPVk71y%3D95WvGJtq0_dWQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
Sent from my phone

[Nick Stolwijk]

Did you already check jira?

Is the MRSP not used that often, and is their a better way to consume artifacts from another build? Or do we just have a strange usecase? 

We are switching from one big code tree to multiple subprojects and we are reorganising our Jenkins builds.

Project A is going to have multiple fases in the build. Unittests and packaging, integrationtests, regressiontests, and if all that succeeds, a deploy to Nexus, so other developers can use the artifacts produced.

We want Project B to start as early as possible (so with the artifacts from the unittest build of Project A), and so I stumbled into the MRSP.

Does anyone have any tips on how to organize this in Jenkins?

With regards,

Nick Stolwijk

~~~ Try to leave this world a little better than you found it and, when your turn comes to die, you can die happy in feeling that at any rate you have not wasted your time but have done your best ~~~

Lord Baden-Powell

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CA%2BnPnMwrHeM9vTEhKMsSeRxPuaVsQ27DreX%3DtvtCQ_y3bkbD5g%40mail.gmail.com.

[Nigel Magnay]

It's probably what stephen says -- when I wrote the plugin was some time ago - either before (or before my awareness of) multi-branches and other non-root project types. Probably not a big fix..

It's also (currently, but am fixing) tied to the 'evil job type' :-)

You might find workflow-plugin better suits your needs these days. I may dig around in MRSP too as we're switching to workflow+multi-branch from a large array of jobs that use MSRP.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAA9S6i43bDPhQchn%3DAN%2BFHbJyOJwzYmvLLETOeT4JOy_KmctVg%40mail.gmail.com.

[Nick Stolwijk]

When you are talking about the "evil job type", do you mean the Maven jobtype?

I'm using those to let Jenkins find out which jobs depend on each other. :S

Should I reconsider my strategy?

I'm currently busy to make little changes to my jobs to fulfill my needs and then automate them with the JobDSL. (I have a few of the same type of jobs). Everywhere I see examples of the Maven job in combination with the JobDSL. (And I find a lot of out of date blog posts and mails to these and other mailinglists).

Is there somewhere a "Best Practices with Jenkens and JobDSL in 2015" reference?

With regards,

Nick Stolwijk

~~~ Try to leave this world a little better than you found it and, when your turn comes to die, you can die happy in feeling that at any rate you have not wasted your time but have done your best ~~~

Lord Baden-Powell

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAPYP83SxqfXogu0Z%3DiUyV3YUxqmNoFzeLd-ysXmNtx8f-5BkKg%40mail.gmail.com.

[Stephen Connolly]

All this talk of the evil job type and the JobDSL plugin makes me a sad panda.

- Stephen

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAA9S6i4jwOLZYgffmwjBB8Ji_xkx8cimv8vf%2BbYz7Ua8BaT_ww%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Nick Stolwijk]

I'm sorry I make you feel like a sad panda. :( Would you set me on the right way as I'm just a young padawan on the road to enlightment.  ;)

With regards,

Nick

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CA%2BnPnMypAo-Mdhrb677y9EYFdzPk_Rj_HHkechZ6gxeYxXkAXQ%40mail.gmail.com.

[Nigel Magnay]

I'll describe my use-case (which led to MRSP), and maybe that'll help.

We do CI builds on every check-in (actually more complicated than that as we only have pre-tested commits).

We have 'customer' projects that use these artifacts.

We don't want to build the 'customer' projects at the same cadence as the CI builds (waaay too many), and we also wanted to break down our build into several stages (expressed as projects). 

Whilst it would be possible to use some kind of maven repo (e.g: nexus) to upload the artifacts to, there are 2 problems:

1) It's very wasteful transferring ~1Gb per CI build.

2) How do you manage 'what artifacts come from what build'. Either you have to massage the version number, or you have to have a repository-per-build - which doesn't fit very well with things like nexus.

Hence Maven-Repository-Server plugin.

How it works is it looks at the attached artifacts declared by Jenkins, and makes them available in a maven-style repository complete with metadata. Point downstream job to repo - job done, maven just uses its existing infrastructure to resolve the artifacts.

One issue is - how is the plugin aware of the artifacts. The resolution above relies on the 'maven' job type -- Jenkins instruments maven in order to (amongst other things) determine what artifacts have been created. If you flip to a 'freestyle' project, that data is missing - so the MSRP has nothing to go on.

The maven job type attracts the sobriquet 'evil job type' because of the allegation that the instrumentation process is unreliable. Personally, it's always worked for me (I.E: works perfectly for n=1); and I think it's sad that maven doesn't seem to be able to be reliably embedded in tooling in this way.

What I am working on for MSRP is an alternative solution where a maven plugin gets added to your build, and this reports back to the jenkins server paths of artifacts to 'attach', which thus allows MSRP to continue functioning for freestyle jobs. Yell if this interests you as I don't think I've pushed it anywhere yet.

For some of our use-cases (build pipeline rather than the customer stuff above) we're moving over to a workflow driven model. This obviates the need to move artifacts around, as steps simply operate on the same workspace.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAA9S6i6ghN%3D%2BLsviGVPpdV9%3DxmVHvTGjqStyvbeZaQzdUE9xHg%40mail.gmail.com.

[Stephen Connolly]

So the JobDSL plugin requires that you have a specific view on the
security model of your Jenkins instance (i.e. only administrators can
explicitly create/configure jobs, all other users get jobs created /
updated via the JobDSL using scripts managed by administrators)

A large chunk of installations do not share that security model (i.e.
letting users create/configure at least some jobs) and thus in those
installations the JobDSL plugin would essentially be a big glaring
security hole waiting for attack...

To be clear, I am not saying that the JobDSL plugin is insecure,
rather that its use mandates a specific security model and if you do
not share that model then it may not be for you.

As to the evil job type...
http://javaadventure.blogspot.ie/2013/11/jenkins-maven-job-type-considered-evil.html

> https://groups.google.com/d/msgid/jenkinsci-users/CAA9S6i6ghN%3D%2BLsviGVPpdV9%3DxmVHvTGjqStyvbeZaQzdUE9xHg%40mail.gmail.com.

[Nick Stolwijk]

That sounds like our usecase.

We have a shared component-tree (ComponentA), which we use in different projects. Some of these projects depend on a released version of the ComponentA and others on a SNAPSHOT version.

We are now looking for a Jenkins process, that when there is a commit to ComponentA all products that depend on a SNAPSHOT of ComponentA gets build. (Only after the build of ComponentA is finished and with the artifacts of that build. 

We can't use the dependencies from Nexus, because we want ComponentA only in Nexus if all the phases succeed. ComponentA build is split into a compile/UT/package phase, an IT phase and an RT/Deploy fase). We want the Product jobs to start after the first phase succeeds, so we quickly have the products to be tested.

So I was looking for a solution to use the artifacts from a previous job and I found the MRSP plugin. We also want the different jobs to be grouped in folders. One folder for ComponentA, one for ProjectB, etc.

@Stephen

I understand the problem with the Maven job type, but for now it seems the only solution to let Jenkins kickoff the right builds, ie only kickoff ProjectB's build if it depends on a SNAPSHOT of ComponentA.

For the JobDSL, we have a small department and everyone with a login can create jobs and security is not the issue. I like the way it circumvents the copy-paste way of maintaining jobs.

@Nigel

I would like to use the MSRP in combination with the Folder plugin. Would it be hard to fix? (I have taken a look at the code,  but couldn't figure it out. Not enough experience with the Jenkins programming model.)

With regards,

Nick Stolwijk

~~~ Try to leave this world a little better than you found it and, when your turn comes to die, you can die happy in feeling that at any rate you have not wasted your time but have done your best ~~~

Lord Baden-Powell

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAPYP83QA1sUifLYoFD6R1-tQ66N7FD_xkMkFLKnOaD9NgJtaCg%40mail.gmail.com.

[Stephen Connolly]

On 30 September 2015 at 12:10, Nick Stolwijk <nick.s...@gmail.com> wrote:
>
> For the JobDSL, we have a small department and everyone with a login can
> create jobs and security is not the issue. I like the way it circumvents the
> copy-paste way of maintaining jobs.
>

Like I said, the plugin assumes a specific security model, and if that
model matches what you need then it is perfect... but it is not good
if you need to restrict users

[Nick Stolwijk]

For now we have chosen to not use the Folder plugin, but go with the Repository Server Plugin.

At the moment the Maven job is good enough for us, but I guess we will move over to the Freestyle job in time. I would be very interested if the MRSP would also work for that type of job.

Also, I wonder how hard it would be to add JobDSL configuration to the MRSP.

Can I do anything to help with implementing these functionalities?

With regards,

Nick Stolwijk

~~~ Try to leave this world a little better than you found it and, when your turn comes to die, you can die happy in feeling that at any rate you have not wasted your time but have done your best ~~~

Lord Baden-Powell

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CA%2BnPnMyTpA2qrUDHAqZV39uJJ1b9qfZGfhnQgf23JhmR-Dnk_w%40mail.gmail.com.