Pipeline external global library in SVN - errors

[Brian Ray]

Despite a quick runthrough from Jesse Glick at Jenkins World, I'm bumping up against two different blockers trying to use an SVN repo as an external global library repository with Pipeline.

All plugins are up to date, and the master is running LTS 2.7.4 on Windows.

In all attempts the script begins with

@Library('pipeline-global-helpers') _

or

@Library('pipeline-global-helpers@trunk') _

Neither variant made a difference.

First Try
This seems like the most logical configuration, with the version token in the Repository URL.

But on running the script, the token is interpreted as a literal and the checkout fails:

Second Try
This configuration didn't make a lot of sense but I just spelled out the Repository URL literally and put the token in the Default version field:

And strangely enough it sort of worked. It checked out the code, but then could not find the subdirectories that are clearly present.

Third Try
Similar to the second try, I put the token in another questionable field and got the same result as the second try.

Can anybody see what I am doing wrong? I feel like I'm missing something elementary, as Jesse Glick has tested this enhancement with an SVN repo.

Thanks!

[Brian Ray]

Evidently I cannot drive this post widget very well. The screenshots are best clicked in reverse order, with the last two corresponding to the First Try, the middle two to Second Try, and the first two to First Try.

[Michael Lasevich]

Is it possible that this is some odd issue with '-' symbols in the library name? I would try a simpler name. Also try to implicit load to simplify the load...

-M

[Brian Ray]

You beat me to the post. The hyphen in the lib name is causing the failure to interpolate, much the same as it would in Groovy GString interpolation. (Though I think the mechanism here is different, since plugins are written in Java ... )

The above lib config fixed the interpolation problem. But the vars/ and src/ subdirectory discovery issue pops up again.

Started by user Brian Ray
Loading library helpers@trunk
Updating https://XXXXXXXXXXXXXXXXXXXXXX/svn/releng/trunk/retools/pipeline-global-libs/pipeline-global-helpers at revision '2016-09-16T15:50:21.511 -0700'
At revision 85052

No changes for https://XXXXXXXXXXXXXXXXXXXXXXXXXX/svn/releng/trunk/retools/pipeline-global-libs/pipeline-global-helpers since the previous build
ERROR: Library helpers expected to contain at least one of src or vars directories
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: Loading libraries failed

1 error

	at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310) [...]

I bet there is something finicky about the root directory of the SVN working copy. But I will give Load implicitly a shot too.

Thanks

[Michael Lasevich]

Implicit load was to work around issues with '@Library' syntax, but I doubt that is your issue here.

I would check that your SVN URL is pointing to directory that has "vars" in it and double-check that it is checking out the right dir. Look for "vars" dir in  <job>/workspace@libs/ dir in yours job and/or "<job>/builds/##/libs/helpers/" inside your specific build)

I would also maybe wipe the local cache of the svn repo and force a full checkout again.

-M

[Brian Ray]

All good tips. I've been blowing away the local copy between attempts and currently have this structure (note: I changed the configured libname from helpers to pipelineGlobalHelpers in the latest attempt, and this is checking out on my local troubleshooter master right now--this is even before the first stage or node blocks):

c:\Jenkins\workspace\Dev-Snippets\x-pipeline-1@libs\pipelineGlobalHelpers>dir
 Volume in drive C has no label.
 Volume Serial Number is 4456-EE0F

 Directory of c:\Jenkins\workspace\Dev-Snippets\x-pipeline-1@libs\pipelineGlobalHelpers

2016-09-16  16:18    <DIR>          .
2016-09-16  16:18    <DIR>          ..
2016-09-16  16:18    <DIR>          resources
2016-09-16  16:18    <DIR>          src
2016-09-16  16:18    <DIR>          vars
               0 File(s)              0 bytes
               5 Dir(s)  249,994,575,872 bytes free

This is likely the method that's complaining; see the throw at the end.

The thing that I am puzzling at is the construction of the directory path:

FilePath libDir = new FilePath(execution.getOwner().getRootDir()).child("libs/" + name);

That to me suggests it yields c:/Jenkins/workspace/Dev-Snippets/x-pipeline-1/libs/pipelineGlobalHelpers, instead of .../x-pipeline-1@libs/pipelineGlobalHelpers, but maybe I am misunderstanding FilePath#child or #getRootDir.

There are still a few more experiments to try.

[Michael Lasevich]

You are probably farther along than me as I have not even looked at the code, but what I am observing on my end is this:

There is a some sort of a shared location, in my case "workspace@libs/" directory, which seems to house the repo (note that name of the library is not there), and then inside individual builds there is a lib/<name>/ directory which contains the snapshot in time of the library that is actually used for this build. I am using git, so there is a git repo in the first place, but only the files in the build - but I suspect there is a similar thing for svn

A few things that I see here that are different in your case:

1 - Your workspace is not named "workspace@libs" but is instead named "x-pipeline-1@libs" - not sure if this is a per-build directory, or a shared repo dir

2 - If #1 is your shared directory,  you have an extra subdirectory in there.

Try searching for "cmd.groovy" under your job directory and see where all the copies are. See where there is a .svn dir vs where there is just a copy and make sure that it is in the right place.

-M

[Brian Ray]

These discrepencies nudged me toward the solution. Until my latest attempt, I only observed the checkout in that shared location (1). There was no checked out library code in the build directories (2). But I now have a clean test build and build library subdirectories as you detailed, in addition to the shared location. The difference in configuration is the Local module directory value above, where the dot strips the parent project directory.

The method source in the plugin's Github repo makes much more sense now.

I wonder if my previous attempts are edge cases that warrant a JIRA. The shared library resolver should not, it seems, get confused if the project/module directory is retained on checkout. At the least the README.md could contain some more guidance on configuration in this case.

Thanks again!

[Brian Ray]

I entered low priority JIRA-38517 for the above issue.