String parameter oddness, something I'm not seeing, or a bug?

[Jonathan Hodgson]

Ok, so I have a project that receives a diff (it's a "try before commit" project) as a string parameter (it's a pipeline project, so I can't send it as a file parameter, or at least I couldn't last time I checked, they don't work in pipelne projects).

This was all working fine, at least for the various test diffs I was using as I was trying to develop the next stage of the project, and then I ran into one that just seems to break things

This is the diff (don't worry too much about the content, this is a test project to develop the idea for the try-before-commit, and  it's mostly me messing about checking various concepts work.

diff -r dee4e38d33db .hgsub
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/.hgsub Fri Jul 01 11:12:01 2016 +0100
@@ -0,0 +1,1 @@
+subrepo = subrepos
\ No newline at end of file
diff -r dee4e38d33db .hgsubstate
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/.hgsubstate Fri Jul 01 11:12:01 2016 +0100
@@ -0,0 +1,1 @@
+0000000000000000000000000000000000000000 subrepo
diff -r dee4e38d33db jenkins-build.groovy
--- a/jenkins-build.groovy Wed Jun 01 17:12:42 2016 +0100
+++ b/jenkins-build.groovy Fri Jul 01 11:12:01 2016 +0100
@@ -1,15 +1,33 @@
-echo "jenkins-build.groovy running"
-
+def checkOutWithPatch(url, credentials, destination, revision, diff) {
 
-node ("VS2015")
+    checkout([$class: 'MercurialSCM', clean:true, credentialsId: credentials, revision: revision, source: url, subdir: destination])
+    if (diff != "")
+    {
+        echo "creating diff file"
+        sh "echo ${rep_diff}"
+        new File(pwd().toString() + "\\patch.diff").write(rep_diff)
+        echo "diff file created"
+        sh "cd ${destination}; patch -p1 -f < ${pwd()}/patch.diff"
+    }
+}
+
+def buildAll()
 {
-    stage "Substage"
-    echo "Substage running on VS2015"
-    echo "Repository revision 8"
-    echo "Added another line"
+    def mercurial = new gforce.Mercurial()
+
+    node ("VS2015")
+    {
+        echo "Substage running on VS2015"
+        echo rep_revision
+        echo pwd().toString()
+        echo rep_diff
+        checkOutWithPatch(url, "82d58684-2a77-4d0a-ab0e-0cdb5141cd8e", "Source", rep_revision, rep_diff)        
+    }
 }
 
 def HelloWorld ()
 {
     echo "Hello World"
-}
\ No newline at end of file
+}
+
+return this;
\ No newline at end of file
diff -r 000000000000 subrepo/Subrepo file.txt
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/subrepo/Subrepo file.txt Fri Jul 01 11:12:01 2016 +0100
@@ -0,0 +1,1 @@
+This is a file in the sub repository
\ No newline at end of file

Now, if I send the same form to a different server that echoes POST information back, I can see that I am sending the right thing.

Also, if I copy and paste from the "Parameters" page of the relevant build of the project, I get the right text (ok, I have to insert carriage returns to get it to match up properly).

FInally, with every diff up to this one, things have worked as expected.

However, with this particular diff, if I put

echo rep_diff

in my pipeline groovy, this is what I see.

diff -r dee4e38d33db .hgsub
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/.hgsub Thu Jun 30 16:55:07 2016 +0100
@@ -0,0 +1,1 @@
+subrepo = subrepos
\ No newline at end of file
diff -r dee4e38d33db .hgsubstate
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/.hgsubstate Thu Jun 30 16:55:07 2016 +0100
@@ -0,0 +1,1 @@
+0000000000000000000000000000000000000000 subrepo
diff -r dee4e38d33db jenkins-build.groovy
--- a/jenkins-build.groovy Wed Jun 01 17:12:42 2016 +0100
+++ b/jenkins-build.groovy Thu Jun 30 16:55:07 2016 +0100
@@ -1,15 +1,33 @@
-echo "jenkins-build.groovy running"
-
+def checkOutWithPatch(url, credentials, destination, revision, diff) {
 
-node ("VS2015")
+    checkout([$class: 'MercurialSCM', clean:true, credentialsId: credentials, revision: revision, source: url, subdir: destination])
+    if (diff != "")
+    {
+        echo "creating diff file"
+        sh "echo diff -r dee4e38d33db .hgsub
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/.hgsub Thu Jun 30 16:55:07 2016 +0100
@@ -0,0 +1,1 @@
+subrepo = subrepos
\ No newline at end of file
diff -r dee4e38d33db .hgsubstate
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/.hgsubstate Thu Jun 30 16:55:07 2016 +0100
@@ -0,0 +1,1 @@
+0000000000000000000000000000000000000000 subrepo
diff -r dee4e38d33db jenkins-build.groovy
--- a/jenkins-build.groovy Wed Jun 01 17:12:42 2016 +0100
+++ b/jenkins-build.groovy Thu Jun 30 16:55:07 2016 +0100
@@ -1,15 +1,33 @@
-echo "jenkins-build.groovy running"
-
+def checkOutWithPatch(url, credentials, destination, revision, diff) {
 
-node ("VS2015")
+    checkout([$class: 'MercurialSCM', clean:true, credentialsId: credentials, revision: revision, source: url, subdir: destination])
+    if (diff != "")
+    {
+        echo "creating diff file"
+        sh "echo ${rep_diff}"
+        new File(pwd().toString() + "\\patch.diff").write(rep_diff)
+        echo "diff file created"
+        sh "cd ${destination}; patch -p1 -f < ${pwd()}/patch.diff"
+    }
+}
+
+def buildAll()
 {
-    stage "Substage"
-    echo "Substage running on VS2015"
-    echo "Repository revision 8"
-    echo "Added another line"
+    def mercurial = new gforce.Mercurial()
+
+    node ("VS2015")
+    {
+        echo "Substage running on VS2015"
+        echo rep_revision
+        echo pwd().toString()
+        echo rep_diff
+        checkOutWithPatch(url, "82d58684-2a77-4d0a-ab0e-0cdb5141cd8e", "Source", rep_revision, rep_diff)        
+    }
 }
 
 def HelloWorld ()
 {
     echo "Hello World"
-}
\ No newline at end of file
+}
+
+return this;
\ No newline at end of file
diff -r 000000000000 subrepo/Subrepo file.txt
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/subrepo/Subrepo file.txt Thu Jun 30 16:55:07 2016 +0100
@@ -0,0 +1,1 @@
+This is a file in the sub repository
\ No newline at end of file
"
+        new File(pwd().toString() + "\\patch.diff").write(rep_diff)
+        echo "diff file created"
+        sh "cd ${destination}; patch -p1 -f < ${pwd()}/patch.diff"
+    }
+}
+
+def buildAll()
 {
-    stage "Substage"
-    echo "Substage running on VS2015"
-    echo "Repository revision 8"
-    echo "Added another line"
+    def mercurial = new gforce.Mercurial()
+
+    node ("VS2015")
+    {
+        echo "Substage running on VS2015"
+        echo rep_revision
+        echo pwd().toString()
+        echo rep_diff
+        checkOutWithPatch(url, "82d58684-2a77-4d0a-ab0e-0cdb5141cd8e", "Source", rep_revision, rep_diff)        
+    }
 }
 
 def HelloWorld ()
 {
     echo "Hello World"
-}
\ No newline at end of file
+}
+
+return this;
\ No newline at end of file
diff -r 000000000000 subrepo/Subrepo file.txt
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/subrepo/Subrepo file.txt Thu Jun 30 16:55:07 2016 +0100
@@ -0,0 +1,1 @@
+This is a file in the sub repository
\ No newline at end of file

notice how part way through it suddenly starts repeating the text from the start of the file.

This is also what I see if I write rep_diff to a file (which is how I use it as a patch).

I can't think of any reason why this should be happening other than a bug, has anybody else got any input?

[Daniel Beck]

> On 01.07.2016, at 13:35, Jonathan Hodgson <j.r.h...@gmail.com> wrote:
>
> I can't think of any reason why this should be happening other than a bug, has anybody else got any input?

This is a quirk in the variable resolution in Jenkins: Any variable references in parameter values are expanded.

So the line that says…

> + sh "echo ${rep_diff}"

will be expanded by Jenkins to read

> + sh "echo ALL OF THE SCRIPT ALL OVER AGAIN BECAUSE YAY RECURSION"

… spanning from line 25 to line 89 in your second snippet. This is done by Jenkins before the (processed) value is passed on to your shell script.

You should be able to prevent this by replacing all occurrences of $whatever by $$whatever (escape by adding a second $) in the value you submit to Jenkins.

[Jonathan Hodgson]

Aha!! I had wondered about something along those lines but I was looking at quotes for the answer and not finding it. 

I will try modifiying my submission code accordingly.

Thanks

[Jonathan Hodgson]

Thanks, that seems to have worked.

Now to get the patch working on slaves as well as the master...