[JIRA] (JENKINS-37492) Propose adding `diff` utility step

25 views
Skip to first unread message

owood@cloudbees.com (JIRA)

unread,
Aug 17, 2016, 11:53:01 PM8/17/16
to jenkinsc...@googlegroups.com
Owen Wood created an issue
 
Jenkins / New Feature JENKINS-37492
Propose adding `diff` utility step
Issue Type: New Feature New Feature
Assignee: rsandell
Components: pipeline-utility-steps-plugin
Created: 2016/Aug/18 3:52 AM
Priority: Minor Minor
Reporter: Owen Wood

Currently, diffing files requires using OS specific implementations (sh, bat).

This has the potential of triggering security approval, which can be an inconvenience.

Proposing add new `diff` utility step, to allow OS agnostic use in PIpeline jobs.
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
Atlassian logo

rndgstn@gmail.com (JIRA)

unread,
Aug 20, 2016, 11:10:01 AM8/20/16
to jenkinsc...@googlegroups.com
Ron Dagostino commented on New Feature JENKINS-37492
 
Re: Propose adding `diff` utility step

At a minimum, it would be helpful if two FileWrapper instances available via the findFiies step from the Pipeline Utility Steps Plugin could be compared, perhaps like this: boolean comparedTheSame = firstFileWrapper.sameContentsAs(secondFileWrapper). A "compareFiles" step would also be helpful, of course.

rndgstn@gmail.com (JIRA)

unread,
Aug 20, 2016, 11:11:01 AM8/20/16
to jenkinsc...@googlegroups.com
Ron Dagostino edited a comment on New Feature JENKINS-37492
At a minimum, it would be helpful if two FileWrapper instances available via the findFiies findFiles step from the Pipeline Utility Steps Plugin could be compared, perhaps like this: boolean comparedTheSame = firstFileWrapper.sameContentsAs(secondFileWrapper).  A "compareFiles" step would also be helpful, of course.

pwolf@cloudbees.com (JIRA)

unread,
Aug 22, 2016, 12:41:03 PM8/22/16
to jenkinsc...@googlegroups.com

Owen Wood why would a sh or bat trigger a security approval? Script security is based on sandbox groovy execution. Anything run in a sh or bat step is executed on the build agent, not the master, and this will work on the files locally on the build agent. After JENKINS-26055 is complete we could build a 'diff' step that is OS agnostic and runs on the build agents but doing all of this work on the master seems inefficient.

rndgstn@gmail.com (JIRA)

unread,
Aug 22, 2016, 1:01:01 PM8/22/16
to jenkinsc...@googlegroups.com

Actually, I think the security issue is that it is not possible to use the Java File API without having to approve a method that – according to the script approval UI – opens a security vulnerability.

pwolf@cloudbees.com (JIRA)

unread,
Aug 22, 2016, 1:12:01 PM8/22/16
to jenkinsc...@googlegroups.com

Steps defined in the Global Library or in a Plugin aren't run in the Pipeline Sandbox. Plugins can do pretty much anything and only Jenkins admins can define functions in the Global library. Admins are intrinsically trusted in Jenkins so these functions are not sandboxed.

Someone could create a new plugin that implements a diff step using the Java File API will no script security issues.

Alternatively, if someone can define a Pipeline step to do this in a Global Library it could be shared here. https://github.com/jenkinsci/pipeline-examples/tree/master/global-library-examples

rndgstn@gmail.com (JIRA)

unread,
Aug 22, 2016, 8:13:02 PM8/22/16
to jenkinsc...@googlegroups.com

I just added vars/getFile.groovy to my Jenkins global library as follows:

String call(name)

{ new java.io.File(name) }

this

Then I tried to invoke it from a Jenkinsfile (e.g. "echo getFile('/tmp').getAbsolutePath()") and received the error "org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use new java.io.File java.lang.String"
with this stack trace:

org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use new java.io.File java.lang.String
at org.jenkinsci.plugins.scriptsecurity.sandbox.whitelists.StaticWhitelist.rejectNew(StaticWhitelist.java:185)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onNewInstance(SandboxInterceptor.java:130)
at org.kohsuke.groovy.sandbox.impl.Checker$3.call(Checker.java:191)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedConstructor(Checker.java:188)
at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.constructorCall(SandboxInvoker.java:19)
etc...

When I look at the script approval UI I see "new java.io.File java.lang.String Approving this signature may introduce a security vulnerability! You are advised to deny it."

Am I missing something? It seems to me that the sandbox applies to the global library plugin at https://github.com/jenkinsci/workflow-cps-global-lib-plugin.

pwolf@cloudbees.com (JIRA)

unread,
Aug 22, 2016, 8:58:01 PM8/22/16
to jenkinsc...@googlegroups.com

https://issues.jenkins-ci.org/browse/JENKINS-34650

Make sure you are running a current update of Pipeline so this change is included. It was a recent change by Kohsuke.

In your example, it seems that you are still reference the File.getAbsolutePath() call in your Jenkinsfile. Rather, what happens if you change your global script to the specific call:

String call(name)

{ new java.io.File(name).getAbsolutePath() }

this

Or you could pass in the name of the file as first param and the action to return on the second param. As long as you are referencing a File object in your Jenkinsfile directly you will get this sandbox warning.

rndgstn@gmail.com (JIRA)

unread,
Aug 22, 2016, 10:00:04 PM8/22/16
to jenkinsc...@googlegroups.com

Thanks. The below script does not work for me (I still get "new java.io.File java.lang.String Approving this signature may introduce a security vulnerability! You are advised to deny it."), and I am not running the latest workflow plugin, so it looks like I will have to upgrade to a newer core to get the fix from https://issues.jenkins-ci.org/browse/JENKINS-34650. I agree that adding a diff script is a relatively easy thing to do when that fix is applied, and the script could be shared at https://github.com/jenkinsci/pipeline-examples/tree/master/global-library-examples as you suggest.

String call (String path)

{ getFilePathNonCPS(path) }

@NonCPS
String getFilePathNonCPS(String path)

{ new java.io.File(path).getAbsolutePath() }

this

pwolf@cloudbees.com (JIRA)

unread,
Aug 22, 2016, 11:48:01 PM8/22/16
to jenkinsc...@googlegroups.com

I apologize. I didn't think about the context before Ron Dagostino. What you are trying to do won't work anyway.

A java.io.File works on the local filesystem but most times you will want to diff two files in your workspace. Pipeline executes on the master in a flyweight executor so java.io.File has a context of the master - hence the security exception.

What you want is this: http://javadoc.jenkins-ci.org/hudson/FilePath.html

The best working example would be the readFile step: https://github.com/jenkinsci/workflow-basic-steps-plugin/blob/master/src/main/java/org/jenkinsci/plugins/workflow/steps/ReadFileStep.java

This takes a hudson.FilePath and streams it. If you wanted to do a diff it would be very similar but take two file parameters and compare them.

mneale@cloudbees.com (JIRA)

unread,
Aug 23, 2016, 12:27:02 AM8/23/16
to jenkinsc...@googlegroups.com

I understand the platform agnostic angle, but "This has the potential of triggering security approval, which can be an inconvenience." - I don't see how this is the case if you are doing the diff inside sh or bat?

mneale@cloudbees.com (JIRA)

unread,
Aug 23, 2016, 12:28:04 AM8/23/16
to jenkinsc...@googlegroups.com
Michael Neale edited a comment on New Feature JENKINS-37492
I understand the platform agnostic angle, but "This has the potential of triggering security approval, which can be an inconvenience." - I don't see how this is the case if you are doing the diff inside sh or bat?


In terms of a diff step - I assume you mean it takes 3 params: 2 input files, and an output file? or would it take 2 strings and produce a string? or some combination?

rndgstn@gmail.com (JIRA)

unread,
Aug 25, 2016, 8:35:01 PM8/25/16
to jenkinsc...@googlegroups.com

Thanks, Patrick Wolf, for the FilePath vs. FIle info. I have also confirmed that the shared global library code is trusted implicitly in recent versions as per https://issues.jenkins-ci.org/browse/JENKINS-34650, so yes, it would be possible to simply write the script and share it at https://github.com/jenkinsci/pipeline-examples/tree/master/global-library-examples. I will do so if I end up writing it.

Michael Neale, I am personally interested in a simple "are two files the same?" step, not full diff functionality.

mneale@cloudbees.com (JIRA)

unread,
Aug 25, 2016, 8:46:02 PM8/25/16
to jenkinsc...@googlegroups.com

Ron Dagostino right - well that would be far easier to do as a step I think, or global var lib.
Reply all
Reply to author
Forward
0 new messages