Groovy script other script tools functions

1,877 views
Skip to first unread message

jer...@bodycad.com

unread,
Jul 28, 2016, 12:00:40 PM7/28/16
to Jenkins Users
Hi,

First of, I'm totaly new to the Groovy thing (I'm not a Java dev either, I'm a C/C++/C#/Python/Bash guys).

I'm trying (without any success so far) to use a common files between my JenkinsFiles as Tools.groovy to have a single implementation for them.

I'm having problems to just find out what I can do into the Jenkins groovy env. Seem like many things are not Groovy standard about it, because many Groovy example just don't want to work or plain are refuse by Jenkins security.

Here's what I'm trying to acheive (pretty basic for any script language):

  1. Have a common file Tools.groovy with static function
  2. Have my JenkinsFile (import, evaluate, ...) that file and use those function

I'm not even able to declare a class with function into my Jenkins file:

-----------------------------------------------------------
Example what I'm trying to delcare here
-----------------------------------------------------------
class BCadTools
{
def findIntoFiles(String path, Pattern fileFilter, Pattern content)
{
def rv = [:]
def srcDir = new File(path)
srcDir.traverse(type: FILES, nameFilter: fileFilter) 
{
def fileName = it.name
def fileContent = readFile(fileName);
def matches = fileContent =~ content
if(matches.count > 0)
{
rv[fileName] = matches
}
}
return rv;
}
}

I get either:
-----------------------------------------------------------
1. Put the class before the node {}
-----------------------------------------------------------

Started by user anonymous
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 30: Class definition not expected here. Please define the class at an appropriate place or perhaps try using a block/Closure instead. at line: 30 column: 2. File: WorkflowScript @ line 30, column 2.
   	class BCadTools
    ^

1 error

	at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
	at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:946)
	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:593)
	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:569)
	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:546)
	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
	at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
	at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:106)
	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:376)
	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:343)
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:212)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:410)
Finished: FAILURE

 we clearly cannot declare it there, but why ??? any information about the context of the Groovy script into JenkinsFile pipeline? any doc (not just a few too basic call examples) ?

-----------------------------------------------------------
2. Put the class inside the node {}
-----------------------------------------------------------

Started by user anonymous
[Pipeline] node
Running on master in d:\JenkinsWorkspace\BuildSystemTest\TestPipeline
[Pipeline] {
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use new java.io.File java.lang.String
	at org.jenkinsci.plugins.scriptsecurity.sandbox.whitelists.StaticWhitelist.rejectNew(StaticWhitelist.java:169)
	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onNewInstance(SandboxInterceptor.java:130)
	at org.kohsuke.groovy.sandbox.impl.Checker$3.call(Checker.java:191)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedConstructor(Checker.java:188)
	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.constructorCall(SandboxInvoker.java:19)
	at WorkflowScript.run(WorkflowScript:38)
	at ___cps.transform___(Native Method)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:93)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:79)
	at sun.reflect.GeneratedMethodAccessor239.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:100)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:79)
	at sun.reflect.GeneratedMethodAccessor239.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:57)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:106)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:79)
	at sun.reflect.GeneratedMethodAccessor239.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
	at com.cloudbees.groovy.cps.Next.step(Next.java:58)
	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:154)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:19)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:33)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:30)
	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:30)
	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:164)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:276)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$000(CpsThreadGroup.java:78)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:185)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:183)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:47)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
Finished: FAILURE

jer...@bodycad.com

unread,
Jul 28, 2016, 5:53:22 PM7/28/16
to Jenkins Users
If this may help someone, here's my finding so far (I must admit I found Groovy inside Jenkins to be painful and by far one of my less favorite scripting language so far, especially loosing my head with a few things. Here's some traps I found:

  • the .each fct bug that iterating only once (work inside all groovy console except into Jenkins project file
  • Editing the pipline script of a pipleine project (when not into from SCM) doesn't show the script box, just right click on the Script tag and inspect content and search for the html tag div class="workflow-editor-wrapper" and uncheck display: none from the CSS property.
  • importing tools file is not funny and I still cannot println() into them so far, the output doesn't display for some reason I don't known.
  • "." path is Jenkins home and not the current workspace, you need to use pwd() to access the workspace
  • printing env options is nearly impossible don't waste your time on this, search for for the available value on google
  • uncheck the sandbox, else you pretty much cannot use anything even basic stuff (regex, string, match)
If anybody want to import a tools file that can be used into multiple JenkinsFile or project, here's an example:

My Jenkinsfile stage look like this
stage 'VcxProj Inspection'
GroovyObject bcadTools = (GroovyObject) (new GroovyClassLoader(getClass().getClassLoader()).parseClass(new File(pwd() + "\\Tools.groovy"))).newInstance();
for(String l : bcadTools.findIntoFiles(true, pwd(), ~/.*\.vcxproj/, ~/Optimization/))
{
  println(l)
}
// Clean up to avoid CPS problems
errorLines = null
bcadTools = null

My Tools.groovy look like this:
/*!
 * \warning     Copyright (c) 2016 Laboratoires Bodycad inc., All rights reserved
 * 
 * \date        2016-07-13
 * \author      Jerome Godbout
 *              
 * \file        Tools.groovy
 * \info        This is common function to use with JenkinsFile or other groovy scripts
 *
 */
import static groovy.io.FileType.*
import java.util.regex.Pattern

class BCadTools
{
/* Find pattern into files from src path that will recursively search
* \param isError, print error or warning
* \param path, path to recursively search for file. Ex: "MyPath\\SubDir"
* \param extension, the file extension to check for. Ex: ~/.*\.vcxproj$/
* \param content, the content regex to seek. Ex: /Optimization/
* \return the match list for each files
*/
def findIntoFiles(boolean isError, String path, Pattern fileFilter, Pattern lineExp)
{
def rv = []
def srcDir = new File(path)
def levelStr = (isError ? "ERROR" : "WARNING")
srcDir.traverse(type: FILES, nameFilter: fileFilter) 
{
def fileName = it.path
if(!(fileName =~ /[\\\\\/]+\.hg[\\\\\/]+/) &&
  !(fileName =~ /[\\\\\/]+external[\\\\\/]+/))
{
def fileCheck = new File(fileName)
def lineNumber = 0
fileCheck.eachLine 
{
def matches = it =~ lineExp
if(matches.count > 0)
{
rv.push("FI " + levelStr + " : " + fileName + " at line " + lineNumber + " : " + it)
}
++lineNumber
}
}
}
return rv
}
}


I should be able to parse the console output to make warning or error with this. I have yet figure out if I can split the consle parsed output results from each stage into different output, but that's something else.

I hope there's a better way to acheive all this, but it already took me way too much time to do this simple thing. Just hope it may help people to start with.

Jerome

jer...@bodycad.com

unread,
Jul 29, 2016, 1:16:28 PM7/29/16
to Jenkins Users
Seem like the pipeline output cannot be parsed by the console log parser (he don't see them inside the node, some flushing or redirection is not properly done).

so I'm move back to Python and did a simple script invoke with batch (this work):
bat "\"${bcad.python_2_exe}\" \"${env.JENKINS_HOME}\\custom_scripts\\FilesInspection.py\" -e -s. -f\".*\\.vcxproj\$\" -g -a\"external\" -c\"<Optimization\""

I'm looking for a way to make multiple console parse with different parameters, I wish I could give it a named for link and have separated results so user known what is where (they are in differrent stage too).

step([$class: 'LogParserPublisher', parsingRulesPath: "${env.JENKINS_HOME}\\console_parse\\CppInspection.txt", useProjectRule: false, failBuildOnError: true]);

step([$class: 'LogParserPublisher', parsingRulesPath: "${env.JENKINS_HOME}\\console_parse\\QmlInspection.txt", useProjectRule: false, failBuildOnError: false]);

right now it doesn't work, I see 2 menu entry but both are pointing to the same data (the last seen).  I will merge them fro now, but would really like to be able to separate this.
Reply all
Reply to author
Forward
0 new messages