withAWS step complains about not being able to support closures in the script console

[Kevin Milner]

I'm having a TON of problems getting the pipeline system setup and working. One issue in particular is troubling me. I have a shared library that has a custom step in vars/publishToS3.groovy. It looks like this:

#!/usr/bin/env groovy
package com.seven10;


def call(hashmap){
 def src_file = hashmap['src_file']
 def target_path = hashmap['target_path']
 def bucket_name = hashmap.get('bucket_name', '...')
 def region = hashmap.get('region', '...')
 def credentials = hashmap.get('credentials', '...')
 
 withAWS(credentials: credentials, region: region) {
 s3Upload bucket: bucket_name, file: src_file, path: target_path
 }
}

But I kept getting weird errors in my build job (like 'groovy.lang.MissingPropertyException: No such property: publish for class: groovy.lang.Binding') So in order to attempt to debug it, I created the following script in the script console.

def call(hashmap){
    def src_file = hashmap['src_file']
    def target_path = hashmap['target_path']
    def bucket_name = hashmap.get('bucket_name', '...')
    def region = hashmap.get('region', '...')
    def credentials = hashmap.get('credentials', '...')
    try {
      withAWS(credentials: credentials, region: region) {
        try {
          s3Upload(bucket: bucket_name, file: src_file, path: target_path)
        }
        catch(error){
           println('inner-try: ' + error)
           throw error
        }
      }
    }
    catch(error){
      println('outer-try: ' + error)
    }
}


call src_file: 'blah', target_path: 'blahblah'

But when I execute this script I get the error:

outer-try: groovy.lang.MissingMethodException: No signature of method: Script1.withAWS() is applicable for argument types: (java.util.LinkedHashMap, Script1$_call_closure1) values: [[credentials:..., region:...], Script1$_call_closure1@47e8f461] Possible solutions: with(groovy.lang.Closure)
The docs for the withAWS step show it being called with a closure block in just this way (see here).
The only other thing I can think of is that its not actually finding the withAWS step (as demonstrated by the fact that it is called "Script1.withAWS") if thats the case, how would I make sure its finding the actual plugin step instead of trying to find a local variable?

[Richard Bywater]

What does your pipeline look like? Are you sure you aren't calling publish instead of publishToS3 (as the step name seems to be the only reference to "publish" in the snippet you provided.

Richard

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/30eea0d8-10a5-4fce-b050-3064c2c475fb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kevin Milner]

Hi Richard, thanks for your response.
publishToS3 is called in another custom step that is then called by the pipeline. The reason for the "publish" bit is I kept getting errors along the lines of "duplicate class load attempted" (see my post here). As a result of a suggestion on that question (which I now think might have been a "disguised" error for not being able to load the publishToS3 custom step) I added a member to a "container" class and initialize it earlier. I will post here the relevant bits of code:



This is the "container" class that I instantiate in my pipeline and it does some stuff, mostly holding variable names and whatnot:

// src/com/seven10/Builder.groovy
#!/usr/bin/env groovy
package com.seven10;

import java.text.SimpleDateFormat
import java.util.Date
import com.seven10.publishToS3

class Builder implements Serializable{

 static final String TARGET_BUCKET = '...'
 static final String ADVANCED_INSTALLER_PATH = 'c:\\advinst\\bin\\x86\\AdvancedInstaller.com'
 String cli_version
 String release_family
 String win_version
 def steps
 final def publisher
 def docker_images = [:]
 final String date

 Builder(steps) {
 println "Builder Constructor Enter"
 this.steps = steps
 def dateFormat = new SimpleDateFormat("yyyyMMdd")
 this.date = dateFormat.format(new Date())
 this.publisher = publishToS3.newInstance()
 }

 def init() {
 def pom = steps.readMavenPom file: 'pom.xml'
 if(pom.version == null) {
 this.cli_version = pom.parent.version
 } else {
 this.cli_version = pom.version
 }
 steps.echo "builder: pom version = " + this.cli_version
 this.win_version = getWinVersionFromVersion(cli_version)
 steps.echo "builder: win_version = " + this.win_version
 this.release_family = getRelFamily(cli_version)
 steps.echo "builder: release_family = " + this.release_family

 steps.echo "Builder Constructor Exit"
 }

 def getFriendlyName(String role_name) {
 "${role_name}_${this.cli_version}_${this.date}"
 }

 static def getRelFamily(String version){
 def rval
 switch (version) {
 case ~/^\d+\.\d+\.\d+$/: rval = (version=~ /^\d+\.\d+/)[0]; break
 case ~/.*-SNAPSHOT$/: rval = 'qa_latest'; break
 case ~/.*SG\d+/: rval = 'sg_latest'; break
 case ~/.*cee\d+/: rval = 'cee_latest'; break
 default: rval = "unknown"
 }
 rval
 }
 static def getWinVersionFromVersion(String version) {
 def rval = (version =~ /^\d+\.\d+\.\d+/)[0]
 println('windows version = ' + rval)
 rval
 }
}

The following snippet is my actual Jenkinsfile (but again, the actual issue I'm having right now seems to be just running the code in the script console to get it working first)

@Library('jenkins-lib') _

final BUILT_INSTALLER_PATH = 'smb_installer/installer/*.exe'
final BUILT_INSTALLER_PATH_WIN = 'smb_installer\\installer\\Storfirst_SMB_Hydrator.exe'
final ADV_INST_PROJECT = 'smb_installer\\Hydrator.aip'

final HYDRATOR_JAR_SRC_PATH = 'lib/hydrator/docker/hydra-hydrator.jar'
final HYDRATOR_JAR_DEST_PATH = 'docker/hydrator/hydra-hydrator.jar'
final HYDRATOR_JAR_DEST_PATH_WIN = 'docker\\hydrator\\hydra-hydrator.jar'

final AUTOMATOR_JAR_SRC_PATH = 'lib/automator/docker/hydra-automator.jar'
final AUTOMATOR_JAR_DEST_PATH = 'docker/automator/hydra-automator.jar'

pipeline {
	agent none
	tools {
		maven 'maven'
		git 'git-install'
	}
	
  	stages {
		stage('Initialize') {
			agent any
			steps {
				script {
					builder = com.seven10.Builder.newInstance(this)
					builder.init()
					println("PATH = " + PATH)
					println("M2_HOME = " + M2_HOME)
					println("BRANCH_NAME = " + env.BRANCH_NAME)
				}
			}
		}
		stage('Build Executables') {
			agent { label 'x64 && mvn && git' }
			steps {
            	cleanWs notFailBuild: true
            	initSubmodules project_url: '...'
				sh 'mvn clean install  --projects lib/parent'
				// install nodejs and other voodoo stuff
				sh 'mvn clean generate-resources  --projects lib/automator -PinstallNode'	
				// install modules
				sh 'mvn install -DskipTests'			
				// build executable jars
            	sh 'mvn package -DskipTests --projects lib/hydrator,lib/automator -PbuildExecutable'
            	
            	// archive hydrator executable
				sh "mv ${HYDRATOR_JAR_SRC_PATH} ${HYDRATOR_JAR_DEST_PATH}"
            	fingerprint HYDRATOR_JAR_DEST_PATH
            	stash includes: HYDRATOR_JAR_DEST_PATH, name: 'hydrator-build'
            	stash includes: HYDRATOR_JAR_DEST_PATH, name: 'hydrator-build-installer'
            	// archive automator executable
            	sh "mv ${AUTOMATOR_JAR_SRC_PATH} ${AUTOMATOR_JAR_DEST_PATH}"
            	fingerprint AUTOMATOR_JAR_DEST_PATH
				stash includes: AUTOMATOR_JAR_DEST_PATH, name: 'automator-build'
			}
		}
		stage('Deployables') {
			parallel {
				stage('Build Automator Docker Image') {
					// builds the actual Automator product as a docker image and publishes it
					agent {label 'docker'}
					when {
						anyOf {
							branch 'feature/gateway'
							branch 'develop'
							branch 'master'
							branch 'cee_jenkinsfile'
						}
					}
					steps {
						unstash 'automator-build'
						doDockerBuild role_name: 'automator', builder: builder, context_path: 'docker/automator'
					}
				}
				stage('Build Hydrator Docker Image') {

					// builds the actual Hydrator product as a docker image and publishes it
					agent {label 'docker'}
					when {
						anyOf {
							branch 'feature/gateway'
							branch 'develop'
							branch 'master'
							branch 'cee_jenkinsfile'
						}
					}
					steps {
						unstash 'hydrator-build'
						doDockerBuild role_name: 'hydrator', builder: builder, context_path: 'docker/hydrator'
					}
				}
				stage('Build Hydra Installer') {
					agent { label 'advanced-installer' }
					when {
						anyOf {
							branch 'feature/gateway'
							branch 'develop'
							branch 'master'
							branch 'cee_jenkinsfile'
						}
					}
					steps {
						//retrieve hydrator jar
						unstash 'hydrator-build'
						//build installer
						bat "copy ${HYDRATOR_JAR_DEST_PATH_WIN} smb_installer\\"
		            	buildInstaller src_file: ADV_INST_PROJECT, win_version: builder.win_version
		            	fingerprint BUILT_INSTALLER_PATH
		            	stash includes: BUILT_INSTALLER_PATH_WIN, name: 'smb-installer'
					}
					post {
						success {
							cleanWs notFailBuild: true	
						}
					}
				}
			}	// parallel
		}	// parallel stage
		stage('Publish everything')
		{
			agent { label 'docker' }
			when {
				anyOf {
					branch 'feature/gateway'
					branch 'develop'
					branch 'master'
					branch 'cee_jenkinsfile'
				}
			}
			steps {
				publishDocker builder: builder, image_list: ['automator','hydrator']
				unstash name: 'smb-installer'
			    // publish installer
			    publishInstaller role_name: 'Storfirst_SMB_Hydrator', target_path: 'installers/hydrator', builder: builder,
			    					  src_path: 'smb_installer\\installer'
		    }
		    post {
		    	success {
		    		cleanWs cleanWhenFailure: false, cleanWhenNotBuilt: false, notFailBuild: true		
		    	}
		    }
		}
		stage('Tag Revisions') {
			agent {label 'git'}
			when {
					anyOf {
						branch 'feature/gateway'
						branch 'develop'
						branch 'master'
						branch 'cee_jenkinsfile'
					}
				}
			steps {
				sh("git tag -a ${env.BUILD_TAG} -m '${repositoryCommiterMessage}'")
				sh("git submodule foreach git tag -a ${env.BUILD_TAG} -m '${repositoryCommiterMessage}'")
			}
		}
	}
}

(sorry for that not being in a codeblock, google doesn't want to properly format the code for some reason)

[Kevin Milner]

Ok, I think I've solved it. Well, I solved my core issue, which was that the vars/publishToS3.groovy script was not being properly identified. I couldn't figure out how to identify publishToS3. It could find neither publishToS3, nor com.seven10.publishToS3. I think, because there was an implied class being generated for publishToS3.call(), it was not able to find the class. I corrected this by making publishToS3 an actual explicit class in the src/<package-name>/ folder.

I'm still not entirely sure if that's the solution that actually resulted in my pipeline working.

The docs for Pipeline in more advanced cases are not...good.