[JIRA] (JENKINS-60223) Pipeline implementing readObject/writeObject fails with ProxyException CpsCallableInvocation

1 view
Skip to first unread message

r.stengel@sap.com (JIRA)

unread,
Nov 20, 2019, 5:14:03 AM11/20/19
to jenkinsc...@googlegroups.com
Roland Stengel created an issue
 
Jenkins / Improvement JENKINS-60223
Pipeline implementing readObject/writeObject fails with ProxyException CpsCallableInvocation
Issue Type: Improvement Improvement
Assignee: Unassigned
Components: workflow-cps-plugin
Created: 2019-11-20 10:13
Environment: Jenkins ver. 2.204
Ubuntu 18.04.2

JSch dependency plugin (jsch): 0.1.55.1
SSH Credentials Plugin (ssh-credentials): 1.18
Rebuilder (rebuild): 1.31
Pipeline: REST API Plugin (pipeline-rest-api): 2.12
Apache HttpComponents Client 4.x API Plugin (apache-httpcomponents-client-4-api): 4.5.10-2.0
Pipeline: Basic Steps (workflow-basic-steps): 2.18
Branch API Plugin (branch-api): 2.5.4
Kubernetes plugin (kubernetes): 1.21.3
SCM API Plugin (scm-api): 2.6.3
Durable Task Plugin (durable-task): 1.33
Warnings Next Generation Plugin (warnings-ng): 7.2.0
FindBugs Plug-in (findbugs): 5.0.0
Mercurial plugin (mercurial): 2.8
Plain Credentials Plugin (plain-credentials): 1.5
Authentication Tokens API Plugin (authentication-tokens): 1.3
Pipeline: Milestone Step (pipeline-milestone-step): 1.3.1
i18n for Blue Ocean (blueocean-i18n): 1.21.0
Docker Commons Plugin (docker-commons): 1.15
GitHub API Plugin (github-api): 1.95
Structs Plugin (structs): 1.20
Pipeline: Stage Tags Metadata (pipeline-stage-tags-metadata): 1.4.0
Design Language (jenkins-design-language): 1.21.0
Pipeline: Declarative (pipeline-model-definition): 1.4.0
Configuration as Code Support Plugin (configuration-as-code-support): 1.18
Pipeline: Declarative Agent API (pipeline-model-declarative-agent): 1.1.1
Pipeline: Job (workflow-job): 2.36
GIT server Plugin (git-server): 1.8
Display URL for Blue Ocean (blueocean-display-url): 2.3.0
Conditional BuildStep (conditional-buildstep): 1.3.6
GitHub Pipeline for Blue Ocean (blueocean-github-pipeline): 1.21.0
Jackson 2 API Plugin (jackson2-api): 2.10.0
Timestamper (timestamper): 1.10
Static Analysis Utilities (analysis-core): 1.96
Run Condition Plugin (run-condition): 1.2
MapDB API Plugin (mapdb-api): 1.0.9.0
GitHub plugin (github): 1.29.5
Pipeline: Declarative Extension Points API (pipeline-model-extensions): 1.4.0
LDAP Plugin (ldap): 1.21
Kubernetes Client API Plugin (kubernetes-client-api): 4.6.2-1
Build Monitor View (build-monitor-plugin): 1.12+build.201809061734
PAM Authentication plugin (pam-auth): 1.6
Docker Pipeline (docker-workflow): 1.21
Events API for Blue Ocean (blueocean-events): 1.21.0
JavaScript GUI Lib: Moment.js bundle plugin (momentjs): 1.1.1
Lockable Resources plugin (lockable-resources): 2.7
Command Agent Launcher Plugin (command-launcher): 1.3
Pipeline: Supporting APIs (workflow-support): 3.3
Pub-Sub "light" Bus (pubsub-light): 1.13
Git client plugin (git-client): 3.0.0
Green Balls (greenballs): 1.15
Pipeline: Input Step (pipeline-input-step): 2.11
Pipeline: Model API (pipeline-model-api): 1.4.0
Git plugin (git): 4.0.0
Subversion Plug-in (subversion): 2.12.2
Pipeline Graph Analysis Plugin (pipeline-graph-analysis): 1.10
Pipeline: GitHub Groovy Libraries (pipeline-github-lib): 1.0
Safe Restart Plugin (saferestart): 0.3
Autofavorite for Blue Ocean (blueocean-autofavorite): 1.2.4
Credentials Binding Plugin (credentials-binding): 1.20
Cucumber json test reporting. (cucumber-testresult-plugin): 0.10.1
AnsiColor (ansicolor): 0.6.2
Pipeline (workflow-aggregator): 2.6
Analysis Model API Plug-in (analysis-model-api): 7.0.2
Gradle Plugin (gradle): 1.34
Parameterized Trigger plugin (parameterized-trigger): 2.35.2
Blue Ocean Core JS (blueocean-core-js): 1.21.0
Trilead API Plugin (trilead-api): 1.0.5
Bitbucket Pipeline for Blue Ocean (blueocean-bitbucket-pipeline): 1.21.0
JUnit Plugin (junit): 1.28
Performance Plugin (performance): 3.17
Ant Plugin (ant): 1.10
JavaScript GUI Lib: ACE Editor bundle plugin (ace-editor): 1.1
SSH Agent Plugin (ssh-agent): 1.17
Token Macro Plugin (token-macro): 2.10
Credentials Plugin (credentials): 2.3.0
Variant Plugin (variant): 1.3
JWT for Blue Ocean (blueocean-jwt): 1.21.0
JavaScript GUI Lib: Handlebars bundle plugin (handlebars): 1.1.1
HTTP Request Plugin (http_request): 1.8.23
Kubernetes Credentials Plugin (kubernetes-credentials): 0.4.1
GitHub Branch Source Plugin (github-branch-source): 2.5.8
Blue Ocean (blueocean): 1.21.0
Pipeline: Build Step (pipeline-build-step): 2.9
Pipeline: Multibranch (workflow-multibranch): 2.21
SSH Slaves plugin (ssh-slaves): 1.31.0
Favorite (favorite): 2.3.2
Checkmarx Plugin (checkmarx): 8.90.4
Configuration as Code Plugin (configuration-as-code): 1.33
Server Sent Events (SSE) Gateway Plugin (sse-gateway): 1.20
Folders Plugin (cloudbees-folder): 6.9
JIRA Integration for Blue Ocean (blueocean-jira): 1.21.0
REST Implementation for Blue Ocean (blueocean-rest-impl): 1.21.0
Forensics API Plugin (forensics-api): 0.6.1
Javadoc Plugin (javadoc): 1.5
Mailer Plugin (mailer): 1.29
Job DSL (job-dsl): 1.76
Pipeline: SCM Step (workflow-scm-step): 2.9
Email Extension Plugin (email-ext): 2.68
Workspace Cleanup Plugin (ws-cleanup): 0.37
Handy Uri Templates 2.x API Plugin (handy-uri-templates-2-api): 2.1.8-1.0
JIRA plugin (jira): 3.0.10
Pipeline: Step API (workflow-step-api): 2.20
Badge (badge): 1.8
Bitbucket Branch Source Plugin (cloudbees-bitbucket-branch-source): 2.6.0
REST API for Blue Ocean (blueocean-rest): 1.21.0
Nexus Artifact Uploader (nexus-artifact-uploader): 2.10
Code Coverage API Plugin (code-coverage-api): 1.1.1
Common API for Blue Ocean (blueocean-commons): 1.21.0
OWASP Markup Formatter Plugin (antisamy-markup-formatter): 1.6
Script Security Plugin (script-security): 1.67
Pipeline: Shared Groovy Libraries (workflow-cps-global-lib): 2.15
JaCoCo plugin (jacoco): 3.0.4
Pipeline: API (workflow-api): 2.37
Matrix Authorization Strategy Plugin (matrix-auth): 2.5
Pipeline: Nodes and Processes (workflow-durable-task-step): 2.35
Pipeline implementation for Blue Ocean (blueocean-pipeline-api-impl): 1.21.0
bouncycastle API Plugin (bouncycastle-api): 2.17
CCM Plug-in (ccm): 3.2
Pipeline: Groovy (workflow-cps): 2.76
JavaScript GUI Lib: jQuery bundles (jQuery and jQuery UI) plugin (jquery-detached): 1.2.1
Matrix Project Plugin (matrix-project): 1.14
GitHub Pull Request Builder (ghprb): 1.42.0
Git Pipeline for Blue Ocean (blueocean-git-pipeline): 1.21.0
Dashboard for Blue Ocean (blueocean-dashboard): 1.21.0
Resource Disposer Plugin (resource-disposer): 0.14
HTML Publisher plugin (htmlpublisher): 1.21
Web for Blue Ocean (blueocean-web): 1.21.0
External Monitor Job Type Plugin (external-monitor-job): 1.7
Maven Integration plugin (maven-plugin): 3.4
Build Timeout (build-timeout): 1.19
Pipeline SCM API for Blue Ocean (blueocean-pipeline-scm-api): 1.21.0
Pipeline Utility Steps (pipeline-utility-steps): 2.3.1
PMD Plug-in (pmd): 4.0.0
Pipeline: Stage View Plugin (pipeline-stage-view): 2.12
Gatling Jenkins Plugin (gatling): 1.2.7
Pipeline: Stage Step (pipeline-stage-step): 2.3
Oracle Java SE Development Kit Installer Plugin (jdk-tool): 1.3
Personalization for Blue Ocean (blueocean-personalization): 1.21.0
Blue Ocean Pipeline Editor (blueocean-pipeline-editor): 1.21.0
Config API for Blue Ocean (blueocean-config): 1.21.0
WMI Windows Agents Plugin (windows-slaves): 1.5
Display URL API (display-url-api): 2.3.2
Cobertura Plugin (cobertura): 1.15
Labels: pipeline serialization CpsCallableInvocation-related
Priority: Minor Minor
Reporter: Roland Stengel

I have a set of steps which share a configuration instance. The configuration is implemented as Singleton. Now, I would like to serialize this @Singleton to support the resume feature of Jenkins pipelines. Currently, all runtime configurations are gone, if the Jenkins is restarted in between.

Since statics aren't serialized, I implemented readObject and writeObject. What ever I tried, i came across the ProxyException: 

hudson.remoting.ProxyException: CpsCallableInvocation{methodName=writeObject, call=com.cloudbees.groovy.cps.impl.CpsFunction@1bc872bf, receiver=my.simple.TestPipelineEnvironment@40f84a32, arguments=[org.jboss.marshalling.river.RiverObjectOutputStream@8ad5752]}

In the code below, I implemented the simple POGO my.simple.TestPipelineEnvironment.groovy with readObject and writeObject. I dropped the @Singleton to reduce complexity.

package my.simple;

public class TestPipelineEnvironment implements Serializable {

	private void writeObject(ObjectOutputStream oos) throws IOException {
		System.out.println("Custom serialization logic invoked.");
		oos.defaultWriteObject();
	}
	private void readObject(ObjectInputStream ois) throws IOException, ClassNotFoundException {
		System.out.println("Custom deserialization logic invoked.");
		ois.defaultReadObject();
	}

	private Map valueMap = [:]

	void setValue(String property, value) {
		valueMap[property] = value
	}

	def getValue(String property) {
		return valueMap.get(property)
	}
}


It is initialized and accessible by the step vars/testPipelineEnvironment.groovy

 

import my.simple.TestPipelineEnvironment

class testPipelineEnvironment implements Serializable {

    private static final long serialVersionUID = -113665469917946598L;

    private TestPipelineEnvironment instance= new TestPipelineEnvironment();

    private TestPipelineEnvironment getCPE() {
        return instance;
    }

    Map getValueMap() {
        getCPE().valueMap
    }

    void setValue(String property, value) {
        valueMap[property] = value
    }

    def getValue(String property) {
        return valueMap.get(property)
    }
}

The Jenkinsfile fails

 

@Library('piper-lib-rodibrin') _
node() {
  stage('prepare') { 
    testPipelineEnvironment.setValue("key","value") echo "value:" + testPipelineEnvironment.getValue("key") 
  }
}

with

 

...
[Pipeline] echo
 value:value
 [Pipeline] }
 [Pipeline] End of Pipeline
 hudson.remoting.ProxyException: CpsCallableInvocation{methodName=writeObject, call=com.cloudbees.groovy.cps.impl.CpsFunction@1bc872bf, receiver=my.simple.TestPipelineEnvironment@40f84a32, arguments=[org.jboss.marshalling.river.RiverObjectOutputStream@8ad5752]}
 Finished: FAILURE

 

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

sparshev@griddynamics.com (JIRA)

unread,
Feb 15, 2020, 8:16:04 PM2/15/20
to jenkinsc...@googlegroups.com
Sergei Parshev commented on Improvement JENKINS-60223
 
Re: Pipeline implementing readObject/writeObject fails with ProxyException CpsCallableInvocation

The same with restoring the pipeline object after the jenkins (2.204.2 with minimal plugins) stop/start:

  • Pipeline log error:
    [Pipeline] input
    continue?
    Proceed or Abort
    Resuming build at Sun Feb 16 01:02:36 UTC 2020 after Jenkins restart
    Waiting to resume part of mpl-build #4: Finished waiting
    [Pipeline] End of Pipeline
    hudson.remoting.ProxyException: CpsCallableInvocation{methodName=readObject, call=com.cloudbees.groovy.cps.impl.CpsFunction@7505dedb, receiver=com.griddynamics.devops.mpl.MPLManager@372524ed, arguments=[org.jboss.marshalling.river.RiverObjectInputStream@6c0dd7d4]}
    Caused: hudson.remoting.ProxyException: java.io.IOException: Failed to load build state
    	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:855)
    	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:853)
    	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$4$1.run(CpsFlowExecution.java:907)
    	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:38)
    	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
    	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
    	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    	at java.base/java.lang.Thread.run(Thread.java:834)
    Finished: FAILURE
    
  • Jenkins output:
    2020-02-16 01:02:36.026+0000 [id=34]    INFO    hudson.WebAppMain$3#run: Jenkins is fully up and running
    2020-02-16 01:02:42.484+0000 [id=161]   WARNING jenkins.security.ClassFilterImpl#notifyRejected: com.cloudbees.groovy.cps.impl.CpsFunction in file:/var/jenkins
    _home/plugins/workflow-cps/WEB-INF/lib/groovy-cps-1.32.jar might be dangerous, so rejecting; see https://jenkins.io/redirect/class-filter/
    2020-02-16 01:02:42.595+0000 [id=179]   WARNING o.j.p.w.cps.CpsThreadGroup#unexport: double unexport of 1
    2020-02-16 01:02:42.597+0000 [id=161]   INFO    o.j.p.workflow.job.WorkflowRun#finish: mpl-build #4 completed: FAILURE
    

sparshev@griddynamics.com (JIRA)

unread,
Feb 15, 2020, 8:42:02 PM2/15/20
to jenkinsc...@googlegroups.com

Roland Stengel, actually @NonCPS helped, got it from `Similar Issues`: JENKINS-55113

  @NonCPS
  private void readObject(java.io.ObjectInputStream inp) throws IOException, ClassNotFoundException {
    System.out.println("Custom deserialization logic invoked.")
    inp.defaultReadObject()
  }

From Jenkins output:

2020-02-16 01:23:44.958+0000 [id=34]    INFO    hudson.WebAppMain$3#run: Jenkins is fully up and running
Custom deserialization logic invoked.

r.stengel@sap.com (JIRA)

unread,
Mar 5, 2020, 10:48:03 AM3/5/20
to jenkinsc...@googlegroups.com

Sergei, many thanks, it's simple and it works.

Interestingly, Jenkins does not serialize a multi branch pipeline, so i can't use read/writeObject to resume from stage, but this another story.... 

This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)
Atlassian logo
Reply all
Reply to author
Forward
0 new messages