Own Cloud Plugin: "ClassNotFoundException: javax.servlet.ServletException" when running job for the first time

[Johannes Schneider]

Hi guys,

I have a strange problem:

I have created a new plugin that provides a cloud implementation.

I create new nodes/computers and connect them using JNLP.

The first time a job runs on the newly created computer it fails with that log output:

```

FATAL: Remote call on JNLP4-connect connection from localhost/127.0.0.1:47026 failed
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection to localhost/127.0.0.1:44145
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
		at hudson.remoting.Request.call(Request.java:202)
		at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
		at com.sun.proxy.$Proxy6.fetch(Unknown Source)
		at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:315)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
		at hudson.util.ProcessTree.get(ProcessTree.java:399)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1090)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1081)
		at hudson.remoting.UserRequest.perform(UserRequest.java:212)
		at hudson.remoting.UserRequest.perform(UserRequest.java:54)
		at hudson.remoting.Request$2.run(Request.java:369)
		at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
		at java.util.concurrent.FutureTask.run(FutureTask.java:266)
		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
		at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
java.lang.ClassNotFoundException: javax.servlet.ServletException
	at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch(RemoteClassLoader.java:808)
	at sun.reflect.GeneratedMethodAccessor209.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:929)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:19)
	at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from localhost/127.0.0.1:47026
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:955)
		at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1078)
		at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
		at hudson.model.Run.execute(Run.java:1798)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
Caused: java.lang.NoClassDefFoundError: javax/servlet/ServletException
	at hudson.util.ProcessTree.get(ProcessTree.java:399)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1090)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1081)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.io.IOException: Remote call on JNLP4-connect connection from localhost/127.0.0.1:47026 failed
	at hudson.remoting.Channel.call(Channel.java:961)
	at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1078)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
	at hudson.model.Run.execute(Run.java:1798)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE
FATAL: Remote call on JNLP4-connect connection from localhost/127.0.0.1:47026 failed
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection to localhost/127.0.0.1:44145
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
		at hudson.remoting.Request.call(Request.java:202)
		at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
		at com.sun.proxy.$Proxy6.fetch(Unknown Source)
		at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:315)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
		at hudson.util.ProcessTree.get(ProcessTree.java:399)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1090)
		at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1081)
		at hudson.remoting.UserRequest.perform(UserRequest.java:212)
		at hudson.remoting.UserRequest.perform(UserRequest.java:54)
		at hudson.remoting.Request$2.run(Request.java:369)
		at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
		at java.util.concurrent.FutureTask.run(FutureTask.java:266)
		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
		at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
java.lang.ClassNotFoundException: javax.servlet.ServletException
	at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch(RemoteClassLoader.java:808)
	at sun.reflect.GeneratedMethodAccessor209.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:929)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:19)
	at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from localhost/127.0.0.1:47026
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:955)
		at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1078)
		at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
		at hudson.model.Run.execute(Run.java:1798)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
Caused: java.lang.NoClassDefFoundError: javax/servlet/ServletException
	at hudson.util.ProcessTree.get(ProcessTree.java:399)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1090)
	at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1081)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.io.IOException: Remote call on JNLP4-connect connection from localhost/127.0.0.1:47026 failed
	at hudson.remoting.Channel.call(Channel.java:961)
	at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1078)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
	at hudson.model.Run.execute(Run.java:1798)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE
```

I tried to debug it further and could narrow it down.

I think the exception is thrown the first time in the Method

hudson.util.ProcessTree#get() (Line 399).

Probably calling "SlaveComputer.getChannelToMaster()" fails.
SlaveComputer references javax.servlet.ServletException which seems to be unavailable there.

The agent.jar that is downloaded from Jenkins does *not* contain the javax.servlet.ServletException class.

Running the job a second time on the same computer works just fine.

The exception occurs after the job has finished.

Any idea how to solve that?

Regards,

Johannes

[Jesse Glick]

This is a longstanding JENKINS-26677 that has never been
satisfactorily resolved, though there have been workarounds. You can
see it without any custom plugin, just using `CommandLauncher` with
`jenkinsci/docker-agent`, depending on how Jenkins is being run and
what build steps you use. The problem relates to how the Servlet API
is loaded from the servlet container hosting the Jenkins master.

Anyway, if you are able to reproduce, please give feedback on
https://github.com/jenkinsci/jenkins/pull/3778. Assuming the build is
successful, it should deploy an incremental version
2.154-rc27545.0e85c9fb206a which you can use as `jenkins.version` in
your POM (or passing via `-D` to `hpi:run`).

[Johannes Schneider]

I will try

 

2.154-rc27545.0e85c9fb206a

as soon as it is build.

One question: What are the workarounds? What can be done?

[Jesse Glick]

On Tue, Nov 27, 2018 at 11:40 AM Johannes Schneider
<mail...@cedarsoft.com> wrote:
> What are the workarounds?

I am not aware of any.

[Johannes Schneider]

Hi Jesse,

could you give me a hint how to try it out?

I tried

mvn hpi:run -Djenkins.version=2.154-rc27545.0e85c9fb206a

which does not work (of course) because I don't have a snapshot repository configured. A quick search didn't give me
the the URL for the snapshot repository. Could you help me out?

[Daniel Beck]

> On 5. Dec 2018, at 18:01, Johannes Schneider <mail...@cedarsoft.com> wrote:
>
> which does not work (of course) because I don't have a snapshot repository configured. A quick search didn't give me
> the the URL for the snapshot repository. Could you help me out?
>

Incrementals are deployed to https://repo.jenkins-ci.org/incrementals/

[Baptiste Mathus]

Note that the Incrementals repo is *not* configured behind http://repo.jenkins-ci.org/public/ group repo on purpose.

So to enable automatic addition of that repository, you must add -Pconsume-incrementals to your command line call, and hence use a recent parent pom. See https://github.com/jenkinsci/plugin-pom

I think some ~3.18+ should be enough, but I'd recommend updating to latest 3.28 or 3.29 to avoid issues.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/A3911619-D4B1-4672-9974-151BC497538F%40beckweb.net.
For more options, visit https://groups.google.com/d/optout.

[Johannes Schneider]

But where are the snapshots published to? I can't find the version 2.154-rc27545.0e85c9fb206a

on https://repo.jenkins-ci.org/incrementals/org/jenkins-ci/main/jenkins-core/

I am running:

mvn hpi:run -Djenkins.version=2.154-rc27545.0e85c9fb206a -Pconsume-incrementals

Which does not work since https://repo.jenkins-ci.org/incrementals/org/jenkins-ci/main/jenkins-core/ doesn't contain that version.

[Jesse Glick]

On Mon, Dec 10, 2018 at 5:07 AM Johannes Schneider
<mail...@cedarsoft.com> wrote:
> But where are the snapshots published to? I can't find the version 2.154-rc27545.0e85c9fb206a

Because that build was unstable, and there has not been a stable build
in this PR. Incrementals only get deployed when a build is stable
_and_ the PR is up to date with the base branch. I filed

https://github.com/jenkinsci/jenkins/pull/3799