[JIRA] (JENKINS-60857) Wildcard certificates rejected by Winstone after Jetty update

86 views
Skip to first unread message

jglick@cloudbees.com (JIRA)

unread,
Jan 24, 2020, 4:15:03 PM1/24/20
to jenkinsc...@googlegroups.com
Jesse Glick updated an issue
 
Jenkins / Bug JENKINS-60857
Wildcard certificates rejected by Winstone after Jetty update
Change By: Jesse Glick
Component/s: winstone-jetty
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

jglick@cloudbees.com (JIRA)

unread,
Jan 24, 2020, 4:15:03 PM1/24/20
to jenkinsc...@googlegroups.com
Jesse Glick updated an issue
Change By: Jesse Glick
Summary: jetty no longer accepts the keystore Wildcard certificates rejected by Winstone after Jetty update

jglick@cloudbees.com (JIRA)

unread,
Jan 24, 2020, 4:15:04 PM1/24/20
to jenkinsc...@googlegroups.com
Jesse Glick updated an issue
Change By: Jesse Glick
Labels: SSL jetty keystore

o.v.nenashev@gmail.com (JIRA)

unread,
Jan 24, 2020, 6:25:03 PM1/24/20
to jenkinsc...@googlegroups.com
Oleg Nenashev commented on Bug JENKINS-60857
 
Re: Wildcard certificates rejected by Winstone after Jetty update

https://github.com/jenkinsci/jenkins/pull/4454 should integrated the fix. Hopefully we will get it released in the next weekly on Sunday

jglick@cloudbees.com (JIRA)

unread,
Jan 24, 2020, 8:21:03 PM1/24/20
to jenkinsc...@googlegroups.com
Change By: Jesse Glick
Status: In Review Fixed but Unreleased
Resolution: Fixed
Released As: 2.218

o.v.nenashev@gmail.com (JIRA)

unread,
Jan 27, 2020, 5:42:03 PM1/27/20
to jenkinsc...@googlegroups.com
Oleg Nenashev updated Bug JENKINS-60857
 

It was released in 2.218
Change By: Oleg Nenashev
Status: Fixed but Unreleased Resolved

michael.litwak@nuix.com (JIRA)

unread,
Jan 27, 2020, 10:30:04 PM1/27/20
to jenkinsc...@googlegroups.com
Michael Litwak commented on Bug JENKINS-60857
 
Re: Wildcard certificates rejected by Winstone after Jetty update

Confirming 2.218 installs/upgrades successfully on my Windows Server, while 2.217 gave the "multiple certificates" error.

Thanks for the fix.

christian.keck@macio.de (JIRA)

unread,
Jan 28, 2020, 2:33:02 AM1/28/20
to jenkinsc...@googlegroups.com

I can also confirm that the fix solves the issue.

Thanks for the great and fast support!

alexander.gaengel@1und1.de (JIRA)

unread,
Jan 28, 2020, 2:35:03 AM1/28/20
to jenkinsc...@googlegroups.com

Thanks for the release it's working again

medianick@gmail.com (JIRA)

unread,
Feb 29, 2020, 7:28:03 AM2/29/20
to jenkinsc...@googlegroups.com

This just bit us with 2.204.3 LTS. I don't see a corresponding bug report for this same error with the LTS release, so expect this should be reopened (and the Environment field updated to cover LTS) until it's resolved on that release line too.
This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)
Atlassian logo

medianick@gmail.com (JIRA)

unread,
Feb 29, 2020, 7:29:03 AM2/29/20
to jenkinsc...@googlegroups.com
Nick Jones reopened an issue
 

2.204.3 LTS has the same issue.
Change By: Nick Jones
Resolution: Fixed
Status: Resolved Reopened

medianick@gmail.com (JIRA)

unread,
Feb 29, 2020, 7:29:06 AM2/29/20
to jenkinsc...@googlegroups.com
Nick Jones updated an issue
Change By: Nick Jones
Environment: CentOS 6.10
Jenkins 2.217 , Jenkins 2.204.3 LTS
Wildcard-SSL-Certificate in Java-Keystore in PKCS12 format

medianick@gmail.com (JIRA)

unread,
Feb 29, 2020, 7:30:03 AM2/29/20
to jenkinsc...@googlegroups.com
Nick Jones updated an issue
Change By: Nick Jones
Comment:
2.204.3 LTS has the same issue.

medianick@gmail.com (JIRA)

unread,
Feb 29, 2020, 7:58:06 AM2/29/20
to jenkinsc...@googlegroups.com
Nick Jones updated an issue
Change By: Nick Jones
Environment: CentOS 6.10 , Windows Server 2012 R2
Jenkins 2.217, Jenkins 2.204.3 LTS

Wildcard-SSL-Certificate in Java-Keystore in PKCS12 format

medianick@gmail.com (JIRA)

unread,
Feb 29, 2020, 7:59:03 AM2/29/20
to jenkinsc...@googlegroups.com
Nick Jones updated an issue
Change By: Nick Jones
Environment: Jenkins 2.217 on CentOS 6.10 , Windows Server 2012 R2
Jenkins 2. 217, Jenkins 2. 204.3 LTS on Windows Server 2012 R2
Wildcard-SSL-Certificate in Java-Keystore in PKCS12 format

dbeck@cloudbees.com (JIRA)

unread,
Feb 29, 2020, 8:44:03 AM2/29/20
to jenkinsc...@googlegroups.com
Daniel Beck updated an issue
Change By: Daniel Beck
Labels: SSL keystore regression

dbeck@cloudbees.com (JIRA)

unread,
Feb 29, 2020, 8:44:04 AM2/29/20
to jenkinsc...@googlegroups.com
Daniel Beck commented on Bug JENKINS-60857
 
Re: Wildcard certificates rejected by Winstone after Jetty update

Looks like JENKINS-60821 was backported into 2.204.3 without considering this regression.

dbeck@cloudbees.com (JIRA)

unread,
Feb 29, 2020, 8:48:02 AM2/29/20
to jenkinsc...@googlegroups.com
Daniel Beck edited a comment on Bug JENKINS-60857
Looks like JENKINS-60821 was backported into 2.204.3 without considering this regression :

[https://github . com/jenkinsci/jenkins/commit/23fce281bd4aa92791ab8e5793ea884e543d841f] is the backport, and only [https://github.com/jenkinsci/winstone/releases/tag/winstone-5.8] has the SSL fix.

dbeck@cloudbees.com (JIRA)

unread,
Feb 29, 2020, 8:49:03 AM2/29/20
to jenkinsc...@googlegroups.com

Oleg Nenashev Oliver Gondža Does this make the cut for a regression that justifies an unscheduled .4 release?

o.v.nenashev@gmail.com (JIRA)

unread,
Feb 29, 2020, 12:18:03 PM2/29/20
to jenkinsc...@googlegroups.com

IMO yes

P.S: Sorry. I was so busy with weekly and other things that I dis not even take a look at the RC

jglick@cloudbees.com (JIRA)

unread,
Feb 29, 2020, 3:46:03 PM2/29/20
to jenkinsc...@googlegroups.com

So IIUC it would suffice for winstone to be bumped from 5.7 to 5.8 in the stable-2.204 branch?

jglick@cloudbees.com (JIRA)

unread,
Feb 29, 2020, 3:46:04 PM2/29/20
to jenkinsc...@googlegroups.com
Jesse Glick updated an issue
 
Change By: Jesse Glick
Labels: SSL keystore lts-candidate regression

Jonathan_Gray@comcast.com (JIRA)

unread,
Mar 1, 2020, 12:53:03 PM3/1/20
to jenkinsc...@googlegroups.com
Jonathan Gray commented on Bug JENKINS-60857
 
Re: Wildcard certificates rejected by Winstone after Jetty update

For what it's worth, this prevented my instance from starting and I had to rollback to 2.204.2.

Jonathan_Gray@comcast.com (JIRA)

unread,
Mar 1, 2020, 1:07:04 PM3/1/20
to jenkinsc...@googlegroups.com
Jonathan Gray edited a comment on Bug JENKINS-60857
For what it's worth, this prevented my instance from starting and I had to rollback to the 2.204.2 docker container .

 
{quote}
webroot: EnvVars.masterEnvVars.get("JENKINS_HOME")
[2020-03-01 17:20:09] [INFO   ] Logging initialized @382ms to org.eclipse.jetty.util.log.JavaUtilLog
[2020-03-01 17:20:09] [INFO   ] Beginning extraction from war file
[2020-03-01 17:20:10] [WARNING] Empty contextPath
[2020-03-01 17:20:10] [WARNING] Using the --httpsPrivateKey/--httpsCertificate options currently relies on unsupported APIs in the Oracle JRE.
Please use --httpsKeyStore and related options instead.
[2020-03-01 17:20:10] [INFO   ] Exclude Ciphers [^.*_(MD5|SHA|SHA1)$, ^TLS_RSA_.*$, ^SSL_.*$, ^.*_NULL_.*$, ^.*_anon_.*$]
[2020-03-01 17:20:10] [INFO   ] jetty-9.4.25.v20191220; built: 2019-12-20T17:00:00.294Z; git: a9729c7e7f33a459d2616a8f9e9ba8a90f432e95; jvm 1.8.0_242-b08
[2020-03-01 17:20:10] [INFO   ] NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet
[2020-03-01 17:20:10] [INFO   ] DefaultSessionIdManager workerName=node0
[2020-03-01 17:20:10] [INFO   ] No SessionScavenger set, using defaults
[2020-03-01 17:20:10] [INFO   ] node0 Scavenging every 660000ms
[2020-03-01 17:20:11] [INFO   ] Jenkins home directory: /var/jenkins_home found at: EnvVars.masterEnvVars.get("JENKINS_HOME")
[2020-03-01 17:20:11] [INFO   ] Started w.@2ad48653{Jenkins v2.204.3,/,file:///var/jenkins_home/war/,AVAILABLE}{/var/jenkins_home/war}
[2020-03-01 17:20:11] [INFO   ] Started ServerConnector@77b52d12{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
[2020-03-01 17:20:11] [INFO   ] x509=X509@17d919b6(hudson,h=[ci.teamlab.domain.invalid, team-jenkins-oshoc-01.team.domain.invalid, cinew.teamlab.domain.invalid, www.ci.teamlab.domain.invalid, www.cinew.teamlab.domain.invalid],w=[]) for SslContextFactory@53f3bdbd[provider=null,keyStore=null,trustStore=null]
[2020-03-01 17:20:11] [INFO   ] Stopped ServerConnector@77b52d12{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
[2020-03-01 17:20:11] [INFO   ] Stopped ServerConnector@100fc185{SSL,[ssl, http/1.1]}{0.0.0.0:8443}
[2020-03-01 17:20:11] [INFO   ] node0 Stopped scavenging
[2020-03-01 17:20:11] [INFO   ] Shutting down a Jenkins instance that was still starting up
[2020-03-01 17:20:11] [INFO   ] Stopped w.@2ad48653{Jenkins v2.204.3,/,null,UNAVAILABLE}{/var/jenkins_home/war}
[2020-03-01 17:20:11] [INFO ] Jetty shutdown successfully java.io.IOException: Failed to start Jetty at winstone.Launcher.<init>(Launcher.java:191) at winstone.Launcher.main(Launcher.java:362) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at Main._main(Main.java:375) at Main.main(Main.java:151) Caused by: java.lang.IllegalStateException: KeyStores with multiple certificates are not supported on the base class org.eclipse.jetty.util.ssl.SslContextFactory. (Use org.eclipse.jetty.util.ssl.SslContextFactory$Server or org.eclipse.jetty.util.ssl.SslContextFactory$Client instead) at org.eclipse.jetty.util.ssl.SslContextFactory.newSniX509ExtendedKeyManager(SslContextFactory.java:1275) at org.eclipse.jetty.util.ssl.SslContextFactory.getKeyManagers(SslContextFactory.java:1256) at org.eclipse.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:374) at org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:245) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:92) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:320) at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81) at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:231) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.server.Server.doStart(Server.java:385) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at winstone.Launcher.<init>(Launcher.java:189) ... 7 more Exception in thread "Jenkins initialization thread" java.lang.NoClassDefFoundError: hudson/util/HudsonFailedToLoad at hudson.WebAppMain$3.run(WebAppMain.java:247) Caused by: java.lang.ClassNotFoundException: hudson.util.HudsonFailedToLoad at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:419) at java.lang.ClassLoader.loadClass(ClassLoader.java:352) at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:543) at java.lang.ClassLoader.loadClass(ClassLoader.java:352) ... 1 more [2020-03-01 17:20:11] [SEVERE ] Container startup failed
{quote}

Jonathan_Gray@comcast.com (JIRA)

unread,
Mar 1, 2020, 1:09:05 PM3/1/20
to jenkinsc...@googlegroups.com
Jonathan Gray edited a comment on Bug JENKINS-60857
For what it's worth, this prevented my instance from starting and I had to rollback to the 2.204.2 docker container.

  
{ quote noformat }
{ quote noformat }

Jonathan_Gray@comcast.com (JIRA)

unread,
Mar 1, 2020, 1:11:04 PM3/1/20
to jenkinsc...@googlegroups.com
Jonathan Gray edited a comment on Bug JENKINS-60857
For what it's worth, this prevented my instance from starting and I had to rollback to the 2.204.2 docker container.

{noformat}
org.eclipse.jetty.util.ssl.SslContextFactory.newSniX509ExtendedKeyManager( SslContextFactory SslContextFact
ory .java:1275) at
org.eclipse.jetty.util.ssl.SslContextFactory.getKeyManagers(SslContextFactory.java:1256) at
org.eclipse.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:374) at
org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:245) at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at
org.eclipse.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:92) at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at
org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:320) at
org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java: 81 8
1 ) at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:231) at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at
org.eclipse.jetty.server.Server.doStart(Server.java:385) at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at
winstone.Launcher.<init>(Launcher.java:189) ... 7 more Exception in thread "Jenkins
initialization thread" java.lang.NoClassDefFoundError: hudson/util/HudsonFailedToLoad at
hudson.WebAppMain$3.run(WebAppMain.java:247) Caused by:
java.lang.ClassNotFoundException: hudson.util.HudsonFailedToLoad at
java.net.URLClassLoader.findClass(URLClassLoader.java:382) at
java.lang.ClassLoader.loadClass(ClassLoader.java:419) at
java.lang.ClassLoader.loadClass(ClassLoader.java:352) at
org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:543) at
java.lang.ClassLoader.loadClass(ClassLoader.java:352) ... 1 more [2020-03-01 17:20:11]
[SEVERE ] Container startup failed
{noformat}

Jonathan_Gray@comcast.com (JIRA)

unread,
Mar 1, 2020, 1:11:11 PM3/1/20
to jenkinsc...@googlegroups.com
ory.java:1275) at
org.eclipse.jetty.util.ssl.SslContextFactory.getKeyManagers(SslContextFactory.java:1256) at
org.eclipse.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:374) at
org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:245) at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at
org.eclipse.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:92) at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at
org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:320) at
org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:8
1) at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:231) at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at
org.eclipse.jetty.server.Server.doStart(Server.java:385) at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at
winstone.Launcher.<init>(Launcher.java:189) ... 7 more Exception in thread "Jenkins
initialization thread" java.lang.NoClassDefFoundError: hudson/util/HudsonFailedToLoad at
hudson.WebAppMain$3.run(WebAppMain.java:247) Caused by:
java.lang.ClassNotFoundException: hudson.util.HudsonFailedToLoad at
java.net.URLClassLoader.findClass(URLClassLoader.java:382) at
java.lang.ClassLoader.loadClass(ClassLoader.java:419) at
java.lang.ClassLoader.loadClass(ClassLoader.java:352) at
org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:543) at
java.lang.ClassLoader.loadClass(ClassLoader.java:352) ... 1 more
[2020-03-01 17:20:11]
[SEVERE ] Container startup failed
{noformat}

Jonathan_Gray@comcast.com (JIRA)

unread,
Mar 1, 2020, 1:15:05 PM3/1/20
to jenkinsc...@googlegroups.com
Jonathan Gray edited a comment on Bug JENKINS-60857
For what it's worth, this prevented my instance from starting and I had to rollback to the 2.204.2 docker container.

{noformat}
Running from: /usr/share/jenkins/jenkins.war
org.eclipse.jetty.util.ssl.SslContextFactory.newSniX509ExtendedKeyManager( SslContextFact
ory SslContextFactory .java:1275)
at
org.eclipse.jetty.util.ssl.SslContextFactory.getKeyManagers(SslContextFactory.java:1256)
at
org.eclipse.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:374)
at
org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:245)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at
org.eclipse.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:92)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at
org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:320)
at
org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:
8
1 81 )

Jonathan_Gray@comcast.com (JIRA)

unread,
Mar 1, 2020, 1:19:04 PM3/1/20
to jenkinsc...@googlegroups.com
at org.eclipse.jetty.util.ssl.SslContextFactory.newSniX509ExtendedKeyManager(SslContextFactory.java:1275)

at org.eclipse.jetty.util.ssl.SslContextFactory.getKeyManagers(SslContextFactory.java:1256)
at org.eclipse.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:374)
at org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:245)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at org.eclipse.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:92)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:320)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81)

at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:231)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
at org.eclipse.jetty.server.Server.doStart(Server.java:385)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
at winstone.Launcher.<init>(Launcher.java:189)
... 7 more
Exception in thread "Jenkins initialization thread" java.lang.NoClassDefFoundError: hudson/util/HudsonFailedToLoad
at hudson.WebAppMain$3.run(WebAppMain.java:247)
Caused by: java.lang.ClassNotFoundException: hudson.util.HudsonFailedToLoad
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:543)
at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
... 1 more
[2020-03-01 17:20:11] [SEVERE ] Container startup failed
{noformat}


edit: Added logs

clarkster82@gmail.com (JIRA)

unread,
Mar 1, 2020, 9:57:02 PM3/1/20
to jenkinsc...@googlegroups.com

Is this just simply updating the winstone version at https://github.com/jenkinsci/jenkins/blob/jenkins-2.204.3/war/pom.xml#L104 to 5.8?

johmart@java.net (JIRA)

unread,
Mar 2, 2020, 2:37:03 AM3/2/20
to jenkinsc...@googlegroups.com
johmart updated an issue
 
Change By: johmart
Environment:
Jenkins 2.217 on CentOS 6.10
Jenkins 2.204.3 LTS on Windows Server 2012 R2

Wildcard-SSL-Certificate in Java-Keystore in PKCS12 format

Jenkins 2.204.3 LTS on Ubuntu 18.04.4 LTS

hp@localhorst.org (JIRA)

unread,
Mar 2, 2020, 4:18:03 AM3/2/20
to jenkinsc...@googlegroups.com
Horst Platz commented on Bug JENKINS-60857
 
Re: Wildcard certificates rejected by Winstone after Jetty update

same for me...

Ubuntu 18.04.4 LTS

:~$ java -version
openjdk version "11.0.6" 2020-01-14
OpenJDK Runtime Environment (build 11.0.6+10-post-Ubuntu-1ubuntu118.04.1)
OpenJDK 64-Bit Server VM (build 11.0.6+10-post-Ubuntu-1ubuntu118.04.1, mixed mode, sharing)

after Update from 2.204.2 to 2.204.3

Mar 2 09:26:39 build01 hpljenkins[23864]: 2020-03-02 08:26:39.735+0000 [id=1]#011SEVERE#011winstone.Logger#logInternal: Container startup failed
Mar 2 09:26:39 build01 hpljenkins[23864]: java.lang.IllegalStateException: KeyStores with multiple certificates are not supported on the base class org.eclipse.jetty.util.ssl.SslContextFactory. (Use org.eclipse.jetty.util.ssl.SslContextFactory$Server or org.eclipse.jetty.util.ssl.SslContextFactory$Client instead)
[...]
Mar 2 09:26:39 build01 systemd[1]: hpljenkins.service: Main process exited, code=exited, status=1/FAILURE
Mar 2 09:26:39 build01 systemd[1]: hpljenkins.service: Failed with result 'exit-code'.

 

Rollback to 2.204.2

o.v.nenashev@gmail.com (JIRA)

unread,
Mar 2, 2020, 4:38:05 AM3/2/20
to jenkinsc...@googlegroups.com

I will submit a patch and coordinate .4 with other stakeholders

o.v.nenashev@gmail.com (JIRA)

unread,
Mar 2, 2020, 5:04:03 AM3/2/20
to jenkinsc...@googlegroups.com
Oleg Nenashev assigned an issue to Oleg Nenashev
 
Change By: Oleg Nenashev
Assignee: Jesse Glick Oleg Nenashev

o.v.nenashev@gmail.com (JIRA)

unread,
Mar 2, 2020, 5:16:03 AM3/2/20
to jenkinsc...@googlegroups.com

jglick@cloudbees.com (JIRA)

unread,
Mar 2, 2020, 8:15:04 AM3/2/20
to jenkinsc...@googlegroups.com
Jesse Glick resolved as Fixed
 

LTS patch merged.
Change By: Jesse Glick
Status: Reopened Resolved
Resolution: Fixed
Released As: 2.218 2.204.4

sf258g@att.com (JIRA)

unread,
Mar 2, 2020, 4:01:21 PM3/2/20
to jenkinsc...@googlegroups.com
Steven Fransen commented on Bug JENKINS-60857
 
Re: Wildcard certificates rejected by Winstone after Jetty update

I update my jenkins to LTS 2.204.3 and it would not even launch 

 

$ cat failed-boot-attempts.txt Mon Mar 02 08:01:43 PST 2020 Mon Mar 02 08:08:07 PST 2020 Mon Mar 02 08:08:54 PST 2020 Mon Mar 02 08:35:52 PST 2020 Mon Mar 02 08:37:10 PST 2020 Mon Mar 02 08:40:31 PST 2020 Mon Mar 02 08:42:34 PST 2020 Mon Mar 02 08:44:01 PST 2020 Mon Mar 02 08:46:54 PST 2020 Mon Mar 02 08:47:53 PST 2020 Mon Mar 02 08:56:15 PST 2020 [08:58:31] root@mtjenkins01/var/lib/jenkins $ cat /var/log/jenkins/jenkins.log Running from: /usr/lib/jenkins/jenkins.war 2020-03-02 16:56:15.111+0000 [id=1] WARNING winstone.Logger#logInternal: Parameter handlerCountMax is now deprecated 2020-03-02 16:56:15.127+0000 [id=1] WARNING winstone.Logger#logInternal: Parameter handlerCountMaxIdle is now deprecated 2020-03-02 16:56:15.132+0000 [id=1] INFO org.eclipse.jetty.util.log.Log#initialized: Logging initialized @498ms to org.eclipse.jetty.util.log.JavaUtilLog 2020-03-02 16:56:15.179+0000 [id=1] INFO winstone.Logger#logInternal: Beginning extraction from war file 2020-03-02 16:56:15.207+0000 [id=1] WARNING o.e.j.s.handler.ContextHandler#setContextPath: Empty contextPath 2020-03-02 16:56:15.412+0000 [id=1] INFO winstone.Logger#logInternal: Exclude Ciphers [^.*_(MD5|SHA|SHA1)$, ^TLS_RSA_.*$, ^SSL_.*$, ^.*_NULL_.*$, ^.*_anon_.*$] 2020-03-02 16:56:15.438+0000 [id=1] INFO org.eclipse.jetty.server.Server#doStart: jetty-9.4.25.v20191220; built: 2019-12-20T17:00:00.294Z; git: a9729c7e7f33a459d2616a8f9e9ba8a90f432e95; jvm 1.8.0_222-b10 2020-03-02 16:56:15.645+0000 [id=1] INFO o.e.j.w.StandardDescriptorProcessor#visitServlet: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet 2020-03-02 16:56:15.679+0000 [id=1] INFO o.e.j.s.s.DefaultSessionIdManager#doStart: DefaultSessionIdManager workerName=node0 2020-03-02 16:56:15.679+0000 [id=1] INFO o.e.j.s.s.DefaultSessionIdManager#doStart: No SessionScavenger set, using defaults 2020-03-02 16:56:15.681+0000 [id=1] INFO o.e.j.server.session.HouseKeeper#startScavenging: node0 Scavenging every 660000ms 2020-03-02 16:56:15.967+0000 [id=1] INFO hudson.WebAppMain#contextInitialized: Jenkins home directory: /var/lib/jenkins found at: SystemProperties.getProperty("JENKINS_HOME") 2020-03-02 16:56:16.038+0000 [id=1] INFO o.e.j.s.handler.ContextHandler#doStart: Started w.@2a7ed1f{Jenkins v2.204.3,/,file:///var/cache/jenkins/war/,AVAILABLE}{/var/cache/jenkins/war} 2020-03-02 16:56:16.084+0000 [id=1] INFO o.e.j.util.ssl.SslContextFactory#load: x509=X509@463fd068(mtcoveritydb01,h=[],w=[]) for SslContextFactory@895e367[provider=null,keyStore=null,trustStore=null] 2020-03-02 16:56:16.084+0000 [id=1] INFO o.e.j.util.ssl.SslContextFactory#load: x509=X509@1b266842(starfieldclass2ca,h=[],w=[]) for SslContextFactory@895e367[provider=null,keyStore=null,trustStore=null] 2020-03-02 16:56:16.085+0000 [id=1] INFO o.e.j.util.ssl.SslContextFactory#load: x509=X509@7a3793c7(taiwangrca,h=[],w=[]) for SslContextFactory@895e367[provider=null,keyStore=null,trustStore=null] 2020-03-02 16:56:16.085+0000 [id=1] INFO o.e.j.util.ssl.SslContextFactory#load: x509=X509@42b3b079(itservices.def.com,h=[gaalpa2adssrv53.itservices.def.com, itservices.def.com, gaalpa2adssrv53, itservices, its-ad-ldap.it.xxx.com, its-ad-ldap.lrns.def.com, its-ad-ldap.cci.xxx.com],w=[]) for SslContextFactory@895e367[provider=null,keyStore=null,trustStore=null] 2020-03-02 16:56:16.086+0000 [id=1] INFO o.e.j.util.ssl.SslContextFactory#load: x509=X509@651aed93(geotrustglobalca,h=[],w=[]) for SslContextFactory@895e367[provider=null,keyStore=null,trustStore=null] 2020-03-02 16:56:16.086+0000 [id=1] INFO o.e.j.util.ssl.SslContextFactory#load: x509=X509@4dd6fd0a(1,h=[mtjenkins01.quantum.xxx.com],w=[]) for SslContextFactory@895e367[provider=null,keyStore=null,trustStore=null] 2020-03-02 16:56:16.087+0000 [id=1] INFO o.e.j.util.ssl.SslContextFactory#load: x509=X509@bb9e6dc(verisignclass3publicprimarycertificationauthority-g3,h=[],w=[]) for SslContextFactory@895e367[provider=null,keyStore=null,trustStore=null] 2020-03-02 16:56:16.087+0000 [id=1] INFO o.e.j.util.ssl.SslContextFactory#load: x509=X509@5456afaa(godaddyclass2ca,h=[],w=[]) for SslContextFactory@895e367[provider=null,keyStore=null,trustStore=null] 2020-03-02 16:56:16.088+0000 [id=1] INFO o.e.j.util.ssl.SslContextFactory#load: x509=X509@6692b6c6(trustisfpsrootca,h=[],w=[]) for SslContextFactory@895e367[provider=null,keyStore=null,trustStore=null] 2020-03-02 16:56:16.088+0000 [id=1] INFO o.e.j.util.ssl.SslContextFactory#load: x509=X509@1cd629b3(epkirootcertificationauthority,h=[],w=[]) for SslContextFactory@895e367[provider=null,keyStore=null,trustStore=null] 2020-03-02 16:56:16.092+0000 [id=1] INFO o.e.j.server.AbstractConnector#doStop: Stopped ServerConnector@5702b3b1{SSL,[ssl, http/1.1]}{0.0.0.0:8443} 2020-03-02 16:56:16.092+0000 [id=1] INFO o.e.j.server.session.HouseKeeper#stopScavenging: node0 Stopped scavenging 2020-03-02 16:56:16.094+0000 [id=1] INFO hudson.WebAppMain#contextDestroyed: Shutting down a Jenkins instance that was still starting up java.lang.Throwable: reason at hudson.WebAppMain.contextDestroyed(WebAppMain.java:388) at org.eclipse.jetty.server.handler.ContextHandler.callContextDestroyed(ContextHandler.java:937) at org.eclipse.jetty.servlet.ServletContextHandler.callContextDestroyed(ServletContextHandler.java:565) at org.eclipse.jetty.server.handler.ContextHandler.stopContext(ContextHandler.java:905) at org.eclipse.jetty.servlet.ServletContextHandler.stopContext(ServletContextHandler.java:367) at org.eclipse.jetty.webapp.WebAppContext.stopWebapp(WebAppContext.java:1450) at org.eclipse.jetty.webapp.WebAppContext.stopContext(WebAppContext.java:1415) at org.eclipse.jetty.server.handler.ContextHandler.doStop(ContextHandler.java:980) at org.eclipse.jetty.servlet.ServletContextHandler.doStop(ServletContextHandler.java:284) at org.eclipse.jetty.webapp.WebAppContext.doStop(WebAppContext.java:547) at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:93) at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:180) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:201) at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:111) at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:93) at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:180) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:201) at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:111) at org.eclipse.jetty.server.Server.doStop(Server.java:454) at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:93) at winstone.Launcher.shutdown(Launcher.java:311) at winstone.Launcher.<init>(Launcher.java:202) at winstone.Launcher.main(Launcher.java:362) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at Main._main(Main.java:375) at Main.main(Main.java:151) 2020-03-02 16:56:16.097+0000 [id=1] INFO o.e.j.s.handler.ContextHandler#doStop: Stopped w.@2a7ed1f{Jenkins v2.204.3,/,null,UNAVAILABLE}{/var/cache/jenkins/war} Exception in thread "Jenkins initialization thread" java.lang.NoClassDefFoundError: hudson/util/HudsonFailedToLoad at hudson.WebAppMain$3.run(WebAppMain.java:247) Caused by: java.lang.ClassNotFoundException: hudson.util.HudsonFailedToLoad at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:543) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 1 more 2020-03-02 16:56:16.097+0000 [id=1] INFO winstone.Logger#logInternal: Jetty shutdown successfully java.io.IOException: Failed to start Jetty at winstone.Launcher.<init>(Launcher.java:191) at winstone.Launcher.main(Launcher.java:362) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at Main._main(Main.java:375) at Main.main(Main.java:151) Caused by: java.lang.IllegalStateException: KeyStores with multiple certificates are not supported on the base class org.eclipse.jetty.util.ssl.SslContextFactory. (Use org.eclipse.jetty.util.ssl.SslContextFactory$Server or org.eclipse.jetty.util.ssl.SslContextFactory$Client instead) at org.eclipse.jetty.util.ssl.SslContextFactory.newSniX509ExtendedKeyManager(SslContextFactory.java:1275) at org.eclipse.jetty.util.ssl.SslContextFactory.getKeyManagers(SslContextFactory.java:1256) at org.eclipse.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:374) at org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:245) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:92) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:320) at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81) at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:231) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.server.Server.doStart(Server.java:385) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at winstone.Launcher.<init>(Launcher.java:189) ... 7 more 2020-03-02 16:56:16.098+0000 [id=1] SEVERE winstone.Logger#logInternal: Container startup failed java.lang.IllegalStateException: KeyStores with multiple certificates are not supported on the base class org.eclipse.jetty.util.ssl.SslContextFactory. (Use org.eclipse.jetty.util.ssl.SslContextFactory$Server or org.eclipse.jetty.util.ssl.SslContextFactory$Client instead) at org.eclipse.jetty.util.ssl.SslContextFactory.newSniX509ExtendedKeyManager(SslContextFactory.java:1275) at org.eclipse.jetty.util.ssl.SslContextFactory.getKeyManagers(SslContextFactory.java:1256) at org.eclipse.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:374) at org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:245) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:92) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:320) at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81) at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:231) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at org.eclipse.jetty.server.Server.doStart(Server.java:385) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72) at winstone.Launcher.<init>(Launcher.java:189) Caused: java.io.IOException: Failed to start Jetty at winstone.Launcher.<init>(Launcher.java:191) at winstone.Launcher.main(Launcher.java:362) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at Main._main(Main.java:375) at Main.main(Main.java:151) [08:58:55] root@mtjenkins01/var/lib/jenkins $

 

 I had to revert back to 2.190.1 

and it messed up my jenkins

Please fix

 

jglick@cloudbees.com (JIRA)

unread,
Mar 2, 2020, 4:32:03 PM3/2/20
to jenkinsc...@googlegroups.com

This is known to be broken in 2.204.3. Either go back to 2.204.2, or wait for 2.204.4. Please do not comment further in JIRA or the pull request.

ogondza@gmail.com (JIRA)

unread,
Mar 6, 2020, 9:45:03 AM3/6/20
to jenkinsc...@googlegroups.com

Winstone component will be reverted in 2.204.5 to a version prior this regression ware introduced.

dbeck@cloudbees.com (JIRA)

unread,
Mar 7, 2020, 9:16:03 PM3/7/20
to jenkinsc...@googlegroups.com
Daniel Beck updated an issue
Change By: Daniel Beck
Labels: 2.204.4-fixed SSL keystore lts-candidate regression

ogondza@gmail.com (JIRA)

unread,
Mar 10, 2020, 9:53:05 AM3/10/20
to jenkinsc...@googlegroups.com
Oliver Gondža updated an issue
Change By: Oliver Gondža
Labels: 2.204.4-fixed 2.222.1-fixed SSL keystore lts-candidate regression
Reply all
Reply to author
Forward
0 new messages