[JIRA] (JENKINS-60409) Cloud Configuration Length Cap

Issue Type:	Bug
Assignee:	Unassigned
Attachments:	Screen Shot 2019-12-09 at 10.21.53 AM.png
Components:	core
Created:	2019-12-09 18:25
Priority:	Major
Reporter:	Joshua Hunter

In Jenkins Version 2.205, PR #4339 moved the cloud configuration from Configure System into is own configuration form on the Manage Nodes page. There is a cap to the length of this form (200000) and prevents me from adding additional docker clouds into the settings.

 
                                                                java.lang.IllegalStateException: Form is larger than max length 200000
	at org.eclipse.jetty.server.Request.extractFormParameters(Request.java:562)
	at org.eclipse.jetty.server.Request.extractContentParameters(Request.java:519)
	at org.eclipse.jetty.server.Request.getParameters(Request.java:430)
Caused: org.eclipse.jetty.http.BadMessageException: 400: Unable to parse form content
	at org.eclipse.jetty.server.Request.getParameters(Request.java:434)
	at org.eclipse.jetty.server.Request.getParameterNames(Request.java:1077)
	at hudson.security.csrf.CrumbFilter.extractCrumbFromRequest(CrumbFilter.java:112)
	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:81)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
	at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:512)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1592)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1296)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1562)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1211)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:500)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:386)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:562)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:378)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:270)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.lang.Thread.run(Thread.java:748)
 
                                                            

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)

ksemtini.mahmoud@gmail.com (JIRA)

unread,

Dec 10, 2019, 9:29:06 AM12/10/19

to jenkinsc...@googlegroups.com

Mahmoud Ksemtini updated an issue

Jenkins /

Change By:	Mahmoud Ksemtini
Attachment:	Screenshot 2019-12-10 at 15.27.37.png

ksemtini.mahmoud@gmail.com (JIRA)

unread,

Dec 10, 2019, 9:29:06 AM12/10/19

to jenkinsc...@googlegroups.com

Mahmoud Ksemtini commented on

I get the same error when trying to replay a job from the UI.

ksemtini.mahmoud@gmail.com (JIRA)

unread,

Dec 10, 2019, 9:29:06 AM12/10/19

to jenkinsc...@googlegroups.com

Mahmoud Ksemtini edited a comment on

I get the same error when trying to replay a job from the UI. (Jenkins Version 2.205) !Screenshot 2019-12-10 at 15.27.37.png|width=732,height=395!

dean.smith@iter8.com.au (JIRA)

unread,

Dec 10, 2019, 5:28:01 PM12/10/19

to jenkinsc...@googlegroups.com

Dean Smith commented on

We have also been experiencing the same error when replaying builds since 2.205.

It seems to only be for jobs with a large Jenkinsfile (84,100 characters/ ~1800 lines) but not jobs with smaller Jenkinsfiles, although even the large ones are below 200,000 characters.

pedro.cuello.orozco@gmail.com (JIRA)

unread,

Dec 11, 2019, 4:16:03 AM12/11/19

to jenkinsc...@googlegroups.com

Pedro Cuello Orozco commented on

Hi guys,

I am facing the same issue when I try to give access to a new user into my permission matrix Role. I guess it happens in cloud version, but I am facing it in the not cloud version 2.207. I am forced to get back to my previous version 2.196

Regards,

jwnmulder@java.net (JIRA)

unread,

Dec 18, 2019, 10:05:03 AM12/18/19

to jenkinsc...@googlegroups.com

jwnmulder commented on

Running jenkins with -Dorg.eclipse.jetty.server.Request.maxFormContentSize=1000000" did resolve the issue for us

o.v.nenashev@gmail.com (JIRA)

unread,

Dec 18, 2019, 2:13:03 PM12/18/19

to jenkinsc...@googlegroups.com

Oleg Nenashev commented on

Daniel Beck

dbeck@cloudbees.com (JIRA)

unread,

Dec 18, 2019, 8:22:02 PM12/18/19

to jenkinsc...@googlegroups.com

Daniel Beck commented on

Oleg Nenashev

A comment from a week ago stated:

We have also been experiencing the same error when replaying builds since 2.205.

Clearly this isn't my PR that only moved some form fields in the global configs around.

I suggest you look at the Winstone/Jetty update. This looks interesting:

+ 3856 Different behaviour with maxFormContentSize=0 if Content-Length header is present/missing

dbeck@cloudbees.com (JIRA)

unread,

Dec 18, 2019, 8:32:02 PM12/18/19

to jenkinsc...@googlegroups.com

Daniel Beck commented on

I just backported 4339 onto 2.204 and the form works like a charm. Whatever is going on here has absolutely nothing to do with that PR.

dbeck@cloudbees.com (JIRA)

unread,

Dec 18, 2019, 8:41:03 PM12/18/19

to jenkinsc...@googlegroups.com

Daniel Beck edited a comment on

I just backported 4339 onto 2.204 and the form works like a charm with the same content that makes 2 . 205 fail. Whatever is going on here has absolutely nothing to do with that PR.

dean.smith@iter8.com.au (JIRA)

unread,

Dec 18, 2019, 9:23:03 PM12/18/19

to jenkinsc...@googlegroups.com

Dean Smith commented on

Running jenkins with -Dorg.eclipse.jetty.server.Request.maxFormContentSize=1000000" did resolve the issue for us

This resolved the issue for us as well.

I agree that it looks to be nothing to do with any core Jenkins changes but instead Winstone/Jetty library changes that curtailed this limit.

o.v.nenashev@gmail.com (JIRA)

unread,

Feb 25, 2020, 10:41:11 AM2/25/20

to jenkinsc...@googlegroups.com

Oleg Nenashev updated an issue

Jenkins /

Change By:	Oleg Nenashev
Component/s:	winstone-jetty

o.v.nenashev@gmail.com (JIRA)

unread,

Feb 25, 2020, 10:43:03 AM2/25/20

to jenkinsc...@googlegroups.com

Oleg Nenashev commented on

We just received a bunch of new reports, linked them to this ticket.

Olivier Lamy any hints on the Jetty side? "+ 3856 Different behaviour with maxFormContentSize=0 if Content-Length header is present/missing" mentioned by Daniel Beck makes me think we need to alter the defaults in Winstone to restore the original behavior

olamy@apache.org (JIRA)

unread,

Feb 25, 2020, 7:24:05 PM2/25/20

to jenkinsc...@googlegroups.com

Olivier Lamy commented on

I guess we need to update the default value. maybe something around 500000? I reckon it's a bit sensible to change that. but a default with no size limit is weird.....

dbeck@cloudbees.com (JIRA)

unread,

Feb 26, 2020, 6:38:05 AM2/26/20

to jenkinsc...@googlegroups.com

Daniel Beck commented on

Would go an order of magnitude up or so to give us breathing room. Given the UI of Jenkins, long form submissions are expected (and 4339 actually helps with that instead of causing the problem…).

I wonder whether there's a good location to document the system property and the need to set it in instances with very complex configurations.

o.v.nenashev@gmail.com (JIRA)

unread,

Feb 26, 2020, 8:36:04 AM2/26/20

to jenkinsc...@googlegroups.com

Oleg Nenashev updated an issue

Jenkins /

Change By:	Oleg Nenashev
Labels:	lts-candidate

o.v.nenashev@gmail.com (JIRA)

unread,

Feb 26, 2020, 8:38:06 AM2/26/20

to jenkinsc...@googlegroups.com

Oleg Nenashev commented on

Marked as an lts-candidate. Taking the number of reports, we would be interested to backport a fix to the next LTS .1 if we have one

brianodriscoll75@gmail.com (JIRA)

unread,

Feb 26, 2020, 11:09:05 AM2/26/20

to jenkinsc...@googlegroups.com

brian odriscoll commented on

Hi Oleg,

So we used the workaround advised in JENKINS-60409.
First our senior picked it up to 400k but it didn't fix the issue.- (still too small for the POST size we are doing), then picked it up to 1M and it began to work properly.
Kind Regards
Brian

raihaan.shouhell@autodesk.com (JIRA)

unread,

Feb 26, 2020, 10:45:03 PM2/26/20

to jenkinsc...@googlegroups.com

Raihaan Shouhell commented on

https://github.com/eclipse/jetty.project/blob/65a22e5e809ecbd20b3475cd8644fdf7aa5e8490/jetty-server/src/main/java/org/eclipse/jetty/server/handler/ContextHandler.java#L148

This looks like where the default size is set. Even if we do bump it, we would require that jetty releases and we then update winstone.

This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)

olamy@apache.org (JIRA)

unread,

Feb 26, 2020, 10:51:03 PM2/26/20

to jenkinsc...@googlegroups.com

Olivier Lamy commented on

Raihaan Shouhell why? we do not need a Jetty release. Jetty has his own default (a bit low to prevent DOS attack and this probably will not be changed in Jetty code).

But we can our own higher default here in Winstone.

raihaan.shouhell@autodesk.com (JIRA)

unread,

Feb 27, 2020, 4:46:03 AM2/27/20

to jenkinsc...@googlegroups.com

Raihaan Shouhell commented on

Olivier Lamy you're right. I did not know we could override that bit. Definitely should be in winstone