[JIRA] (JENKINS-60409) Form Submission Length Cap

0 views
Skip to first unread message

dbeck@cloudbees.com (JIRA)

unread,
Feb 27, 2020, 10:34:03 PM2/27/20
to jenkinsc...@googlegroups.com
Daniel Beck updated an issue
 
Jenkins / Bug JENKINS-60409
Form Submission Length Cap
Change By: Daniel Beck
Summary: Cloud Configuration Form Submission Length Cap
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)
Atlassian logo

dbeck@cloudbees.com (JIRA)

unread,
Feb 29, 2020, 1:38:03 PM2/29/20
to jenkinsc...@googlegroups.com
Daniel Beck updated an issue
Change By: Daniel Beck
Labels: lts-candidate regression

jglick@cloudbees.com (JIRA)

unread,
Mar 4, 2020, 7:07:03 PM3/4/20
to jenkinsc...@googlegroups.com
Jesse Glick commented on Bug JENKINS-60409
 
Re: Form Submission Length Cap

So Winstone does attempt to set this parameter to be unlimited; see JENKINS-20327 and winstone #20. Perhaps something about jetty.project #3899 broke this hack, though it is not obvious what: the same key is being used, and in the same way from what I can tell.

(Note that maven-hpi-plugin, jenkins-test-harness, and jenkinsfile-runner all attempt to set this too but with apparently obsolete property names. The comment in RunMojo is obsolete as well: we no longer accept client-side update center JSON.)

jglick@cloudbees.com (JIRA)

unread,
Mar 4, 2020, 7:29:06 PM3/4/20
to jenkinsc...@googlegroups.com
Jesse Glick started work on Bug JENKINS-60409
 
Change By: Jesse Glick
Status: Open In Progress

jglick@cloudbees.com (JIRA)

unread,
Mar 4, 2020, 7:29:06 PM3/4/20
to jenkinsc...@googlegroups.com
Jesse Glick assigned an issue to Jesse Glick
Change By: Jesse Glick
Assignee: Jesse Glick

jglick@cloudbees.com (JIRA)

unread,
Mar 4, 2020, 8:18:02 PM3/4/20
to jenkinsc...@googlegroups.com

Working on a fix in winstone #95.

jglick@cloudbees.com (JIRA)

unread,
Mar 4, 2020, 9:05:05 PM3/4/20
to jenkinsc...@googlegroups.com
Change By: Jesse Glick
Status: In Progress Review

olamy@apache.org (JIRA)

unread,
Mar 5, 2020, 2:39:04 AM3/5/20
to jenkinsc...@googlegroups.com
Change By: Olivier Lamy
Status: In Review Resolved
Resolution: Fixed

alexhraber@gmail.com (JIRA)

unread,
Mar 5, 2020, 7:21:04 AM3/5/20
to jenkinsc...@googlegroups.com

Jesse Glick fyi this is happening in 2.204.4 LTS as well, can we get a patch bump to 5 with your winstone fix please.

alexhraber@gmail.com (JIRA)

unread,
Mar 5, 2020, 7:21:09 AM3/5/20
to jenkinsc...@googlegroups.com
Alex Raber edited a comment on Bug JENKINS-60409
[~jglick] fyi this is happening in 2.204.4 LTS as well, can we get a patch bump to 2.204. 5 with your winstone fix please.

o.v.nenashev@gmail.com (JIRA)

unread,
Mar 5, 2020, 8:19:07 AM3/5/20
to jenkinsc...@googlegroups.com

Winstone 5.9 with fixes was released: https://github.com/jenkinsci/winstone/releases/tag/winstone-5.9

Jenkins Core update pull request: https://github.com/jenkinsci/jenkins/pull/4542

o.v.nenashev@gmail.com (JIRA)

unread,
Mar 5, 2020, 8:19:09 AM3/5/20
to jenkinsc...@googlegroups.com
Oleg Nenashev reopened an issue
Change By: Oleg Nenashev
Resolution: Fixed
Status: Resolved Reopened

o.v.nenashev@gmail.com (JIRA)

unread,
Mar 5, 2020, 8:43:04 AM3/5/20
to jenkinsc...@googlegroups.com
Oleg Nenashev updated an issue
Change By: Oleg Nenashev
Environment: Jenkins 2.205+, Jenkins LTS 2.204.3 and 2.204.4
Winstone 5.5 to 5.8

o.v.nenashev@gmail.com (JIRA)

unread,
Mar 5, 2020, 8:47:08 AM3/5/20
to jenkinsc...@googlegroups.com
Oleg Nenashev updated an issue
**Jenkins LTS Notice**: Jenkins LTS 2.204.3 and 2.204.4 are also affected due to the Winstone upgrade which was backported as a part of the JENKINS-57888 fix backporting. Please see [https://groups.google.com/forum/#!topic/jenkinsci-dev/M_RtDuDXtbU] for the discussion and retrospective

 In Jenkins Version 2.205, PR #4339 moved the cloud configuration from Configure System into is own configuration form on the Manage Nodes page. There is a cap to the length of this form (200000) and prevents me from adding additional docker clouds into the settings.
{code:java}
java.lang.IllegalStateException: Form is larger than max length 200000
at org.eclipse.jetty.server.Request.extractFormParameters(Request.java:562)
at org.eclipse.jetty.server.Request.extractContentParameters(Request.java:519)
at org.eclipse.jetty.server.Request.getParameters(Request.java:430)
Caused: org.eclipse.jetty.http.BadMessageException: 400: Unable to parse form content
at org.eclipse.jetty.server.Request.getParameters(Request.java:434)
at org.eclipse.jetty.server.Request.getParameterNames(Request.java:1077)
at hudson.security.csrf.CrumbFilter.extractCrumbFromRequest(CrumbFilter.java:112)
at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:81)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:512)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1592)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1296)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1562)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1211)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:500)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:386)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:562)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:378)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:270)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
at java.lang.Thread.run(Thread.java:748)
{code}

o.v.nenashev@gmail.com (JIRA)

unread,
Mar 5, 2020, 8:50:10 AM3/5/20
to jenkinsc...@googlegroups.com
Oleg Nenashev updated an issue
**Jenkins LTS Notice**: Jenkins LTS 2.204.3 and 2.204.4 are also affected due to the Winstone upgrade which was backported introduced as a part of the JENKINS-57888 fix backporting. Please see [https://groups.google.com/forum/#!topic/jenkinsci-dev/M_RtDuDXtbU] for the discussion and retrospective

jglick@cloudbees.com (JIRA)

unread,
Mar 5, 2020, 3:38:08 PM3/5/20
to jenkinsc...@googlegroups.com
Change By: Jesse Glick
Status: Reopened Open

jglick@cloudbees.com (JIRA)

unread,
Mar 5, 2020, 3:38:11 PM3/5/20
to jenkinsc...@googlegroups.com
Change By: Jesse Glick
Status: In Progress Review

jglick@cloudbees.com (JIRA)

unread,
Mar 5, 2020, 3:38:13 PM3/5/20
to jenkinsc...@googlegroups.com
Jesse Glick started work on Bug JENKINS-60409
 
Change By: Jesse Glick
Status: Open In Progress

o.v.nenashev@gmail.com (JIRA)

unread,
Mar 5, 2020, 9:27:03 PM3/5/20
to jenkinsc...@googlegroups.com
Oleg Nenashev updated an issue

I have just released an alternate Winstone 5.4.1 release for 2.204.x LTS. This patch reverts Jetty to older versions https://github.com/jenkinsci/winstone/releases/tag/winstone-5.4.1 but keeps other regression fixes. It should be more stable than upgrade to Winstone 5.9 with just another Jetty upgrade and a risk of new regressions. 

Pull request with the 2.204.x baseline update: https://github.com/jenkinsci/jenkins/pull/4545
Change By: Oleg Nenashev
Released As: Winstone 5.4.1, Winstone 5.9

o.v.nenashev@gmail.com (JIRA)

unread,
Mar 5, 2020, 9:47:07 PM3/5/20
to jenkinsc...@googlegroups.com
Oleg Nenashev updated an issue
Change By: Oleg Nenashev
Priority: Major Critical

dbeck@cloudbees.com (JIRA)

unread,
Mar 5, 2020, 9:53:05 PM3/5/20
to jenkinsc...@googlegroups.com
Daniel Beck updated an issue
Change By: Daniel Beck
Labels: lts-candidate lts-candidate-for-2.204-and-2.222 regression

i+jenkins@always.fail (JIRA)

unread,
Mar 6, 2020, 10:15:03 AM3/6/20
to jenkinsc...@googlegroups.com

Will this fix be included in 2.224?

o.v.nenashev@gmail.com (JIRA)

unread,
Mar 6, 2020, 10:59:05 AM3/6/20
to jenkinsc...@googlegroups.com

I believe so. https://github.com/jenkinsci/jenkins/pull/4542 is waiting for the 24hrs merge timeout which ends in 1 hour or so. Taking the approvals, I am pretty confident that the next weekly release will include the fix. LTS is a separate story, I am waiting for responses from Oliver Gondža and Kohsuke Kawaguchi about out-of-order releases

o.v.nenashev@gmail.com (JIRA)

unread,
Mar 6, 2020, 10:59:08 AM3/6/20
to jenkinsc...@googlegroups.com
Oleg Nenashev edited a comment on Bug JENKINS-60409
I believe so. [https://github.com/jenkinsci/jenkins/pull/4542] is waiting for the 24hrs merge timeout which ends in 1 hour or so. Taking the approvals, I am pretty confident that the next weekly release will include the fix. LTS is a separate story, I am waiting for responses from [~olivergondza] and [~ci_jenkinsci_org] about out-of-order releases 2.204.5 LTS 

bmathus+ossjira@cloudbees.com (JIRA)

unread,
Mar 6, 2020, 2:23:03 PM3/6/20
to jenkinsc...@googlegroups.com

Curious, did someone ever test

-Dorg.eclipse.jetty.server.Request.maxFormContentSize=-1

to remove any limit instead of just bumping it higher?

bmathus+ossjira@cloudbees.com (JIRA)

unread,
Mar 6, 2020, 2:33:03 PM3/6/20
to jenkinsc...@googlegroups.com
Baptiste Mathus edited a comment on Bug JENKINS-60409
Curious, did someone ever test
{code:java}
-Dorg.eclipse.jetty.server.Request.maxFormContentSize=-1{code}

to remove any limit instead of just bumping it higher?


 

{color:#de350b}*UPDATE: just tested it. This works. I think this should be strong recommendation. FWIW, this is what Jenkins normally does internally.*{color}

bmathus+ossjira@cloudbees.com (JIRA)

unread,
Mar 6, 2020, 2:34:02 PM3/6/20
to jenkinsc...@googlegroups.com
Baptiste Mathus edited a comment on Bug JENKINS-60409
Curious, did someone ever test
{code:java}
-Dorg.eclipse.jetty.server.Request.maxFormContentSize=-1{code}
to remove any limit instead of just bumping it higher?

 

{color:#de350b}*UPDATE: just tested it. This works. I think this should be strong the recommendation instead of any high number . FWIW, this is what Jenkins normally does internally.*{color}

jglick@cloudbees.com (JIRA)

unread,
Mar 6, 2020, 2:35:03 PM3/6/20
to jenkinsc...@googlegroups.com

Minimal test in context to reproduce: run Jenkins with Winstone 5.8 on a fresh user dir. Go through setup wizard, installing no plugins. Create an API token for admin. Then run

x=1; while :; do echo trying $x; (echo description=; seq -s. $x | tr -d '[:digit:]') > /tmp/$x-dots.txt; curl -f -u admin:YOURTOKEN -d @/tmp/$x-dots.txt http://localhost:8080/submitDescription || break; x=$((x * 3)); done

You should see it fail after 200000:

trying 1
trying 3
trying 9
trying 27
trying 81
trying 243
trying 729
trying 2187
trying 6561
trying 19683
trying 59049
trying 177147
trying 531441
curl: (22) The requested URL returned error: 500 Server Error

If you now start Jenkins with -Dorg.eclipse.jetty.server.Request.maxFormContentSize=-1, or with Winstone 5.9, it keeps on going.

ByteEnable@protonmail.com (JIRA)

unread,
Mar 6, 2020, 7:14:04 PM3/6/20
to jenkinsc...@googlegroups.com

I am experiencing this issue as well.  Source file is 200K in size.  I whittled it down to 177K and still experience the error.  Is there a work-around?

jglick@cloudbees.com (JIRA)

unread,
Mar 6, 2020, 8:41:06 PM3/6/20
to jenkinsc...@googlegroups.com
Change By: Jesse Glick
Status: In Review Fixed but Unreleased
Resolution: Fixed

dbeck@cloudbees.com (JIRA)

unread,
Mar 8, 2020, 2:13:04 AM3/8/20
to jenkinsc...@googlegroups.com
Daniel Beck updated an issue
Change By: Daniel Beck
Labels: 2.204.5-fixed lts-candidate lts-candidate-for-2.204-and-2.222 regression

o.v.nenashev@gmail.com (JIRA)

unread,
Mar 8, 2020, 12:12:06 PM3/8/20
to jenkinsc...@googlegroups.com

 

o.v.nenashev@gmail.com (JIRA)

unread,
Mar 8, 2020, 12:13:03 PM3/8/20
to jenkinsc...@googlegroups.com
Oleg Nenashev updated an issue
Change By: Oleg Nenashev
Released As: Jenkins 2.204.5, Jenkins 2.224, Winstone 5.4. 1 3 , Winstone 5.9

o.v.nenashev@gmail.com (JIRA)

unread,
Mar 9, 2020, 12:46:05 PM3/9/20
to jenkinsc...@googlegroups.com
Change By: Oleg Nenashev
Status: Fixed but Unreleased Resolved

ogondza@gmail.com (JIRA)

unread,
Mar 10, 2020, 1:47:04 PM3/10/20
to jenkinsc...@googlegroups.com
Oliver Gondža updated an issue
Change By: Oliver Gondža
Labels: 2.204.5-fixed lts-candidate lts-candidate-for- 2. 204-and-2. 222 .1-fixed regression

jonesbusy@gmail.com (JIRA)

unread,
Mar 24, 2020, 11:15:03 AM3/24/20
to jenkinsc...@googlegroups.com

Hi,

I have the same issue on 2.222 (non-LTS) with websocket connection payload. I know the feature is still in beta, but I guess the 2.221. should fix the websocket layer as well.

2020-03-24 09:47:21.616+0000 [id=219730] INFO j.s.DefaultJnlpSlaveReceiver#channelClosed: Jetty (winstone)-219730 for ************* terminated: java.nio.channels.ClosedChannelException
2020-03-24 09:47:27.142+0000 [id=219727] WARNING j.agents.WebSocketAgents$Session#error
org.eclipse.jetty.websocket.api.MessageTooLargeException: Binary message size [69632] exceeds maximum size [65536]
at org.eclipse.jetty.websocket.api.WebSocketPolicy.assertValidBinaryMessageSize(WebSocketPolicy.java:128)
at org.eclipse.jetty.websocket.common.message.SimpleBinaryMessage.appendFrame(SimpleBinaryMessage.java:57)
at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:61)
at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.onContinuationFrame(AbstractEventDriver.java:183)
at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onContinuationFrame(JettyListenerEventDriver.java:255)
at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:155)
at org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:322)
at org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:202)
at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:225)
at org.eclipse.jetty.websocket.common.Parser.parseSingleFrame(Parser.java:259)
at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:460)
at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:441)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
at java.lang.Thread.run(Thread.java:748)

Will try again when the 222.1 is released

Thanks

dbeck@cloudbees.com (JIRA)

unread,
Mar 24, 2020, 11:41:03 AM3/24/20
to jenkinsc...@googlegroups.com

Valentin Delaye

same issue

It's not. Note how the error message is completely different. You're looking for JENKINS-61409.

jonesbusy@gmail.com (JIRA)

unread,
Mar 24, 2020, 12:06:03 PM3/24/20
to jenkinsc...@googlegroups.com

Jesse Glick Ok thanks
Reply all
Reply to author
Forward
0 new messages