[JIRA] [core] (JENKINS-25046) Cookie header too long, causing a 413 HTTP error

13 views
Skip to first unread message

tangtheone@gmail.com (JIRA)

unread,
Jul 6, 2015, 5:40:01 AM7/6/15
to jenkinsc...@googlegroups.com
Pei-Tang Huang updated an issue
 
Jenkins / Bug JENKINS-25046
Cookie header too long, causing a 413 HTTP error

Same problem here, request information screenshots attached.

Change By: Pei-Tang Huang
Attachment: 413_on_requests.png
Attachment: full_of_session_cookie.png
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)
Atlassian logo

james.sharpe@zenotech.com (JIRA)

unread,
May 19, 2016, 12:13:09 PM5/19/16
to jenkinsc...@googlegroups.com
James Sharpe updated an issue
Change By: James Sharpe
Priority: Major Critical

james.sharpe@zenotech.com (JIRA)

unread,
May 19, 2016, 12:14:02 PM5/19/16
to jenkinsc...@googlegroups.com

james.sharpe@zenotech.com (JIRA)

unread,
May 19, 2016, 12:14:02 PM5/19/16
to jenkinsc...@googlegroups.com
James Sharpe updated an issue
Change By: James Sharpe
Labels: 2.0

o.v.nenashev@gmail.com (JIRA)

unread,
May 28, 2016, 3:20:02 AM5/28/16
to jenkinsc...@googlegroups.com
Oleg Nenashev commented on Bug JENKINS-25046
 
Re: Cookie header too long, causing a 413 HTTP error

Seems to be the issue, but I doubt it's 2.0-specific. CC Daniel Beck

dbeck@cloudbees.com (JIRA)

unread,
May 28, 2016, 5:02:02 AM5/28/16
to jenkinsc...@googlegroups.com
Daniel Beck updated an issue
 

Nothing critical about this.

It's both rare (only affecting those restarting Jenkins like crazy) and trivial to fix (remove the cookie).
Change By: Daniel Beck
Priority: Critical Minor

dbeck@cloudbees.com (JIRA)

unread,
May 28, 2016, 5:03:01 AM5/28/16
to jenkinsc...@googlegroups.com
Daniel Beck updated an issue
Change By: Daniel Beck
Labels: 2.0

marius@gedmin.as (JIRA)

unread,
Jun 1, 2016, 5:50:01 AM6/1/16
to jenkinsc...@googlegroups.com
Marius Gedminas commented on Bug JENKINS-25046
 
Re: Cookie header too long, causing a 413 HTTP error

I restart Jenkins once a week – when you release a new version to the apt repository, or when an OS security update requires a reboot. Does that count as "restarting like crazy"? Seems to me it's not as much rare as "inevitable, once enough time passes.

Diagnosing the problem is not trivial: currently, with 156 session cookies, I get a completely blank page when I attempt to install or upgrade some plugins (because cookie size + form data size hits some limit). The error is only visible if I open up the dev tools and have sufficient web developer-fu to investigate.

It would be nice if Jenkins could automatically clear stale cookies when there are too many of them (say, over 50 – I imagine if you have two Jenkins instances available at the same domain, you may not want them interfering with each other's sessions).

dbeck@cloudbees.com (JIRA)

unread,
Jul 19, 2016, 9:35:01 AM7/19/16
to jenkinsc...@googlegroups.com

Culprit (with explanation):
https://github.com/jenkinsci/extras-executable-war/blob/5a5f01362e7649da3b39a3a45940ba3ce1ab65e8/src/main/java/Main.java#L235

FWIW I think generating a hash based on JENKINS_HOME path, or maybe the instance ID would be sufficiently unique, and not get regenerated on every startup.
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
Atlassian logo

dbeck@cloudbees.com (JIRA)

unread,
Jul 19, 2016, 9:38:01 AM7/19/16
to jenkinsc...@googlegroups.com

The easiest solution is probably to add an argument that allows overriding this behavior. Unfortunately if we make this the default in installers, it'll break users who reverse-proxy multiple instances from the same proxy host.

dbeck@cloudbees.com (JIRA)

unread,
Jul 19, 2016, 9:46:02 AM7/19/16
to jenkinsc...@googlegroups.com

Unfortunately if we make this the default in installers

Seems the reasonable default in service scripts is to initialize it with a hash of hostname + actual port to be sufficiently unique. Not sure how to handle the Windows service, but that would at least take care of the Linuxes.

sorin.sbarnea@gmail.com (JIRA)

unread,
Jul 27, 2016, 11:34:03 AM7/27/16
to jenkinsc...@googlegroups.com

Considering that Jenkins is releasing a new version every ~5 days and that if you use 20-30 you will get daily updates, there is a high chance of you to reach this issue quite fast.

This is not "restarting jenkins like crazy" - is just trying to keep a Jenkins instance up-to-date! Quite a different approach.

This is a serious bug and cleaning up the cookies doesn't seem to help, I tried doing this on Chrome and they suddenly reappeared.

sorin.sbarnea@gmail.com (JIRA)

unread,
Aug 2, 2016, 12:00:01 PM8/2/16
to jenkinsc...@googlegroups.com

Can we fix this please? This bug is making Jenkins almost useless as the server is replying with 413 to some AJAX requests and the browsers do not display any error.

For example now we get some live console logs empty! The text-view returns the output but the live output fails because if this.

I don't want to reset cookies as I re-logging in to so many web sites is really painful as many of them do have 2FA!.

benoit.donneaux@gmail.com (JIRA)

unread,
Aug 25, 2016, 3:10:02 AM8/25/16
to jenkinsc...@googlegroups.com

Same here with the LTS (2.7.1) and I see a (Windows/JNLP) slave also cumulating those cookies:

Aug 25 08:24:48.116 localhost httpd[ssl]: INFO <master> <slave> - - TLSv1.2 ECDHE-RSA-DES-CBC3-SHA \"GET /jenkins/tcpSlaveAgentListener/ HTTP/1.1\" 200 12 \"-\" \"Java/1.8.0_101\" \"JSESSIONID.fff7b3dd=11l7hv2dfznc3184d4is3wbu10; JSESSIONID.968f4893=1uwkng7acyzl76hadp68n8yqq; JSESSIONID.ca951ea0=wvur34byo02hakkwrq7n88gg; JSESSIONID.738879c3=ahclxy0pqfeh1ptgpsld0kut; JSESSIONID.e4031fa7=1fdr8gl8ywl2i6qqpntj0m9vf; JSESSIONID.2b52b2d7=nyrhdvz48z28begkc4vgkoo5; JSESSIONID.9d172e35=902aok32vjqcceaj80ndqvv7; JSESSIONID.9a44300d=1xeqmrzml9zmk1esvdtzyawbln; JSESSIONID.943f4575=17jml3nitbi352urb499w998z; JSESSIONID.38029651=10ndnhjicmvav1ldit0phkpmc6; JSESSIONID.1fb8e86c=1cott3tydzw6qijta32q6f4wn; JSESSIONID.c187efae=1cx6anjacypkw15hkka0en0sas; JSESSIONID.21c6cb9e=1hp54um5nxa5zvp76qwebr4ck; JSESSIONID.e3579812=1gfr59v4cr9sigobm2sc6k9ma; JSESSIONID.0ff2be94=1947yt88jearx142kr33bxcz18; JSESSIONID.d5d6f77a=ie8vrevmwvksk7l9rj2fjdcx; JSESSIONID.d927eaf9=1b0z10l3uhus5q03fxgjfsx26; JSESSIONID.eb158e24=bl6yte8nzkxq1cxfqj94d1lug; JSESSIONID.df59a26e=k5a356tpfguk1jfy88okv27h8; JSESSIONID.bf86d1fa=htu0skwixvyx1dz8rury4zy5h; JSESSIONID.e00c23d9=66tbk20lgawe1vgr42kfzuvy6; JSESSIONID.eca7f3ed=1fn4vunf6lb5p1e04lht1yylte; JSESSIONID.eea69353=1sufj78yvlg2k1jil96s51aw6t; JSESSIONID.c69bdaf8=1lrxn49sph8t0qcxzc8jxtlje; JSESSIONID.a9f71cc4=150jt1zaycb5m7vut4s819mv7; JSESSIONID.730c5a61=1ocsh9yy95rj99775cju7o14y; JSESSIONID.a3c1e801=1nuvij8odw28jnhsqlxt4lmq4; JSESSIONID.c4b45b1d=131jfczqa71c81ggev69tz0fit; JSESSIONID.8023cc01=j2x42se0fj6u1j4m8e291ozzo; JSESSIONID.d3f64b53=1uurqkjgwcrbk1kfjqp7o7v6gp; JSESSIONID.e3aabb49=nepfr2sdwpwb2pxfim51ghi3; JSESSIONID.86f1a901=1n01r80jookai1dqvylcsrghxk; JSESSIONID.ba8f9d35=1umbvucj9k2a1vbu0yerqt8k7; JSESSIONID.12989142=fz65vrkhy2cvonbxbt758vf; JSESSIONID.c738909a=11kysf3hezdbpfxkf6i1rm635; JSESSIONID.ecfdce60=yd6ufbwaku8n13yaoxb5jb5e8; JSESSIONID.d9040e39=1ffmb9o2zym5z1vqiaz9vpv5ze; JSESSIONID.25dc9165=1wmqe68gwsolnhxiv525ltmux; JSESSIONID.1041b5cc=ktg3c081xmzsh040wx39x7ve; JSESSIONID.4b267dcd=q037pjax8vhce9jf648q5xpl; JSESSIONID.b3613560=m105j0hpr319ag95lestlohx; JSESSIONID.d6140975=10vbk07hf3hi6u11tjxdc9cuv\"

I suspect this to slow down Jenkins in general (clients, slave and master), as the overhead per request goes up and the master may have to process all those cookies to find out which one is valid, right?

benoit.donneaux@gmail.com (JIRA)

unread,
Aug 25, 2016, 3:18:03 AM8/25/16
to jenkinsc...@googlegroups.com
Benoit Donneaux updated an issue
 

Due to the possible performance aspect, I don't think this issue is so Minor: promoting to Major...
Change By: Benoit Donneaux
Priority: Minor Major

benoit.donneaux@gmail.com (JIRA)

unread,
Aug 25, 2016, 3:57:02 AM8/25/16
to jenkinsc...@googlegroups.com
 
Re: Cookie header too long, causing a 413 HTTP error

I'm probably wrong about the master processing all the cookies though.
But I'm pretty confident there is difference in term of responsiveness when we get rid of those cookies for every client.

REM: I our case, cookie is changing every night because of a backup job that stop Jenkins for a few seconds in order to get consistent data.

dbeck@cloudbees.com (JIRA)

unread,
Aug 25, 2016, 4:08:03 AM8/25/16
to jenkinsc...@googlegroups.com

Benoit Donneaux Sorin Sbarnea There's no point in repeating the same things others wrote before you. You're just making the comments that e.g. explain how to go about fixing this issue more difficult to find. Try adding something of value to the discussion instead.

dbeck@cloudbees.com (JIRA)

unread,
Aug 25, 2016, 5:23:06 AM8/25/16
to jenkinsc...@googlegroups.com
Daniel Beck edited a comment on Bug JENKINS-25046
[~bdonneaux] [~ssbarnea] There's no point in repeating the same things others wrote before you. You're just making the comments that e.g. explain how to go about fixing this issue more difficult to find. Try adding something of value original to the discussion instead.

benoit.donneaux@gmail.com (JIRA)

unread,
Aug 25, 2016, 11:07:05 AM8/25/16
to jenkinsc...@googlegroups.com

Reading the previous comments once again, I don't see anyone referring the impact on slaves and the performance.
Sorry if my attempt to give more feedback is annoying some. Not coding too much my self, I'm trying to support community in other ways.

Daniel Beckham, since you've already pointed at the cause and proposed a possible fix, do you expect us (the not very original users) to send a pull request with the solution you're proposing (hash port number)???

Meanwhile, I'd like to find workaround for our situation that does not require me patching and compiling Jenkins from source or restarting every browser and slave after each restart of the master...

Since Daniel Beckham wants something original, here is my modest contribution that works for us for Apache (with mod_rewrite and mod_header):

  1. Detect multiple session cookies and extract the oldest (assuming it's the first set in the Cookie string)
    RewriteCond "% {HTTP_COOKIE}

    " ".(JSESSIONID\.[^=])=.(JSESSIONID\.[^=])=[^;]+.*"

  2. Store the name of this session cookie in an environment variable
    RewriteRule /jenkins/ - [env=OLDJSESSIONID:%1]
  3. Expire this obsolete session cookie if found
    Header add Set-Cookie "% {OLDJSESSIONID}

    e=;Expires=Thu, 01 Jan 1970 00:00:01 GMT;Path=/jenkins;Secure;HttpOnly" env=OLDJSESSIONID

I've tested this with Firefox and Chromium and it looks ok for now...

But, in my opinion, randomizing the cookie name is a bad idea from the first place.
I'm in deeply in favor of a static name, based on the context_path and/or the port number by default, or a custom property.

benoit.donneaux@gmail.com (JIRA)

unread,
Aug 25, 2016, 11:09:06 AM8/25/16
to jenkinsc...@googlegroups.com
Benoit Donneaux edited a comment on Bug JENKINS-25046
Reading the previous comments once again, I don't see anyone referring the impact on slaves and the performance.
Sorry if my attempt to give more feedback is annoying some. Not coding too much my self, I'm trying to support community in other ways.

[~dbeckham], since you've already pointed at the cause and proposed a possible fix, do you expect us (the not very original users) to send a pull request with the solution you're proposing (hash port number)???


Meanwhile, I'd like to find workaround for our situation that does not require me patching and compiling Jenkins from source or restarting every browser and slave after each restart of the master...

Since [~dbeckham] wants something original, here is my modest contribution that works for us for Apache (with mod_rewrite and mod_header):

 {code:java}
         # Detect multiple session cookies and extract the oldest (assuming it's the first set in the Cookie string)
        RewriteCond "%{HTTP_COOKIE}" ".*(JSESSIONID\.[^=]+)=.*(JSESSIONID\.[^=]+)=[^;]+.*"
        # Store the name of this session cookie in an environment variable

        RewriteRule /jenkins/ - [env=OLDJSESSIONID:%1]
        # Expire this obsolete session cookie if found

        Header add Set-Cookie "%{OLDJSESSIONID}e=;Expires=Thu, 01 Jan 1970 00:00:01 GMT;Path=/jenkins;Secure;HttpOnly" env=OLDJSESSIONID
{code}


I've tested this with Firefox and Chromium and it looks ok for now...

But, in my opinion, randomizing the cookie name is a bad idea from the first place.
I'm in deeply in favor of a static name, based on the context_path and/or the port number by default, or a custom property.

benoit.donneaux@gmail.com (JIRA)

unread,
Aug 25, 2016, 11:10:02 AM8/25/16
to jenkinsc...@googlegroups.com

dbeck@cloudbees.com (JIRA)

unread,
Aug 25, 2016, 11:17:05 AM8/25/16
to jenkinsc...@googlegroups.com

Benoit Donneaux Please note that I am Daniel Beck, not Daniel Beckham.

since you've already pointed at the cause and proposed a possible fix, do you expect us … to send a pull request with the solution you're proposing (hash port number)???

No, I recorded my earlier investigation into this issue. While I'd really like to fix this, it doesn't rank highest on my (long and growing) to-do list. Meanwhile, anyone is free to tackle this, and my comments may even help them.

But, in my opinion, randomizing the cookie name is a bad idea from the first place.

Right – the issue here is multiple Jenkinses on the same host, possibly behind a reverse proxy. Then, you may have same Jenkins home directory (different machine), same port (different host / different context), same context (different port), same (front-end) host name, depending at what stage you check. And suddenly, they'd share cookies.

So while I'd like to keep some uniqueness to the cookie name, ideally it would be in a way that reuses the same key over multiple restarts, while still being unique per instance.

the not very original users

I apologize for the choice of words. It wasn't meant disparaging.

benoit.donneaux@gmail.com (JIRA)

unread,
Aug 25, 2016, 11:30:01 AM8/25/16
to jenkinsc...@googlegroups.com
Benoit Donneaux edited a comment on Bug JENKINS-25046
Reading the previous comments once again, I don't see anyone referring the impact on slaves and the performance.
Sorry if my attempt to give more feedback is annoying some. Not coding too much my self, I'm trying to support community in other ways.

[~ dbeckham danielbeck ], since you've already pointed at the cause and proposed a possible fix, do you expect us (the not very original users) to send a pull request with the solution you're proposing (hash port number)???


Meanwhile, I'd like to find workaround for our situation that does not require me patching and compiling Jenkins from source or restarting every browser and slave after each restart of the master...

Since [~ dbeckham danielbeck ] wants something original, here is my modest contribution that works for us for Apache (with mod_rewrite and mod_header):


{code:java}
# Detect multiple session cookies and extract the oldest (assuming it's the first set in the Cookie string)
RewriteCond "%{HTTP_COOKIE}" ".*(JSESSIONID\.[^=]+)=.*(JSESSIONID\.[^=]+)=[^;]+.*"
# Store the name of this session cookie in an environment variable
RewriteRule /jenkins/ - [env=OLDJSESSIONID:%1]
# Expire this obsolete session cookie if found
Header add Set-Cookie "%{OLDJSESSIONID}e=;Expires=Thu, 01 Jan 1970 00:00:01 GMT;Path=/jenkins;Secure;HttpOnly" env=OLDJSESSIONID
{code}

I've tested this with Firefox and Chromium and it looks ok for now...

But, in my opinion, randomizing the cookie name is a bad idea from the first place.
I'm in deeply in favor of a static name, based on the context_path and/or the port number by default, or a custom property.

marc.esher@gmail.com (JIRA)

unread,
Nov 4, 2016, 2:40:02 PM11/4/16
to jenkinsc...@googlegroups.com

Definitely a problem for us. We run jenkins behind nginx, and for various reasons, we're required to have a fairly strict setting on cookie size, which means this problem hits our users every week or two. It's incredibly annoying.

I'd be happy to submit a PR for the behavior requested above if the Jenkins folks think it'd be accepted

dbeck@cloudbees.com (JIRA)

unread,
Nov 4, 2016, 7:58:03 PM11/4/16
to jenkinsc...@googlegroups.com

I'd be happy to submit a PR for the behavior requested above if the Jenkins folks think it'd be accepted

If by that you're referring to my comments from Jul 19, at least this "Jenkins folk" thinks it's a reasonable approach No guarantees it'll pass the review gauntlet though.

yixiaol@medallia.com (JIRA)

unread,
Jan 17, 2019, 6:03:03 PM1/17/19
to jenkinsc...@googlegroups.com

Bump this issue.

We use Jenkins enterprise and use the operation center to manage the masters.

With more masters, we run into this issue quite frequently. 
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

bmathus+ossjira@cloudbees.com (JIRA)

unread,
Jul 3, 2019, 11:08:07 AM7/3/19
to jenkinsc...@googlegroups.com
Baptiste Mathus assigned an issue to Josh Soref
 
Change By: Baptiste Mathus
Assignee: Josh Soref

bmathus+ossjira@cloudbees.com (JIRA)

unread,
Jul 3, 2019, 11:08:10 AM7/3/19
to jenkinsc...@googlegroups.com
Baptiste Mathus started work on Bug JENKINS-25046
 
Change By: Baptiste Mathus
Status: Open In Progress

bmathus+ossjira@cloudbees.com (JIRA)

unread,
Jul 3, 2019, 11:08:13 AM7/3/19
to jenkinsc...@googlegroups.com
Change By: Baptiste Mathus
Status: In Progress Review

o.v.nenashev@gmail.com (JIRA)

unread,
Jul 8, 2019, 8:48:03 AM7/8/19
to jenkinsc...@googlegroups.com
Oleg Nenashev updated an issue
Change By: Oleg Nenashev
Labels: lts-candidate

o.v.nenashev@gmail.com (JIRA)

unread,
Jul 8, 2019, 8:49:06 AM7/8/19
to jenkinsc...@googlegroups.com
Oleg Nenashev updated Bug JENKINS-25046
 

https://github.com/jenkinsci/jenkins/pull/3939 was integrated and released in 2.184. I marked it as the LTS candidate, but I am not sure it will be considered for backporting by Oliver Gondža talking the rebase issues in the pull request. Whomever is interested, please feel free to suggest a clean pull request against the LTS branch

 
Change By: Oleg Nenashev
Status: In Review Resolved
Resolution: Fixed
Released As: Jenkins 2.184

ogondza@gmail.com (JIRA)

unread,
Aug 1, 2019, 7:03:04 AM8/1/19
to jenkinsc...@googlegroups.com
Oliver Gondža updated an issue
Change By: Oliver Gondža
Labels: 2.176.3-rejected lts-candidate

dbeck@cloudbees.com (JIRA)

unread,
Sep 5, 2019, 5:54:18 AM9/5/19
to jenkinsc...@googlegroups.com
Daniel Beck updated an issue
Change By: Daniel Beck
Labels: 2.176.3-rejected lts-candidate
Reply all
Reply to author
Forward
0 new messages