[JIRA] [core] (JENKINS-12875) "No valid crumb was included in the request" errors all around

2,208 views
Skip to first unread message

p.giarrusso@gmail.com (JIRA)

unread,
Aug 4, 2015, 9:08:04 AM8/4/15
to jenkinsc...@googlegroups.com
Paolo Giarrusso commented on Bug JENKINS-12875
 
Re: "No valid crumb was included in the request" errors all around

I ran into this yesterday. What's the status? Somebody proposed an X-Jenkins-Crumb header (in

JENKINS-7518 ), which doesn't seem a legal JavaScript identifier (though it might be "the start of a legal JavaScript statement").
Would X-Jenkins-Crumb-0NoJS help, since `0NoJs` is an illegal token?

Also, SECURITY-47 isn't visible in the bug tracker, apparently on purpose — I've only found the fix (https://github.com/jenkinsci/jenkins/commit/0de3e9b14ed75f70279435e78eb9f6a3a1a179df) and maybe the advisory (https://wiki.jenkins-ci.org/display/SECURITY/Jenkins+Security+Advisory+2013-02-16).

For my part, I tried to document this on https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+behind+an+NGinX+reverse+proxy; I only mentioned the most secure workaround, not the -Dhudson.security.csrf.requestfield= setting. Improvements are welcome.
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)
Atlassian logo

p.giarrusso@gmail.com (JIRA)

unread,
Aug 4, 2015, 9:09:08 AM8/4/15
to jenkinsc...@googlegroups.com
Paolo Giarrusso edited a comment on Bug JENKINS-12875
I ran into this yesterday. What's the status? Somebody proposed an  {{  X-Jenkins-Crumb }}  header (in JENKINS-7518), which doesn't seem a legal JavaScript identifier (though it might be "the start of a legal JavaScript statement").
Would  {{  X-Jenkins-Crumb-0NoJS }}  help, since  `  {{ 0NoJs ` }}  is an illegal  JS  token?


Also, SECURITY-47 isn't visible in the bug tracker, apparently on purpose — I've only found the fix (https://github.com/jenkinsci/jenkins/commit/0de3e9b14ed75f70279435e78eb9f6a3a1a179df) and maybe the advisory (https://wiki.jenkins-ci.org/display/SECURITY/Jenkins+Security+Advisory+2013-02-16).

For my part, I tried to document this on https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+behind+an+NGinX+reverse+proxy; I only mentioned the most secure workaround, not the {{-Dhudson.security.csrf.requestfield=}} setting. Improvements are welcome.

niftiness@gmail.com (JIRA)

unread,
Sep 2, 2015, 3:22:02 AM9/2/15
to jenkinsc...@googlegroups.com

Running into this too, with NginX, on https://timboudreau.com/builds

I've set ignore_invalid_headers to off, but I'm not keen on leaving it that way, especially for an NginX instance that is proxying far more than just Jenkins - there are other forms of nastiness that would be better not to pass on to back end applications that makes possible.

tomreyn@megaglest.org (JIRA)

unread,
Sep 3, 2015, 10:30:02 AM9/3/15
to jenkinsc...@googlegroups.com

The number of people watching this issue is pretty high compared to that of other issues. This may provide an indication as to how much of an issue it is.

C.f. http://seclists.org/bugtraq/2015/Aug/161

Adding a couple HTTP headers would also help to harden the security of a Jenkins installation (may break some Jenkins plugins - works for me). Assuming your Domain name is example.org and you use HTTPS:

Content-Security-Policy: "default-src 'self' EXAMPLE.ORG *.EXAMPLE.ORG; script-src 'self' 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; img-src *; media-src *; object-src 'self'; frame-src 'self' updates.jenkins-ci.org;"
X-Frame-Options: "sameorigin, DENY"
X-XSS-Protection: "1; mode=block"
X-Content-Type-Options:"nosniff"
Strict-Transport-Security:"max-age=15768000"

tomreyn@megaglest.org (JIRA)

unread,
Sep 3, 2015, 10:31:02 AM9/3/15
to jenkinsc...@googlegroups.com
Tom Reynolds edited a comment on Bug JENKINS-12875
The number of people watching this issue is pretty high compared to that of other issues. This may provide an indication as to how much of an issue it is.

C.f. http://seclists.org/bugtraq/2015/Aug/161

Adding a couple HTTP headers would also help to harden the security of a Jenkins installation (may break some Jenkins plugins - works for me). Assuming your  Domain  domain  name is  example  EXAMPLE . org ORG  and you use HTTPS:

{noformat}

Content-Security-Policy: "default-src 'self' EXAMPLE.ORG *.EXAMPLE.ORG; script-src 'self' 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; img-src *; media-src *; object-src 'self'; frame-src 'self' updates.jenkins-ci.org;"
X-Frame-Options: "sameorigin, DENY"
X-XSS-Protection: "1; mode=block"
X-Content-Type-Options:"nosniff"
Strict-Transport-Security:"max-age=15768000"
{noformat}

niftiness@gmail.com (JIRA)

unread,
Sep 3, 2015, 12:58:03 PM9/3/15
to jenkinsc...@googlegroups.com

> as crumb is a valid Javascript identifier that would open you to attack

Would like to understand better how a header name being a Javascript identifier opens one to attack - there are plenty of standard HTTP header names that are valid identifiers - if I make an HTTP request containing

Date: eval("alert('Hey')")

is something foolish enough to execute it?

At any rate, the problem here is simple - ".crumb" is not a legal HTTP header, but there are plenty of options that are legal HTTP headers and illegal as js identifiers - one has already been suggested. I don't think any of us want to leave our NginX installs set to accept whatever garbage a client decides to turn into headers.

Re the suggested additional headers: Some are not a bad idea, but I could write a NodeJS proxy that would strip those out in about 10 minutes and go hog wild (or just use a programmatic HTTP client) - you can't rely on that sort of thing to give you real security, just deter low-hanging-fruit bad behavior.

jglick@cloudbees.com (JIRA)

unread,
Sep 4, 2015, 5:01:06 PM9/4/15
to jenkinsc...@googlegroups.com

Would like to understand better how a header name being a Javascript identifier opens one to attack

It is complicated but has to do with how the crumb is offered to legitimate clients.

Anyway the fix is well understood and filed, “just” needs to be tested and reviewed by someone qualified in web security.

dbeck@cloudbees.com (JIRA)

unread,
Sep 4, 2015, 6:56:03 PM9/4/15
to jenkinsc...@googlegroups.com

Tim Boudreau You wrote:

".crumb" is not a legal HTTP header, but there are plenty of options that are legal HTTP headers and illegal as js identifiers

Just curious, where did you get this information from? Neither RFC 2822 (section 2.6.8) nor RFC 7230 seem to impose any header name restrictions beyond being printable ascii and not including spaces or colons. (Not saying this shouldn't be fixed due to compatibility, but I looked repeatedly, even back when I worked on this, and couldn't find anything like what you're saying).

p.giarrusso@gmail.com (JIRA)

unread,
Sep 4, 2015, 7:26:01 PM9/4/15
to jenkinsc...@googlegroups.com

On header names: I mostly agree with Daniel Beck about the spec (after reading both specs, I've also looked at http://stackoverflow.com/a/3569667/53974 — beyond colons, other "separators" are forbidden, but not dots). I think our source is just what the nginx docs imply without giving any reference (http://nginx.org/en/docs/http/ngx_http_core_module.html#ignore_invalid_headers).

> Re the suggested additional headers: Some are not a bad idea, but I could write a NodeJS proxy that would strip those out in about 10 minutes and go hog wild (or just use a programmatic HTTP client) - you can't rely on that sort of thing to give you real security, just deter low-hanging-fruit bad behavior.

At least Strict-Transport-Security is there to offer some protection for clients (browsers) that do honor them (and I think some other ones too). Not sure how you propose to use the proxy, but that headers assumes Jenkins is behind HTTPS, and then attackers can't drop the header.

However, please don't silently enable Strict-Transport-Security — that's something for the sysadmin to decide, and it's not fun if one HTTP application enables it on a host where other services don't support HTTPS.

Anyway, this is OT here, maybe somebody (Tom Reynolds) can open a different bug for them? (I don't understand the headers well enough to do that).

jglick@cloudbees.com (JIRA)

unread,
Mar 16, 2016, 11:36:18 AM3/16/16
to jenkinsc...@googlegroups.com
Jesse Glick updated an issue
 
Jenkins / Bug JENKINS-12875
Change By: Jesse Glick
Labels: 2.0

jglick@cloudbees.com (JIRA)

unread,
Mar 16, 2016, 11:36:18 AM3/16/16
to jenkinsc...@googlegroups.com
Jesse Glick updated an issue
Change By: Jesse Glick
Labels: 2.0  crumb

dbeck@cloudbees.com (JIRA)

unread,
Mar 27, 2016, 5:11:09 PM3/27/16
to jenkinsc...@googlegroups.com
Daniel Beck updated an issue
Change By: Daniel Beck
Labels: 2.0  2.0-planned  crumb

dbeck@cloudbees.com (JIRA)

unread,
Mar 27, 2016, 5:11:12 PM3/27/16
to jenkinsc...@googlegroups.com
Daniel Beck updated an issue
Change By: Daniel Beck
Priority: Major Critical

dbeck@cloudbees.com (JIRA)

unread,
Mar 27, 2016, 9:39:03 PM3/27/16
to jenkinsc...@googlegroups.com
Daniel Beck commented on Bug JENKINS-12875
 
Re: "No valid crumb was included in the request" errors all around

We already got some enthusiastic feedback how broken Jenkins is with nginx. Looks like we actually should address this before shipping with CSRF protection enabled by default.

jglick@cloudbees.com (JIRA)

unread,
Mar 28, 2016, 4:51:05 PM3/28/16
to jenkinsc...@googlegroups.com
Jesse Glick assigned an issue to Daniel Beck
 

Assigning to Daniel Beck on the basis that he filed PR 1319 which possibly just needs to have some merge conflicts resolved and be retested, but feel free to reassign if you think someone else should pick it up.
Change By: Jesse Glick
Assignee: Daniel Beck

jglick@cloudbees.com (JIRA)

unread,
Mar 28, 2016, 4:51:12 PM3/28/16
to jenkinsc...@googlegroups.com
Jesse Glick started work on Bug JENKINS-12875
 
Change By: Jesse Glick
Status: Open In Progress

scm_issue_link@java.net (JIRA)

unread,
Apr 1, 2016, 7:58:06 PM4/1/16
to jenkinsc...@googlegroups.com
SCM/JIRA link daemon commented on Bug JENKINS-12875
 
Re: "No valid crumb was included in the request" errors all around

Code changed in jenkins
User: Daniel Beck
Path:
core/src/main/java/hudson/security/csrf/CrumbFilter.java
core/src/main/java/hudson/security/csrf/CrumbIssuer.java
core/src/main/java/hudson/security/csrf/CrumbIssuerDescriptor.java
core/src/main/java/hudson/security/csrf/DefaultCrumbIssuer.java
test/src/test/groovy/hudson/model/SlaveTest.groovy
http://jenkins-ci.org/commit/jenkins/ed0ea631cd4af0e678da96d7297910ba1612749d
Log:
[FIXED JENKINS-12875] Change default crumb name to Jenkins-Crumb

scm_issue_link@java.net (JIRA)

unread,
Apr 1, 2016, 7:58:09 PM4/1/16
to jenkinsc...@googlegroups.com
SCM/JIRA link daemon resolved as Fixed
 
Change By: SCM/JIRA link daemon
Status: In Progress Resolved
Resolution: Fixed

dbeck@cloudbees.com (JIRA)

unread,
Apr 1, 2016, 7:58:13 PM4/1/16
to jenkinsc...@googlegroups.com
Daniel Beck commented on Bug JENKINS-12875
 
Re: "No valid crumb was included in the request" errors all around

Fixed towards Jenkins 2.0 which is scheduled to be released in April.

scm_issue_link@java.net (JIRA)

unread,
Apr 1, 2016, 7:58:14 PM4/1/16
to jenkinsc...@googlegroups.com

Code changed in jenkins
User: Daniel Beck
Path:
test/src/test/java/lib/form/RowSetTest.java
http://jenkins-ci.org/commit/jenkins/5b1642252ebd977c21832eb12cae633b3616429f
Log:

JENKINS-12875 Fix test

scm_issue_link@java.net (JIRA)

unread,
Apr 1, 2016, 7:58:19 PM4/1/16
to jenkinsc...@googlegroups.com

Code changed in jenkins
User: Daniel Beck
Path:

core/src/main/java/hudson/security/csrf/CrumbIssuer.java
http://jenkins-ci.org/commit/jenkins/8b6d38de67154fb96c59791082d5cbe743b2b693
Log:

JENKINS-12875 Restrict access to constant

scm_issue_link@java.net (JIRA)

unread,
Apr 1, 2016, 7:58:19 PM4/1/16
to jenkinsc...@googlegroups.com

Code changed in jenkins
User: Daniel Beck
Path:

core/src/main/java/hudson/security/csrf/CrumbFilter.java
core/src/main/java/hudson/security/csrf/CrumbIssuer.java
core/src/main/java/hudson/security/csrf/CrumbIssuerDescriptor.java
core/src/main/java/hudson/security/csrf/DefaultCrumbIssuer.java
test/src/test/groovy/hudson/model/SlaveTest.groovy

test/src/test/java/lib/form/RowSetTest.java
http://jenkins-ci.org/commit/jenkins/a87c38c5052cba745127a0205c40490b30c8bdbf
Log:
Merge pull request #2198 from daniel-beck/

JENKINS-12875

[FIX JENKINS-12875] Change default crumb name to Jenkins-Crumb

Compare: https://github.com/jenkinsci/jenkins/compare/4649e04cbe58...a87c38c5052c

knurek.j@gmail.com (JIRA)

unread,
Apr 14, 2016, 4:17:02 AM4/14/16
to jenkinsc...@googlegroups.com
J Knurek commented on Bug JENKINS-12875

Maybe it's worth noting in this ticket:
I installed Jenkins 2.0-rc-1 and created a view using Build Monitor View plugin (1.8+build.201601112328), and get this error (which leaves the UI in an infinite "sorry" loop

Apr 14, 2016 8:16:03 AM hudson.security.csrf.CrumbFilter doFilter
WARNING: No valid crumb was included in request for /$stapler/bound/18964be5-e86d-4242-a57b-35ee573d52c9/fetchJobViews. Returning 403.

dbeck@cloudbees.com (JIRA)

unread,
Apr 14, 2016, 4:36:02 AM4/14/16
to jenkinsc...@googlegroups.com

J Knurek That's unrelated to this issue and looks like a bug in the plugin that cannot handle the CSRF protection option (around since 2010 or so) being enabled.

knurek.j@gmail.com (JIRA)

unread,
Apr 15, 2016, 11:34:02 AM4/15/16
to jenkinsc...@googlegroups.com
J Knurek commented on Bug JENKINS-12875

My first thought was that it was a bug specific to the Build Monitor View plugin, but when I run Jenkins 1.651.1, the same configuration works ok. I'm only getting this error with the 2.0-rc-1.
(but I'm not sure what the underlying issue is in this case, so it is likely a separate issue)

jan.molak@smartcodeltd.co.uk (JIRA)

unread,
Apr 29, 2016, 5:48:02 PM4/29/16
to jenkinsc...@googlegroups.com

Hey Daniel and thanks for your suggestion about Build Monitor not handling the CSRF protection option.
The CSRF protection option works perfectly fine with Build Monitor on Jenkins 1.x.

Has anything changed regarding CSRF and the Crumb/.crumb headers handling on Jenkins 2.0? I'd be grateful for suggestions.

Kind regards,
Jan (author of Build Monitor)

jan.molak@smartcodeltd.co.uk (JIRA)

unread,
Apr 29, 2016, 6:20:03 PM4/29/16
to jenkinsc...@googlegroups.com
Jan Molak updated an issue
 
Change By: Jan Molak
Comment:
Hey Daniel and thanks for your suggestion about Build Monitor not handling the CSRF protection option. 
The CSRF protection option works perfectly fine with Build Monitor on Jenkins 1.x. 

Has anything changed regarding CSRF and the Crumb/.crumb headers handling on Jenkins 2.0? I'd be grateful for suggestions.

Kind regards,
Jan (author of Build Monitor)

jan.molak@smartcodeltd.co.uk (JIRA)

unread,
Apr 29, 2016, 6:26:02 PM4/29/16
to jenkinsc...@googlegroups.com
 
Re: "No valid crumb was included in the request" errors all around

Daniel Beck Build Monitor has supported CSRF protection since its early days

The problem is caused by change https://github.com/jenkinsci/jenkins/compare/4649e04cbe58...a87c38c5052c , which replaced the hard-coded '.crumb' header with 'Jenkins-Crumb', that's why they no longer match.

I can work around that in Build Monitor.

J Knurek you can watch the progress of this issue on github at https://github.com/jan-molak/jenkins-build-monitor-plugin/issues/215, which is the official issue tracker for Build Monitor.

Thanks!
Jan

dbeck@cloudbees.com (JIRA)

unread,
Apr 29, 2016, 7:34:02 PM4/29/16
to jenkinsc...@googlegroups.com

which replaced the hard-coded '.crumb' header with 'Jenkins-Crumb', that's why they no longer match.

The crumb name has been configurable since 1.310 (hudson.security.csrf.requestfield), and that was necessary for anyone running Jenkins behind nginx, as it dropped the .crumb header (

JENKINS-12875 ). So we renamed the default to something nginx accepts. Maybe it's a good idea to check which crumb name is used by Jenkins in your plugin to adapt to user configuration.

jan.molak@smartcodeltd.co.uk (JIRA)

unread,
Apr 29, 2016, 8:44:05 PM4/29/16
to jenkinsc...@googlegroups.com

J Knurek - Fixed in Build Monitor 1.9+build.2016043001​28, which you can [download from my Jenkins CI server](https://smartcode-opensource.ci.cloudbees.com/job/build-monitor/37/artifact/build-monitor-plugin/target/build-monitor-plugin.hpi) before it hits the official update centre.

jan.molak@smartcodeltd.co.uk (JIRA)

unread,
Apr 29, 2016, 8:44:06 PM4/29/16
to jenkinsc...@googlegroups.com
Jan Molak edited a comment on Bug JENKINS-12875
[~jknurek] - Fixed in Build Monitor 1.9+build.2016043001​28, which you can [download  from my Jenkins CI server](  here| https://smartcode-opensource.ci.cloudbees.com/job/build-monitor/37/artifact/build-monitor-plugin/target/build-monitor-plugin.hpi ) ]  before it hits the official update centre.

jan.molak@smartcodeltd.co.uk (JIRA)

unread,
Apr 29, 2016, 8:49:02 PM4/29/16
to jenkinsc...@googlegroups.com

Thanks Daniel Beck, that's exactly what I did. The latest Build Monitor picks up the crumb name from CrumbIssuer::getCrumbRequestField so it should be a more appropriate solution

jan.molak@smartcodeltd.co.uk (JIRA)

unread,
Apr 29, 2016, 8:52:02 PM4/29/16
to jenkinsc...@googlegroups.com
Jan Molak edited a comment on Bug JENKINS-12875
Thanks Awesome, thanks  [~danielbeck], that's exactly what I did. The latest Build Monitor picks up the crumb name from CrumbIssuer::getCrumbRequestField so it should be a more appropriate solution :-)

mr.naveen22@gmail.com (JIRA)

unread,
Jun 20, 2016, 7:55:02 AM6/20/16
to jenkinsc...@googlegroups.com

Hi,
I have Jenkins 2.7, I have created the service hook in TFS to trigger the automatic build but while testing the hook I get " No valid crumb was included in the request (403)". I can run the same build from Jenkins dashboard with no issue.
I have also installed the downgrade Build - Plugin-Version: 1.9+build.201606171408 as suggested by Jan Molak above but I am still facing the issue.

Thanks,
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
Atlassian logo

mr.naveen22@gmail.com (JIRA)

unread,
Jun 20, 2016, 7:57:03 AM6/20/16
to jenkinsc...@googlegroups.com
Naveen Sharma edited a comment on Bug JENKINS-12875
Hi,
I have Jenkins 2.7, I have created the service hook in TFS to trigger the automatic build but while testing the hook I get " No valid crumb was included in the request (403)". I can run the same build from Jenkins dashboard with no issue.
I have also installed the downgrade Build - Plugin-Version: 1.9+build.201606171408 as suggested by Jan Molak above but I am still facing the issue.
P.s - I have installed Jenkins on my local windows 2007 machine.

 Thanks,

jan.molak@smartcodeltd.co.uk (JIRA)

unread,
Jun 20, 2016, 8:05:04 AM6/20/16
to jenkinsc...@googlegroups.com

Hey Naveen - from what you're saying it seems like the problem is related to Jenkins and not the Build Monitor plugin?
If you think it's related to Build Monitor please feel free to raise a ticket on github at https://github.com/jan-molak/jenkins-build-monitor-plugin

Thanks,
Jan

mr.naveen22@gmail.com (JIRA)

unread,
Jun 20, 2016, 9:02:03 AM6/20/16
to jenkinsc...@googlegroups.com

Hi Jan, yeah I think the issue is either with Jenkins or may be with Tfs-jenkins plugin, I am not sure. Pls suggest.

Thanks

ryan@ryancocks.net (JIRA)

unread,
Oct 16, 2016, 9:54:13 AM10/16/16
to jenkinsc...@googlegroups.com

Having just wasted a few hours debugging why I was seeing this: if you re-install Jenkins you need to clear your cookies or you'll hit this.

dbeck@cloudbees.com (JIRA)

unread,
Oct 17, 2016, 4:30:02 AM10/17/16
to jenkinsc...@googlegroups.com

Ryan Cocks Too many cookies is tracked as JENKINS-25046.
Reply all
Reply to author
Forward
0 new messages