CRS version 4.0.0 release candidate 2 available

[Andrew Howe]

The OWASP ModSecurity Core Rule Set (CRS) team is proud to announce
the availability of release candidate 2 (RC2) of the upcoming CRS
v4.0.0 release. The release candidate is available for download as a
'release' from our GitHub repository:

* https://github.com/coreruleset/coreruleset/releases/tag/v4.0.0-rc2

This new release candidate includes over 230 changes. Some of the
important changes include:

* Add new rule 920620 to explicitly detect multiple Content-Type abuse
(CVE-2023-38199) (Andrea Menin)
* Extend definition of restricted headers to include Content-Encoding
and Accept-Charset by default (Walter Hop)
* Migrate application exclusions and less-used functionality to
plugins (Christian Folini, Max Leske, Jozef Sudolský, Andrew Howe)
* Add support for HTTP/3 (Jozef Sudolský)
* Add enable_default_collections flag to not initialize collections by
default (Matteo Pace)
* Switch to using wordnet instead of spell for finding English words
in spell.sh utility (Max Leske)

Refer to the CHANGES.md file in the release for the full list of changes.

It is important to note that this new release candidate is
*significantly different to the first release candidate* which was
announced and made available[1] back in April 2022. Two days after the
v4.0.0 RC1 release, the CRS project participated in a bug bounty
program[2] in April-May 2022 which resulted in 175 security findings
being reported. The decision was made to fix the findings in full for
the v4.0.0 release, rather than release a half-baked version 4.0 with
many newly discovered holes.

The fixes required a *significant* amount of work over many months. It
was sometimes the case that adding the required new detection would
cause unforeseen problems, such as introducing new false positives
which then needed to be addressed. Fixing all of the security findings
in full required the development of new tooling, new rules, new tests,
and new approaches. This all took a lot of time to complete to the
high standard expected from the CRS project, resulting in an
unfortunate delay to v4.0.0.

As a result of fixing the security findings, the RC2 release features
*a lot of new detection capability*. It is highly likely that *new
false positives will continue to appear* as a result, so it is very
important for this new release candidate to be tested as widely as
possible. *Please test this new release candidate* and report any
false positives encountered via GitHub
(https://github.com/coreruleset/coreruleset/). All feedback and
reports are gratefully received and will help to make the final v4.0.0
release the best and most comprehensive CRS release ever!

Sincerely,
Andrew Howe on behalf of the Core Rule Set development team

---
[1]: https://coreruleset.org/20220428/coreruleset-v4-rc1-available
[2]: https://coreruleset.org/20230509/what-we-learnt-from-our-bug-bounty-program-its-not-for-the-faint-of-heart

[Christian Folini]

Thank you Andrew.

Getting this out the door was a huge undertaking. Getting CRSv3 done pales
in comparison.

The biggest problem that remains is the new false positives that CRSv4
introduces because of hundreds and hundreds of rule changes.

And given we have but little traffic among the team members, we need
the wider community to test this with us. Every false positive we
solve before the real release is a false positive that does not hit
the millions of CRS end users afterwards.

Here is your shortcut to write a report: https://t.ly/A61yS

(Full link: https://github.com/coreruleset/coreruleset/issues/new?assignees=&labels=%3Aheavy_plus_sign%3A+False+Positive&projects=&template=01_false-positive.md&title=FP%20with%20RC2:%20...)

Best regards,

Christian

> --
> You received this message because you are subscribed to the Google Groups "ModSecurity Core Rule Set project" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to modsecurity-core-rule-...@owasp.org.
> To view this discussion on the web visit https://groups.google.com/a/owasp.org/d/msgid/modsecurity-core-rule-set-project/CAO2fWAbXPjbQsObT2VBLqMbn2LMLXd_bnWpU_tE-HZ5Sdf4EFA%40mail.gmail.com.

[Emilio Campos]

Hi team, we are using this core ruleset, and today we detected the following: 

Some CRS v4 rules check the phase 2, for example in the file:

https://github.com/coreruleset/coreruleset/blob/v4.0/dev/rules/REQUEST-930-APPLICATION-ATTACK-LFI.conf

secrule:

SecRule REQUEST_URI_RAW|ARGS|REQUEST_HEADERS|!REQUEST_HEADERS:Referer|FILES|XML:/* "@rx (?i)(?:[/\x5c]|%(?:2(?:f|5(?:2f|5c|c(?:1%259c|0%25af))|%46)|5c|c(?:0%(?:[2aq]f|5c|9v)|1%(?:[19p]c|8s|af))|(?:bg%q|(?:e|f(?:8%8)?0%8)0%80%a)f|u(?:221[5-6]|EFC8|F025|002f)|%3(?:2(?:%(?:%6|4)6|F)|5%%63)|1u)|0x(?:2f|5c))(?:\.(?:%0[0-1]|\?)?|\?\.?|%(?:2(?:(?:5(?:2|c0%25a))?e|%45)|c0(?:\.|%[25-6ae-f]e)|u(?:(?:ff0|002)e|2024)|%32(?:%(?:%6|4)5|E)|(?:e|f(?:(?:8|c%80)%8)?0%8)0%80%ae)|0x2e){2,3}(?:[/\x5c]|%(?:2(?:f|5(?:2f|5c|c(?:1%259c|0%25af))|%46)|5c|c(?:0%(?:[2aq]f|5c|9v)|1%(?:[19p]c|8s|af))|(?:bg%q|(?:e|f(?:8%8)?0%8)0%80%a)f|u(?:221[5-6]|EFC8|F025|002f)|%3(?:2(?:%(?:%6|4)6|F)|5%%63)|1u)|0x(?:2f|5c))" \
    "id:930100,\
    phase:2,\
    block,\
    capture,\
    t:none,\
    msg:'Path Traversal Attack (/../) or (/.../)',\
    logdata:'Matched Data: %{TX.0} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}',\
    tag:'application-multi',\
    tag:'language-multi',\
    tag:'platform-multi',\
    tag:'attack-lfi',\
    tag:'paranoia-level/1',\
    tag:'OWASP_CRS',\
    tag:'capec/1000/255/153/126',\
    ver:'OWASP_CRS/4.0.0-rc2',\
    severity:'CRITICAL',\
    setvar:'tx.inbound_anomaly_score_pl1=+%{tx.critical_anomaly_score}',\
    setvar:'tx.lfi_score=+%{tx.critical_anomaly_score}'"

This rule is evaluated in phase2 (Request body) but in a particular case where there isn't body, then the rule isn't evaluated. I just wanted to do mention that our proxy sends the content to evaluate to the modsecurity in case the REQUEST BODY isn't empty. Also, what happens if the request body is disabled in the global directive section? We don't want to evaluate the body but the HTTP headers, in this use-case those rules aren't going to work as well.

My question is, why this rule and others where the Header content is evaluated is not moved to phase1 instead of phase2, in a quick view I think that the same should be done for rules 930110, 930120.

Thanks in advance.

Regards!

To view this discussion on the web visit https://groups.google.com/a/owasp.org/d/msgid/modsecurity-core-rule-set-project/ZTtsvP1U6M2Rs9Yf%40leander.

[Christian Folini]

Hello Emiliom

Nobody has been picking this up, so let's give it a shot.

On Fri, Dec 29, 2023 at 12:41:16PM +0100, Emilio Campos wrote:
> Some CRS v4 rules check the phase 2, for example in the file:
> https://github.com/coreruleset/coreruleset/blob/v4.0/dev/rules/REQUEST-930-APPLICATION-ATTACK-LFI.conf

That is correct.

> REQUEST_URI_RAW|ARGS|REQUEST_HEADERS|!REQUEST_HEADERS:Referer|FILES|XML:/*
> "@rx
> (?i)(?:[/\x5c]|%(?:2(?:f|5(?:2f|5c|c(?:1%259c|0%25af))|%46)|5c|c(?:0%(?:[2aq]f|5c|9v)|1%(?:[19p]c|8s|af))|(?:bg%q|(?:e|f(?:8%8)?0%8)0%80%a)f|u(?:221[5-6]|EFC8|F025|002f)|%3(?:2(?:%(?:%6|4)6|F)|5%%63)|1u)|0x(?:2f|5c))(?:\.(?:%0[0-1]|\?)?|\?\.?|%(?:2(?:(?:5(?:2|c0%25a))?e|%45)|c0(?:\.|%[25-6ae-f]e)|u(?:(?:ff0|002)e|2024)|%32(?:%(?:%6|4)5|E)|(?:e|f(?:(?:8|c%80)%8)?0%8)0%80%ae)|0x2e){2,3}(?:[/\x5c]|%(?:2(?:f|5(?:2f|5c|c(?:1%259c|0%25af))|%46)|5c|c(?:0%(?:[2aq]f|5c|9v)|1%(?:[19p]c|8s|af))|(?:bg%q|(?:e|f(?:8%8)?0%8)0%80%a)f|u(?:221[5-6]|EFC8|F025|002f)|%3(?:2(?:%(?:%6|4)6|F)|5%%63)|1u)|0x(?:2f|5c))"
> \
> "id:930100,\

...

> This rule is evaluated in phase2 (Request body) but in a particular case
> where there isn't body, then the rule isn't evaluated.

No, that's not how this works. I'm afraid it seems you do not understand how
ModSecurity phases work, or your custom implementation is doing it wrong.

You can evaluate various targets in all the phases when the information
is available. That means REQUEST_HEADERS can be evaluated in phases 1 - 5.
REQUEST_BODY / ARGS and friends in phases 2 - 5.

That's all there is to know about this.

> I just wanted to do
> mention that our proxy sends the content to evaluate to the modsecurity in
> case the REQUEST BODY isn't empty.

I do not get this. What is your proxy server and if you were able to observe
the behaviour explained above on your custom implementation, then the
implementation is wrong.

> Also, what happens if the request body
> is disabled in the global directive section?

Little changes. Rule 930100 is evaluated for targets

REQUEST_URI_RAW
REQUEST_HEADERS (minus Referrer)

but not for

ARGS
FILES
XML (= xml payload)

> We don't want to evaluate the
> body but the HTTP headers, in this use-case those rules aren't going to
> work as well.

See above.

> My question is, why this rule and others where the Header content is
> evaluated is not moved to phase1 instead of phase2, in a quick view I think
> that the same should be done for rules 930110, 930120.

When we introduced early blocking for CRS v4 the idea was to shift as many
rules as possible into phase 1. However rules that are being applied to
request body as well would stay in phase 2. This is a pragmatic approach.

An alternative would be to duplicate the rule and to run it in phase 1
for the header stuff and in phase 2 for the HTTP request body. This would
lead to a performance impact and I think that's not worth it.

Best,

Christian

> > https://groups.google.com/a/owasp.org/d/msgid/modsecurity-core-rule-set-project/ZTtsvP1U6M2Rs9Yf%40leander
> > .

> >
>
> --
> You received this message because you are subscribed to the Google Groups "ModSecurity Core Rule Set project" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to modsecurity-core-rule-...@owasp.org.

> To view this discussion on the web visit https://groups.google.com/a/owasp.org/d/msgid/modsecurity-core-rule-set-project/CAKWgN_qkrFsq9KG1%3Dfr-Sup33oreMG3HQJBcGP3MVPBGkh_%3Ddw%40mail.gmail.com.