Request For Guides on Testing ModSecurity C.R.S. Deployment Effectiveness...

[Michael Bullut]

Good Afternoon Team,

I hope this e-mail finds you well.

I am writing to ask for your help. I am in the process of deploying ModSecurity with the Core Rule Set and am keen to ensure its effectiveness is thoroughly evaluated.

While the documentation provides excellent guidance on installation and configuration, I am seeking resources or best practices on how to specifically test and validate the effectiveness of the deployment once it is in place. Specifically, I am interested in learning:

1. Methodologies: Recommended approaches for testing the ruleset (e.g., controlled testing with benign traffic vs. simulated attack traffic).
2. Testing Tools: What tools are commonly used and recommended by the community for this purpose (e.g., custom scripts, open-source vulnerability scanners, specialized testing suites)?
3. Success Metrics: How to define and measure success? What key metrics or log indicators should we look for to confirm the rules are triggering correctly on malicious requests and, just as importantly, not triggering false positives on legitimate traffic?
4. Baseline Establishment: How to establish a performance and security baseline before and after deployment to measure impact.

I believe guidance on this topic would be incredibly valuable not only for me but for many others in the community looking to confidently deploy and tune CRS. If such guides, whitepapers, or community knowledge already exist, I would be very grateful if you could point me in the right direction.

If this does not yet exist, perhaps this e-mail could serve as a starting point for a discussion on the subject.

Thank you for your time and for your continued work on this fantastic project.

Warm regards,

Michael Bullut.

---

Cellphone: +254 723 393 114.

Twitter: @MichaelBullut

Blog: http://www.kipsang.com/
E-mail Address: ma...@kipsang.com

---

Virus-free.www.avast.com

[Andrew Howe]

Hi Michael,

What is the context for this? Are you approaching this as a hobbyist?
For a personal CRS deployment? For a professional/business deployment?
Or maybe you're looking to validate a CRS integration in a commercial
product? Academic testing interest? Something else? I think the
approach and advice given would vary quite a bit...

Both main versions of CRS (v3 and v4) have full test suites that
include positive and negative tests for every* rule (the "benign"
versus "attack" traffic you mentioned.) You can execute these test
suites to ensure a CRS-powered WAF is behaving correctly. Some
deviation is expected between WAF engines (e.g. Apache vs nginx vs
standalone, ModSecurity v2 vs v3, other engines can behave quite
differently, etc.) but, in general, if you find that *many* tests are
failing then that may indicate mis-configuration or an engine problem.
If you get, say 99% passing tests then your deployment is likely
correct (and maybe this is the confirmation you're looking for?).
(*I think the CRS v3.3.7 tests cover *all* rules... But certainly, the
v4 tests are fuller and more complete!)

As for tooling: the go-ftw testing tool is your friend. This is used
by CRS to check for regressions and was written specifically for CRS
and its requirements, written by CRS developers.

* Project page + documentation: https://github.com/coreruleset/go-ftw
* Releases: https://github.com/coreruleset/go-ftw/releases
* How we set up containers for testing:
https://github.com/coreruleset/coreruleset/blob/main/tests/docker-compose.yml

*Please note*: You'll almost certainly need an *older version* of the
go-ftw testing tool if you're working with CRS v3. Check out the
version we pin for building the official Docker containers: if you
follow the same testing version we use then you should be able to get
the same results (i.e. passing results).

Alternatively, if you're working with a *specific* web application /
deployment type, it's a very good idea to write your own positive and
negative tests. For example: each rule exclusion you add, you should
include tests to make sure that the relevant false positive stays
fixed! But also add tests to make sure that the 'hole' you open in
your WAF is the correct size and not accidentally too big (e.g. test
against the wrong URI or with the wrong HTTP method and make sure that
a block correctly occurs.)

I hope these thoughts can help you out on your testing journey.

Thanks,
Andrew Howe

[Christian Folini]

Hi Michael,

This is an excellent question or rather request.

I am hosting CRS Community Call later today and while preparing it, we thought
this would be the perfect agenda for that call.

Monday September 22, 8:30 PM - 10:00 PM GMT+2

Registration and details at https://luma.com/8yc1p543

Best regards,

Christian

On Wed, Sep 17, 2025 at 09:59:44AM +0300, 'Michael Bullut' via OWASP CRS project wrote:
> Good Afternoon Team,
>
> I hope this e-mail finds you well.
>
> I am writing to ask for your help. I am in the process of deploying
> ModSecurity with the Core Rule Set and am keen to ensure its effectiveness
> is thoroughly evaluated.
>
> While the documentation provides excellent guidance on installation and
> configuration, I am seeking resources or best practices on how to
> specifically test and validate the effectiveness of the deployment once it
> is in place. Specifically, I am interested in learning:
>

> 1. *Methodologies:* Recommended approaches for testing the ruleset

> (e.g., controlled testing with benign traffic vs. simulated attack traffic).

> 2. *Testing Tools:* What tools are commonly used and recommended by the

> community for this purpose (e.g., custom scripts, open-source vulnerability
> scanners, specialized testing suites)?

> 3. *Success Metrics:* How to define and measure success? What key

> metrics or log indicators should we look for to confirm the rules are
> triggering correctly on malicious requests and, just as importantly, not
> triggering false positives on legitimate traffic?

> 4. *Baseline Establishment:* How to establish a performance and security

> baseline before and after deployment to measure impact.
>
> I believe guidance on this topic would be incredibly valuable not only for
> me but for many others in the community looking to confidently deploy and
> tune CRS. If such guides, whitepapers, or community knowledge already
> exist, I would be very grateful if you could point me in the right
> direction.
>
> If this does not yet exist, perhaps this e-mail could serve as a starting
> point for a discussion on the subject.
>
> Thank you for your time and for your continued work on this fantastic
> project.
>
> Warm regards,
>
> Michael Bullut.
>
> ---
>

> *Cellphone:* *+254 723 393 114.*
> *Twitter:* *@MichaelBullut* <https://x.com/MichaelBullut>
> *Blog:* *http://www.kipsang.com/ <http://www.kipsang.com/>*
> *E-mail Address:* *ma...@kipsang.com* <ma...@kipsang.com>
>
> ---
>
> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
> Virus-free.www.avast.com
> <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
> <#m_5302983944335740208_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>
> --
> You received this message because you are subscribed to the Google Groups "OWASP CRS project" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to modsecurity-core-rule-...@owasp.org.
> To view this discussion visit https://groups.google.com/a/owasp.org/d/msgid/modsecurity-core-rule-set-project/CAGy%2BNY0xgx7QEtwJ5R-STCCqc2ug_ug428%2BvKx9yB%2B-AJGuvYw%40mail.gmail.com.