913100 Detecting but not Blocking

Larry David

unread,

Mar 9, 2021, 5:10:27 AM3/9/21

to ModSecurity Core Rule Set project

I have added the following text to the end of scanners-user-agents.data

zgrab
crawler4j
Dispatch
Mb2345Browser
LieBaoFast
zh-CN
zh_CN
MicroMessenger
Kinza
aiohttp
node-superagent
BrandVerity
GarlikCrawler
netEstate
pimeyes
Barkrowler
WikiDo
proximic
Adsbot
Accompanybot

and restarted httpd.

913100 is detecting these strings but it is not blocking.

In the httpd error log is this:

[Tue Mar 09 09:24:10.402953 2021] [:error] [pid 51142:tid 139939832395520] [client xx.xx.xx.xx:35963] [client xx.xx.xx.xx] ModSecurity: Warning. Matched phrase "aiohttp" at REQUEST_HEADERS:User-Agent. [file "/etc/httpd/modsecurity.d/rules/REQUEST-913-SCANNER-DETECTION.conf"] [line "57"] [id "913100"] [msg "Found User-Agent associated with security scanner"] [data "Matched Data: aiohttp found within REQUEST_HEADERS:User-Agent: python/3.6 aiohttp/3.6.2"] [severity "CRITICAL"] [ver "OWASP_CRS/3.2.0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-reputation-scanner"] [tag "OWASP_CRS"] [tag "OWASP_CRS/AUTOMATION/SECURITY_SCANNER"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"] [hostname "www.example.com"] [uri "/index.html"] [unique_id "YEc@utgwhZqZBarrzrh97AAAAAc"]

The modsecurity audit file shows this:

--b261d270-A--
[09/Mar/2021:09:24:10 +0000] YEc@utgwhZqZBarrzrh97AAAAAc xx.xx.xx.xx 35963 yy.yy.yy.yy 443
--b261d270-B--
GET /index.html HTTP/1.1
Host: www.example.com
Accept: */*
Accept-Encoding: gzip, deflate
User-Agent: Python/3.6 aiohttp/3.6.2

--b261d270-F--
HTTP/1.1 200 OK
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Strict-Transport-Security: max-age=63072000; includeSubDomains
Set-Cookie: PHPSESSID=48286d81908636e802890c4a59e738f6; path=/; secure
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
Vary: Accept-Encoding
Content-Encoding: gzip
X-UA-Compatible: IE=edge
Content-Security-Policy: default-src 'self' 'unsafe-inline' 'unsafe-eval' https://www.paypalobjects.com https://seal.starfieldtech.com https://platform.twitter.com http://www.youtube.com https://www.youtube.com https://$
X-Content-Security-Policy: default-src 'self' 'unsafe-inline' 'unsafe-eval' https://www.paypalobjects.com https://seal.starfieldtech.com https://platform.twitter.com http://www.youtube.com https://www.youtube.com https:$
Content-Length: 8654
Content-Type: text/html; charset=UTF-8

--b261d270-E--

--b261d270-H--
Message: Warning. Matched phrase "aiohttp" at REQUEST_HEADERS:User-Agent. [file "/etc/httpd/modsecurity.d/rules/REQUEST-913-SCANNER-DETECTION.conf"] [line "57"] [id "913100"] [msg "Found User-Agent associated with secur$
Apache-Error: [file "apache2_util.c"] [line 273] [level 3] [client xx.xx.xx.xx] ModSecurity: Warning. Matched phrase "aiohttp" at REQUEST_HEADERS:User-Agent. [file "/etc/httpd/modsecurity.d/rules/REQUEST-913-SCANNER-D$
Stopwatch: 1615281850402449 4510 (- - -)
Stopwatch2: 1615281850402449 4510; combined=792, p1=355, p2=282, p3=19, p4=64, p5=53, sr=32, sw=19, l=0, gc=0
Response-Body-Transformed: Dechunked
Producer: ModSecurity for Apache/2.9.3 (http://www.modsecurity.org/); OWASP_CRS/3.2.0.
Server: Apache
Engine-Mode: "ENABLED"

and in the httpd access log there is this:

xx.xx.xx.xx - - [09/Mar/2021:09:24:10 +0000] "GET /index.html HTTP/1.1" 200 8654 "-" "Python/3.6 aiohttp/3.6.2" 0 www.example.com "-" "-"

Why is it detecting but then not serving a 403?

Ervin Hegedüs

unread,

Mar 9, 2021, 8:14:04 AM3/9/21

to Larry David, ModSecurity Core Rule Set project

Hi Larry,

On Tue, Mar 09, 2021 at 02:10:26AM -0800, Larry David wrote:
> I have added the following text to the end of scanners-user-agents.data
>
> zgrab
> crawler4j

[...]

> Adsbot
> Accompanybot
>
> and restarted httpd.
>
> 913100 is detecting these strings but it is not blocking.
>
> In the httpd error log is this:
>

> [Tue Mar 09 09:24:10.402953 2021] [:error] [pid 51142:tid 139939832395520] [client xx.xx.xx.xx:35963] [client xx.xx.xx.xx] ModSecurity: Warning. Matched phrase "aiohttp" at REQUEST_HEADERS:User-Agent. [file ...
[...]

Rule 913100 has a disruptive action "block" - see here:
https://github.com/coreruleset/coreruleset/blob/v3.2/dev/rules/REQUEST-913-SCANNER-DETECTION.conf#L36

"Block" follows the described behavior in SecDefaultAction:
https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual-(v2.x)#block

which is "pass" in crs-setup.conf:

https://github.com/coreruleset/coreruleset/blob/v3.2/dev/crs-setup.conf.example#L96-L97

CRS v3.2 is a bit old (released on Sep 24, 2019), and I don't
have it anywhere, so I can test it.

But I'm wondering why 949110 wasn't triggered...

Regards,

a.

Larry David

unread,

Mar 9, 2021, 2:49:54 PM3/9/21

to ModSecurity Core Rule Set project, air...@gmail.com, ModSecurity Core Rule Set project, Larry David

Yes of course - SecDefaultAction.

I had all SecDefaultAction lines # out such as

#SecDefaultAction "phase:1,log,auditlog,pass"

#SecDefaultAction "phase:2,log,auditlog,pass"

I have now set this

SecDefaultAction "phase:1,log,auditlog,deny,status:403"
SecDefaultAction "phase:2,log,auditlog,deny,status:403"