Anthropic’s ClaudeBot is aggressively scraping DSpace

[James Holobetz]

Has anyone in the DSpace community/tech seen a large spike in queries for Anthropic’s ClaudeBot?

User Agent: compatible; "ClaudeBot/1.0; +claudebot\@anthropic.com"

Here is some other references lately:

https://www.reddit.com/r/singularity/comments/1cdm97j/anthropics_claudebot_is_aggressively_scraping_the/

We have been getting hit hard for a couple days now.

Best regards,

James

[Andrew K]

Yes, pretty aggressively, Stats for today:

78575 Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +clau...@anthropic.com)

Googlebot is the next in the top crawlers with decent 5500 requests.

But I've seen worse load.

пʼятницю, 26 квітня 2024 р. о 18:51:54 UTC+3 James Holobetz пише:

[emilio lorenzo]

Yes,

not only  Anthropic’s ClaudeBot  but also some (or many)  IA bots are "collapsing" the repositories system,  we have even noticed  a huge increase in traffic from Google bots.🙁  Many installations  (large and small, without no discrimination) are experiencing downtimes for these reasons.

 

Hard days...

Emiolio

--
All messages to this mailing list should adhere to the Code of Conduct: https://www.lyrasis.org/about/Pages/Code-of-Conduct.aspx
---
You received this message because you are subscribed to the Google Groups "DSpace Technical Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dspace-tech...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dspace-tech/CAAosP7VXJqUwxDs3q2wSBvE0Q9H6Ms%3DPd9Gvyo8G5KP0OqsQBQ%40mail.gmail.com.