suspected bot traffic

[Karen Kohn]

Hello,

I am doing my annual gathering of usage data for ARL and am finding a lot of the numbers in the TR_B1 report hugely inflated in ways that make me suspect some kind of automated traffic. A particularly clear example would be in Springer, where we had a Dictionary of Gems and Gemology with zero downloads from July to February, then 23,000 Total_Item_Requests each in March and April, then back to zero.

 

I have many questions! If you know why/how this happens, I’d be interested to learn. My more practical questions are: Do you have a strategy for identifying when there is suspected automated traffic, other than just noticing when the usage seems ridiculously high? Do you report the numbers in the COUNTER report or try to remove the bot numbers somehow? I can make an adjusted estimate, though it still won’t necessarily be accurate enough to be meaningful.

 

Thanks for your thoughts.

Karen

 

Karen Kohn, Collections Analysis Librarian
Temple University Libraries, Charles Library
1900 N. 13th St, Philadelphia, PA 19122-6082

215-204-4428 | karen...@temple.edu

pronouns: she/her/hers

 

[Harker, Karen]

We have had the same problem, although I'm not sure if it's gotten worse (as in, more titles or more frequently). Since most platforms don't provide access to the data until after monthly compilations, there really is no way for the libraries to track this real-time or near real-time.  That is something only the platforms/providers can monitor.  

We haven't yet decided on how we should handle these obviously inflated usage metrics - delete them, interpolate, or leave them as-is?  The first and the last options will cause significant fluctuations (or not, depending on the past reports), but estimating based on last month's or same month last year(s) data isn't the truth.  

Karen R. Harker

Collection Assessment Librarian

University of North Texas Libraries

Denton, Texas 75287

Karen....@unt.edu

 

---

From: 'Karen Kohn' via ARL ASSESS <arl-a...@arl.org>
Sent: Tuesday, July 28, 2026 9:26 AM
To: arl-a...@arl.org <arl-a...@arl.org>
Subject: [EXT] [ARL-ASSESS] suspected bot traffic

 

--
To post to this group, send email to arl-a...@arl.org
To unsubscribe from this group, send email to arl-assess+...@arl.org
For more options, visit https://groups.google.com/a/arl.org/d/forum/arl-assess?hl=en
For instructions on logging in visit https://sites.google.com/a/arl.org/techguides_arl/login.
Discussions on this list are subject to ARL's Code of Conduct: https://www.arl.org/who-we-are/#section-codeofconduct.
To unsubscribe from this group and stop receiving emails from it, send an email to arl-assess+...@arl.org.

[Jennifer Sweeney]

What is the vendors' explanation ? 

Jen

Jennifer K. Sweeney, MSLS, PhD

Library Planning & Evaluation

Information School, San Jose State University

[Savage, Devin]

Hey, I tried to post this response earlier but it appeared to bounce. Hopefully you all are not getting a duplicate response. 

I think what you do with this information largely matters in what you are planning to do with it. For cost-usage/renewal decisions, etc. I would simply drop those aberrations - or if the case seemed more reasonable (the above case does not) I might look at a median monthly number to extrapolate and adjudicate from. 

However! I also feel like we should be recording and noting the massive impact of these bots, internal and external, upon our operations and infrastructure. The only downside I can see to including these stats in our annual roll-ups, reports, and benchmarking surveys is that the comparison data gets a bit more uneven and perhaps unreliable. The thing that I come back to is that almost all of us are getting impacted in some way in the last couple of years by this bot traffic, and so as long as those of us who handle and use the data are aware of it, I feel like it's absolutely defensible to include these bot traffic numbers. I'm sure there are more caveats that I've not yet processed, though. 

Devin

Get Outlook for iOS

---

From: Jennifer Sweeney <jkswee...@gmail.com>
Sent: Tuesday, 28 July 2026 13:50:33
To: Harker, Karen <Karen....@unt.edu>
Cc: arl-a...@arl.org <arl-a...@arl.org>; Karen Kohn <karen...@temple.edu>
Subject: [EXTERNAL] Re: [EXT] [ARL-ASSESS] suspected bot traffic

 

Caution: This is an external email and may be malicious. Please use caution before clicking links or opening attachments.

[Karen Kohn]

Thanks, Devin and Karen. To Jennifer’s question, I haven’t heard back from the vendors yet. I appreciate the points about the inflated number not being any less accurate than an estimate of “true use”  that I could potentially make. And also that we can treat the numbers differently depending on whether we are reporting them out or using them for internal decision-making.

 

I think we will probably report the data as-is, including suspected automated traffic. But if anyone else has considerations, please share!

 

Thanks,

Karen

 

Karen Kohn, Collections Analysis Librarian
Temple University Libraries, Charles Library
1900 N. 13th St, Philadelphia, PA 19122-6082

215-204-4428 | karen...@temple.edu

pronouns: she/her/hers

 

[Cowan, Susanna]

My two cents after the great comments so far...

If we all start to correct, then we’ll lose the “shape” of these mass attacks (that’s what they are) on our systems.  If we report the numbers as we have them - even knowing they’re incorrect - we can see that shape across all of our members.  Which then will give us traction when we talk to vendors about how they will address this (stopping the bots or deflating the data or whatever).

Just my data ponderings on this Wednesday morning.

Love the discussion this has generated!

Susanna

 

-----------------------------------

SUSANNA M. COWAN PHD MLS

Library Assessment Strategist

(she/her/hers)

UConn Library | Babbidge Library

369 Fairfield Way, Box 1005 | Storrs, CT 06269

lib.uconn.edu

From: 'Karen Kohn' via ARL ASSESS <arl-a...@arl.org>

Date: Wednesday, July 29, 2026 at 6:29 AM
To: arl-a...@arl.org <arl-a...@arl.org>
Subject: RE: [EXTERNAL] Re: [EXT] [ARL-ASSESS] suspected bot traffic

*External sender: This message was not sent through the UConn email system. It might be safe, but use caution before interacting with links, attachments, or requests.*

[Selena Chau]

Hi everyone, and thanks Karen for the initial prompt.

I like everyone's feedback. My actions have been similar to Devin's. The purpose guides the action. I drop aberrations, and when I have time I report to the vendor. Most often there is no action on their end, so the only benefit is to confirm it is an aberration, and then one is back in the place of having to decide what to do with the data.

Recording and noting the massive impact of these bots is a nice idea. I wonder if Project COUNTER may have started this work (they know and field Qs about this in many of their webinars) and if they can be a support for anyone interested in starting this initiative.

Slightly related, I started the book This is how they tell me the world ends: the cyberweapons arms race, by Nicole Perlroth. Although libraries are a distant target from other more valuable places that hackers and bots can put at risk, I am thinking of how much more difficult it is, and will be, to uphold values of protecting patron privacy and providing legitimate access to scholarly content.

Selena Chau | she/her

Collection Strategist Librarian,
Humanities and Social Sciences

UCSB Library - selen...@ucsb.edu

Rm 5511, 805-893-4719