Request for guidance on arXiv API rate limits / higher-throughput access

[Bharath Sethupathi]

Hello arXiv API team,

I’m writing to ask for guidance on API rate limits and whether there is a way to request higher throughput (or a dedicated access path) for our application.

We use the arXiv API in a production literature review workflow. Specifically, our literature review agent searches arXiv and fetches paper metadata . This has worked reliably before, but starting around February 25–26, 2026, we began seeing repeated HTTP 429 responses despite adding delays between requests.

Current behavior:

• We use the export.arxiv.org/api/query endpoint.
• We have client-side delay/retry logic (roughly 3–4 seconds between requests per client).
• We still see 429s when multiple literature review workflows run concurrently.

Could you please clarify:

1. Whether there have been any recent changes to effective rate limits or enforcement behavior.
2. Recommended best practices for multi-workflow/production usage.
3. Whether there is any option for higher rate limits, dedicated API access, or a registration/approval process for sustained application usage.

Thank you for your time and support.

[Brian Maltzan]

Hi Bharath,

1. Yes, we have made changes recently and over time, as the total use of our endpoints change.
2. Our recommendations are here: https://info.arxiv.org/help/api/tou.html

3. We'll look into options for you and others, and get back to you in the next day or so.

Thanks,
Brian

--
You received this message because you are subscribed to the Google Groups "arXiv API Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api+uns...@arxiv.org.
To view this discussion visit https://groups.google.com/a/arxiv.org/d/msgid/api/19a2af0e-d495-4448-a0b2-7e0048d1fca3n%40arxiv.org.

[Serguei Kalentchouk]

Hi Brian,

I wanted to follow up on this question. I am working on an internal application that uses the query end point in very light workflows and following the public recommendations, however I frequently run into rate limiting.

Is it possible that my workflow has to contend with anyone else hitting this API in my company? At what granularity is rate limiting enforced?

Likewise, would be interested in learning if there are options for dedicated API access.

Thank you!

[Lukas Schwab]

I'm not sure this is actually relevant to anyone reaching out about rate limits in the last couple days, but my recent exploration of switching from GET to POST requests for the /api/query endpoint revealed POST requests don't hit the Fastly cache, and therefore lead much more immediately (esp. for repeated queries, which would cache-hit) to 429 responses.

If you're using POST requests, you might explore switching so you can hit the cache.

One caveat. My usage pattern here — integration tests for an API wrapper — run the same queries repeatedly, and therefore benefit more from cached GET responses at baseline than a real-world application (which should cache results locally until the next arXiv mailing instead of repeating the same API call; I think arXiv has published their own guidance about this somewhere).

Cheers,

Lukas

To view this discussion visit https://groups.google.com/a/arxiv.org/d/msgid/api/5605757c-e942-4296-82c0-81f4ee2d33ffn%40arxiv.org.