Request for guidance on arXiv API rate limits / higher-throughput access

[Bharath Sethupathi]

Hello arXiv API team,

I’m writing to ask for guidance on API rate limits and whether there is a way to request higher throughput (or a dedicated access path) for our application.

We use the arXiv API in a production literature review workflow. Specifically, our literature review agent searches arXiv and fetches paper metadata . This has worked reliably before, but starting around February 25–26, 2026, we began seeing repeated HTTP 429 responses despite adding delays between requests.

Current behavior:

• We use the export.arxiv.org/api/query endpoint.
• We have client-side delay/retry logic (roughly 3–4 seconds between requests per client).
• We still see 429s when multiple literature review workflows run concurrently.

Could you please clarify:

1. Whether there have been any recent changes to effective rate limits or enforcement behavior.
2. Recommended best practices for multi-workflow/production usage.
3. Whether there is any option for higher rate limits, dedicated API access, or a registration/approval process for sustained application usage.

Thank you for your time and support.

[Brian Maltzan]

Hi Bharath,

1. Yes, we have made changes recently and over time, as the total use of our endpoints change.
2. Our recommendations are here: https://info.arxiv.org/help/api/tou.html

3. We'll look into options for you and others, and get back to you in the next day or so.

Thanks,
Brian

--
You received this message because you are subscribed to the Google Groups "arXiv API Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api+uns...@arxiv.org.
To view this discussion visit https://groups.google.com/a/arxiv.org/d/msgid/api/19a2af0e-d495-4448-a0b2-7e0048d1fca3n%40arxiv.org.

[Serguei Kalentchouk]

Hi Brian,

I wanted to follow up on this question. I am working on an internal application that uses the query end point in very light workflows and following the public recommendations, however I frequently run into rate limiting.

Is it possible that my workflow has to contend with anyone else hitting this API in my company? At what granularity is rate limiting enforced?

Likewise, would be interested in learning if there are options for dedicated API access.

Thank you!

[Lukas Schwab]

I'm not sure this is actually relevant to anyone reaching out about rate limits in the last couple days, but my recent exploration of switching from GET to POST requests for the /api/query endpoint revealed POST requests don't hit the Fastly cache, and therefore lead much more immediately (esp. for repeated queries, which would cache-hit) to 429 responses.

If you're using POST requests, you might explore switching so you can hit the cache.

One caveat. My usage pattern here — integration tests for an API wrapper — run the same queries repeatedly, and therefore benefit more from cached GET responses at baseline than a real-world application (which should cache results locally until the next arXiv mailing instead of repeating the same API call; I think arXiv has published their own guidance about this somewhere).

Cheers,

Lukas

To view this discussion visit https://groups.google.com/a/arxiv.org/d/msgid/api/5605757c-e942-4296-82c0-81f4ee2d33ffn%40arxiv.org.

[Mohammed Mehdi]

We are also getting persistent 429 Rate Exceeded errors on all our api requests, even though we are well within the rate limit of 1 request every 3 seconds.

After repeatedly trying same request search_query parameter, We finally get a response which is then cached. But the api stops responding for other search_query parameter and still returns 429.

Many times we are also getting the 503 Service Unavailable response from the api.

Attached screenshots from postman for reference purposes.

[Justus (Ju)]

Same issues here.

[Shivam Ralli]

Same issues here, even though we are under 1 request every three seconds, we get rate limit exceeded

[LVM Tools]

Is there a way to get higher throughput access ??

[Juneleung Chan]

Same issue here. 429 rate exceeded when requesting with 5-mins waiting 

[Dominik Wolf]

Unfortunately same for me.

[arXiv API Discussion]

Hi all,

Thank you for reporting the issue. We see it and are working on it. Apologies for the inconvenience in the meantime.

A short summary,  we've already increased capacity, and there's one fix still needed, which we expect to roll out today.

A bit of context on what you might be seeing. There are two types of rate limiting in play:

1. The normal rate limiting you've seen for years.
2. A more recent system-level rate limit that returns "rate exceeded" as the response text.

The second one is what we're fixing. If you're seeing "rate exceeded," that's the issue we're addressing today.

We'll follow up once the fix is live. Thanks for your patience.

Thanks,

On Friday, June 5, 2026 at 10:11:06 AM UTC-4 Juneleung Chan wrote:

Same here, 429 with a 5-minute wait per request.

在2026年6月5日星期五 UTC+12 00:42:15<LVM Tools> 写道：

[arXiv API Discussion]

Hi all,

The api endpoint work is done, and it seems to be working normally again.

Thanks,