Minor update to arxiv api on Tuesday.

77 views

Skip to first unread message

Brian Maltzan

unread,

Nov 7, 2025, 5:31:41 PMNov 7

to a...@arxiv.org

Short Summary:

We are replacing the software and hardware for the arXiv api:
https://export.arxiv.org/api/query

The replacement does not change the url or parameters or how it works, so there is nothing you need to do, to continue using it.

Upgrade Date and Time:
Tuesday, 2025-11-11 8AM ET, and should be only a few seconds.

API Documentation:
https://info.arxiv.org/help/api/user-manual.html

Some Details:

We are moving this service to the cloud, and standardizing technologies.
At some point in the future, we'll look into new features, but our plan for this migration is not to change the functionality.

You may notice an increase in errors returned. The current api does not always validate the input, and sometimes will return an http status of 200 with 0 search results. Sometimes there really are no results. Sometimes the input is invalid, and the error message is suppressed. For example, unbalanced parens:
curl -is https://export.arxiv.org/api/query\\?search_query\\=(ti:space

Because the search_query parameter is not validated against the api as we have defined it, users have gradually found alternative syntaxes that work. Here are some examples of syntax that works, but is not defined in our documentation:
ti:(electron proton)
apple fruit

The correct syntax for these are:
(ti:electron OR ti:proton)
all:apple OR all:fruit

We will start validating input. In many cases, we will be able to adjust your query, to match the api. You can see how we adjusted your input in the new api's search results, like this:
curl -si http://localhost:8080/api/query\\?search_query\\=pineapple | gi '<title' | grep arXiv
...
<title>arXiv Query: search_query=all:pineapple&id_list=&start=0&max_results=10</title>

So in this case, we adjusted pineapple, to all:pineapple.

You can disable our modification to your search_query input, by adding a parameter, raw=1:
curl -is http://localhost:8080/api/query\\?search_query\\=pineapple\\&raw\\=1 | g
rep summary
<summary>Invalid query string: 'pineapple'</summary>

Here are some examples that we won't correct, and will now return an error:
all:(title:(apple))
au:
ti:(fruit AND AND AND pineapple)

Nesting the all and title prefixes is ambiguous, and we will not make changes. The author search is missing content to find. The title search has too many operators. The last two patterns look like bugs in user’s automated searching, maybe from using input from a database, to construct queries for the arXiv api.

We took 2 weeks of web logs, adjusted the search_query, and replayed the query using the new search. About 1 in 1000 return an error. Many are these returned errors are helpful, in that they were previously errors that returned success with 0 results. And others are the nested syntax, field:(field:(value)), that you will need to adjust.

There are also some minor differences in the search results output:

Some xml elements include a namespace. For example, the feed.entry.author.affiliation element currently includes the namespace: <arxiv:affiliation xmlns:arxiv="http://arxiv.org/schemas/atom">. Namespaces may move, and be defined in the root xml element “feed”, or may stay in xml elements.
The xml content will not be word wrapped at 80 characters. This is easy to notice in the text of the abstract field. Basically, text fields will have fewer carriage returns.