How often to call export.arxiv.org/api/query so-as to not get timedout?

[Noon Silk]

Hi,

So I've written a little script to search for individual titles and
then look at the categories. I have this running on ~1k entries. My
code previously waited 3 seconds between calls (which perhaps I read
on the api site, but I can't find anything about it now) as a suitable
period over which to spread calls so-as to not overload the API. But
this doesn't seem to have done the trick (indeed, I'm currently timed
out from my current location).

Is there any advice on how to politely call the API repetitively? Or
am I forced to deal with the requests in bulk?

--
Noon

[Thorsten]

There is a Note in: http://arxiv.org/help/api/user-manual

"In cases where the API needs to be called multiple times in a row, we encourage you to play nice and incorporate a 3 second delay in your code. The detailed examples below illustrate how to do this in a variety of languages."

You should avoid lots of rapid fire requests and either space them out accordingly or change the queries to provide the desired results in bulk.

A timeout is different from access denied (HTTP status 403). What is the error you see?

Cheers
Thorsten

[Noon Silk]

Hi Thorsten,

  Thanks for the response.

  This link - http://export.arxiv.org/api/query?search_query=ti:electron&start=0&max_results=1 - is timing out (i.e. loading in the browser for 30 seconds, and the browser informs me of a connection timeout)  from both the networks I am able to try it on.

  I'll try and convert the code into one that uses the bulk system, but I did think that if I obeyed the 3-second rule it would be okay. This url - http://export.arxiv.org/oai2 - also times out (it perhaps goes without saying that I *can* access the internet generally, and the arxiv site proper.)
 

--

Noon

--
You received this message because you are subscribed to the Google Groups "arXiv api" group.
To unsubscribe from this group and stop receiving emails from it, send an email to arxiv-api+...@googlegroups.com.
To post to this group, send email to arxi...@googlegroups.com.
Visit this group at http://groups.google.com/group/arxiv-api?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Noon Silk

Fancy a quantum lunch? https://sites.google.com/site/quantumlunch/

"Every morning when I wake up, I experience an exquisite joy — the joy
of being this signature."

[Simeon Warner]

Hi Noon,

I see timeouts from neither of these URLs. As Thorsten wrote, if you see
a 403 (denied response) from export.arxiv.org please contact arXiv
admins (http://arxiv.org/help/contact) stating the IP address you are
coming from.

Cheers,
Simeon

> <mailto:arxiv-api%2Bunsu...@googlegroups.com>.

> To post to this group, send email to arxi...@googlegroups.com

> <mailto:arxi...@googlegroups.com>.

> Visit this group at http://groups.google.com/group/arxiv-api?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
>
>
> --
> Noon Silk
>
> Fancy a quantum lunch? https://sites.google.com/site/quantumlunch/
>

> "Every morning when I wake up, I experience an exquisite joy � the joy
> of being this signature."
>