Limit of requests

[francesc...@unitn.it]

Hi everyone,

I'm trying to retrieve UniParc entries by UniParc Proteomes ID using Python, requests and asyncio. For example, when retrieving entries from UPID UP000025660, I can make only 33 requests (size 100), and when I try to make the 34th request, I get a HTTP 500 Server Error. Is there any kind of limitation of requests per minute?

The code I use is the following: https://pastebin.com/Wx1u1hb8

Thanks

[G Qi]

Hello there.

Currently we don't limit the requests but your particular query is causing our server very high CPU usage and that is why some of your requests failed.

Could you please limit your request under 20 each time to see works or not? Don't continue to try if not working. We are working a best way for you to get your data.

Thank you very much!

Guoying

[Francesco Beghini]

Hi Guoying,

thank you for the quick response. I now tried using size 10 and now it seems working. Do you any advice for gathering the data in the best way possible?

Thank you in advance.

Francesco

Francesco Beghini

Laboratory of Computational Metagenomics

University of Trento

--
You received this message because you are subscribed to the Google Groups "ebi-proteins-api" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ebi-proteins-api+unsubscribe@googlegroups.com.
To post to this group, send email to ebi-proteins-api@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/ebi-proteins-api/9c40a2e7-02a2-4892-b01b-3001caa4f01b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[G Qi]

Hi Francesco.

The small size of the page makes things worse. Can you please stop this kind of requests for the moments? 

We are trying a best way for you to retrieve the data?

Thanks.

Guoying

On Wednesday, 2 May 2018 12:23:27 UTC+1, Francesco Beghini wrote:

Hi Guoying,

thank you for the quick response. I now tried using size 10 and now it seems working. Do you any advice for gathering the data in the best way possible?

Thank you in advance.

Francesco

Francesco Beghini

Laboratory of Computational Metagenomics

University of Trento

On Wed, May 2, 2018 at 11:58 AM, G Qi <guoy...@gmail.com> wrote:

Hello there.

Currently we don't limit the requests but your particular query is causing our server very high CPU usage and that is why some of your requests failed.

Could you please limit your request under 20 each time to see works or not? Don't continue to try if not working. We are working a best way for you to get your data.

Thank you very much!

Guoying

On Wednesday, 2 May 2018 08:38:20 UTC+1, francesc...@unitn.it wrote:

Hi everyone,

I'm trying to retrieve UniParc entries by UniParc Proteomes ID using Python, requests and asyncio. For example, when retrieving entries from UPID UP000025660, I can make only 33 requests (size 100), and when I try to make the 34th request, I get a HTTP 500 Server Error. Is there any kind of limitation of requests per minute?

The code I use is the following: https://pastebin.com/Wx1u1hb8

Thanks

--
You received this message because you are subscribed to the Google Groups "ebi-proteins-api" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ebi-proteins-a...@googlegroups.com.
To post to this group, send email to ebi-prot...@googlegroups.com.

[G Qi]

Hi Francesco.

Can you please don't send any request with small page size? this completely bring our servers down.

I have tried to stream the data from the server. It works fine. You just using -1 for the size.

curl -X GET --header 'Accept:application/xml' http://ves-pg-c8:9090/proteins/api/uniparc/proteome/UP000025660?size=-1 >> UP000025660.xml

It took 1 hour to finish the 11.7G data download.

Please let us know how it goes.

Thank you very much!

Regards,

Guoying

[Francesco Beghini]

Hi Guoying,

I stopped sending requests more than 1 hour ago. 

I'll try using the -1 size.

Thank you so much

Francesco

Francesco Beghini

Laboratory of Computational Metagenomics

University of Trento

To unsubscribe from this group and stop receiving emails from it, send an email to ebi-proteins-api+unsubscribe@googlegroups.com.
To post to this group, send email to ebi-proteins-api@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/ebi-proteins-api/f1cb4ac8-3005-40f9-b48e-dc65512ba35d%40googlegroups.com.