500s from Zotero API for certain bibtex entries

[John Lees-Miller]

Hello,

This is John from Overleaf. One of our users reported getting an error from the Overleaf-Zotero integration, and we tracked it down to the Zotero API returning a 500 when certain entries in the user's library were included in the export.

This was from a request like:

https://api.zotero.org/users/$USER_ID/items?v=3&key=$API_TOKEN&format=bibtex&limit=100&start=200 

Since this is a public group, I'm not sure if I should post the user ID, but I'm happy to provide that privately if it would be helpful. (Or you may have these 500s in your logs already; the timestamps would be from Oct 27 at ~10:25 UK time.)

If there's anything else you'd like to know, happy to provide more information if I can.

Best,

John

--

Dr John Lees-Miller

Cofounder, CTO

Overleaf

[Dan Stillman]

On 10/28/20 6:22 AM, 'John Lees-Miller' via zotero-dev wrote:
> One of our users reported getting an error from the Overleaf-Zotero
> integration, and we tracked it down to the Zotero API returning a 500
> when certain entries in the user's library were included in the export.
>
> This was from a request like:
>
> https://api.zotero.org/users/$USER_ID/items?v=3&key=$API_TOKEN&format=bibtex&limit=100&start=200
>

If you're exporting to BibTeX (or most other export formats), you'll
want to use /items/top, not /items.

Basically, this person has many child attachments with large embedded
notes (with extracted ZotFile annotations), and if you try to generate
BibTeX for 100 items at a time, a large batch of those get sent to the
translation server, and the request ends up being too large for AWS
Lambda and rejected internally with a 413, which isn't currently handled
and results in an external 500. We don't really have a great way of
fixing that, other than returning a clearer error saying that you should
use a smaller batch size. And you probably should, because parent items
themselves could potentially have large abstracts, but the main thing
here would be using /top.

When you don't use /top, these items are functionally standalone
attachments, which aren't supported by most export formats anyway, so
there's no reason to include them. (If you try to export a standalone
attachment to BibTeX in the client, you'll get an empty file.)

(Note that the web API doesn't support the "Export Notes" and "Export
Files" options for translators that exist in the client, so it's not
possible to get child notes in the "annote" field in BibTeX export.)

[John Lees-Miller]

Hi Dan,

Great, thanks for this helpful info. I will pass it on to the team.

Best,
John

--
You received this message because you are subscribed to a topic in the Google Groups "zotero-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/zotero-dev/GICslY_OLyU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to zotero-dev+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/zotero-dev/010001756ee18f2d-2c15f75c-6f55-4eea-92c7-c06f7ccac733-000000%40email.amazonses.com.