API can find papers in separate queries, but not together

[Robert C]

I ran into a very weird case when using the API.

When I do a query for PMIDs 37090539 and 38293224 as a combined query, the API does not return any hits. However, when I query them separately, the API is able to find the papers.

Combined query: https://www.ebi.ac.uk/europepmc/webservices/rest/search?query=EXT_ID:(37090539 OR 38293224)+SRC:MED&format=json&resultType=core&pageSize=100

Separate queries:

https://www.ebi.ac.uk/europepmc/webservices/rest/search?query=EXT_ID:(37090539)+SRC:MED&format=json&resultType=core&pageSize=100

https://www.ebi.ac.uk/europepmc/webservices/rest/search?query=EXT_ID:(38293224)+SRC:MED&format=json&resultType=core&pageSize=100

It appears to only happen with these two papers. They also won't show up if I combine them in a query with other papers. Has anyone run into anything like this before?

[Robert C]

So it appears that both of these papers are pre-prints. I then tried other pre-prints and had the same problem. Was this done on purpose?

[Madhumiethaa Jayaprabha Palanisamy]

Hi,

Thank you for your patience, and apologies for the delayed response.
Yes, this behavior is related to preprint articles. The reason the combined query for PMIDs 37090539 and 38293224 doesn't return results is that these preprints are excluded from Solr, our main search system. This is intentional to avoid duplication, as we already index preprints via Crossref.  As a result, these records don't appear in the standard search index.

To retrieve these preprints from PubMed, you need to include SRC:MED for each EXT_ID separately in the query. For example:

EXT_ID:(37090539) + SRC:MED

EXT_ID:(38293224) + SRC:MED

Unfortunately, you cannot combine these IDs in a single query with SRC:MED because each ID needs to be searched individually to pull the data from PubMed.

Let me know if you have any other questions or need further assistance!

Kind regargs,
Madhu

[Robert C]

Ahh, so I can still do a batch query using the DOIs and without SRC:MED!

I noticed that having DOIs with parantheses seem to disrupt the entire API batch query. Is there a way around this?

[Madhumiethaa Jayaprabha Palanisamy]

Yes, you can absolutely use the DOIs in the query. For example: DOI:(10.1101/2023.03.10.23287127 OR 10.1101/2024.01.17.575414).
Could you please share the DOI query you're using to check the issue?

Kind regards,
Madhu

[Robert C]

https://www.ebi.ac.uk/europepmc/webservices/rest/search?query=DOI:(10.1016/S0140-6736(24)00476-8 OR 10.1016/j.xjtc.2024.08.005 OR 10.1016/j.xjon.2024.06.016 OR 10.1038/s41591-024-03402-y OR 10.3310/YJWQ4299 OR 10.1101/2023.12.26.573390 OR 10.1101/2024.03.18.585452 OR 10.1101/2024.10.18.24314709 OR 10.1101/2024.10.24.620152 OR 10.1101/2024.10.22.618529 OR 10.1007/978-3-031-50675-8_17 OR 10.1007/978-3-031-50675-8_10 OR 10.1101/2024.10.27.620437 OR 10.1101/2024.11.01.621414 OR 10.1101/2024.10.30.621013 OR 10.1101/2024.10.26.24316190)&format=json&resultType=core&pageSize=100

The above query returns no results. However, if I remove the first DOI (the one that has parantheses in it), the query returns 11 results.

[Michael Parkin]

Hi Robert,

I expect the parentheses in the DOI are causing the search some trouble here. You can enclose the DOIs in double quotes, e.g. DOI:"10.1016/S0140-6736(24)00476-8" and that should help.

Best wishes,

Michael

[Robert C]

Amazing! Thanks so much, Michael!