Bulk search queries & documentation

[Andrew Green]

Hello,

I've been working on some of our pipelines at RNAcentral, that use the API to find mentions of ncRNAs. The first step in this is using the search API to search for every RNA identifier that needs updating. This is a _lot_ of searches.

Is there a bulk search functionality we could use? We currently have to hit the API pretty hard, which I don't like. An endpoint where we could search for a batch of queries and get an idlist result for each would be great. Alternatively, since this is internal to EBI, if there's a better way to do this, could someone give me a hand figuring it out?

Aside from that, I'd like to request a bit of documentation - I think I can remove more API calls by parsing the PMCLiteMetadata, but it would be super helpful to have a small example of the xml files contained in the tarball. Could that be added to the FTP alongside those? I know its the lite format, but there also isn't actually an example of that that I could find, nor does the current documentation page obviously cover what is in it.

Thanks!

Andrew

[Madhumiethaa Jayaprabha Palanisamy]

Hi Andrew,

Thanks for reaching out.

For the lite metadata format, you can refer to the Web Service Reference Guide here: (pages 18-29)
https://europepmc.org/docs/EBI_Europe_PMC_Web_Service_Reference.pdf
It's also linked from our Articles RESTful API page as Web Service Reference Guide

That said, a few fields may not always be fully up to date in the document. In practice, the most reliable way to understand the current structure is to query the Search API directly (which returns lite by default) and inspect the response fields. I have also attached an example XML file content of PMCLiteMetadata archive.

Regarding your pipeline, it would be helpful if you could share examples of your current search queries to better understand your use case and advise on the right api and optimisations.

Thanks,
Madhu