May snapshot available, search without stemming

[Casey Meyer]

Hi all,

The May snapshot is available to download! It contains some awesome improvements:

1. Over the past two months we added 151M new references to works, which is a 7.61% increase compared to our previous count. Our new method finds references without a DOI by using the title, author, and publication year. This affects a lot of older references that do not have DOIs, but can affect new articles as well.
2. About 17.9M works received author updates, based on syncing all changes from Crossref. This is one of three improvements we're making to authors. The first is to constantly stay in sync with author changes in Crossref (done!), the second is to stay in sync with ORCID (working on it), and finally implement a self-service system for curating author profiles.
3. We added four new work types, reclassifying existing works: “preprint” (5.7M), “libguides” (1.8M), “review” (820k), and “supplementary-materials” (50k).
4. Finally, the DataCite ingest continues with a focus on datasets, with 1.07M datasets added from Cambridge Structural Database and 709k added from Harvard Dataverse.

And one change:

1. We removed "super system" institutions from institution lineage that occurs within works, institutions, and authors. These large systems such as University of California System affect group-bys in the API, often rising to the top of queries. The full list is here. We may add a "super system lineage" later on. But for now we removed these institutions from the lineage portion of results.

Quick API announcement: you can now search in a more precise manner by disabling stemming and the removal of stop words. If you want to search for something like "surgery" and not get "surgeries" too, you can do that with .no_stem added to title and abstract queries:

https://api.openalex.org/works?filter=display_name.search.no_stem:surgery
https://api.openalex.org/works?filter=title.search.no_stem:surgery
https://api.openalex.org/works?filter=abstract.search.no_stem:surgery
https://api.openalex.org/works?filter=title_and_abstract.search.no_stem:surgery

Thanks,

Casey

--

Casey Meyer, CTO

OurResearch: We build tools to make scholarly research more open, connected, and reusable—for everyone.