2nd Call for Participation: TREC 2021 Clinical Trials Track

[Kirk Roberts]

Update: topics and corpus now ready.  Please register soon if you're considering participating.

---------------------------------------------------
TREC 2021 Clinical Trials Track
---------------------------------------------------

Most clinical trials fail to meet their patient recruitment goal. NIH has estimated that 80% of clinical trials fail to meet their patient recruitment timeline and, more critically, many (or most) fail to recruit the minimum number of patients to power the study as originally anticipated. Efficient patient trial recruitment is thus one of the major barriers to medical research, both delaying trials and forcing others to terminate entirely.

An important solution to this problem is to utilize the vast amounts of patient data that is already available in the form of the electronic health record (EHR). EHRs maintain medical records for routine medical care, but their secondary use for research such as clinical trial recruitment is well-known (Hersh, 2007). This was part of the inspiration for the TREC Medical Records track (2011-2012) (Voorhees and Hersh, 2012). However, that track was ultimately discontinued due to the difficulty in obtaining an EHR dataset of sufficient size (due to privacy issues) to merit a reasonable evaluation. The TREC Clinical Trials track flips the trial-to-patients paradigm to a patient-to-trials paradigm to enable the evaluation of patient matching systems and the building of a test collection for clinical trial search. That is, the query/topic will be (synthetic) patient descriptions and the corpus will be a large set of clinical trial descriptions.

Participants of the track will be challenged with retrieving clinical trials from ClinicalTrials.gov, a required registry for clinical trials in the United States. Clinical trial descriptions can be quite long, but the core aspect of the trial description are the inclusion/exclusion criteria. These are not all-inclusive statements about the trial to the point that other trial information can be ignored, but they are key aspects to defining trial eligibility. The topics present a lengthy (5-10 sentence) patient case description that simulates an admission statement in an EHR. The evaluation will further be broken down into eligible, excludes, and not relevant to allow retrieval methods to distinguish between patients that do not have sufficient information to qualify for the trial (not relevant) and those that are explicitly excluded (excludes). The topics are limited to just the free text description of a patient record, as the structured data in EHRs, while helpful, is more routinely used for clinical trial matching and therefore better-studied.

--- Task Description ---
* Topics/Queries: Synthetic patient EHR admission notes
* Data Collection: ClinicalTrials.gov
* Website: http://www.trec-cds.org/2021.html
* Mailing List: http://groups.google.com/d/forum/trec-cds

--- Important Dates (Tentative) ---
April 2021: Document collection available for download  

May 2021: TREC Registration: https://ir.nist.gov/trecsubmit.open/application.html
May 2021: Topics available for download
August 18, 2021: Results submission deadline
October 2021: Relevance judgements and individual evaluation scores released
Late October 2021: Initial system description papers due
November 17-19, 2021: TREC 2021 conference (at NIST and/or virtual)

--- Organizing Committee ---
Kirk Roberts (UTHealth)
William Hersh (OHSU)
Dina Demner-Fushman (NLM)
Ellen Voorhees (NIST)