vote: Clinical TempEval 2018?

109 views
Skip to first unread message

clinical-tempeval

unread,
Apr 7, 2017, 4:38:23 PM4/7/17
to clinical-tempeval
We were considering possibilities for Clinical TempEval 2018. Here are the current options:
  1. Don't run a Clinical TempEval in 2018. The 2017 CodaLab site would remain up, so people would still be able to evaluate new systems on the 2017 test data.
  2. Do another domain adaptation shared task, this time with a different institution. We're not yet sure if the SemEval organizers will allow this (they have a new rule that the "same task" may not run for more than 3 years, and we don't know if this counts as "the same" or not.) But assuming it is allowed, it would be something like:
    • Train on Mayo data (colon cancer + brain cancer)
    • Test on UPMC data (breast cancer)
  3. Do a time normalization task. The THYME data is currently being annotated with the annotation scheme proposed here: http://www.lrec-conf.org/proceedings/lrec2016/pdf/288_Paper.pdf. Essentially, it turns time normalization into a tagging + linking task; once the tags and links are assigned correctly, the normalized time intervals can be inferred deterministically. The THYME colon train data is almost complete; the test data would be finished well before January 2018 when it would be needed.
Which option(s) would you prefer? If you have some other idea of what we should do in 2018, please share that too!

Thanks,

The Clinical TempEval Organizers

sm3...@georgetown.edu

unread,
Apr 8, 2017, 5:22:03 PM4/8/17
to clinical-tempeval, clinical...@googlegroups.com
We like 2 and/or 3.
Message has been deleted

André Lamúrias

unread,
Apr 12, 2017, 5:14:47 AM4/12/17
to clinical-tempeval
We also agree that options 2 and 3 would be better, and we also suggest an entity linking task of events to UMLS. Perhaps it could be another subtask if option 3 is chosen? 

Best regards,

André Lamúrias

--
You received this message because you are subscribed to the Google Groups "clinical-tempeval" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clinical-tempeval+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply all
Reply to author
Forward
0 new messages