CrisisFACTS Automatic Evaluation Code Available and ISCRAM Reminder

24 views

Skip to first unread message

Cody Buntain

unread,

Jan 9, 2023, 11:57:46 AM1/9/23

to TREC-IS

Good morning/afternoon, everyone! I've just returned for the winter break, and we've made the code for automatic evaluation for CrisisFACTS available on our GitHub repository here:

https://github.com/crisisfacts/utilities/tree/main/03-Evaluation/01-AutoEvaluation

You should be able to run both ROUGE and BERTScore evaluations against Wikipedia-, ICS 209-, and CrisisFACTS-facts-based summaries using the provided code. We are working to make the matching-based evaluation available as soon as possible!

Also, as a reminder, ISCRAM's deadline has been extended to 15 January. We hope to see you there!