1. Download the code from here:
https://github.com/TREMA-UNH/car-convert-ranking-to-ordering
2. Download, don't unpack! place as-is in working directory (or
set the path on the command line)
http://trec-car.cs.unh.edu/datareleases/v2.3/paragraph_ids.txt.xz
3. Download and unpack, then identify the file called benchmarkY3test.cbor-outlines.cbor
http://trec-car.cs.unh.edu/datareleases/v2.3/benchmarkY3test.public.tar.gz
4. Install python requirement with
python3 setup.py install
You can avoid using sudo through the use of anaconda, or you can also install it into user space with this command
python3 setup.py install --user
python3 y3_validate_submission.py --json-file ${submission} --submission-check-y3 --outline-cbor benchmarkY3test.cbor-outlines.cbor
Cheers,
Laura