Example of "Evaluation Method" in COMIC ?

[matt.c...@gmail.com]

Hi,

would anyone have a simple example showing how to set up an evaluation within COMIC?

This idea to automatically generate this from the user submissions: https://camelyon17.grand-challenge.org/evaluation/results/

I have not found anything in the Grand Challenge doc (https://grand-challenge.org/support).

From the evaluation/methods of our challenge, and the github doc (https://github.com/comic/grand-challenge.org/blob/master/docs/evaluation.rst#id1), I understand:

- we need to create a "container image" with Docker

- this "image" must read user inputs, evaluate them with respect to a reference standard,and generates a file /output/metrics.json with the data to be displayed in the Results tab of the challenge

- this image will be automatically executed after each new submission

Would anyone (especially the COMIC developers?) could provide an example of a "container image"

- how to create it?

- a simple (Python?) file to read an input and compute a score

- how to export the metrics.json file

Thanks for your help

Matt

[james...@gmail.com]

Please take a look at our evautils library which can help you get this set up, the getting started documentation should help.

https://github.com/comic/evalutils

Any other queries please contact sup...@grand-challenge.org

[matt.c...@gmail.com]

Sounds exactly what I am looking for, thanks!

Maybe adding this info to the main doc (https://grand-challenge.org/support) could be relevant?

Best,

Matt

[James Bibi]

Thanks for the good ideas to bring it. I know a lot more. :)ufabet :J

[ufabet...@gmail.com]

เมื่อ วันเสาร์ที่ 30 มิถุนายน ค.ศ. 2018 3 นาฬิกา 53 นาที 14 วินาที UTC+7, matt.c...@gmail.com เขียนว่า:

[patipon garena]

thank you  ufabet