Regarding submission for perceptual metrics track

[simone sun]

Dear Organizers,

I have some questions about the submission of perceptual metrics track:

1) When testing, do we need to submit the selection result for each pair (i.e., 0 or 1), or do we need to submit an absolute score for each distorted image? 

2) Do we need to submit code?

3) Is the release time of the test set the same as other tracks?

Thanks!

[George Toderici]

Hi Simone,

1) For now we want just binary decisions because that's what we advertised in the beginning. Likely for next year, we'll want a contiguous score. 

We believe that even with binary decisions, the metric used in making that decision might be good enough to be used in a training setup, which is really what we want to encourage here (i.e., have researchers experiment with what metrics work in a wider enough range of distortions so that hopefully they'lll be able to use it to train a compression algorithm for next year's challenge).

2) We're trying to make it easier this year, so no code needed. Next year we'll require code.

3) The paper deadline is the same. We will release the files for the test phase once we have done all the human ratings, which means that we'll *necessarily* release this data in ~1 month after the compression challenge is over (hopefully sooner). We'll give an opportunity to all contestants to submit an updated paper that contains the final results.

  

George

--
You received this message because you are subscribed to the Google Groups "Workshop and Challenge on Learned Image Compression (CLIC)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clic-list+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/clic-list/c26d0048-9957-4207-91c7-d23759f00fd4n%40googlegroups.com.

[simone sun]

Thanks for your prompt reply, it is of great help to us!