ANNOUNCEMENT - Instructions and important dates for the test phase

[etagust]

Dear Participants,

The test phase is approaching soon!

To ensure that the decoder is not optimized for the test set, we will require the teams to use one of the decoders submitted during the validation phase of the challenge. Please ensure that the decoders are exactly the same, as we will use file hashes to test whether a decoder matches a previously uploaded one.

If there are any issues with the validation server, you can send a decoder to lth...@twitter.com before the deadline. In that case, please make sure the decoder runs locally in our docker environment.

The deadline for submitting decoders is April 14th, 23:59:59 PST.

Any decoders submitted after the deadline can not be used.

The test set will be released on April 15th (one day later).

You will have seven days (until April 22th) to submit a compressed version of the test set along with a previously submitted decoder.

Finally, by April 26th the participants are expected to submit factsheets describing their challenge submission.

We wish you luck for the final sprint!

The Organizers

[li.don...@zte.com.cn]

Hello,

     our decoder has four files (decode.py、Decoder、input.txt and input2.txt).

     decoder.py is a Script file.

     Decoder is encoder kernel.These two files will not change.

     However, These two files (input.txt and input2.txt) contain the picture information, such as the name and resolution of the picture. We have to change the file content according to the name and resolution of the images after the test set is released.

     I want to know if this is in line with the rules.

     Thank you!

Dongyang Li
在 2018年4月6日星期五 UTC+8下午4:57:33，etagust写道：

[chunlei...@gmail.com]

Hello, 

     In my opinion, metadata (such as image height and width) should be encoded into bit-stream, and the decoder can parse them from bit-stream. 

     In your method,  input.txt and input2.txt should be considered as part of encoded image files rather than the decoder. And the Images Size should contain the bytes of input.txt and input2.txt.

在 2018年4月8日星期日 UTC+8上午11:05:10，li.don...@zte.com.cn写道：

[li.don...@zte.com.cn]

Yes, we can make the files (input.txt) contain only the name of the picture.

 However, Because of the different names of pictures from validation set and test set,  we also have to update the content of the files (input.txt) in the test phase according to the name of pictures in the test data set.
在 2018年4月8日星期日 UTC+8上午11:35:47，chunlei...@gmail.com写道：

[George Toderici]

The spirit of this benchmark is to be accurate. We expect ALL INFORMATION NECESSARY FOR DECODING to be contained in each compressed file. The compressed file and the decoder by itself (here decoder may also mean model files) are not sufficient for successfully decompressing an image, then we're not doing compression!

To reiterate:

- all information necessary to decode a compressed an image should be contained in the compressed file

- no additional information may be stored in temporary files of any kind

The idea to store metadata in other files does not fit with the spirit of the contest, and I strongly suggest all participants to NOT DO THAT.

--
You received this message because you are subscribed to the Google Groups "Challenge on Learned Image Compression 2018" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clic-2018+unsubscribe@googlegroups.com.
To post to this group, send email to clic...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/clic-2018/3475a55a-6d59-4d56-9690-798b46b1c029%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Çağlar Aytekin]

Hi, 

We have a different situation then the one written here. Can you kindly read the description of our process below and check if the following is OK for submission?

In the decoder folder, we do not keep any additional files or metadata, only our decoder models and necessary python scripts to run our decoder.
In our test submission, decoder folder will NOT change in any way, nor we will use any additional folder (other than encoded folder) to send metadata of any sort. 
YET in the folder containing our encoded files (which will of course be changed with test set) we have three separate file for each image, each file necessary for our decoding process.
Moreover all our files (306 files for 102 images) are within the limits of the 4.72 MB encoded folder.
Another thing is DURING decoding we write and remove some temporary files, again these are not taken from any other folder we just generate them during decoding and remove afterwards.
In our opinion, this process does not contradict with the spirit of challenge or the idea of image compression
In the end there is one decoder fixed at device side and we send three files in order to decompress an image.
If you think this is still a problem, then we can simply put three files in a single zip and extract and use them during decoding process.
Please let us know if the current settings are OK with you,  so that we can submit a new submission if not.

BR,

Caglar

8 Nisan 2018 Pazar 09:08:46 UTC+3 tarihinde George Toderici yazdı:

To unsubscribe from this group and stop receiving emails from it, send an email to clic-2018+...@googlegroups.com.

To post to this group, send email to clic...@googlegroups.com.

[George Toderici]

I (personally) think having 3 files per image is OK since we can measure their size (i.e., we know where they are). I'll let Lucas chime in, because I want to make sure that our evaluation script handles this case.

To unsubscribe from this group and stop receiving emails from it, send an email to clic-2018+unsubscribe@googlegroups.com.

To post to this group, send email to clic...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/clic-2018/d12545b7-c189-47bc-aade-9469f1b88595%40googlegroups.com.

[Çağlar Aytekin]

Hi,

Thank you for the quick answer. We have submitted one with these settings yesterday and it seems like it worked.

BR,

Caglar

8 Nisan 2018 Pazar 10:51:23 UTC+3 tarihinde George Toderici yazdı:

[li.don...@zte.com.cn]

OK,Thank you!

I think there are some misunderstandings in it.

we must follow the rules of the competition!

We use resolution to transform decoded yuv files into png files，and our compressed file contain the resolution information. So input.txt file is not necessary.

Of course, we can get all necessary information from the compressed files.

However, I have a question about Intermediate data of decoder.

Our decoder will decode yuv file from compressed file firstly. Then, Decoder convert the yuv file to png file.

These yuv files come from decoder and compressed files. But these YUV files belong to the intermediate data.

So,I want to know whether this is in line with the rules of the competition.

在 2018年4月8日星期日 UTC+8下午2:08:46，George Toderici写道：

To unsubscribe from this group and stop receiving emails from it, send an email to clic-2018+...@googlegroups.com.

To post to this group, send email to clic...@googlegroups.com.

[ltheis]

Your decoder may generate any number of intermediate files. These files will not be counted towards the size of the compressed images.

Your compressed images may be stored in any number of files. It could be a single file storing all images or multiple files per image.

Importantly, when you make your submission, 

docker run [...] clic2018/submit [...] decoder.zip file1 file2 [...] fileN

you should not have to change decoder.zip if the images change! If input.txt is different for the training and test set, then it cannot be part of decoder.zip but should be part of the other files that follow it. It will then also count towards the total file size. N can be different from the number of PNGs in the validation or test set. The size of the submission will be the total size of the N files. The decoder or any files generated by the decoder will not be counted towards the size.

P.S.: I don't think this will be an issue in this competition, but please try to avoid storing any significant information (e.g., resolution) in the file names or the number of files itself.

[li.don...@zte.com.cn]

OK, thank you~

在 2018年4月8日星期日 UTC+8下午6:59:37，ltheis写道：

[Çağlar Aytekin]

Hi,

A quick question. Let us say that we have submitted 5 decoders so far with our team name. Are we allowed to use any one of these decoders in the test set or only the one that gave best validation score? I mean do you keep all the submitted decoders from each time or do you only store the best one so far?

BR,

Caglar

6 Nisan 2018 Cuma 11:57:33 UTC+3 tarihinde etagust yazdı:

[ltheis]

You can use any of the 5 successfully submitted decoders during the test phase. While we do not store all decoders, we store their file hashes.

[ltheis]

One clarification:

When you submit your decoder to me via email, it will not show up on the leaderboard for the validation set. I.e., I won't be able to run it. However, it will be available to you during the test phase.

Please make extra sure that it works locally and uses reasonable amounts of memory :).