Is it normal still stuck at 'decoding images' even when my submission result has appeared?

[不爱吃猫的鱼]

Is it normal  for the submit client still stuck at 'decoding images'  even when my submission result has appeared on the Leaderboard?

[ltheis]

Yes. There is an issue with keeping the connection alive for long enough. Unfortunately I currently don't have the time to fix it, but your decoder will be evaluated correctly anyway.

You can try to submit your decoder again to check if it's still running. If there's already an evaluation with your team name going on, you should get a message from the server saying that.

[okirm...@ku.edu.tr]

Hi,

Can you check if my submission is still running? I tried resubmitting again to check if it continues but all I see is connection reset by peer. My team name is MVGL. Thank you.

[li.don...@zte.com.cn]

Having the same problem for 24 hours....

在 2018年4月12日星期四 UTC+8下午3:34:40，okirm...@ku.edu.tr写道：

[ltheis]

Your decoder is running for 43 hours now @MVGL. How long does it take locally for you? This is approaching the limits of where it is practical to run the decoder on the test set, which will be larger than the validation set.

[okirm...@ku.edu.tr]

It takes around 23-24 minutes for a single image on my Intel i7-3630QM machine. Is this too much?

[ltheis]

Hi,

If it takes about 2x as long on the server, or 50 minutes per image, that would mean ~10 days to decode 300 images (approximate size of test set). This is okay in principle, but it seems impractical given that we will have to run a larger number of decoders and will also need time for the human observer evaluations.

I will stop the decoder so that you can submit another one. Up to 5 minutes per image on your i7 seems reasonable, which will probably still mean ~2 days to decode the test set on the server.

[chunlei...@gmail.com]

Hi,

    Can you tell us the maximum resolution of the image in test set? Whether is it larger than the one of validation set?

在 2018年4月13日星期五 UTC+8下午5:33:00，ltheis写道：

[Lucas Theis]

Great question. All images of both sets are downsampled to be below 2048x2048.

--
You received this message because you are subscribed to the Google Groups "Challenge on Learned Image Compression 2018" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clic-2018+...@googlegroups.com.
To post to this group, send email to clic...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/clic-2018/09a73455-dea3-4acb-b2e8-d21a1728f8d6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[okirm...@ku.edu.tr]

Hi,

We understand that the challenge is organized for the first time therefore some troubles are to be expected. However, although we knew that the decoder would work on CPUs, we did not really give a thought on decoding time since we were told there would not be a limitation on decoding time. We act accordingly and our network is not too deep for this kind of application. Enforcing a constraint now puts us at a really bad place because training a neural net takes time. 

We will be making a submission via email if we can make the deadline.

[ltheis]

Thank you for understanding. But we actually did state from the beginning that we expect the decoders to use reasonable amounts of resources, including computation time. From the website:

"We will impose reasonable limitations for compute and memory of the decoder executable."

We tried not to enforce hard limits where possible to allow some flexibility, but perhaps it would be better for everyone to be precise in future competitions. Please note that the resources we allow are already well above what would be considered practical.