Is there any baseline for our reference?

[Albert ZHU]

Dear organizers,

Thanks for this grand challenge.

OpenNetLab’s website is not available at this time. So, I wonder is there some updates about the Evaluation Criteria?

What's more, could you please tell me if there is some evaluation baseline for our reference?

Thank you!

Best regards,

Albert

[Zhixiong]

Hi Albert,

We will use VMAF and DNSMOS to do video and audio metrics. We also consider some delays to measure if your model can deliver the frames in time. 

[OpenNetLab Challenge]

Dear Albert,

    We have an challenge environment (https://github.com/OpenNetLab/Challenge-Environment) that includes the evaluation system. Actually, we will have two evaluation system, online and offline.

    The online system will just provide a crude score which will not be used as rank but used to tell the contestant the submittal is accepted to our system. This part have been published in the github repository.

    The offline system will be used as rank, it's almost same as the online evaluation system but will add more testcases and complex frame alignment algorithm. It will also be published in this repository about the end of May.

      I hope my answer can help you. If you have more concerns or question, please don't hesitated to contact us.

Thanks,

Ze