Re: AVA Active Speaker: video FPS

[Sourish Chaudhuri]

Hi Baptiste,

                   Glad that you find this dataset useful!

                  You're correct that the videos in the dataset are not all at the same FPS. The best way to use this is a bit of an open question, but for all the experiments we did when we released the dataset and our initial benchmark models (detailed in this paper), we resampled all the video to 20 FPS. However, we did provide labels at the original frame rate of the videos, so experiments and learnings on different ways of normalizing this aspect are welcome.

                  You mentioned ensuring "sync between audio and video", which should be less of an issue once you've made a decision on how you approach the video FPS difference, since whatever rates you resample the video and audio to, the timestamps should enable accurate sync-ing between the audio and video.

                 We look forward to seeing more of your work on this! And of course, please reach out with any questions!

Thanks,

 Sourish

On Mon, Jun 22, 2020 at 3:02 AM Baptiste Pouthier <baptiste...@gmail.com> wrote:

Hello, thank you for this great dataset!

Unless i downloaded the dataset in a wrong way, i observed not all video have the same FPS (see attached .png).

I'm currently pre-processing the data and i wonder how to ensure sync between audio and video regarding this fact. 

For the moment, i'm considering the following options:

• resample all videos to the same FPS (cons: loss of quality)
• use only the labeled frames (31 labels per second of videos) as features (cons1: time intervals between consecutive labels are not constant, so if i want to use less than 31 consecutive frames i get inconsistent audio samples length... and i cannot keep sync between audio and video. cons2: 31fps is not a standard fps)

I am missing something? Is there a standard way to deal with this problem?

Thank you,

Baptiste Pouthier

--
You received this message because you are subscribed to the Google Groups "AVA Dataset Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ava-dataset-us...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ava-dataset-users/690c03ec-8240-40b0-a528-522f960482dao%40googlegroups.com.