Using the AV Speech Dataset

Jayvardhan Rathi

unread,

Sep 5, 2019, 5:00:43 AM9/5/19

to avspeech-users

I wish to use the AV Speech dataset, and truncate the videos up till a certain length of time. Is there any way I can get a code for reference. Working on python 3.

Have the csv file which has 200k + youtube video ids, Need to know a simpler way to use all of that.

nabarun goswami

unread,

Sep 5, 2019, 5:23:09 AM9/5/19

to avspeech-users

Hi,

I have created a simple python script to download the videos(including audio) which downloads only the relevant part of the video according to the csv files.

You can find it here: https://github.com/naba89/AVSpeechDownloader

Best regards

Nabarun

Jayvardhan Rathi

unread,

Sep 5, 2019, 6:32:43 AM9/5/19

to avspeech-users

Hi, I ran the code and it could not download any of the videos. It is flagging an error for all the videos, none of them are getting downloaded

nabarun goswami

unread,

Sep 5, 2019, 6:34:16 AM9/5/19

to avspeech-users

What error are you getting?

Jayvardhan Rathi

unread,

Sep 5, 2019, 6:40:13 AM9/5/19

to avspeech-users

Screenshot (2).png

nabarun goswami

unread,

Sep 5, 2019, 6:45:30 AM9/5/19

to avspeech-users

Well, you can try removing the try...except blocks and see where the error is coming from. I have a feeling it might be due to write permissions or something like that. I can download it without any problems on Ubuntu 16.04 and 18.04. By the way, are you inside a proxy server or something like that? I haven't tested inside proxy.

nabarun goswami

unread,

Sep 5, 2019, 6:46:48 AM9/5/19

to avspeech-users

I have also tested on Windows 10 and was working fine.

Jayvardhan Rathi

unread,

Sep 5, 2019, 7:26:51 AM9/5/19

to avspeech-users

Tried without a vpn , still have the same error. What should I try removing in the code blocks

nabarun goswami

unread,

Sep 5, 2019, 7:31:09 AM9/5/19

to avspeech-users

Can you try removing the try...except part..

line 35 and 47-49. And also don't forget to adjust the indentation after removing those lines.

Do you have write permission on the folder you are downloading to?

Jayvardhan Rathi

unread,

Sep 5, 2019, 8:32:46 AM9/5/19

to avspeech-users

I also wanted to know if you had to use a gpu for downloading the dataset. How long does it take to download both the train and test dataset.

nabarun goswami

unread,

Sep 5, 2019, 8:41:56 AM9/5/19

to avspeech-users

You don't need a gpu for downloading. I am also currently downloading, it'll easily take a couple of days I think. It is a lot of videos. Did you manage to get the script working?

Jayvardhan Rathi

unread,

Sep 6, 2019, 3:23:04 AM9/6/19

to avspeech-users

I think the administrator has blocked downloading of youtube videos, is there any other way of using this dataset remotely or any alternate way to download it

nabarun goswami

unread,

Sep 6, 2019, 3:29:40 AM9/6/19

to avspeech-users

Oh, well, maybe you can ask your administrator to unblock it temporarily. If it's for research purpose I am sure they would not mind. I don't think there is any way to use it remotely. Another option might be subscribe for some cloud compute system and download and work on it there.

Jayvardhan Rathi

unread,

Sep 9, 2019, 3:41:07 AM9/9/19

to avspeech-users

Hey since you've downloaded the dataset, is there any way you can share the whole set with me?

Administrator will not unblock the network firewall, so I'm stuck and getting nowhere with my project.