GSoC 2015 - Audio

[Trang HO]

This thread is the main thread for the GSoC 2015 "Audio" project.
http://en.wiki.tatoeba.org/articles/show/gsoc_2015_ideas#audio

I will write in this first post as much information as possible about audio in Tatoeba.

If you are a student interested to work on this project for GSoC, please post your questions/introductions in this thread.

The current workflow for adding new audio

1. People contact us via te...@tatoeba.org to let us know they are interested in contributing audio[1].

2. Once we have validated that their audio is good, they record more audio and they send it to CK.

3. CK uploads them on the server (he only has SFTP access), then notifies us via te...@tatoeba.org that a new batch of audio has been uploaded.

4. Someone with SSH access (99% of the times it will be Alan) unzips the file that CK uploaded and runs the audio script[2]. The script takes care of moving the audio to the corresponding folder, and updates the database accordingly.

[1] http://en.wiki.tatoeba.org/articles/show/contribute-audio2
[2] https://github.com/Tatoeba/tatoeba2/blob/dev/docs/add_audio.sh

The way audio is handled in Tatoeba

* The database has a table `sentences` in which there is a field `hasaudio`.
* As mentioned in the doc[3], the field can have 3 values: no, shtooka or from_users. At this point, we don't use from_users.

* When hasaudio = shtooka, the interface displays the "audio available" icon. When hasaudio = no, it displays the "audio unavailable" icon.

* Audio files for sentences are stored in a folder that is divided into several subfolder: one for each language.
* Each audio file is named after the sentence's id and stored in the corresponding language folder.
* From the web, the files in this folder can be access via the subdomain audio.tatoeba.org. For instance, the audio for sentence #61, which is a Chinese sentence, is accessible via http://audio.tatoeba.org/sentences/cmn/61.mp3.

[3] https://github.com/Tatoeba/tatoeba2/blob/dev/docs/database/tables/sentences.sql#L16

The problems we have

Some of the problems were briefly mentioned in the wiki page[4] and on Kakul's Wall thread[5], but I will add some more things here.

1. It requires at least 2 people to be available to complete the task of making new audio available. As a result, it can take a while between the time a contributor submits their audio and the time it becomes available on Tatoeba. This can make people a bit frustrated.

2. Since it requires someone with SSH access to eventually make the audio available, it means it's more difficult to delegate the task of managing audio because we can't give SSH access to anyone.

3. Lately CK has started to create lists[6] in order to keep track of who has recorded what, when, and which license is attributed to the audio. All this information was previously not organized at all and was spread through all the emails sent by contributors. CK's page is "enough" for now but it's not sustainable. If we want to grow our audio content, we need to make this information part of the system, rather than keeping track of it externally.

[4] http://en.wiki.tatoeba.org/articles/show/gsoc_2015_ideas#audio
[5] http://tatoeba.org/eng/wall/show_message/21921#message_21921
[6] http://a4esl.com/temporary/tatoeba/lists/audio.html