Live speech-to-speech translation via AI hooks

[M P]

Hi All,

I did comb through earlier posts on this subject, but there is nothing recent.  Given the pace  of how AI has advanced in key technologies, it seems live speech translation should be a possibility.

First, some background:
I have created a couple Moodle plugins (local and filter) that handle bidirectional translation of all course content and asynchronous student interactions, making mixed language cohorts possible.  The results are impressive and work on all text, and asynchronous audio and video.

The next step is to target live, synchronous course activities, such as a BBB lecture.  You may have seen Google's own demo of something like this in Google Meet last year: https://youtu.be/hyXqcsWOONo.

We have used AWS bedrock services for all our other AI translation, and AWS' recent Nova Sonic model seems a well-suited, cost-effective solution for live speech-to-speech translation.

All that to say, has anyone else had a look at this? I might have a go at creating a BBB plugin for something like this, but I wanted to test the waters with experienced devs to see how feasible this would be. Thoughts?

A small caveat to the above is that language support for any kind of translation is limited to "high-resource" languages.  I have tried to target a few African languages in the Niger-Congo phylum, but they are "low resource" languages, i.e. there is insufficient AI model content to make translation reliable enough for use.