You ask, I Deliver: Language switching via Oral Command and a Promise of custom commands to come in the Text-to-Speech Plugin!

[Finn Lancaster]

Text-to-speech-in-TW5 Version 1.0.4 is here!

As requested, I have implemented language switching to the following languages:

• "african
• "indonesian"
• "malaysian",
• catalonian"
• "spanish"
• "basque"
• "french"
• "croatian"
• "icelandic"
• "italian"
• "hungarian"
• "dutch"
• "norwegian"
• "polish",
• "portuguese",
• "romanian",
• "slavic"
• "finnish",
• "swedish",
• "turkish",
• "bulgarian",
• "russian"
• "serbian"
• "korean"
• "chinese"
• "japanese"
• "latin"

But Wait ... It's Even Better than That!

v1.0.4 now makes this language switching EVEN EASIER! All you have to do is tell the plugin to 

Change language to "X". 

Also, please wait a few seconds after saying this, the plugin can take 1-3 seconds to register the command

All out loud! That's right, just click the microphone button added by the plugin, and tell it to switch the language using the syntax given above!

Install

The homepage of the plugin has been moved to speech-to-text.finnsoftware.net, so the old domain will NO LONGER WORK. From there, you can do the necessary install via drag-and-drop!

Coming Soon ...

- (me and BTC): custom commands via TW input fields. Just type out the syntax of your "new" command and the TW action it should take, and BOOM! it happens.

- (BTC) Better TW look for v1.0.4 in next update, BTC will be handling making the new language change feature "look better." Right now, everything is sort of spliced together in terms of UI since the latest update. Expect some changes such as notifications of when command completed, as well as the transcript to be discarded if it is a command.

- and any other amazing ideas you people have!

[BurningTreeC]

Hi @Finn,

you should change "dutch" to "german" :)

[BurningTreeC]

It's worth mentioning that one needs to say: Command change language to <<your language>>

[PMario]

On Wednesday, August 11, 2021 at 8:41:12 AM UTC+2 BurningTreeC wrote:

Hi @Finn,

you should change "dutch" to "german" :)

 

Hi,

IMO it's not necessary to remove something, if it works but you should definitely add German.

The TW full edition https://tiddlywiki.com/editions/full/ shows all the text translations we have. ... Not all of them are complete ... but anyway.

-mario

[R²]

Hi Franc,

Awesome stuff!

Just a bit worried about the exact commands though. African and Slavic aren't languages so I wonder what they correspond to. Having putonghua or Mandarin would be more appropriate than Chinese as Chinese is a disparate group of languages — and it's worth mentioning that Mandarin doesn't account for a whopping 400 million speakers of languages from that group. Malaysian is a nationality, Malay being the language you're looking for, and finally Catalan should be used instead of Catalonian.

Regarding the selection, I do hope this is also easily modifiable as a lot of languages supported by the API are missing, including some of the "big" ones. Not only German, as already mentioned, but Hindi, Tamil, Kannada (of Bangalore's fame)… I feel TW's existing text translations shouldn't be the limit here. I almost never apply localizations to my programs and just because my email interface is in English for example doesn't mean I'm not going to want to write my emails in the languages I happen to be able to write. I also want to point out that the STT capabilities for some of the "minor" languages are incredible and this tool would be extremely empowering for their typically neglected users.

I assume that accent can be specified manually in the settings and there is less sense in being able to switch accents on the fly (though some use cases could require that too I imagine) but it bears reminding this is extremely important. Try the Nigerian accent for English with your standard accent and see how that works out for you. The differences are probably more extreme between the forms of French from Québec and France, among the many forms of Spanish (especially from the Cono de Oro as it was strongly influenced by Italian), between Swiss or Bavarian German and standard German…

Best,

R²

[TiddlyTweeter]

Ciao flanc ... I have lots to comment as this tool is fantastic! I'll start with a few general points ...

LANGUAGE CHANGE ISSUES 

1 - BTC already clarified "one needs to say: Command change language to <<your language>>" and that actually, at the moment, "dutch" starts "german." :-)

2 - An issue is that once you changed language you can't change it back!  Yes? WHY? Because, at the moment "command" gets translated from English. What do I mean? Well I say "Command change language to Italian". Works perfectly! BUT, once in Italian, saying "Command change language to English" does not work. The only way to get English back at the moment is to reload? Yes? 

FYI, in terms of my specific end-use I do need to frequently switch languages several times a day.

3 - The list of potential languages available under the API I don't know. I really would appreciate to have SLOVENIAN (I think the code is: sl-SI (that is s followed by "l" (as in "eel") and S followed by capital "I" (me))) . Slovenian will be useful for some end-users here in North-East Italy whom I work with. Just a request.

Best wishes

TT

[TiddlyTweeter]

Ciao Flanc... & BTC

SIDE NOTES ON API BEHAVIOR

This does not effect anything YOU two are doing. Merely observations on the "behaviours" of the recognitions. 

But I do think it is useful to understand the end-product the API issues forth.

1 - There are occasional QUIRKY things that happen that have no clear rationale. For instance if you dictate "cat dog mouse horse" you get back ... "cat dog Mouse horse". Maybe mice, or mouses, actually rule the universe as "The Hitchhiker's Guide To The Galaxy" contended"?

2 - Sometimes words that are not understood by the api are simply DISCARDED. (For instance in Italian mode most dictated English words are just ignored).

3 -  Sometimes words that are not understood by the api are capitalized as Proper Names. There is some very clever thing going on inside the recog. machine that decides when to do this.

None of these minor points change anything other than the need to be slightly aware the API does stuff and awareness of that is helpful I think.

I'll may add more comments about these behaviors after I used it more, if there any other obvious ones. 

Best wishes

TT

[BurningTreeC]

Hello all, I've now changed the Command command to Ok Wiki

I've made it also detect Okay Vicky, Ok WC, and more keywords ... If those of you who are testing this would report any other keywords we should add, that would be a great help

What should work right now is Ok Wiki change language to <<language>> and Ok Wiki stop talking

[Finn Lancaster]

Love the feedback! 

Something to do with the language names. Each of those language names were decided rather carefully, since, for example, catalan was understood in a different word every time, but the word "catalonian" was transcribed with perfect accuracy. One possible solution would be to test for a variety of different words that sound like the keyword, similar to what BTC did with the new "Ok Wiki" functionality, and that would require you awesome folks to test the words, and report back what they are transcribed as.

In addition to this, I forgot to add, there are multiple variants of several languages, such as english US, UK, Australian, South African; spanish Spain, etc. etc. I plan to add all of this functionality sometime soon, as well.

For all of the languages the API supports that my little wrapper program could easily be changed to modify, they are listed in this StackOverflow post: https://stackoverflow.com/a/23733589/14719982. Feel free to speak the translated language names into the plugin, and see what they are transcribed as.

(Since the default lang I have set is English-US, the language names must sadly be translated into English to be understood, such as Espanol to Spanish. This also accounts for maybe "improper" languages names, as pointed out.)

[Finn Lancaster]

New idea I would like to share: Because of spoken commands in languages other than English to
change the language again, I believe a better method would be to allow the plugin to get the current language of TW, and auto-default to that setting. When the wiki language was changed, the plugin could also change.

With this idea, English custom spoken commands would still be available.

Let me know what you think!

[R²]

Using TW's default language seems like a great idea. Also storing that value in wikitext would be even better, as anyone could easily change it without having to overriding any JS and therefore make it more user-friendly.

Interesting explanations… So "African" was simply "Afrikaans"! Makes sense, though I guess this wasn't exactly a happy choice to have a whole continent represented by a recent colonial language ;) I guess "OK wiki change language to A-F" (for 'af-ZA': Afrikaans) could be a more neutral and efficient way of covering all languages without running into recognition issues. You could also do accents that way too by the way ("OK wiki change E-N-A-U" → 'English, Australian').

The StackOverflow post you linked to is seven years old. Based on the Web Speech API demo at https://www.google.com/intl/en/chrome/demos/speech.html, the full list should in fact be the following:

[['Afrikaans',       ['af-ZA']],

 ['አማርኛ',           ['am-ET']], → Amharic

 ['Azərbaycanca',    ['az-AZ']],

 ['বাংলা',            ['bn-BD', 'বাংলাদেশ'], → Bengali, Bangladeshi

                     ['bn-IN', 'ভারত']], → Bengali, Indian

 ['Bahasa Indonesia',['id-ID']], → Indonesian

 ['Bahasa Melayu',   ['ms-MY']], → Malay

 ['Català',          ['ca-ES']],

 ['Čeština',         ['cs-CZ']], → Czech

 ['Dansk',           ['da-DK']],

 ['Deutsch',         ['de-DE']],

 ['English',         ['en-AU', 'Australia'],

                     ['en-CA', 'Canada'],

                     ['en-IN', 'India'],

                     ['en-KE', 'Kenya'],

                     ['en-TZ', 'Tanzania'],

                     ['en-GH', 'Ghana'],

                     ['en-NZ', 'New Zealand'],

                     ['en-NG', 'Nigeria'],

                     ['en-ZA', 'South Africa'],

                     ['en-PH', 'Philippines'],

                     ['en-GB', 'United Kingdom'],

                     ['en-US', 'United States']],

 ['Español',         ['es-AR', 'Argentina'],

                     ['es-BO', 'Bolivia'],

                     ['es-CL', 'Chile'],

                     ['es-CO', 'Colombia'],

                     ['es-CR', 'Costa Rica'],

                     ['es-EC', 'Ecuador'],

                     ['es-SV', 'El Salvador'],

                     ['es-ES', 'España'],

                     ['es-US', 'Estados Unidos'],

                     ['es-GT', 'Guatemala'],

                     ['es-HN', 'Honduras'],

                     ['es-MX', 'México'],

                     ['es-NI', 'Nicaragua'],

                     ['es-PA', 'Panamá'],

                     ['es-PY', 'Paraguay'],

                     ['es-PE', 'Perú'],

                     ['es-PR', 'Puerto Rico'],

                     ['es-DO', 'República Dominicana'],

                     ['es-UY', 'Uruguay'],

                     ['es-VE', 'Venezuela']],

 ['Euskara',         ['eu-ES']], → Basque

 ['Filipino',        ['fil-PH']],

 ['Français',        ['fr-FR']],

 ['Basa Jawa',       ['jv-ID']], → Javanese

 ['Galego',          ['gl-ES']], → Galician

 ['ગુજરાતી',           ['gu-IN']], → Gujarati

 ['Hrvatski',        ['hr-HR']], → Croatian

 ['IsiZulu',         ['zu-ZA']], → Zulu, South Africa

 ['Íslenska',        ['is-IS']], → Icelandic

 ['Italiano',        ['it-IT', 'Italia'],

                     ['it-CH', 'Svizzera']],

 ['ಕನ್ನಡ',             ['kn-IN']], → Kannada

 ['ភាសាខ្មែរ',          ['km-KH']], → Khmer

 ['Latviešu',        ['lv-LV']], → Latvian

 ['Lietuvių',        ['lt-LT']], → Lithuanian

 ['മലയാളം',          ['ml-IN']], → Malayam

 ['मराठी',             ['mr-IN']], → Marathi

 ['Magyar',          ['hu-HU']], → Hungarian

 ['ລາວ',              ['lo-LA']], → Lao

 ['Nederlands',      ['nl-NL']], → Dutch

 ['नेपाली भाषा',        ['ne-NP']], → Nepali

 ['Norsk bokmål',    ['nb-NO']],

 ['Polski',          ['pl-PL']],

 ['Português',       ['pt-BR', 'Brasil'],

                     ['pt-PT', 'Portugal']],

 ['Română',          ['ro-RO']],

 ['සිංහල',          ['si-LK']], → Sinhala

 ['Slovenščina',     ['sl-SI']], → Slovenian

 ['Basa Sunda',      ['su-ID']], → Sundanese

 ['Slovenčina',      ['sk-SK']],

 ['Suomi',           ['fi-FI']], → Finnish

 ['Svenska',         ['sv-SE']],

 ['Kiswahili',       ['sw-TZ', 'Tanzania'],

                     ['sw-KE', 'Kenya']],

 ['ქართული',       ['ka-GE']], → Georgian

 ['Հայերեն',          ['hy-AM']], → Armenian

 ['தமிழ்',            ['ta-IN', 'இந்தியா'], → Tamil, from India

                     ['ta-SG', 'சிங்கப்பூர்'], → Tamil, from Singapore

                     ['ta-LK', 'இலங்கை'], → Tamil, from Sri Lanka

                     ['ta-MY', 'மலேசியா']], → Tamil, from Malaysian

 ['తెలుగు',           ['te-IN']], → Telugu

 ['Tiếng Việt',      ['vi-VN']],

 ['Türkçe',          ['tr-TR']],

 ['اُردُو',            ['ur-PK', 'پاکستان'], → Urdu

                     ['ur-IN', 'بھارت']],

 ['Ελληνικά',         ['el-GR']], → Greek

 ['български',         ['bg-BG']], → Bulgarian

 ['Pусский',          ['ru-RU']], → Russian

 ['Српски',           ['sr-RS']], → Serbian

 ['Українська',        ['uk-UA']], → Ukrainian

 ['한국어',            ['ko-KR']], → Korean

 ['中文',             ['cmn-Hans-CN', '普通话 (中国大陆)'], → Mandarin, from China

                     ['cmn-Hans-HK', '普通话 (香港)'], → Mandarin, from HK

                     ['cmn-Hant-TW', '中文 (台灣)'], → Mandarin, from Taiwan

                     ['yue-Hant-HK', '粵語 (香港)']], Cantonese, from HK

 ['日本語',           ['ja-JP']], → Japanese

 ['हिन्दी',             ['hi-IN']], → Hindi

 ['ภาษาไทย',         ['th-TH']]]; → Thai

(Do please include Lao! The Lao STT is terrific!)

++

[strikke...@gmail.com]

Hi flanc,

I followed the link to Web Speech API Demonstration. Danish can be used, will it also be possible to use it using your plugin with Danish?

Birthe

[Finn Lancaster]

@Strikke…, yes. Any language shown on the demo is theoretically be supported by the plug-in. I’m not sure if there is a Danish translation of TW. With my new idea to just use the TW language, there may be some limitation. 

--
You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/3O0eT4K5mkI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/1059a0eb-2708-4946-b04f-84a6e320c4dcn%40googlegroups.com.

[R²]

Random idea: using the NATO phonetic alphabet "OK Wiki (Whiskey??) Lima [L for language switch] Charly Alfa [CA for Catalan]".

You wouldn't only win on the "cool" factor. According to the Wikipedia, The final choice of code words for the letters of the alphabet and for the digits was made after hundreds of thousands of comprehension tests involving 31 nationalities. The qualifying feature was the likelihood of a code word being understood in the context of others.

Biased towards human listening and probably also Western nationalities given the context, but could be interesting ;)

++

[ludwa6]

My €0.02 /worth from sunny Algarve PT: your plugin nailed my "Olá, como estás?" pretty darn close, except it put an H at the front, which makes it look Spanish to me. 

A bit wide of the mark tho with "Eu tenho muito saudades de ti," which it translated as "And you want to sell that to teeth". :-)

Still: i am amazed, but i guess active language selection is needed -especially if one wants to distinguish between Brazilian and European Portuguese, which distinction should be added, i would say (can sound pretty similar at times, but differences are really significant).

Rock on, @flanc/ @BTC/ et al!

/walt

[strikke...@gmail.com]

@flanc, I tested it once more- after importing the danish language plugin and it did write some sentences correct...and some rather amusing. It will be very useable though. In fact it is much better than I had hoped for, Danish being used by so few people. So nice, I know I will use this a lot.

Thank you so much for all the work you and BurningTreeC are doing on this.

Birthe

[Finn Lancaster]

I know I’ve said it so many times already, but thank you so much for your support! This is truly the first GitHub project that I’ve had this many users for, and I along with BTC intend to keep the plug-in updated and always cutting-edge in features! 

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/af5ec459-8703-41f4-bb09-77f68353ed52n%40googlegroups.com.

[BurningTreeC]

Hi @R² ... I kinda like the idea of the NATO phonetic alphabet :)

[Finn Lancaster]

The NATO Phonetic Alphabet is a great example of something that may never be implemented as part of the plug-in, but could easily be achieved by custom commands! 

I think there’s just so much flexibility with the plug-in to do new and amazing things, so I really love when people like @R2 give ideas like that! 

BurningTreeC and I will continue to make these crazy ideas a reality, as we have been doing! 

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/f499da9f-69bf-4e49-ae28-40b8c4ae0019n%40googlegroups.com.

[TW Tones]

If while learning a language or practicing translation;

We could speak words out loud in the language we are learning and get immediate feedback, in that language, and translated to our own language, it would be a way to practice without exposing ourselves in front of others. If the translator recognises our words then perhaps we know how to speak?

Regards

Tones

[TW Tones]

Flanc

Just some thoughts while you are working on this.

• As I hinted at before, in a similar manner to changing language or issuing commands I would love to be able to "OK tiddlywiki" from a listening state. Apart from the fact is allows a keyword activation it does celebrate the integration of these tools with tiddlywiki and in somways "personifies" tiddlywiki. 
• I am not sure if you noticed yet but even having a keyword entry can cause problems. I have heard both Siri and Google start responding out of context when they have misheard and they have activation phrases. I imagine a stop word would be good too, for when someone enters the room.
• My thoughts would be if in a edit text field transcribe there, if not create the new tiddler and transcribe in there. The button activation would be an alternative to "OK Tiddlywiki" when this is inactive.

In a recent post I am also looking at a UI addition of indicators, Thus you could leverage this with a tiddler tagged "$:/tags/indicator" to display listening or dictation states as well as a trigger in a diminutive icon on the side bar or other locations like the menu bar. All you need do is create the matching button/indicators.

Regards

Tones

[Finn Lancaster]

Seems like some pretty solid ideas to look at, particularly the indicators. I can imagine a simple indicator as just a loop to send a tw notify event to display some text.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/cb7fac57-3d38-4476-8dbe-b480ae36777bn%40googlegroups.com.

[TiddlyTweeter]

flanc...@gmail.com wrote (my emphasis):

New idea I would like to share: Because of spoken commands in languages other than English to
change the language again, I believe a better method would be to allow the plugin to get the current language of TW, and auto-default to that setting. When the wiki language was changed, the plugin could also change.

With this idea, English custom spoken commands would still be available.

Let me know what you think!

Ciao Flanc...

I think that is a great approach!

Best

TT 

[TiddlyTweeter]

flanc...@gmail.com wrote:

@Strikke…, yes. Any language shown on the demo is theoretically be supported by the plug-in. I’m not sure if there is a Danish translation of TW. With my new idea to just use the TW language, there may be some limitation. 

Ciao Flanc ...

After using this a few days I do think that being able to switch via "OK Wiki" to ANY API supported language would make good sense.

I don't see the need to limit the available recognitions to only languages that TW has editions for. 

WHY? Because it is often the case we use software in English UIs to do things in other languages. 

It is not an impediment to DICTATING to TRANSCRIPTION since the process is self-contained and has no UI implications.

FYI, if you limit language switch it to ONLY the TW Language Editions I think you will get endless requests to add more! 

For instance, I could, as I mentioned before, do with access to SLOVENIAN recognition. :-)

An opinion. 

Best wishes 

TT 

[BurningTreeC]

Hi @TiddlyTweeter, we're working on making ANY API-supported language work.

We could imagine adding a tiddler $:/config/speech-to-text/language and users can set the text of that tiddler to en-US, de-DE or whatever they like

Some news from the front:

users can now define their own keywords, which means that they can add words to "Ok" and "Wiki" using a config tiddler - like "Pasta" for "Ok" and "Maccheroni" for "Wiki" ... then you can trigger the listening for commands using "Pasta Maccheroni"

users can now define their own actions triggered by voice-commands. See the actualized Usage section on https://speech-to-text.finnsoftware.net/

best wishes, BTC

[TiddlyTweeter]

BurningTreeC wrote:

Hi @TiddlyTweeter, we're working on making ANY API-supported language work.

We could imagine adding a tiddler $:/config/speech-to-text/language and users can set the text of that tiddler to en-US, de-DE or whatever they like

Sounds great!

Just FYI. A side point. Whatever is going on in the background with the API still slightly confuses me. Particularly on what version of the (variant) language gets returned in text. For instance I'm testing on Chrome and Windows tablets. Even though the default is en-US the spellings that come back are IN en-GB (e.g. colour, not color; and honour, not honor). I have NO idea where in the chain of things that happens. It may be because I set machine & browser defaults to British English and that interacts with the recognition somewhere?? 

I should emphasise this it NOT a problem for me since I dominantly write in British English. But it is a bit confusing! :-)

Thank you!

TT

[BurningTreeC]

@TiddlyTweeter, I have changed the default so that it uses the websites lang attribute or the system-default language

[TiddlyTweeter]

BurningTreeC wrote:

@TiddlyTweeter, I have changed the default so that it uses the websites lang attribute or the system-default language

Thanks for such a swift reply! 

I'll test and report back if it don't work. I'm pretty sure it will since you know what you doing, unlike me :-)

 Best, TT

[BurningTreeC]

>I'm pretty sure it will since you know what you doing, unlike me :-)

I don't know if I know what I'm doing :D

Best, BTC

[BurningTreeC]

The cool thing is, that now if it recognizes keywords wrongly, users can just add keywords to the $:/config/speech-to-text/keywords tiddler and recognition gets better

[BurningTreeC]

Update from the front: One can now change the recognized language to the BCP-47 language codes specified in https://cloud.google.com/speech-to-text/docs/languages

Additionally, there's a setting ($:/config/speech-to-text/continuous set to "yes") that allows for continuous listening (listening doesn't stop automatically, one must stop it by hand or voice command)

all the best, BTC