Speech-to-Text v1.0.5 is here!

[Finn Lancaster]

What's New?

New language features:

• auto switch when TW language switched (opt-in on plugin settings)
• more settings, such as continuous listening option!
• Loads of more languages!
• minor BUG fixes
• and lots more!

[BurningTreeC]

... And custom VoiceCommands are possible!

Create a tiddler tagged $:/tags/VoiceCommand

Fill its text field with your desired actions

Use the field voice-commands to add a list of commands that should trigger the actions

[TW Tones]

Links etc... here in this thread please

[BurningTreeC]

The Link to the GitHub project page https://github.com/flancast90/Speech-To-Text-in-TW5

The Link to the Demo page https://speech-to-text.finnsoftware.net/

[TW Tones]

Thanks BTC, and especially Flanc...

This is seriously cool, I will try and contribute more moving forward.

I can't believe how I could switch to the French language and even my poor French was recognised as I spoke it. Wow.  As I said before to practice a new language and succeed at having the correct words transcribed is a revolutionary tool. No need to always talk to native speakers to practice :)

Some thoughts

1. I feel perhaps I need to identify easy ways to have tiddlers read back to us as a complement, I am sure there are similar tools for browsers, whether or not they can be built in like this transcription process. There was a plugin that reads text with a progress indication over the text as it reads, I will share when I locate it. 
   1. Windows already has text to voice readers, 
2. I do look forward, if possible to inserting transcriptions into existing text, which I understand is on the list of features.
3. The commands seem to be working well. 
4. I do not follow how to add multiple new keywords, I am just a little confused.
5. Is it possible to capture the actual audio and save that in a tiddler as well? 
   1. ie using voice not only to transcribe but also to trigger a recording. 

Speculation

• The reason I ask question 5 above, is along with 1. above we could build a wiki for managing reading and playing spoken word as a script or like karaoke. Podcasters, radio producers etc... may find this useful. Then editing content could be as simple as rearranging tiddlers, be it textural and/or audio read what you hear, and see what is up next.
  
• If this was running on top of a node server we could imagine the operating system responding when detecting a new tiddler containing instructions.
• I listen to a lot of spoken work podcasts. Perhaps one day I can just play a favorite quote into my microphone, to capture it in tiddlywiki, a bit like how I can copy and past a types transcript.

Regards

Your excited tiddlywikian

Tones

[BurningTreeC]

Hi @Tones,

To answer 4)

There's the tiddler $:/config/speech-to-text/keywords that can be used to add multiple new keywords

Use its fields "ok-keywords" and "wiki-keywords" for that

There's the rule that we're using two keywords which are "Ok" and "Wiki"

But the $:/config/speech-to-text/keywords can be used to extend them

An example:

ok-keywords: bonjour Bonjour

wiki-keywords: madame Madame

[TiddlyTweeter]

BurningTreeC wrote:

The Link to the GitHub project page https://github.com/flancast90/Speech-To-Text-in-TW5

The Link to the Demo page https://speech-to-text.finnsoftware.net/

Thanks BTC & Flanc for the update beta.

I will check over next few days & comment back

Best wishes, TT 

[TiddlyTweeter]

Ciao Flanc & BTC ...

I agree with @Tones "Wow!". 

And the tool has already changed how I use some wiki.

ONE point I'd like to quickly make whilst the iron is hot. (Which I will fill in details for later with more precision.)

From MY point of view, I want to get my DICTATION/TRANSCRIPTION so it is optimally efficient. 

(FYI, I'm far less concerned with controlling the wiki outwith dictation. I'm also not that interested in whether it can "read-back" or record to an audio file. 

First I need to get basic transcription working really well! :)

To give simple examples of, I think, main dictation focused needs ...

1 - A way to verbally insert PUNCTUATION. (Without it, long dictation produces unwieldy texts.)

2 - A way to verbally INSERT "boilerplate text". 

For instance: "Ok Wiki insert exercise intro". Which might(?) be text in a Tiddler called "Exercise Intro" like "Lie on you back and rest a moment, scanning for how your back contacts the floor. Notice which parts touch it and which parts don't."

You get the idea? 

What I don't currently understand is how in dictation you'd execute a command to INSERT "boilerplate" text into the ongoing transcription?

3 - A way to CONCATENATE transcript fragments. 

WHAT do I mean? In a test I dictated a long text in small segments. The result being that I ended up with ... 

  - New Transcript

  - New Transcript 1

... through to ...
   - New Transcript 23

Manually joining those would be a fag. Maybe we need a way to click a button to CONCATENATE (latest last) all those new transcript fragments?

Just a thought, and likely YOU don't need to devise that as a standard TW macro could likely do it?

BUT I did think it a real functional need for folk interested in serious dictation to be able to manipulate Transcripts, and this is one of the needs?

It is also fun to play with! :-)

Thanks & best wishes

TT

[Finn Lancaster]

@TiddlyTweeter: 

To answer just your question about punctuation, I have tested the plugin with various punctuation words. The result is, by saying "comma", you have a comma. The same works for question marks, periods, etc. Some of your other points, I felt were really good, as well. Hopefully @BurningTreeC and I can implement them in later versions of the plugin!

[BurningTreeC]

@TiddlyTweeter

at https://cloud.google.com/speech-to-text/docs/languages you can check if your chosen language supports spoken punctuation

the insertion of boilerplate text may be possible, we'll have to think about it a little bit

concatenating transcript fragments could be done by a simple button. Maybe we should add $:/tags/TranscriptFragment to a new Transcript Tiddler so that filtering for transcript tiddlers gets easier?

Speech Recognition is not perfect, it sometimes doesn't get the text fully right. That's a limitation we cannot change

best wishes

Simon

[TiddlyTweeter]

Ciao Simon ...

...Speech Recognition is not perfect, it sometimes doesn't get the text fully right. That's a limitation we cannot change

Right! For me it is not a big issue as the recognition (in the US & UK English I have tested most) is a zillion times better than other systems I have used in the past which were just ineffective and frustratingly useless. 

Basically the recognition is high. It is a lot, lot quicker than typing, whatever the residual (cloud intelligence) issues.

Best wishes

Josiah 

[TiddlyTweeter]

Simon wrote:

... the insertion of boilerplate text may be possible, we'll have to think about it a little bit

Thank you! For my use cases it would be invaluable. And maybe for others too!

(Side note: I am a bit disappointed you are not getting more feedback. FYI, I posted notes about the your tool on Twitter and to two TW Telegram groups to try and get a bit more feedback.)

 

... concatenating transcript fragments could be done by a simple button. Maybe we should add $:/tags/TranscriptFragment to a new Transcript Tiddler so that filtering for transcript tiddlers gets easier?

Sounds like it could be useful to do that! 

Best wishes

Josiah

[BurningTreeC]

@TiddlyTweeter

Thank you for the advertisement, I think it's needed. I don't know how many people are already using this plugin, feedback would be very welcome and would also be motivation for further development

best wishes,

Simon

[TiddlyTweeter]

Simon (BTC) wrote ...

Thank you for the advertisement, I think it's needed. I don't know how many people are already using this plugin, feedback would be very welcome and would also be motivation for further development.

One thing I commented on Telegram that I think is TRUE, that might interest you ...

• ... Speech input in TW could be a game changer as the TW UI system is ideal to leverage speech-to-text interface elements fully.

Best wishes

Josiah (@TiddlyTweeter)

[TiddlyTweeter]

Flanc ...

I can confirm you can sing in the lyrics for VIP IN TRIP (Italiano [it-IT], Fabri Fibri) and the output is basically ...

  Federico, mi guardi su Wikipedia
  Se sotto la voce di "rapper superfighissimo" c'è la mia faccia?
  Ah, non c'è? E che faccia c'è, scusa?
  Fabri Fibra è tanta roba, come il detto
  Comandare è meglio che scopare, di chi era questa?

  Io lo so, lo diceva (come non detto) ... etc

https://www.youtube.com/watch?v=9CxSDqNrxPM

It needs a bit of Layout Massage after for new lines and missing punctuation & a couple of accents.

TT, researcher

[TiddlyTweeter]

Ciao Flanc

Following your comment I tried PUNCTUATING with "comma". 

It worked ONCE. 

BUT I could not get it to work afterwards, though the cloud machine can ...

     ... recognise the difference between a comma and a coma

Any idea why, & how to get it working reliably?

Best wishes, TT

[Finn Lancaster]

@TiddlyTweeter, 

It’s strange that punctuation only worked once. I can look at the code and try to replicate your situation, but if that is truly happening, it is most likely an API issue, and can’t be fixed through the plug-in. 

Thanks!

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/0034663d-e89c-4620-998a-0490da779c3cn%40googlegroups.com.

[TiddlyTweeter]

DUNNO! I tried looking at the Google Cloud docs for their implementation of the API for clues ... but I get lost as it quickly gets too technical for my modest brain ;-(.

Leave it with me a few days. I can test on Chromebook later to see if it works better on it.

So ... later

TT

flanc...@gmail.com wrote ... 

It’s strange that punctuation only worked once. ...