Enabling ChromiumOS Dictation DLC (SODA)

[Kevin Ho]

I am currently working on trying to enable the accessibility feature Dictation on ChromiumOS. From my system settings, the toggle exists to turn it on and off, but when I toggle it on, nothing happens.

I believe this is an issue related to the SODA DLC? From the googlesource pages, I found these two lines:

1. Line where I can see that checkout_soda is set to "False" : https://chromium.googlesource.com/chromium/src/+/refs/heads/main/DEPS#135, and 2. Lines where I see that the package for SODA is set behind chrome_internal : https://chromium.googlesource.com/chromium/src/+/refs/heads/main/DEPS#4281 I am currently trying to build this on ChromiumOS R114, and am hoping someone can provide guidance for getting it to work on ChromiumOS

[K. Moon]

If a package is part of chrome_internal, then it's Google-proprietary and it's not available for use with (open source) Chromium.

This list is for Chromium only, by the way; there's a separate list for Chromium OS.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/ba351dc8-a708-4fee-9128-a16cbf6d92e3n%40chromium.org.

[Mike Frysinger]

for CrOS code in the browser, largely chromium-dev@ is better

-mike

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CACwGi-4k%2Bw_Td6XgGjDkME1Sw%3D8cJbwZcQkQMvVdYZ%3DuiyXo6Q%40mail.gmail.com.

[Kevin Ho]

Got it. This would be trying to enable it device wide. i'll make a separate post in the ChromiumOS channel for the question.

Thanks!

Kevin

[Akihiro Ota]

Hi Kevin,

Could you provide a few more details about what's not working? For example, do you have a specific list of steps that you're taking that will reproduce your issue?

One thing to verify is that you're focused on an editable field (a text field, contenteditable, input, omnibox, etc.) before you activate Dictation. Otherwise, Dictation won't turn on. Another thing to try is to switch your Dictation language in ChromeOS settings to something other than en-US. You can do this by going to Settings > Accessibility > keyboard and text input > Dictation > Language > Change. Just for some context, en-US is the only locale that uses SODA, all others will use network speech recognition. So switching to a different locale and trying again will help us understand if the issue is SODA-specific, or if there's an issue with our logic that chooses the type of speech recognition to use.

Thanks!

[Kevin Ho]

Hi Akihiro,

Thanks for the feedback. I just wanted to clarify - I am not using a vanilla Chromebook.

I am wondering whether it is possible to use on-device dictation with an instance of ChromiumOS that is self-built using the open source code.

I get that Libsoda itself is closed-source, I merely want to use the Libsoda binary+language model in my own instance of ChromiumOS. The primary issue here is that the Libsoda library is protected by an API key: https://source.chromium.org/chromiumos/chromiumos/codesearch/+/main:src/platform2/ml/soda_recognizer_impl.cc;l=167?q=soda_recognizer_impl.cc&ss=chromiumos which is not provided in the public code: https://source.chromium.org/chromium/chromium/src/+/main:google_apis/google_api_keys.cc;l=83;drc=f4a00cc248dd2dc8ec8759fb51620d47b5114090;bpv=1;bpt=1 so even if you set up the DLCs in your own self-built ChromiumOS the Libsoda library will not process anything.

[Kevin Ho]

Hi Akihiro,

Appreciate the feedback regarding SODA not supported on public Chromium/ChromiumOS.

I have tried building enabling dictation without SODA and using Network based dictation as per your advice and I still can't build dictation and get it to work.  It appears network-based dictation may also be behind proprietary pieces of code / not supported on public Chromium/ChromiumOS?

In the build file, i can see that it is looking for the "ENABLE_SERVER_BASED_RECOGNITION" flag here : https://source.chromium.org/chromium/chromium/src/+/main:chrome/services/speech/buildflags/BUILD.gn;l=16?q=ENABLE_SERVER_BASED_RECOGNITION_RECOGNIZER&ss=chromium%2Fchromium%2Fsrc . 

However, to build it, and to set it to true, I need a specific header file for it. Based on this line here -  https://source.chromium.org/chromium/chromium/src/+/main:chrome/browser/speech/cros_speech_recognition_service.cc;l=29 - it looks like this points to another set of "internal" folder (similar as to which I saw on SODA), which i don't have access too?

My question is: is there a workaround to enable network dictation? Any guidance here would be appreciated, or if you could advise if I've misunderstood something from my findings above.

A second question I have is with regards to PUMPKIN. I haven't gotten to this step yet, but I am expecting to have the same issue with SODA since both are DLCs. My understanding is that Pumpkin is needed to help process the text that the network recognition software returns. Do you have any guidance here that you can provide?

Thanks,

Kevin

On Mon, Mar 4, 2024 at 10:37 AM Akihiro Ota <akihi...@google.com> wrote:

Hi Kevin,

Based on the error messages and the responses above, it doesn't seem that SODA is supported on public chromium/chromiumos. However, you don't need SODA in order to use Dictation. Dictation will fallback to network speech recognition if SODA isn't available. So while you won't be able to use on-device Dictation with your self-built image, you can still use network Dictation.

Thanks and hopefully that helps!

[Akihiro Ota]

Hi Kevin,

Based on the error messages and the responses above, it doesn't seem that SODA is supported on public chromium/chromiumos. However, you don't need SODA in order to use Dictation. Dictation will fallback to network speech recognition if SODA isn't available. So while you won't be able to use on-device Dictation with your self-built image, you can still use network Dictation.

Thanks and hopefully that helps!

[Akihiro Ota]

Hi Kevin,

I don't have much background on CrosSpeechRecognitionService, so I asked a teammate about what it's used for and if it's internal-only, and will follow-up when I hear back from them.

In the meantime, I'd recommend looking at SpeechRecognitionPrivateRecognizer, which is the class that handles speech recognition for Dictation. It's a part of a private chrome extension API, but it should be available on a public build of Chrome/ChromeOS. The class has logic to choose between on-device and network speech recognition, based on what's available. In the case of network speech recognition, it will use a class called NetworkSpeechRecognizer, which also seems to be publicly available.

Regarding pumpkin, you're right that it's a DLC so you may face the same challenges as you did with SODA. However, Dictation will still work without Pumpkin by using regular expressions to parse the text that is returned by the speech recognition service. SimpleParseStrategy has more implementation details, but the TLDR is that Dictation will still work without pumpkin.

I hope that helps and let me know if you have additional questions!