Disable Live Caption for duplicate web audio streams [chromium/src : main]
Evan Liu (Gerrit)
Evan Liu added 1 comment![]( "Open in Gerrit")
Related details
- Dale Curtis
Code-Coverage
Code-Owners
Code-Review
Review-Enforcement
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Dale Curtis (Gerrit)
Dale Curtis added 2 comments![]( "Open in Gerrit")
What do you think of this approach?
I worry this is disabling Live Caption on WebAudio for cases which are transformative of the media element content and include many other graph nodes. I forget exactly how the WebAudio graph is structured, so defer to the OWNERS though.
Disabling live captions just for the media element would be safer. Did you identify why the approach I suggested wasn't setting the values expected?
As soon as WebAudio is attached to the element it should report the right value:
https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/media/web_audio_source_provider_impl.cc;l=349;drc=9ec1ad7139cca8bacfd91fc05dac7907b0ba7fba
That may require checking it periodically instead of just once.
virtual void SetDisableLiveCaption(bool disable_live_caption) {}Interface is currently pure virtual. Defer to WebAudio OWNERS if they want to keep it that way.
Related details
- Evan Liu
Code-Coverage
Code-Owners
Code-Review
No-Unresolved-Comments
Review-Enforcement
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Michael Wilson (Gerrit)
Michael Wilson added 1 comment![]( "Open in Gerrit")
virtual void SetDisableLiveCaption(bool disable_live_caption) {}Interface is currently pure virtual. Defer to WebAudio OWNERS if they want to keep it that way.
Thanks for pointing this out, I do prefer to keep this pure virtual.
Evan Liu (Gerrit)
Evan Liu added 1 comment![]( "Open in Gerrit")
Dale CurtisWhat do you think of this approach?
I worry this is disabling Live Caption on WebAudio for cases which are transformative of the media element content and include many other graph nodes. I forget exactly how the WebAudio graph is structured, so defer to the OWNERS though.
Disabling live captions just for the media element would be safer. Did you identify why the approach I suggested wasn't setting the values expected?
As soon as WebAudio is attached to the element it should report the right value:
https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/media/web_audio_source_provider_impl.cc;l=349;drc=9ec1ad7139cca8bacfd91fc05dac7907b0ba7fbaThat may require checking it periodically instead of just once.
It looks like RendererWebAudioDeviceImpl is using a real AudioOutputDevice so IsOptimizedForHardwareParameters() always returns true. Is that not how things are supposed to work?
Related details
- Dale Curtis
Code-Coverage
Code-Owners
Code-Review
No-Unresolved-Comments
Review-Enforcement
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Dale Curtis (Gerrit)
Dale Curtis added 1 comment![]( "Open in Gerrit")
Dale CurtisWhat do you think of this approach?
Evan LiuI worry this is disabling Live Caption on WebAudio for cases which are transformative of the media element content and include many other graph nodes. I forget exactly how the WebAudio graph is structured, so defer to the OWNERS though.
Disabling live captions just for the media element would be safer. Did you identify why the approach I suggested wasn't setting the values expected?
As soon as WebAudio is attached to the element it should report the right value:
https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/media/web_audio_source_provider_impl.cc;l=349;drc=9ec1ad7139cca8bacfd91fc05dac7907b0ba7fbaThat may require checking it periodically instead of just once.
It looks like RendererWebAudioDeviceImpl is using a real AudioOutputDevice so IsOptimizedForHardwareParameters() always returns true. Is that not how things are supposed to work?
I think that means WebAudio isn't being connected to the media element if that's what you see on TikTok. Is it working properly on this demo? https://webaudioapi.com/samples/audio-tag/
Related details
- Evan Liu
Code-Coverage
Code-Owners
Code-Review
No-Unresolved-Comments
Review-Enforcement
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Hongchan Choi (Gerrit)
Hongchan Choi added 1 comment![]( "Open in Gerrit")
if (destination()) {
destination()->GetAudioDestinationHandler().SetHasMediaElementSource(true);
}So the problem is caused because:
```
MediaElement ----------------> Live Caption back-end
\-> WebAudio ---> Live Caption back-end
```
Did I understand right?
Evan Liu (Gerrit)
Evan Liu added 2 comments![]( "Open in Gerrit")
Dale CurtisWhat do you think of this approach?
Evan LiuI worry this is disabling Live Caption on WebAudio for cases which are transformative of the media element content and include many other graph nodes. I forget exactly how the WebAudio graph is structured, so defer to the OWNERS though.
Disabling live captions just for the media element would be safer. Did you identify why the approach I suggested wasn't setting the values expected?
As soon as WebAudio is attached to the element it should report the right value:
https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/media/web_audio_source_provider_impl.cc;l=349;drc=9ec1ad7139cca8bacfd91fc05dac7907b0ba7fbaThat may require checking it periodically instead of just once.
Dale CurtisIt looks like RendererWebAudioDeviceImpl is using a real AudioOutputDevice so IsOptimizedForHardwareParameters() always returns true. Is that not how things are supposed to work?
I think that means WebAudio isn't being connected to the media element if that's what you see on TikTok. Is it working properly on this demo? https://webaudioapi.com/samples/audio-tag/
I'm seeing the same behavior on https://webaudioapi.com/samples/audio-tag/ where AudioOutputDevice is used with web audio and so both `AudioRendererImpl::TranscribeAudio()` and `RendererWebAudioDeviceImpl::Render()` end up sending the same audio to be transcribed.
if (destination()) {
destination()->GetAudioDestinationHandler().SetHasMediaElementSource(true);
}So the problem is caused because:
```
MediaElement ----------------> Live Caption back-end
\-> WebAudio ---> Live Caption back-end
```Did I understand right?
- Dale Curtis
- Hongchan Choi
Code-Coverage
Code-Owners
Code-Review
No-Unresolved-Comments
Review-Enforcement
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Dale Curtis (Gerrit)
Dale Curtis added 1 comment![]( "Open in Gerrit")
Dale CurtisWhat do you think of this approach?
Evan LiuI worry this is disabling Live Caption on WebAudio for cases which are transformative of the media element content and include many other graph nodes. I forget exactly how the WebAudio graph is structured, so defer to the OWNERS though.
Disabling live captions just for the media element would be safer. Did you identify why the approach I suggested wasn't setting the values expected?
As soon as WebAudio is attached to the element it should report the right value:
https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/media/web_audio_source_provider_impl.cc;l=349;drc=9ec1ad7139cca8bacfd91fc05dac7907b0ba7fbaThat may require checking it periodically instead of just once.
Dale CurtisIt looks like RendererWebAudioDeviceImpl is using a real AudioOutputDevice so IsOptimizedForHardwareParameters() always returns true. Is that not how things are supposed to work?
Evan LiuI think that means WebAudio isn't being connected to the media element if that's what you see on TikTok. Is it working properly on this demo? https://webaudioapi.com/samples/audio-tag/
I'm seeing the same behavior on https://webaudioapi.com/samples/audio-tag/ where AudioOutputDevice is used with web audio and so both `AudioRendererImpl::TranscribeAudio()` and `RendererWebAudioDeviceImpl::Render()` end up sending the same audio to be transcribed.
I just tested this and it works as intended, though as I mentioned depending on how the page attaches WebAudio you need to check `IsOptimizedForHardwareParameters()` periodically.
```
[558030:558030:1205/231320.593487:ERROR:third_party/blink/renderer/platform/media/web_audio_source_provider_impl.cc:140] SetClient: 0
[558030:558118:1205/231320.631384:ERROR:media/renderers/audio_renderer_impl.cc:932] DecodedAudioReady: 1
[558030:558118:1205/231320.631738:ERROR:media/renderers/audio_renderer_impl.cc:932] DecodedAudioReady: 1
<snip.../snip>
[558030:558118:1205/231320.719447:ERROR:media/renderers/audio_renderer_impl.cc:932] DecodedAudioReady: 1
[558030:558030:1205/231320.746547:ERROR:third_party/blink/renderer/platform/media/web_audio_source_provider_impl.cc:140] SetClient: 0xb54004a3870
[558030:558118:1205/231320.761838:ERROR:media/renderers/audio_renderer_impl.cc:932] DecodedAudioReady: 0
```
Related details
- Evan Liu
- Hongchan Choi
Code-Coverage
Code-Owners
Code-Review
No-Unresolved-Comments
Review-Enforcement
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Evan Liu (Gerrit)
Evan Liu voted and added 1 comment![]( "Open in Gerrit")
Votes added by Evan Liu
| Commit-Queue | +1 |
Dale CurtisWhat do you think of this approach?
Evan LiuI worry this is disabling Live Caption on WebAudio for cases which are transformative of the media element content and include many other graph nodes. I forget exactly how the WebAudio graph is structured, so defer to the OWNERS though.
Disabling live captions just for the media element would be safer. Did you identify why the approach I suggested wasn't setting the values expected?
As soon as WebAudio is attached to the element it should report the right value:
https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/media/web_audio_source_provider_impl.cc;l=349;drc=9ec1ad7139cca8bacfd91fc05dac7907b0ba7fbaThat may require checking it periodically instead of just once.
Dale CurtisIt looks like RendererWebAudioDeviceImpl is using a real AudioOutputDevice so IsOptimizedForHardwareParameters() always returns true. Is that not how things are supposed to work?
Evan LiuI think that means WebAudio isn't being connected to the media element if that's what you see on TikTok. Is it working properly on this demo? https://webaudioapi.com/samples/audio-tag/
Dale CurtisI'm seeing the same behavior on https://webaudioapi.com/samples/audio-tag/ where AudioOutputDevice is used with web audio and so both `AudioRendererImpl::TranscribeAudio()` and `RendererWebAudioDeviceImpl::Render()` end up sending the same audio to be transcribed.
I just tested this and it works as intended, though as I mentioned depending on how the page attaches WebAudio you need to check `IsOptimizedForHardwareParameters()` periodically.
```
[558030:558030:1205/231320.593487:ERROR:third_party/blink/renderer/platform/media/web_audio_source_provider_impl.cc:140] SetClient: 0
[558030:558118:1205/231320.631384:ERROR:media/renderers/audio_renderer_impl.cc:932] DecodedAudioReady: 1
[558030:558118:1205/231320.631738:ERROR:media/renderers/audio_renderer_impl.cc:932] DecodedAudioReady: 1
<snip.../snip>
[558030:558118:1205/231320.719447:ERROR:media/renderers/audio_renderer_impl.cc:932] DecodedAudioReady: 1
[558030:558030:1205/231320.746547:ERROR:third_party/blink/renderer/platform/media/web_audio_source_provider_impl.cc:140] SetClient: 0xb54004a3870
[558030:558118:1205/231320.761838:ERROR:media/renderers/audio_renderer_impl.cc:932] DecodedAudioReady: 0
```
Ahh my bad I completely misread your original suggestion--I thought you were suggesting disabling captioning from the web audio side so I was checking that sink instead. I've updated this CL with the fix that you recommended.
Related details
- Dale Curtis
- Hongchan Choi
Code-Coverage
Code-Owners
Code-Review
No-Unresolved-Comments
Review-Enforcement
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |