Is it possible, with CameraX, to separate and process the audio stream during video capture?

[skeg0]

We have recently migrated our video taking application from Camera API 2 to CameraX and are wanting to implement voice isolation to remove background noise from the video, as well as potentially analysing what's being spoken to provide subtitles - in real time during video capture.

The only way we can do this, we believe, is to separate the audio stream and run processing on it separately - but our research with CameraX is telling us that this isn't possible to do.

Has anyone had any experience with this and is dropping back to Camera API 2 the only way? This seems highly unfortunate seeing as CameraX is the recommended library to use, and assuming, going to be receiving updates as time goes on. So it seems a bit backwards to have to revert back to Camera API 2!

[Aubrey Haskett]

We have wanted to separate audio as well - just as the Image Analysis use case lets us separate and process frames while still allowing preview and video capture use cases. But I don't believe there is currently a way to do this with CameraX unfortunately. I do hope that if enough people desire this functionality, They might add it (which is why I'm chiming in). :)

[Leo Huang]

Hi, 

Thanks for the quesiton. 

You probably can use ImageAnalysis/Preview and framework classes AudioRecord/MediaCodec/MediaMuxer to do the recording.

Something like:

Camera -> ImageAnalysis(or Preview)  -> MediaCodec \ 

                                                                                                    -> MediaMuxer -> Video file

AudioRecord -> Your audio processing -> MediaCodec /

But this requires app to do the encoding and muxing.

To clarify, do you want to get/modify the audio raw data from CameraX and then feeding it to the CameraX Recorder?

The current CameraX video recording is like

Camera -> VideoCapture \
                                                -> Recorder -> Video file
                     AudioSource /

Is the following CustomProcessor what you want?

                Camera -> VideoCapture   \
                                                                  -> Recorder -> Video file
AudioSource  -> CustomProcessor  /

or a CustomAudioSource?

                Camera -> VideoCapture   \
                                                                  -> Recorder -> Video file
                     CustomAudioSource    /

or ?

Any samples are appreciated.

aubrey....@gc.com 在 2025年7月30日 星期三晚上8:20:48 [UTC+8] 的信中寫道：

[Scott Nien]

Thanks for raising this request. I think it's a good one! 
Filed a public bug https://issuetracker.google.com/436444582 to track this feature request.

--
You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.
To view this discussion visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/2439ca45-2c17-4c7b-b491-bcf9e0d15804n%40android.com.