Hi Ajay, there are a couple of ways to do this. However, there are trade-offs between the performance and the complexity. If you are looking for something simple, I would recommend the following:
1. Use
ML Kit to get the segmentation mask.
Set up the effect to target
PREVIEW
| VIDEO_CAPTURE.
This is for setting up a listener for frame drawing.
3. Transform the segmentation mask, and draw a different background on top of the camera frame.
The mask you get from ML Kit is in a different coordinate system with the video stream. The transformation from camera sensor to ML Kit output is
ImageProxy#getImageInfo#getSensorToBufferTransformMatrix. The transformation from camera sensor to the video frame is
Frame#getSensorToBufferTransform. You need to combine those two values to get the transformation from ML Kit output to the video frame, then use it to transform the segmentation mask. After that, you can draw the mask on top of the frame using the
canvas API.