Get stream video capture camerax

[Mohammad Goudarzi]

Hello Recently I am working on a video calling project I need to enable video recording of the front camera and send the data stream to the server through the socket
I had not worked with the camera before and recently read the official CameraX documents but thought I do not think this is possible through CameraX.
Although the data stream makes sense to the preview, I do not know at all how to receive this data, and if it is received, can the data stream received from the server be previewed to display?
Can anyone help me?

[tonyt...@google.com]

Hi,

Thanks for the question. Currently it's not supported by CameraX. We'll consider this as a feature request. To be clear, do you just want to access the video stream or do you want CameraX to handle the communication as well?

[Mohammad Goudarzi]

Thanks for your reply

I only need the stream of video capture data, I can do the coding and send To be honest, accessing the data stream of a recorded video should not be so difficult

Do you have another way?

[Trevor McGuire]

Do you need to display the video while it is being sent? CameraX's Recorder class can record a video to a ParcelFileDescriptor, which could come from a remote socket. However, since Recorder is meant for archival purposes, and is not built for streaming, the video may not be playable until it has completely been sent. 

We are looking into ways to improve the streaming experience. It would be great to hear more about your use case so we can see if our future plans align with what you are looking for.

--
You received this message because you are subscribed to the Google Groups "Android CameraX Discussion Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to camerax-develop...@android.com.
To view this discussion on the web visit https://groups.google.com/a/android.com/d/msgid/camerax-developers/82adae60-cccb-4bb1-95c0-60a61682c9d3n%40android.com.

[Mohammad Goudarzi]

Thanks for your reply

Actually what I want:

I want to make a video call and i implemented the server and socket structure. I have already made a voice call.

When camera is recording, for each frame give me Image(ByteArray) in a callback input And I send it to the server and the server send it to the destination device so that the destination device will receive the frame and display it in the PreviewView (it is better that the developer is not involved in media encoding like h264 and ...).

[Trevor McGuire]

To be clear, are you asking for a ByteArray that contains encoded frames, i.e., has already gone through an h264 encoder? Or are you looking for a ByteArray that contains RGB frames (or YUV)? ]

The former we can't currently provide from CameraX, though it is possible to hook CameraX into MediaCodec to provide encoded frames. I'm assuming you would like to avoid this since writing MediaCodec code can be tedious.

If you're just looking for RGB/YUV frames, you can use CameraX ImageAnalysis to get Image objects that contain that data.

[Mohammad Goudarzi]

Thank you for your time and your answer

I meant the first case, but if I want to use camerax, If I receive the frames on the first device in RGB / YUV format, can I re-display the frames on the second device?

Can you provide a sample source or code to help me send a frame to another device using CameraX?

[Trevor McGuire]

If I receive the frames on the first device in RGB / YUV format, can I re-display the frames on the second device?

You can do this, but you'll almost certainly want to encode the frames before sending them to the second device. If you want to do this with CameraX, your best bet would be to use something like Preview + MediaCodec. But we don't currently have any samples for that type of flow.

Based on the description of what you're trying to do, you might want to check out WebRTC for Android. There are a few samples out there which could get you started. CameraX is currently evaluating how we can better support this type of use case, and WebRTC is certainly something we're considering. But for now, the samples which do not use CameraX will likely get you unblocked.

[Gasol Wu]

Hi Team,

I'm glad to hear CameraX cares about the cases of streaming. I have tried before to implement a video streaming app on the new CameraX library, which has a preview and pushes a video stream to the remote server via protocols like RTMP. It also allows users to capture an image while streaming. Also, the app needs to draw overlay graphics on both preview and video stream. After digging the sources, especially VideoCapture use case in camera-video. It seems like the Recorder is hard to implement features like this. It's easy to use as a high level component. But it also means it's lack of customization. So I use the camera2 library and combine it with GLSurfaceView and MediaCodec instead. I still have a hope to replace it with the new CameraX library. 

[Android CameraX Discussion Group]

Thanks for your feedback. We will carefully consider your suggestion when we are planning for the streaming use case.

[Mohammad Goudarzi]

Hello again, I did exactly what you said and it worked I could not use MdiaCodec So I converted each frame to base64 and sent it I did my university project but I am still interested in learning new things I also tried Camera2 and found that CameraX was a great idea from the start. thanks all

[Mohammad Goudarzi]

I'm trying to use MediaCodec Do you have a good resource for learning it?

Just convert the YUV to h264 at the source and display it on a Surface at the destination.