Live camera streaming using grpc python

[Sanket Kumar Mali]

Hi,

I am trying to implement a live camera streaming setup using grpc python. I was able to stream camera frame(1280x720) in 30 fps to a single client. But whenever I try to consume the stream from multiple client it seems frame rate is getting divided (e.g if I connect two client frame rate becomes 15fps).

I am looking for a direction where I am doing wrong. Appreciate any clue in the right way to achieve multi client streaming.

Thanks

[torpido]

What happens if you run the same process in parallel, and serve in each one a different client?

just to make sure that there is no issue with the bandwidth in the server.

I would also set debug logs for gRPC to get more info

Can you share the RPC and server code you are using? Seems like it should be A request-streaming RPC 

ב-יום שבת, 13 במאי 2023 בשעה 16:21:41 UTC+3, Sanket Kumar Mali כתב/ה:

[Sanket Kumar Mali]

my server code

camera_buffer = queue.Queue(maxsize=20)

# Define the gRPC server class
class CameraStreamServicer(camera_stream_pb2_grpc.CameraStreamServicer):
    def __init__(self):
        self.clients = []

    def CameraStream(self, request, context):
        global camera_buffer
        # Add the connected client to the list
        self.clients.append(context)
        try:
            while True:
                    print("size: ",camera_buffer.qsize())
                    frame = camera_buffer.get(timeout=1)  # Get a frame from the buffer

                    # Continuously send frames to the client
                    for client in self.clients:
                        try:
                            response = camera_stream_pb2.Frame()
                            response.frame = frame
                            response.timestamp = int(time.time())
                            yield response
                        except grpc.RpcError:
                            # Handle any errors or disconnections
                            self.clients.remove(context)
                            print("Client disconnected")
        except Exception as e:
            print("unlnown error: ", e)

in a seperate thread I am getting frames from camera and populating the buffer

[Sanket Kumar Mali]

my proto file

syntax = "proto3";

package camera_stream;

// Camera frame message
message Frame {
  bytes frame = 1;
  int64 timestamp = 2;
}

// Camera stream service definition
service CameraStream {
  // Method to connect and start receiving camera frames
  rpc CameraStream(Empty) returns (stream Frame) {}
}
// Empty message
message Empty {}

[yas...@google.com]

Are you tied to gRPC Python or could you also experiment with another language?

[Richard Belleville]

Sanket, you've shared your servicer, but you haven't shared the code that's enqueueing data into your camera_buffer. If you have a single writer enqueueing objects into a global buffer at a constant rate, then when you have one active RPC, you'll be getting the expected result -- each frame is delivered to the client. But if you have two connected clients you'll have two readers on your queue and each frame will be received by only one of them. So you'll basically be sending even frames to one client and odd frames to another.

What you do instead will depend on the sort of behavior you want. If you are okay with potential loss of frames on a stream, then you can do something very simple. Keep a global variable with a single frame, along with a threading.Lock and a threading.Condition. The writer will signal all awaiting threads each time it changes the frame. Each stream will be waiting on the condition variable, read the frame when signalled, and send it on its stream.

If you can't tolerate the potential loss of frames, then you'll need a data structure a little more complicated. Each frame needs to be kept in memory until all readers have received it. Only then can it be purged. You'd keep frames in a global buffer. Each time you pull a frame from the buffer, increase a count on the frame. Whichever reader happens to read it last will observe that the count has reached the global number of readers and can purge it from memory. Then you'll need to factor in the fact that clients can come and go, so that the total number of readers may change at any time.

Regardless, the issue isn't really with gRPC. This is more about multithreading in Python.

[Sanket Kumar Mali]

Hi,

I was able achieve the expected behaviour using a global frame buffer.

Thanks for the support.

Regards,

Sanket

--
You received this message because you are subscribed to a topic in the Google Groups "grpc.io" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/grpc-io/BqlPVum23gE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to grpc-io+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/cb8b93bb-20b6-4087-b2d3-a7facc2d1640n%40googlegroups.com.