How to convert opencv mat from c# to python with gRPC

[yuz...@gmail.com]

My applications needs to send a gRPC request from the WPF program to the Python service, request data is an opencv mat The contents of my proto file are as follows:

Service gRPCRecognition {
  Rpc Compare (CompareRequest) returns (CompareReply) {}
}
Message CompareRequest {
  Bytes CurrImage = 1;
  Bytes ToolImage = 2;
}
Message CompareReply {
  Bool IsMatched = 1;
}

In the wpf program I use opencvsharp to get the CurrImage from the camera, and the ToolImage is read from the image file using opencvsharp. In the python server, opencv is used to process the business. How do I convert data on the client and server?

C# client code:

Private void sendRequest(Mat curr, Mat tool)

{

    Var reply = client.Compare(new CompareRequest

                    {

                        CurrImage = // how to convert?

                        ToolImage = // how to convert?

                    });

}

python server code:

Class gRPCRecognition(Recognition_pb2_grpc.gRPCRecognitionServicer):

    Def Compare(self, request, context):

        Cv2.imshow('curr', /* how to convert */)

        Cv2.imshow('user', /* how to convert */)

[hopki...@gmail.com]

If the bytes are already encoded as a jpg or png, you can send the byte array from c# and on the python side, change it back into a Mat with with the instructions here: https://stackoverflow.com/questions/17170752/python-opencv-load-image-from-byte-string .

You need to remember though:

- protobuf helps you convert integers/strings/arrays/maps in language A to language B (the parcel)

- grpc helps you create connections to send the protobuf around (the messengers and messaging system)

- protobuf does not support any complex data structures like numpy or cv::Mat, it only support primitives and objects (structs) made of the primitives.

An illustrative example: you can definitely attempt to create a pb Message that includes all the primitives that a Mat has (e.g. byte array, width, height, pointer to first block of memory, total length,  endianess etc.) and write conversion code for multiple languages. This is probably not worth doing!