avoid memory copy in bidi stream

41 views
Skip to first unread message

yanch...@gmail.com

unread,
Oct 22, 2020, 2:48:36 AM10/22/20
to grpc.io
Hi, 

I have a simple client and server using gRPC to exchange information. For now, client side transfer big binary data to server by stream. At server side, server reads one by one from stream and copy the content to another pre-allocated memory to deal with. 
My question is, is there any way to avoid coping data from stream into pre-allocated space and use stream data directly in other thread. 

Here is the code snip 
proto:
message my_data{
    sint32 data_len;
    string data;
}

rpc transfer(stream my_data) returns (stream other)

server:
Status  transfer  ServerContext* context,
                   ServerReaderWriter< my_data  ,  other  >* stream
    my_data d;
    while (stream->Read(&d)) {
        char* p = malloc(d.data_len);
        strncpy(p, d.data.c_str(), d.data_len);
        //add p to list 
    }
    .....
}

is there any way to avoid above strncpy?

Thanks.

veb...@google.com

unread,
Jan 15, 2021, 4:52:36 PM1/15/21
to grpc.io
This is more related to the way to use protobuf message efficiently. One thing you can try is to use `release_data()` which gives you an ownership of the data so that you can reuse it. Note that this doesn't necessary guarantee it won't involve any memory copy because it works differently depending on whether it's allocated on the arena. But it's worth trying it.
Reply all
Reply to author
Forward
0 new messages