RST_STREAM problem! who can give some suggestions

[wuliju...@gmail.com]

Hi, all

I wrote a server and client in Python, grpcio version is 1.0.0, protobuf is 3.0.0, os is centos 6.

my proto file is 

-------------------------

service AudioClassification {

rpc Classify (AudioUnit) returns (Prob) {

}

}

message AudioUnit {

    string id = 1;

    bytes audio = 2;

}

message Prob {

    float prob = 1;

}

---------------------------

when I run my client, I will put an audio file content to the field audio. If the file size is 841772 bytes, the return will be ok, but the file size is large, for example,  4482088 bytes, it will return an error, the stacktrace is bellow:

------------------------------------------------------

  File "./client2.py", line 12, in run

    reply = stub.Classify(AudioUnit(id='1', audio=bytes(open('../data/1018465138.wav', 'rb').read())), timeout=3600)

  File "/home/web_server/dlpy2/dlpy/lib/python2.7/site-packages/grpc/_channel.py", line 481, in __call__

    return _end_unary_response_blocking(state, False, deadline)

  File "/home/web_server/dlpy2/dlpy/lib/python2.7/site-packages/grpc/_channel.py", line 432, in _end_unary_response_blocking

    raise _Rendezvous(state, None, None, deadline)

grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.INTERNAL, {"created":"@1472528410.219728253","description":"RST_STREAM","file":"src/core/ext/transport/chttp2/transport/frame_rst_stream.c","file_line":107,"http2_error":2})>

------------------------------------------------------

my client code is:

---------------

def run():

    channel = grpc.insecure_channel('localhost:10411')

    stub = AudioClassificationStub(channel)

    reply = stub.Classify(AudioUnit(id='1', audio=bytes(open('../data/1018465138.wav', 'rb').read())), timeout=3600)

    print(reply.prob)

if __name__ == '__main__':

    run()

---------------

Anyone can give me some tips?

[Nathaniel Manista]

StatusCode.INTERNAL could mean one of a few different things, but I suspect the problem is that you're hitting the message size limit. How much thought have you given to making your Classify method response-streaming and breaking up your audio data into each-no-larger-than-some-size messages?

-Nathaniel

[wuliju...@gmail.com]

how can i show the message size limit? 

在 2016年8月30日星期二 UTC+8下午12:24:55，Nathaniel Manista写道：

[wuliju...@gmail.com]

when I set environment variable GRPC_TRACE=all,  the server will output log:

E0830 14:22:27.692563175  134215 completion_queue.c:254]     Operation failed: tag=0x7f5ca50ed590, error={"created":"@1472538147.692532429","description":"Error in HTTP transport completing operation","file":"src/core/ext/transport/chttp2/transport/chttp2_transport.c","file_line":1097,"referenced_errors":[{"created":"@1472538147.692530892","description":"Attempt to send initial metadata after stream was closed","file":"src/core/ext/transport/chttp2/transport/chttp2_transport.c","file_line":1212}{"created":"@1472538147.692534751","description":"Attempt to send trailing metadata after stream was closed","file":"src/core/ext/transport/chttp2/transport/chttp2_transport.c","file_line":1270}],"target_address":"ipv6:[::1]:44603"}

在 2016年8月30日星期二 UTC+8下午12:01:27，wuliju...@gmail.com写道：

[Nathaniel Manista]

On Mon, Aug 29, 2016 at 9:28 PM, <wuliju...@gmail.com> wrote:

how can i show the message size limit?

The limit's default value is located deep inside the gRPC Core. We don't yet have APIs for setting application-specific maximum message sizes in all languages and on both the invocation-side and service-side of an RPC, but we intend to provide that capability in future development of the implementation. Between now and then, the best approach to take is to keep your messages under that limit.

... but knowing a maximum and a minimum doesn't necessarily help with the question of into what size of messages you should break up your data, so I had to ask around for that: 16KiB to 64KiB.

Given that there's currently no way for one side of an RPC to ask the other side what maximum message size is supported, it makes sense to have the protocol require that all implementations support a certain minimum message size. I couldn't find any such minimum message size specified, so here's an issue for that too.

-N

[wuliju...@gmail.com]

Thanks. In 1.0.0 version, the max message size is 4MiB, prior version is 100MiB.

Now my solution as below:

At client side, we can provide an option max_message_length, for example.

    options = [(cygrpc.ChannelArgKey.max_message_length, 100 * 1024 * 1024)]

    channel = grpc.insecure_channel('localhost:10419', options)

At server side, there is no options param for server interface. 

But we can modify the library, for example: https://github.com/grpc/grpc/pull/7926  or you  can extend the Server class. I select the second solution:

from grpc._cython import cygrpc

from grpc import _server

class MyServer(_server.Server):

    _state = None

    def __init__(self, thread_pool, generic_handlers, args=None):

        completion_queue = cygrpc.CompletionQueue()

        server = cygrpc.Server(args)

        server.register_completion_queue(completion_queue)

        self._state = _server._ServerState(

            completion_queue, server, generic_handlers, thread_pool)

在 2016年8月30日星期二 UTC+8下午12:01:27，wuliju...@gmail.com写道：

Hi, all