Re: [tornado] ERROR:root:Reached maximum read buffer size

[Roey Berman]

10 MB is the default IOStream.max_buffer_size param.
I think the best way to override this is to subclass HTTPServer and override handle_stream() like so (not tested):

class LargeFileHTTPServer(HTTPServer):
    def handle_stream(self, stream, address):
        stream.IOStream.max_buffer_size = newsize
        super(LargeFileHTTPServer, self). handle_stream( stream, address)

If you're expecting heavy loads you might consider to use nginx for handling uploads, it's better at it.

Hth.

On Oct 21, 2012 12:26 AM, "Shrikar archak" <shri...@gmail.com> wrote:

Hi All,

I was trying to upload a file of 10MB to my tornado server and I seeing these error

ERROR:root:Reached maximum read buffer size

ERROR:root:Error in connection callback

Traceback (most recent call last):

  File "/Library/Python/2.7/site-packages/tornado-2.4-py2.7.egg/tornado/netutil.py", line 235, in _handle_connection

    self.handle_stream(stream, address)

  File "/Library/Python/2.7/site-packages/tornado-2.4-py2.7.egg/tornado/httpserver.py", line 146, in handle_stream

    self.no_keep_alive, self.xheaders)

  File "/Library/Python/2.7/site-packages/tornado-2.4-py2.7.egg/tornado/httpserver.py", line 172, in __init__

    self.stream.read_until(b("\r\n\r\n"), self._header_callback)

  File "/Library/Python/2.7/site-packages/tornado-2.4-py2.7.egg/tornado/iostream.py", line 153, in read_until

    self._try_inline_read()

  File "/Library/Python/2.7/site-packages/tornado-2.4-py2.7.egg/tornado/iostream.py", line 387, in _try_inline_read

    if self._read_to_buffer() == 0:

  File "/Library/Python/2.7/site-packages/tornado-2.4-py2.7.egg/tornado/iostream.py", line 436, in _read_to_buffer

    raise IOError("Reached maximum read buffer size")

IOError: Reached maximum read buffer size

Am I missing something?

Thanks,

Shrikar

[Shrikar archak]

I saw that the default read buffersize is 100MB .

http://www.tornadoweb.org/documentation/_modules/tornado/iostream.html

I hope I am not missing any configurations.

Thanks,

Shrikar

[Ben Darnell]

From this stack trace, you're hitting the limit (100MB) while reading
the headers, not the uploaded file. How are you making the upload?
Are you sure it's a well-formed request?

-Ben

[Roey Berman]

Sorry, missed that extra zero.