[CherryPy] #966: environ["wsgi.input"] may hang indefinitely if read()

[CherryPy]

#966: environ["wsgi.input"] may hang indefinitely if read()
------------------------------------------+---------------------------------
Reporter: tuure.la...@indagon.com | Owner: fumanchu
Type: defect | Status: new
Priority: normal | Milestone:
Component: wsgiserver | Keywords:
------------------------------------------+---------------------------------
If read() is called on environ["wsgi.input"] in a WSGI app on a request
that was not chunked (i.e. SizeCheckWrapper.rfile is a CP_fileobject),
CherryPy attempts to recv CP_fileobject.default_bufsize or more, i.e. at
least 8192 bytes on the underlying socket. This obviously blocks unless
the request body is huge.

If read() without specific size argument is not supposed to be supported,
failing earlier and with a descriptive error message would be prefarable
to current situation.

--
Ticket URL: <http://www.cherrypy.org/ticket/966>
CherryPy <http://www.cherrypy.org>
CherryPy - a pythonic, object-oriented HTTP framework

[CherryPy]

#966: environ["wsgi.input"] may hang indefinitely if read()
------------------------------------------+---------------------------------
Reporter: tuure.la...@indagon.com | Owner: fumanchu
Type: defect | Status: new
Priority: normal | Milestone:

Component: wsgiserver | Resolution:
Keywords: |
------------------------------------------+---------------------------------
Comment (by Graham.D...@gmail.com):

If your WSGI application is calling read() without an argument you are
violating the WSGI 1.0 specification regardless of what CherryPy does. In
other words the WSGI specification requires that you supply a size and
that you never attempt to read more data than specified by CONTENT_LENGTH
variable in WSGI environment.

[CherryPy]

#966: environ["wsgi.input"] may hang indefinitely if read()
------------------------------------------+---------------------------------
Reporter: tuure.la...@indagon.com | Owner: fumanchu
Type: defect | Status: new
Priority: normal | Milestone:

Component: wsgiserver | Resolution:
Keywords: |
------------------------------------------+---------------------------------

Comment (by tuure.la...@indagon.com):

Yes, it seems to break the specification. What about my other point of
failing fast and with descriptive error rather than slowly and erratically
if CherryPy intends to not support read()?

[CherryPy]

#966: environ["wsgi.input"] may hang indefinitely if read()
------------------------------------------+---------------------------------
Reporter: tuure.la...@indagon.com | Owner: fumanchu
Type: defect | Status: new
Priority: normal | Milestone:

Component: wsgiserver | Resolution:
Keywords: |
------------------------------------------+---------------------------------

Comment (by lawouach):

I'm not sure I understand what you expect CherryPy to do. You attempt to
read any number of bytes, it seems rather logical the call blocks unless
it receives at least buffer size of data. What's erratic in that behavior?
Is this failing randomly?

[CherryPy]

#966: environ["wsgi.input"] may hang indefinitely if read()
------------------------------------------+---------------------------------
Reporter: tuure.la...@indagon.com | Owner: fumanchu
Type: defect | Status: new
Priority: normal | Milestone:

Component: wsgiserver | Resolution:
Keywords: |
------------------------------------------+---------------------------------

Comment (by tuure.la...@indagon.com):

I originally expected read() to read the whole body of the current HTTP
message and nothing more. Trying to read *more* will obviously either a)
block or b) read something that doesn't belong to the request that is
being processed and could only be a pipelined request or something silly.
I don't see why it would ever be useful for CherryPy to do that.

Then Graham noted that I shouldn't be using read() in the first place.
After another read of the WSGI spec I agree that it is clearly out-of-
spec. For some reason CherryPy still supports it. I think that support
should either be made useful (change whatever reads from the actual socket
to not read more than Content-Length specified) or removed (remove the
default parameter from read() in SizeCheckWrapper and maybe elsewhere).

[CherryPy]

#966: environ["wsgi.input"] may hang indefinitely if read()
------------------------------------------+---------------------------------
Reporter: tuure.la...@indagon.com | Owner: fumanchu
Type: defect | Status: new
Priority: normal | Milestone:

Component: wsgiserver | Resolution:
Keywords: |
------------------------------------------+---------------------------------

Comment (by lawouach):

I don't read Graham's message as you do. Graham only says that the WSGI
spec tells you to provide the size you want to read from the socket. In
other words, you can call read without a size but that is against the
spec. If you are not trying to be WSGI compliant, you can call read and
block as long as you know this is what you expect.

What would be useful probably is to document more the behavior and
indicates reducing the bufsize is advised when dealing with small
requests.

[CherryPy]

#966: environ["wsgi.input"] may hang indefinitely if read()
------------------------------------------+---------------------------------
Reporter: tuure.la...@indagon.com | Owner: fumanchu
Type: defect | Status: new
Priority: normal | Milestone:

Component: wsgiserver | Resolution:
Keywords: |
------------------------------------------+---------------------------------

Comment (by Graham.D...@gmail.com):

The reason it blocks is because for the version of CherryPy WSGI server
being used there is no end sentinel of an empty string being returned when
all content as specified by Content-Length has been read. That CherryPy
does this is its right within WSGI 1.0 because an end sentinel is not
mandatory.

You will find that CherryPy WSGI server in either repository trunk or
branch (not sure which) behaves differently and will return an empty
string when content as specified by Content-Length is exhausted. This is
because that latter version is implementing guarantees proposed as later
revision of WSGI specification and as I describe in
'http://blog.dscpl.com.au/2009/10/details-on-
wsgi-10-amendmentsclarificat.html'.

The Apache/mod_wsgi module has always provided those guarantees but since
was outside of scope of WSGI 1.0 specification, technically a WSGI 1.0
compliant application couldn't rely on them. The intent is that WSGI 1.1
make those guarantees and clarifications. There has though be no mandate
from Python WEB-SIG to agree to that though, so CherryPy and mod_wsgi are
getting out on a limb and doing it anyway as far as saying they implement
WSGI 1.1.

[CherryPy]

#966: environ["wsgi.input"] may hang indefinitely if read()
------------------------------------------+---------------------------------
Reporter: tuure.la...@indagon.com | Owner: fumanchu

Type: defect | Status: closed
Priority: normal | Milestone: 3.2
Component: wsgiserver | Resolution: fixed
Keywords: |
------------------------------------------+---------------------------------
Changes (by fumanchu):

* milestone: => 3.2
* resolution: => fixed
* status: new => closed

Comment:

Fixed in 3.2