CherryPy 3.2, Python 3.1 and default io buffer size

[Glenn]

I tried running CherryPy 3.2.0 rc1 on Ubuntu 9.04 with Python 3.1.1
and got this error which prevented it from starting up:

[22/Oct/2009:15:21:18] ENGINE Listening for SIGHUP.
[22/Oct/2009:15:21:18] ENGINE Listening for SIGTERM.
[22/Oct/2009:15:21:18] ENGINE Listening for SIGUSR1.
[22/Oct/2009:15:21:18] ENGINE Bus STARTING
[22/Oct/2009:15:21:18] ENGINE Started monitor thread 'Autoreloader'.
[22/Oct/2009:15:21:18] ENGINE Started monitor thread
'_TimeoutMonitor'.
[22/Oct/2009:15:21:18] ENGINE Serving on 127.0.0.1:8080
[22/Oct/2009:15:21:18] ENGINE Bus STARTED
8192
[22/Oct/2009:15:21:18] ENGINE Error in HTTP server: shutting down
Traceback (most recent call last):
File "/home/glenn/scrap/usenet_scraper/cherrypy/process/servers.py",
line 75, in _start_http_thread
self.httpserver.start()
File "/home/glenn/scrap/usenet_scraper/cherrypy/wsgiserver/
__init__.py", line 1344, in start
self.tick()
File "/home/glenn/scrap/usenet_scraper/cherrypy/wsgiserver/
__init__.py", line 1418, in tick
conn = self.ConnectionClass(self, s, makefile)
File "/home/glenn/scrap/usenet_scraper/cherrypy/wsgiserver/
__init__.py", line 912, in __init__
self.rfile = makefile(sock, "rb", self.rbufsize)
File "/home/glenn/scrap/usenet_scraper/cherrypy/wsgiserver/
__init__.py", line 891, in CP_makefile
return io.BufferedReader(socket.SocketIO(sock, mode), bufsize)
File "/opt/python-3.1.1/lib/python3.1/_pyio.py", line 875, in
__init__
raise ValueError("invalid buffer size")
ValueError: invalid buffer size

[22/Oct/2009:15:21:18] ENGINE Bus STOPPING
[22/Oct/2009:15:21:18] ENGINE HTTP Server
cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 8080)) shut down
[22/Oct/2009:15:21:18] ENGINE Stopped thread '_TimeoutMonitor'.
[22/Oct/2009:15:21:18] ENGINE Stopped thread 'Autoreloader'.
[22/Oct/2009:15:21:18] ENGINE Bus STOPPED
[22/Oct/2009:15:21:18] ENGINE Bus EXITING
[22/Oct/2009:15:21:18] ENGINE Bus EXITED
Exception in thread HTTPServer Thread-3:
Traceback (most recent call last):
File "/opt/python-3.1.1/lib/python3.1/threading.py", line 509, in
_bootstrap_inner
self.run()
File "/opt/python-3.1.1/lib/python3.1/threading.py", line 462, in
run
self._target(*self._args, **self._kwargs)
File "/home/glenn/scrap/usenet_scraper/cherrypy/process/servers.py",
line 75, in _start_http_thread
self.httpserver.start()
File "/home/glenn/scrap/usenet_scraper/cherrypy/wsgiserver/
__init__.py", line 1344, in start
self.tick()
File "/home/glenn/scrap/usenet_scraper/cherrypy/wsgiserver/
__init__.py", line 1418, in tick
conn = self.ConnectionClass(self, s, makefile)
File "/home/glenn/scrap/usenet_scraper/cherrypy/wsgiserver/
__init__.py", line 912, in __init__
self.rfile = makefile(sock, "rb", self.rbufsize)
File "/home/glenn/scrap/usenet_scraper/cherrypy/wsgiserver/
__init__.py", line 891, in CP_makefile
return io.BufferedReader(socket.SocketIO(sock, mode), bufsize)
File "/opt/python-3.1.1/lib/python3.1/_pyio.py", line 875, in
__init__
raise ValueError("invalid buffer size")
ValueError: invalid buffer size

[22/Oct/2009:15:21:18] ENGINE Waiting for child threads to
terminate...


So I saw the default bufsize parameter for CP_makefile() and makefile
() is set to -1 which seems to be no good on my machine. I went and
set these to the default supplied in io.DEFAULT_BUFFER_SIZE. The diffs
are below:

--- CherryPy-3.2.0rc1/cherrypy/wsgiserver/__init__.py 2009-10-18
21:38:25.000000000 -0400
+++ ./cherrypy/wsgiserver/__init__.py 2009-10-22 15:27:13.000000000
-0400
@@ -885,7 +885,7 @@
del self._write_buf[:n]


-def CP_makefile(sock, mode='r', bufsize=-1):
+def CP_makefile(sock, mode='r', bufsize=io.DEFAULT_BUFFER_SIZE):
if 'r' in mode:
return io.BufferedReader(socket.SocketIO(sock, mode),
bufsize)
else:
@@ -909,7 +909,7 @@
self.server = server
self.socket = sock
self.rfile = makefile(sock, "rb", self.rbufsize)
- self.wfile = makefile(sock, "wb", -1)
+ self.wfile = makefile(sock, "wb")

def communicate(self):
"""Read each request and respond appropriately."""
@@ -968,7 +968,7 @@
except NoSSLError:
if req and not req.sent_headers:
# Unwrap our wfile
- self.wfile = CP_makefile(self.socket._sock, "wb", -1)
+ self.wfile = CP_makefile(self.socket._sock, "wb")
req.simple_response("400 Bad Request",
"The client sent a plain HTTP request, but "
"this server only speaks HTTPS on this port.")
@@ -1400,7 +1400,7 @@
"Content-Type: text/plain\r\n\r\n",
msg]

- wfile = CP_makefile(s, "wb", -1)
+ wfile = CP_makefile(s, "wb")
try:
wfile.write("".join(buf).encode
('ISO-8859-1'))
except socket.error as x:




--- CherryPy-3.2.0rc1/cherrypy/wsgiserver/ssl_builtin.py 2009-09-17
11:23:56.000000000 -0400
+++ ./cherrypy/wsgiserver/ssl_builtin.py 2009-10-22 15:28:34.000000000
-0400
@@ -64,6 +64,6 @@
}
return ssl_environ

- def makefile(self, sock, mode='r', bufsize=-1):
+ def makefile(self, sock, mode='r',
bufsize=io.DEFAULT_BUFFER_SIZE):
return wsgiserver.CP_makefile(sock, mode, bufsize)

[Robert Brewer]

Glenn wrote:
> I tried running CherryPy 3.2.0 rc1 on Ubuntu 9.04 with Python 3.1.1
> and got this error which prevented it from starting up:
>

> File "/opt/python-3.1.1/lib/python3.1/_pyio.py", line 875, in
> __init__
> raise ValueError("invalid buffer size")
> ValueError: invalid buffer size

Applied in http://www.cherrypy.org/changeset/2561. Thanks!

Robert Brewer
fuma...@aminus.org