Slow delivery of css files by CherryPy server - CherryPy bug
JLIST
I'm using the built-in web server for development on Windows with
web.py 0.22. I find that when a page references a css file,
the page takes about 10 second to load. But if I remove the css link,
the page loads immediately (w/o style.) I then access the css file
directly like this: http://localhost:8080/static/style.css
and the css file takes about 10 seconds to load.
I tried with an image file in static folder, which loads
immediately. I tried both IE and firefox, same behavior. The server
command line shows an entry for style.css immediately but it
won't be loaded until after about 10 seconds in browser.
I tried retrieving the css with Python urllib, I get the file
immediately every time. So it seems that the problem is in the
browser - it seems that the browser doesn't think it's got the whole
file.
I went ahead and studied the headers that got sent between IE and
CherryPy. I notice that CherryPy is doing three things wrong here:
1. It replies HTTP/1.0 with Keep-Alive with HTTP/1.1 reply
2. It sends "Connection: Keep-Alive" which is not necessary
because HTTP/1.1 defaults to keep-alive.
3. It eats 0xd from 0xd 0xa in the original file, and only sends 0xa,
but the Content-Length it reports includes 0xd so the browser was
expecting more data. Contnet-Length says 1178 bytes but it actually
sends 1100 bytes.
Problem 1 and 2 are not too bad. Problem 3 is bad and seems to be
what's causing the problem. So the css file won't be displayed until
CherryPy times out the connection and sends a FIN packet.
Hopefully someone who's familiar with the code can double-check and
come up with a fix :)
(Request Method) GET /static/style.css HTTP/1.0
Accept image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*
Accept-Language en-us
User-Agent Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727)
Host localhost:8080
Connection Keep-Alive
(Response Status) HTTP/1.1 200
Content-type text/css
Content-Length 1178
Last-Modified Tue, 25 Dec 2007 22:50:41 GMT
Connection Keep-Alive
Date Wed, 26 Dec 2007 02:59:26 GMT
Server CherryPy/3.0.1
Yingbo Qiu
Anand Chitipothu
>
> I think next major release should bundle the newest wsgiserver/__init__.py
The newest version available is 3.1beta.
http://www.cherrypy.org/log/trunk/cherrypy/wsgiserver/__init__.py
We can include it once they release 3.1.
Anand Chitipothu
SimpleHTTPHandler for handling static files. There could be a problem
there.
JLIST
with the ajax errors.
http://www.cherrypy.org/log/trunk/cherrypy/wsgiserver/__init__.py
Sławek Tuleja
Tzury Bar Yochay
CRLF vs. LF ('\r\n' vs. '\n') seems to be the problem on windows when
a text editor uses CR+LF.
I looked through the web.py code (v 0.22) and here are the results:
do_GET calls copyfile which says in its docstring that it should be
override when
using CRLF (see below).
Is this is the approach taken in version 0.3?
# /usr/lib/python2.5/SimpleHTTPServer.py line: 161
def copyfile(self, source, outputfile):
"""Copy all data between two file objects.
The SOURCE argument is a file object open for reading
(or anything with a read() method) and the DESTINATION
argument is a file object open for writing (or
anything with a write() method).
The only reason for overriding this would be to change
the block size or perhaps to replace newlines by CRLF
-- note however that this the default server uses this
to copy binary data as well.
"""
shutil.copyfileobj(source, outputfile)
Anand Chitipothu
yes.
Jack
shutils.copyfileobj works fine if the file is open as binary. The
problem seems to be in SimpleHTTPServer.send_head(), which opens "css"
files as text file.
I wonder if it's necessary to differentiate a text file and a binary
file here at all. Why not always open as "rb".
Anand Chitipothu
>
> Looks like someone else has been bitten as well.
>
> shutils.copyfileobj works fine if the file is open as binary. The
> problem seems to be in SimpleHTTPServer.send_head(), which opens "css"
> files as text file.
>
> I wonder if it's necessary to differentiate a text file and a binary
> file here at all. Why not always open as "rb".
Here is a monkey patch to fix it.
# httppatch.py
from SimpleHTTPServer import SimpleHTTPRequestHandler
send_head = SimpleHTTPRequestHandler.send_head
def new_send_head(*a, **kw):
print 'new_send_head', a, kw
f = send_head(*a, **kw)
return f and open(f.name, 'rb')
SimpleHTTPRequestHandler.send_head = new_send_head
import this module before calling web.run.
JLIST
The ajax problem that I mentioned in my other email seems to be a
trickier problem because I have problem with both cherrypy server
and SimpleHTTPServer, but the symptom is different. I haven't digged
deeper but I noticed that cherrypy's responses involve HTTP chunking.
Yingbo Qiu
> The newest version available is 3.1beta.
>
> http://www.cherrypy.org/log/trunk/cherrypy/wsgiserver/__init__.py
>
> We can include it once they release 3.1.
3.1 have been released ... please merge it into 0.3