Setting Content-Length on a streamed file
Charles Duffy
length to the client. This being the case, I set
cherrypy.response.headers['Content-Length'] to the known target length
before returning the generator which actually creates the content.
However, it never makes it to the client: Content-Length is actively
deleted if it exists and we're streaming content. This happens in
_cphttptools.py's finalize() at line 397 in CP2.2.1. I'm presuming
there's a reason this is done -- but would it be safe to delete
Content-Length only if its value is None? (If it's safe, is there
anything I should do to help make this default behavior in future
versions -- file a bug, submit a patch or such?)
Robert Brewer
> I'm sending a very large, dynamically generated file with
> a fixed, known length to the client. This being the case,
> I set cherrypy.response.headers['Content-Length'] to the
> known target length before returning the generator which
> actually creates the content.
I assume you want it so the user-agent can inform the user about progress?
> However, it never makes it to the client: Content-Length
> is actively deleted if it exists and we're streaming content.
> This happens in _cphttptools.py's finalize() at line 397
> in CP2.2.1. I'm presuming there's a reason this is done
> -- but would it be safe to delete Content-Length only if
> its value is None? (If it's safe, is there anything I
> should do to help make this default behavior in future
> versions -- file a bug, submit a patch or such?)
The Content-Length header is deleted in order to more-gracefully handle errors that occur in your generator while the content is being streamed out. If you set Content-Length to 10000, and an exception is raised on byte 10, for example, the client may hang on the socket. As http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.4 says:
"When a message-body is included with a message, the
transfer-length of that body is determined by one of
the following (in order of precedence):
...
3.If a Content-Length header field (section 14.13) is
present...
...
5.By the server closing the connection."
...and _cpwsgiserver does indeed close the connection if there is no CL response header (see _cpwsgiserver.HTTPRequest.send_headers). So it would be "safe" to send it only if your code has no bugs. ;) So IMO we can either provide a safe mechanism or an unsafe mechanism (given the various places one might unwittingly have Content-Length set for them), and IMO we're going to stick with the safe mechanism for now.
The solution I've wanted to pursue for a while now is this: provide a different "safe" mechanism via the "chunked" transfer-coding (see http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6.1), which would require cooperation on the WSGI server side. If we had that, then we could make that the norm and allow "streaming" content to be unsafe in rare cases. But until we have that, I have a hard time only providing an "unsafe" mechanism.
So a ticket would be a good thing. :)
Robert Brewer
System Architect
Amor Ministries
fuma...@amor.org