Can you change the WSGI script file from having:
application = trac.web.main.dispatch_request
to:
_application = trac.web.main.dispatch_request
def application(environ, start_response):
del environ['wsgi.file_wrapper']
return _application(environ, start_response)
I just want to eliminate wsgi.file_wrapper extension as being issue.
This presumes you are using mod_wsgi 2.0 or later.
Graham
2008/7/15 Matt Stevens <smas...@gmail.com>:
So just let me know which version of mod_wsgi is being used and I'll
think of other tests you can do.
In interim at least, enable 'debug' LogLevel for Apache. See:
http://code.google.com/p/modwsgi/wiki/DebuggingTechniques
Documentation says 'info', but I want 'debug'.
Look for anything strange in log file for the problematic request.
Graham
2008/7/15 Graham Dumpleton <graham.d...@gmail.com>:
Also, can you use example code in that debugging document I referenced
to capture request WSGI environ and response headers and post them so
can see what the request looks like and what response headers also
look like so can get a feel for what the application is doing.
The problem could be one of two things.
The first is that mod_wsgi is simply broken somehow.
The second is that Trac is not opening files in 'raw' module. The
consequences of this are that if the file contains CRLF pairs, they
would get returned as just LF. For each of these this would have the
effect of decreasing the actual returned content length. Since the
expected content length and actual returned content length are so
different, that however would have to be a lot of CRLF pairs in the
original file, which seems unlikely unless zip algorithm generates a
lot for a reason.
If you really wanted to try something, you could in a Python script do:
fd = open("nant-0.86-beta1-bin.zip", 'r')
data = fd.read()
print len(data)
fd = open("nant-0.86-beta1-bin.zip", 'rb')
data = fd.read()
print len(data)
and tell me what it outputs.
Graham
2008/7/16 Matt Stevens <smas...@gmail.com>:
Have been slowly looking at this, so don't think I have forgotten.
What is weird is that it appears that Trac doesn't appear to use
_FileWrapper when there is no wsgi.file_wrapper when working through a
true WSGI server. It does however use _FileWrapper when running with
its own internal web server.
I need to pose this as a question on Trac developers list to confirm
my reading of source code as don't have version of Trac to run where
am at the moment to try myself.
What you might do for me is use the debugging middleware in section
'Poorly Performing Code' of:
http://code.google.com/p/modwsgi/wiki/DebuggingTechniques
to wrap Trac application. Modify the middleware to remove
'wsgi.file_wrapper' from environ like you did before. Then do your
requests and monitor Apache error log file. Tell me if the middleware
complains about Trac returning an actual file object for the
attachment response. If it is, it can perform badly as described in
that section.
The code in Trac which looks suspect is in web/api.py.
if self.method != 'HEAD':
self._response = file(path, 'rb')
file_wrapper = self.environ.get('wsgi.file_wrapper')
if file_wrapper:
self._response = file_wrapper(self._response, 4096)
raise RequestDone
What this is saying, is if wsgi.file_wrapper exists then use it,
otherwise just return file object. I believe that code should use
_FileWrapper when it doesn't exist.
Other than finding that, my tests on MacOS X haven't found a problem
with returning StringIO instance with wsgi.file_wrapper, which would
trigger same code as on Windows. So, when get a chance, may get you to
try some other things for me. Will need to work out what to suggest
first though. :-)
Graham
BTW, forgot to mention that the reason that non binary is very short,
is that text mode in Windows from memory will still stop reading when
it encounters a ctrl-z in file. This harks back to old DOS days. At
least I think this still happens.
More below .....
As well as that test, can you try test application:
def application(environ, start_response):
status = '200 OK'
response_headers = [('Content-Type', 'text/plain'),]
start_response(status, response_headers)
file = open('/some/absolute/path/nant-0.86-beta1-bin.zip', 'rb')
length = 0
wrapper = environ['wsgi.file_wrapper'](file, 4096)
for block in wrapper:
length += len(block)
return 'LENGTH %d' % length
Fix up path as appropriate. Is the length displayed as expected?
Graham
http://groups.google.com/group/trac-dev/browse_frm/thread/a813b85ed3073b93?hl=en
No response at this time.
Graham
2008/7/20 Graham Dumpleton <graham.d...@gmail.com>:
Haven't had a chance to look further at original issue as reported and
haven't see a response to my request to run the test and provided
result.
Graham
2008/7/21 Graham Dumpleton <graham.d...@gmail.com>:
Graham
2008/7/20 Graham Dumpleton <graham.d...@gmail.com>:
I was offline yesterday, so just catching up on emails etc. With this
I should be able to duplicate it with just wsgi.file_wrapper and not
even require Trac. I'll get on to this one today.
Thanks again.
Graham
2008/8/22 illuminarti <si...@illuminarti.com>:
To save you the bother, I'm attaching said text file. This is the one
with it aligned, so it breaks the download.
Cheers,
Simon
--
Simon J. Oliver
CISSP-ISSAP, ISSMP
Applied Information Technology Center
University of Memphis
Very odd. I have no problem with your file, but can trigger problems
in other ways. Just the following is sufficient:
def application(environ, start_response):
status = '200 OK'
response_headers = [('Content-Type', 'text/plain'),]
start_response(status, response_headers)
path = os.path.join('/tmp/null.txt')
fd = open(path, 'w')
#fd.write(8192*'\0')
fd.write('\0')
fd.seek(0)
return environ['wsgi.file_wrapper'](fd)
Doing HEAD using telnet get:
$ telnet localhost 8224Trying ::1...
Connected to localhost.
Escape character is '^]'.
HEAD /wsgi/scripts/file.py HTTP/1.0
HTTP/1.1 200 OK
Date: Fri, 22 Aug 2008 00:57:38 GMT
Server: Apache/2.2.4 (Unix) mod_wsgi/3.0-TRUNK Python/2.3.5
Content-Length: 1
Connection: close
Content-Type: text/plain
Connection closed by foreign host.
If use curl it just hangs waiting for data.
Don't even need 4k of data. The first character of any 4k block,
including the first, being a null is possibly enough.
I'll have to check the code path which is used, but the whole point of
wsgi.file_wrapper is that it passes control for sending file off to
Apache functions to do. So first impression would be that I can't see
how this can be an issue in mod_wsgi and that it would have to be in
Apache.
Anyway, time for some digging.
Graham
> Cheers,
>
> Simon
>
> --
>
> Simon J. Oliver
> CISSP-ISSAP, ISSMP
> Applied Information Technology Center
> University of Memphis
>
>
> >
>
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!
>
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
> The quick brown fox jumped over the lazy dog!12345The quick brown fox jumped over the lazy dog!1234
>> --~--~---------~--~----~------------~-------~--~----~
>> This message is part of the topic "Strange connection drops from Trac
>> via mod_wsgi" in the Google Group "modwsgi" for which you requested
>> email
>> updates.
>> To stop receiving email updates for this topic, please visit the topic
>> at http://groups.google.com/group/modwsgi/t/90d169459a651120
>> -~----------~----~----~----~------~----~------~--~---
>>
>
>
>
Even if I write a file containing 8192 null characters, can't reproduce it.
This is with Python 2.3, Apache 2.2.4 and MacOS X 10.4.11.
And then something clicks. You are using Apache 1.3. For both Apache
1.3 and Windows Apache 2.X, use of sendfile techniques isn't used and
so any problem would lie in mod_wsgi somewhere. I'll need to disable
the sendfile code manually and then see what happens.
Graham
2008/8/22 Graham Dumpleton <graham.d...@gmail.com>:
With your file and test harness of:
import os
def application(environ, start_response):
status = '200 OK'
response_headers = [('Content-Type', 'text/plain'),]
start_response(status, response_headers)
path = os.path.join(os.path.dirname(__file__), 'null4k.txt')
return environ['wsgi.file_wrapper'](open(path, 'rb'), 4096)
Whatever is causing it, it is already fixed in mod_wsgi 3.0
development version as don't have the problem there. :-)
Just need to work out what changed.
As it happens. I'm using the macports python 2.5 on that that machine,
but that probably doesn't make any difference, given what you're
seeing. I am still using the 1.3.41 apache that came with Tiger, though.
Simon
if (*PyString_AsString(result) == '\0') {
PyErr_SetObject(PyExc_StopIteration, Py_None);
Py_DECREF(args);
Py_DECREF(result);
return 0;
}
Meant to be checking length of string, not whether first character is a null.
Thus:
if (PyString_Size(result) == 0) {
PyErr_SetObject(PyExc_StopIteration, Py_None);
Py_DECREF(args);
Py_DECREF(result);
return 0;
}
It was done correctly in mod_wsgi 3.0 although code in that area had
been changed around to handle Python 3.0 unicode strings. Not sure why
I didn't notice that it was wrong and fix it in mod_wsgi 2.X.
Anyway, the problem would affect mod_wsgi 2.0 and mod_wsgi 2.1 on
Windows or Apache 1.3 on UNIX.
Working version of mod_wsgi 2.2 which includes this and other fixes
can be checked out from subversion at:
https://modwsgi.googlecode.com/svn/branches/mod_wsgi-2.X
More details in:
http://code.google.com/p/modwsgi/wiki/ChangesInVersion0202
Haven't yet updated this with details of this fix yet though.
Graham
import StringIO
data = StringIO.StringIO(8192*'\0')
return environ['wsgi.file_wrapper'](data, 4096)
Thank you so much. I downloaded 2.2 and it does indeed seem to have
fixed the problem.
I appreciate the prompt attention to this issue.
Simon
It always helps when people can give a good pointer, like you did, as
to what the problem is and are also happy to do a bit of debugging if
I have further questions. There have been separate cases in the past
where people have indicated they were having a problem but did little
to debug it themselves nor were they supplying the exact information I
needed when I asked various questions. In one case person came out and
said I was being too difficult when I kept asking questions.
So, it gets frustrating sometimes, but generally more than happy to
help sort problems out, especially when it is a problem in mod_wsgi
code itself. :-)
BTW, since I haven't heard any complaints about other fixes pending
there in 2.2 and haven't had time to investigate further some of the
graceful shutdown issues, I'll try and roll out a version 2.2 now.
I know there are some other potentially worrying issues outstanding at
the moment, but don't have simple answers for those now, plus haven't
had a huge amount of time lately to work on it.
Graham