(9)Bad file descriptor: core_output_filter: writing data to the network
gert
if 'wsgi.file_wrapper' in environ: return environ['wsgi.file_wrapper']
(f, 8192)
else: return iter(lambda: f.read(8192), '')
For some reason it doesn't work anymore ?
Graham Dumpleton
Which version of Python 2.5 or 3.0?
Does it fail on both or only one?
Does it still work for same program under mod_wsgi 2.3 rather than
mod_wsgi 3.0 trunk you are probably using?
Graham
gert
On Feb 4, 2:29 am, Graham Dumpleton <graham.dumple...@gmail.com>
wrote:
> 2009/2/2 gert <gert.cuyk...@gmail.com>:
At revision 1177.
If I recall correctly it did work just before you did the last update
1161 or something.
What was the revision number of 2.3 again ?
Graham Dumpleton
Changes made since don't touch wsgi.file_wrapper code.
> What was the revision number of 2.3 again ?
Use tar ball, or separate svn branch.
https://modwsgi.googlecode.com/svn/branches/mod_wsgi-2.X
Graham
gert
usr/python/bin/python3.0 --with-apxs=/usr/httpd/bin/apxs
checking Apache version... 2.2.11
File "<string>", line 1
from distutils import sysconfig; print sysconfig.get_config_var
("VERSION")
^
SyntaxError: invalid syntax
File "<string>", line 1
from distutils import sysconfig; print "-I" +
sysconfig.get_config_var("INCLUDEPY")
^
SyntaxError: invalid syntax
File "<string>", line 1
from distutils import sysconfig; print " ".join(filter(lambda
x: x.startswith("-D"), sysconfig.get_config_var("CFLAGS").split
()))
^
SyntaxError: invalid syntax
File "<string>", line 1
from distutils import sysconfig; print sysconfig.get_config_var
("PYTHONFRAMEWORKDIR")
^
SyntaxError: invalid syntax
File "<string>", line 1
from distutils import sysconfig; print sysconfig.get_config_var
("PYTHONFRAMEWORKPREFIX")
^
SyntaxError: invalid syntax
File "<string>", line 1
from distutils import sysconfig; print sysconfig.get_config_var
("PYTHONFRAMEWORK")
^
SyntaxError: invalid syntax
File "<string>", line 1
from distutils import sysconfig; print
sysconfig.get_config_var("LINKFORSHARED").replace( "/
Versions//", '')
^
SyntaxError: invalid syntax
File "<string>", line 1
from distutils import sysconfig; print
sysconfig.get_config_var("LIBS")
^
SyntaxError: invalid syntax
configure: creating ./config.status
config.status: creating Makefile
root@r12276:~/mod_wsgi-2.3#
2.3 is sooo 80's like in not going to work with py3000 :-)
Graham Dumpleton
I am not expecting you to build mod_wsgi 2.3 with Python 3.0, just
confirm it works for what you are doing with any older version of
Python. This will eliminate the possibility that there is something
flaky with your Apache setup.
Graham
gert
wrote:
> 2009/2/6 gert <gert.cuyk...@gmail.com>:
(9)Bad file descriptor: core_output_filter: writing data to the
network
doh!
gert
from os import path
f=open(path.join(path.dirname(__file__),'../bin/
picture.png'),'rb')
# if 'wsgi.file_wrapper' in environ: return environ
['wsgi.file_wrapper'](f, 8192)
# else:
return iter(lambda: f.read(8192), '')
Graham Dumpleton
Use the test script from genro's post on February 4th in discussion:
http://groups.google.com/group/modwsgi/browse_frm/thread/69d3076689a40aa9
In particular:
def application(environ, start_response):
size=environ.get('QUERY_STRING')
if size=='': size='255'
size=int(size)
file=open('tmp/wsgitest','wb')
for i in range(size):
file.write('*')
file.close()
file=open('tmp/wsgitest','rb')
file_wrapper = environ.get('wsgi.file_wrapper', None)
start_response('200 OK',[])
return file_wrapper(file, 4096*16)
See if you find similar results where whether it works depends on file size.
It is interesting that genro sees a difference at 256, as that is a
key value in Apache for which below it doesn't try and use sendfile.
#define AP_MIN_SENDFILE_BYTES (256)
Another interesting constant in Apache is:
#define AP_MIN_BYTES_TO_WRITE 8000
Which is close to the other end of the range where they see a problem.
I haven't myself yet had a chance since getting back home yesterday to
try that test script.
PS. When replying, please do not discard the previous posts of the
discussion so context isn't lost.
Graham
Graham Dumpleton
And before you complain the script doesn't work, fix the obvious of
adding '/' at front of paths so uses /tmp, rather than 'tmp' directory
of what ever current working directory of Apache may be.
Graham
gert
wrote:
> 2009/2/9 gert <gert.cuyk...@gmail.com>:
> > when i comment out wsgi.file it works
>
> > from os import path
> > f=open(path.join(path.dirname(__file__),'../bin/
> > picture.png'),'rb')
> > # if 'wsgi.file_wrapper' in environ: return environ
> > ['wsgi.file_wrapper'](f, 8192)
> > # else:
> > return iter(lambda: f.read(8192), '')
>
> Use the test script from genro's post on February 4th in discussion:
>
> In particular:
>
> def application(environ, start_response):
> size=environ.get('QUERY_STRING')
> if size=='': size='255'
> size=int(size)
> file=open('tmp/wsgitest','wb')
> for i in range(size):
> file.write('*')
> file.close()
> file=open('tmp/wsgitest','rb')
> file_wrapper = environ.get('wsgi.file_wrapper', None)
> start_response('200 OK',[])
> return file_wrapper(file, 4096*16)
>
> See if you find similar results where whether it works depends on file size.
>
> It is interesting that genro sees a difference at 256, as that is a
> key value in Apache for which below it doesn't try and use sendfile.
>
> #define AP_MIN_SENDFILE_BYTES (256)
>
> Another interesting constant in Apache is:
>
> #define AP_MIN_BYTES_TO_WRITE 8000
>
> Which is close to the other end of the range where they see a problem.
>
> I haven't myself yet had a chance since getting back home yesterday to
> try that test script.
>
> PS. When replying, please do not discard the previous posts of the
> discussion so context isn't lost.
> adding '/' at front of paths so uses /tmp, rather than 'tmp' directory
> of what ever current working directory of Apache may be.
http://91.121.53.159/test.wsgi?1 = OK
...
http://91.121.53.159/test.wsgi?255 = OK
http://91.121.53.159/test.wsgi?256 = (9)Bad file descriptor:
def application(environ, start_response):
size=environ.get('QUERY_STRING')
if size=='': size='255'
size=int(size)
for i in range(size):
file.write(b'*') # <- Could have been me complaining :P
file.close()
file=open('/tmp/wsgitest','rb')
start_response('200 OK',[])
return file_wrapper(file, 4096*16)
this list :)
Graham Dumpleton
try with Apache 2.2.9 and Python 2.5.2 that ship with MacOS X
(Leopard) instead of your separate versions.
Ie., for mod_wsgi use:
./configure --with-apxs=/usr/sbin/apxs --with-python=/usr/bin/python
if PATH is picking up other versions first.
At the moment the thing that links both of you is that you are using
Apache 2.2.11. My quick test on older version supplied with MacOS X
works fine.
I will have to try and install Apache 2.2.11 and see if I get a problem with it.
Graham
2009/2/10 gert <gert.c...@gmail.com>:
Graham Dumpleton
>
> Il giorno 10/feb/09, alle ore 01:05, Graham Dumpleton ha scritto:
>
>> Giovanni, if you are still reading, would it be possible for you to
>> try with Apache 2.2.9 and Python 2.5.2 that ship with MacOS X
>> (Leopard) instead of your separate versions.
>>
>> Ie., for mod_wsgi use:
>>
>> ./configure --with-apxs=/usr/sbin/apxs --with-python=/usr/bin/python
>>
>> if PATH is picking up other versions first.
>>
>> At the moment the thing that links both of you is that you are using
>> Apache 2.2.11. My quick test on older version supplied with MacOS X
>> works fine.
>>
>> I will have to try and install Apache 2.2.11 and see if I get a problem
>> with it.
>
>
>
> In this way it works.
>
> Is still a way to use a different Apache and Python ?
Possibly just don't use Apache 2.2.11. Try with Apache 2.2.10 and
whatever Python version you want.
I am going to try and duplicate issue with Apache 2.2.11 tonight, so
will see what I find.
Graham
Graham Dumpleton
Duplicated problem.
What is odd is that Apache 2.2.9 that comes with MacOS X is fine.
A version of 2.2.9 that I had installed previously from source code also worked.
I then installed 2.2.11 and 2.2.10 and neither worked.
I then rebuilt 2.2.9 from source again and it also then stopped working.
When compiling from source code they are using APR that ships with
MacOS X which is an older version and probably shouldn't be used. Even
when I build Apache with option --with-included-apr, it still fails,
although the type of page that displays in Safari is different,
getting a blank screen rather than a page from Safari saying
connection had been dropped.
Definitely something fishy going on, but have something to work with now.
If this affects so many versions, don't understand why people haven't
been screaming about it.
Can you both tell me what options you are supplying to 'configure'
when building Apache from source code.
Graham
Graham Dumpleton
Possible fix/workaround (not sure which yet).
Edit mod_wsgi.c and go to function Adapter_output_file().
Search down for:
b = apr_bucket_eos_create(r->connection->bucket_alloc);
APR_BRIGADE_INSERT_TAIL(bb, b);
Add in front of those two lines:
/* XXX */
b = apr_bucket_flush_create(r->connection->bucket_alloc);
APR_BRIGADE_INSERT_TAIL(bb, b);
/* XXX */
Thus, you end up with:
/* XXX */
b = apr_bucket_flush_create(r->connection->bucket_alloc);
APR_BRIGADE_INSERT_TAIL(bb, b);
/* XXX */
b = apr_bucket_eos_create(r->connection->bucket_alloc);
APR_BRIGADE_INSERT_TAIL(bb, b);
Need to work out why a flush is required as thought eos would achieve
same result ans that has worked before.
Graham
Graham Dumpleton
Odd, leaving out both, ie.,
#if 0
/* XXX */
b = apr_bucket_flush_create(r->connection->bucket_alloc);
APR_BRIGADE_INSERT_TAIL(bb, b);
/* XXX */
b = apr_bucket_eos_create(r->connection->bucket_alloc);
APR_BRIGADE_INSERT_TAIL(bb, b);
#endif
also results in correct behaviour. Thus it is not so much the lack of
the flush, but that the eos is being inserted.
So, as fix while I work out what it should do, can also not add the
flush, but comment out the two eos lines.
Graham
gert
wrote:
> 2009/2/10 Graham Dumpleton <graham.dumple...@gmail.com>:
>
>
> > 2009/2/10 Porcari Giovanni <giovanni.porc...@softwell.it>:
enable-mime=shared --enable-deflate=shared --enable-authz_host=shared
--enable-autoindex=shared --enable-dir=shared
Graham Dumpleton
Okay, think I have worked it out. Quite serious problem and not sure
why people haven't been complaining about it left right and centre.
First off, Apache has this bit of code:
/* It doesn't make any sense to use sendfile for a file bucket
* that represents 10 bytes.
*/
else if (APR_BUCKET_IS_FILE(e)
&& (e->length >= AP_MIN_SENDFILE_BYTES)) {
What this does is say that if length is less than 256 bytes, then
don't bother using sendfile as waste of time, so it just sends it out
without trying to optimise it.
Next relevant thing it has is:
if (nbytes + flen < AP_MIN_BYTES_TO_WRITE
&& !AP_BUCKET_IS_EOC(last_e)
&& ((!fd && !more && !APR_BUCKET_IS_FLUSH(last_e))
|| (APR_BUCKET_IS_EOS(last_e)
&& c->keepalive == AP_CONN_KEEPALIVE))) {
This is part of some optimisation. If the amount of data is less than
8000 bytes, minus nbytes whatever that works out as, then if next
bucket is EOS, it decides that it will not actually send the data
straight away but hold it over in case there is more data later that
can be sent at the same time.
The problem with this is that mod_wsgi is assuming that all data will
be sent straight away. It relies on this as the Python fle object
would then be destroyed. Because the data hasn't been sent, Apache
still holds a reference to the file descriptor, yet Python file object
destroyed and file descriptor closed. End result is that when at end
of request Apache finally tries to send the data, the file descriptor
is no longer valid and the error occurs.
It would seem therefore that correct thing to do is to send a flush
instead of an eos.
So have:
b = apr_bucket_flush_create(r->connection->bucket_alloc);
APR_BRIGADE_INSERT_TAIL(bb, b);
instead of:
b = apr_bucket_eos_create(r->connection->bucket_alloc);
APR_BRIGADE_INSERT_TAIL(bb, b);
My only concern about this is that documentation says about flush:
/**
* Create a flush bucket. This indicates that filters should flush their
* data. There is no guarantee that they will flush it, but this is the
* best we can do.
* @param list The freelist from which this bucket should be allocated
* @return The new bucket, or NULL if allocation failed
*/
APU_DECLARE(apr_bucket *) apr_bucket_flush_create(apr_bucket_alloc_t *list);
The bit that obviously isn't good is 'There is no guarantee that they
will flush it'.
Annoyingly, for eos documentation says:
**
* Create an End of Stream bucket. This indicates that there is no more data
* coming from down the filter stack. All filters should flush at this point.
* @param list The freelist from which this bucket should be allocated
* @return The new bucket, or NULL if allocation failed
*/
APU_DECLARE(apr_bucket *) apr_bucket_eos_create(apr_bucket_alloc_t *list);
Thus, it says that flush should occur, which was what I was relying
upon, but as shown above it doesn't as stupid output filter attempts
to optimise it.
I need to figure how you can gaurantee that flush has actually occurred. :-(
I'll think about it and may be post on one of the httpd developer lists.
Graham
Graham Dumpleton
http://code.google.com/p/modwsgi/issues/detail?id=132
Fix for 3.0 in subversion trunk at revision 1197.
I still need to look at back porting this to 2.X branch.
Because I regard this as a quite serius issue, looks like I'll now
need to hurry along with getting 2.4 out with a number of accumulated
bug fixes.
BTW, the change is to add the flush bucket before the eos. Apache dev
list suggested that inserting flush should be okay.
Graham
2009/2/13 Graham Dumpleton <graham.d...@gmail.com>:
Graham Dumpleton
The 2.X branch can be checked out from:
http://modwsgi.googlecode.com/svn/branches/mod_wsgi-2.X
For more details about this what is being fixed in 2.4 see:
http://code.google.com/p/modwsgi/wiki/ChangesInVersion0204
Graham
2009/2/14 Graham Dumpleton <graham.d...@gmail.com>:
gert
mbertoldi
I'm going to try the new version with python 2.5.4.
Michele
On Feb 14, 12:11 pm, Graham Dumpleton <graham.dumple...@gmail.com>
wrote:
>
> The 2.X branch can be checked out from:
>
> http://modwsgi.googlecode.com/svn/branches/mod_wsgi-2.X
>
> For more details about this what is being fixed in 2.4 see:
>
> http://code.google.com/p/modwsgi/wiki/ChangesInVersion0204
>
> Graham
>
> > Issue logged as:
>
> > http://code.google.com/p/modwsgi/issues/detail?id=132
>
> > Fix for 3.0 in subversion trunk at revision 1197.
>
> > I still need to look at back porting this to 2.X branch.
>
> > Because I regard this as a quite serius issue, looks like I'll now
> > need to hurry along with getting 2.4 out with a number of accumulated
> > bug fixes.
>
> > BTW, the change is to add the flush bucket before the eos. Apache dev
> > list suggested that inserting flush should be okay.
>
> > Graham
>
> >> 2009/2/9 Graham Dumpleton <graham.dumple...@gmail.com>:
> >>> 2009/2/9 gert <gert.cuyk...@gmail.com>:
> >>>> when i comment out wsgi.file it works
>
> >>>> from os import path
> >>>> f=open(path.join(path.dirname(__file__),'../bin/
> >>>> picture.png'),'rb')
> >>>> # if 'wsgi.file_wrapper' in environ: return environ
> >>>> ['wsgi.file_wrapper'](f, 8192)
> >>>> # else:
> >>>> return iter(lambda: f.read(8192), '')
>
> >>> Use the test script from genro's post on February 4th in discussion:
>
Graham Dumpleton
useless back and forth in the discussion, but from that has now come a
post that identifies there is a potential problem in Apache, even if
to the extent of documenting how the APIs needs to be used and/or
giving a gaurantee that a flush actually does flush.
Graham
2009/2/13 Graham Dumpleton <graham.d...@gmail.com>: