webob tests broken...

[Chris McDonough]

Recent checkins have broken the automated tests...

http://lists.repoze.org/pipermail/pyramid-checkins/2012-April/003876.html

Coverage is also no longer 100%.

- C

[Sergey Schetinin]

Yup, we've noticed too. Immediate attempt to fix the tests didn't work
out unfortunately: https://github.com/Pylons/webob/pull/41

> --
> You received this message because you are subscribed to the Google Groups "Paste Users" group.
> To post to this group, send email to paste...@googlegroups.com.
> To unsubscribe from this group, send email to paste-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/paste-users?hl=en.
>

[Ian Bicking]

I might have to open the server on port 0 and figure out how to find the resulting real port.

I'll see what I can do about the coverage. I'm not sure how I feel about 100%, but I guess I'll find out.

[Sergey Schetinin]

The following works well (used to open FTP PASV ports in
http://www.maluke.com/gate )

sock.bind((ip, 0))
port = sock.getsockname()[1]

[Ian Bicking]

Well, I tried this and I am still getting errors on the CI server that I'm not getting locally, that appear to be a case of the local server not working.  I'm not sure what to do at this point to figure it out, as the failure seems to be specific to the CI server.  (Or success is local to my machine?)

[Sergey Schetinin]

The test suite has some other tests that use a real server, so maybe
you can just use some code from those:
https://github.com/Pylons/webob/blob/master/tests/test_in_wsgiref.py

[Chris McDonough]

I see these failures on my machine as well (it's not the CI server)
whether I do:

tox -epy26

Or

somevenv/bin/python setup.py nosetests

Under what circumstances do the tests failures not occur?

[Ian Bicking]

Given your report, apparently they succeed just on my machine.  I'll look into copying the form of those other tests, though it might take me a couple days.

[Chris McDonough]

On 04/17/2012 11:46 AM, Ian Bicking wrote:
> Given your report, apparently they succeed just on my machine. I'll
> look into copying the form of those other tests, though it might take me
> a couple days.

Nudge.

- C

[Ian Bicking]

Changes committed; works locally, fingers crossed it works on the CI server

[Chris McDonough]

On 04/26/2012 10:33 AM, Ian Bicking wrote:
> Changes committed; works locally, fingers crossed it works on the CI server

Sorry to report no...

http://jenkins.pylonsproject.org/job/webob/245/console

I also tried it on my local system with the same result (via "setup.py
nosetests). Your machine is magical ;-)

- C

>
> On Thu, Apr 26, 2012 at 9:17 AM, Chris McDonough <chr...@plope.com
> <mailto:chr...@plope.com>> wrote:
>
> On 04/17/2012 11:46 AM, Ian Bicking wrote:
>
> Given your report, apparently they succeed just on my machine. I'll
> look into copying the form of those other tests, though it might
> take me
> a couple days.
>
>
> Nudge.
>
> - C
>
>

[Ian Bicking]

What are the exceptions you are getting locally?  The one that seems to fail consistently is where it requests http://laksjdfkajwoeifknslkasdflkjasdflaksjdf.eu and this somehow succeeds.  I can't imagine why it would though.  Then there's one other failure on PyPy with timeouts, which might very well be a PyPy-specific problem.

On Thu, Apr 26, 2012 at 9:43 AM, Chris McDonough <chr...@plope.com> wrote:

On 04/26/2012 10:33 AM, Ian Bicking wrote:

Changes committed; works locally, fingers crossed it works on the CI server

Sorry to report no...

http://jenkins.pylonsproject.org/job/webob/245/console

I also tried it on my local system with the same result (via "setup.py nosetests).  Your machine is magical ;-)

- C

On Thu, Apr 26, 2012 at 9:17 AM, Chris McDonough <chr...@plope.com

<mailto:chr...@plope.com>> wrote:

   On 04/17/2012 11:46 AM, Ian Bicking wrote:

       Given your report, apparently they succeed just on my machine.  I'll
       look into copying the form of those other tests, though it might
       take me
       a couple days.


   Nudge.

   - C

--
You received this message because you are subscribed to the Google
Groups "Paste Users" group.
To post to this group, send email to paste...@googlegroups.com.
To unsubscribe from this group, send email to

paste-users+unsubscribe@googlegroups.com.

For more options, visit this group at
http://groups.google.com/group/paste-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "Paste Users" group.
To post to this group, send email to paste...@googlegroups.com.

To unsubscribe from this group, send email to paste-users+unsubscribe@googlegroups.com.

[Chris McDonough]

On 04/26/2012 11:09 AM, Ian Bicking wrote:
> What are the exceptions you are getting locally? The one that seems to
> fail consistently is where it requests
> http://laksjdfkajwoeifknslkasdflkjasdflaksjdf.eu and this somehow
> succeeds. I can't imagine why it would though. Then there's one other
> failure on PyPy with timeouts, which might very well be a PyPy-specific
> problem.
>

Here's the output on my own system using a new 2.6 virtualenv:


"""
[chrism@thinko webob]$ env26/bin/python setup.py nosetests
running nosetests
running egg_info
writing requirements to WebOb.egg-info/requires.txt
writing WebOb.egg-info/PKG-INFO
writing top-level names to WebOb.egg-info/top_level.txt
writing dependency_links to WebOb.egg-info/dependency_links.txt
reading manifest file 'WebOb.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching '*.pyc'
no previously-included directories found matching '*.pyo'
writing manifest file 'WebOb.egg-info/SOURCES.txt'
running build_ext
.........................................................................................F.F...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
======================================================================
FAIL: tests.test_client.test_client
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/home/chrism/projects/webob/env26/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/case.py",
line 197, in runTest
self.test(*self.arg)
File "/home/chrism/projects/webob/tests/test_client.py", line 35, in
test_client
assert resp.status_code == 502, resp.status
AssertionError: 200 OK
assert resp.status_code == 502, resp.status
<Request at 0x1b45b90 GET
http://laksjdfkajwoeifknslkasdflkjasdflaksjdf.eu> =
Request.blank('http://laksjdfkajwoeifknslkasdflkjasdflaksjdf.eu')
resp = <Request at 0x1b45b90 GET
http://laksjdfkajwoeifknslkasdflkjasdflaksjdf.eu>.send(None)
>> assert resp.status_code == 502, resp.status

-------------------- >> begin captured logging << --------------------
tests.test_in_wsgiref: DEBUG: server started on http://localhost:48375
tests.test_in_wsgiref: DEBUG: shutting server down
tests.test_in_wsgiref: DEBUG: server stopped
--------------------- >> end captured logging << ---------------------

======================================================================
FAIL: tests.test_client.test_client_slow
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/home/chrism/projects/webob/env26/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/case.py",
line 197, in runTest
self.test(*self.arg)
File "/home/chrism/projects/webob/tests/test_client.py", line 66, in
test_client_slow
assert resp.status_code == 504, resp.status
AssertionError: 200 OK
<Request at 0x1b458d0 GET http://localhost:50949> =
Request.blank(<tests.test_in_wsgiref.QuietServer instance at 0x1b4b098>.url)
<Request at 0x1b458d0 GET
http://localhost:50949>.environ['webob.client.timeout'] = 0.1
resp = <Request at 0x1b458d0 GET http://localhost:50949>.send(None)
>> assert resp.status_code == 504, resp.status

-------------------- >> begin captured logging << --------------------
tests.test_in_wsgiref: DEBUG: server started on http://localhost:50949
tests.test_in_wsgiref: DEBUG: shutting server down
tests.test_in_wsgiref: DEBUG: server stopped
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 967 tests in 6.093s

FAILED (failures=2)
"""


I only get one of those failures with a new 2.7 virtualenv however:


"""
[chrism@thinko webob]$ env27/bin/python setup.py nosetests
running nosetests
running egg_info
writing requirements to WebOb.egg-info/requires.txt
writing WebOb.egg-info/PKG-INFO
writing top-level names to WebOb.egg-info/top_level.txt
writing dependency_links to WebOb.egg-info/dependency_links.txt
reading manifest file 'WebOb.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching '*.pyc'
no previously-included directories found matching '*.pyo'
writing manifest file 'WebOb.egg-info/SOURCES.txt'
running build_ext
.........................................................................................F.............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
======================================================================
FAIL: tests.test_client.test_client
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/home/chrism/projects/webob/env27/lib/python2.7/site-packages/nose-1.1.2-py2.7.egg/nose/case.py",
line 197, in runTest
self.test(*self.arg)
File "/home/chrism/projects/webob/tests/test_client.py", line 35, in
test_client
assert resp.status_code == 502, resp.status
AssertionError: 200 OK
assert resp.status_code == 502, resp.status
<Request at 0x1c87f10 GET
http://laksjdfkajwoeifknslkasdflkjasdflaksjdf.eu> =
Request.blank('http://laksjdfkajwoeifknslkasdflkjasdflaksjdf.eu')
resp = <Request at 0x1c87f10 GET
http://laksjdfkajwoeifknslkasdflkjasdflaksjdf.eu>.send(None)
>> assert resp.status_code == 502, resp.status

-------------------- >> begin captured logging << --------------------
tests.test_in_wsgiref: DEBUG: server started on http://localhost:34794
tests.test_in_wsgiref: DEBUG: shutting server down
tests.test_in_wsgiref: DEBUG: server stopped
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 967 tests in 4.047s

FAILED (failures=1)
"""

[Ian Bicking]

Oops, I realized I was testing a timeout feature that Python 2.6 and below doesn't have.  That I believe is fixed.

I can't imagine why a request to a clearly bad URL would succeed though...? 

"""

[Chris McDonough]

On 04/26/2012 12:12 PM, Ian Bicking wrote:
> Oops, I realized I was testing a timeout feature that Python 2.6 and
> below doesn't have. That I believe is fixed.
>
> I can't imagine why a request to a clearly bad URL would succeed though...?

Maybe because some ISPs (including my own, and presumably the CI box's)
take a request for an irresolveable host as an opportunity to display a
miserable spam page.

And indeed. This is what happens when I visit that test URL in my
browser... http://plope.com/static/misc/spamisp.png

- C

[Taavi Burns]

On Thu, Apr 26, 2012 at 12:12 PM, Ian Bicking <i...@ianbicking.org> wrote:
> Oops, I realized I was testing a timeout feature that Python 2.6 and below
> doesn't have.  That I believe is fixed.
>
> I can't imagine why a request to a clearly bad URL would succeed though...?

Some residential ISPs around here (Rogers) will return a spammy bunch
of linkbait for domains that don't exist. People who know better
choose to use different ISPs, or at least different DNS servers. Maybe
it's that?

--
taa
/*eof*/

[Sergey Schetinin]

My ISP doesn't do that, but their DNS service is down a lot of the
time. Google's public DNS (8.8.8.8, 8.8.4.4) worked very well for me
and is also way faster than the one provided by the ISP. Might be
worth a try.

> --
> You received this message because you are subscribed to the Google Groups
> "Paste Users" group.
> To post to this group, send email to paste...@googlegroups.com.
> To unsubscribe from this group, send email to

> paste-users...@googlegroups.com.

[Ian Bicking]

I'm not exactly sure how to test an error at the DNS level then (there's a separate connection refused test).  Is there any domain they won't try to intercept?

[Chris McDonough]

On 04/26/2012 12:59 PM, Ian Bicking wrote:
> I'm not exactly sure how to test an error at the DNS level then (there's
> a separate connection refused test). Is there any domain they won't try
> to intercept?

No. I think you're going to have to monkeypatch something to raise an
exception temporarily in the test to emulate it.

- C

>
>
> On Thu, Apr 26, 2012 at 11:22 AM, Chris McDonough <chr...@plope.com
> <mailto:chr...@plope.com>> wrote:
>
> On 04/26/2012 12:12 PM, Ian Bicking wrote:
>
> Oops, I realized I was testing a timeout feature that Python 2.6 and
> below doesn't have. That I believe is fixed.
>
> I can't imagine why a request to a clearly bad URL would succeed
> though...?
>
>
> Maybe because some ISPs (including my own, and presumably the CI
> box's) take a request for an irresolveable host as an opportunity to
> display a miserable spam page.
>
> And indeed. This is what happens when I visit that test URL in my

> browser... http://plope.com/static/misc/__spamisp.png

> <http://plope.com/static/misc/spamisp.png>
>
> - C
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Paste Users" group.
> To post to this group, send email to paste...@googlegroups.com

> <mailto:paste...@googlegroups.com>.

> To unsubscribe from this group, send email to

> paste-users+unsubscribe@__googlegroups.com
> <mailto:paste-users%2Bunsu...@googlegroups.com>.

> For more options, visit this group at

> http://groups.google.com/__group/paste-users?hl=en
> <http://groups.google.com/group/paste-users?hl=en>.
>
>

[Ian Bicking]

On Thu, Apr 26, 2012 at 12:00 PM, Chris McDonough <chr...@plope.com> wrote:

On 04/26/2012 12:59 PM, Ian Bicking wrote:

I'm not exactly sure how to test an error at the DNS level then (there's
a separate connection refused test).  Is there any domain they won't try
to intercept?

No.  I think you're going to have to monkeypatch something to raise an exception temporarily in the test to emulate it.

I don't think there's any reason for the test then, trying to monkeypatch socket errors defeats the point.  I've commented it out.  Oh well.

  Ian

[Chris McDonough]

Thanks. The tests are now passing again, albeit with poor coverage in
the client module. Is there any reason you can't get coverage for this
stuff? We did for everything else.

- C


>
> Ian

>
> --
> You received this message because you are subscribed to the Google
> Groups "Paste Users" group.

> To post to this group, send email to paste...@googlegroups.com.

> To unsubscribe from this group, send email to

> paste-users...@googlegroups.com.

> For more options, visit this group at

> http://groups.google.com/group/paste-users?hl=en.

[Taavi Burns]

On Thu, Apr 26, 2012 at 1:13 PM, Chris McDonough <chr...@plope.com> wrote:
> On 04/26/2012 01:05 PM, Ian Bicking wrote:
>>
>>    No.  I think you're going to have to monkeypatch something to raise
>>    an exception temporarily in the test to emulate it.
>>
>> I don't think there's any reason for the test then, trying to
>> monkeypatch socket errors defeats the point.  I've commented it out.  Oh
>> well.
>
> Thanks.  The tests are now passing again, albeit with poor coverage in the
> client module.  Is there any reason you can't get coverage for this stuff?
>  We did for everything else.

Would it be worth checking to see if the DNS record exists (e.g. using
urllib, unrelated to the actual code under test) and skipping the test
if it does? That way at least it'll run when it can.

--
taa
/*eof*/

[Ian Bicking]

I've added more tests, most of what is left is a bit of defensive programming (error cases that I expect httplib will catch on its own, but still want to check), and code paths that are Python 2 or 3 specific (I'm not sure how to see a combined report).

  Ian

paste-users+unsubscribe@googlegroups.com.

For more options, visit this group at
http://groups.google.com/group/paste-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "Paste Users" group.
To post to this group, send email to paste...@googlegroups.com.

To unsubscribe from this group, send email to paste-users+unsubscribe@googlegroups.com.

[Sergey Schetinin]

Could you please add pragmas to exclude the py3 & defensive code from
coverage reports, so it's easier to spot missing coverage in the
future?

>>> paste-users...@googlegroups.com.

>>> For more options, visit this group at
>>> http://groups.google.com/group/paste-users?hl=en.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Paste Users" group.
>> To post to this group, send email to paste...@googlegroups.com.
>> To unsubscribe from this group, send email to

>> paste-users...@googlegroups.com.

>> For more options, visit this group at
>> http://groups.google.com/group/paste-users?hl=en.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Paste Users" group.
> To post to this group, send email to paste...@googlegroups.com.
> To unsubscribe from this group, send email to

> paste-users...@googlegroups.com.