1.0 and 1.1 testing
Mark Ramm
into 1.0 and 1.1 so that you can write application tests that are
"framework independent." WebTest is the tool we use for testing in
TG2 and I think it has a much nicer API than TG1. The WebTest tests
look like this:
def test_redirect_absolute(self):
resp = self.app.get('/redirect_me?target=/')
assert resp.status == 302, resp.status
assert resp.header('location') == 'http://localhost/', resp.headers
resp = resp.follow()
assert 'hello world' in resp
This checks that we get response (which is a WebOb response object)
which is a redirect, and the does resp.follow() to get the redirected
page and then verifies that 'hello world' is in the rendered template.
We also plan to make resp.namespace contain the values that are passed
into the template (basically the controller's returned dictionary).
I've looked into it and it turns out to be kind of hard to emulate the
create_request and call that are in TG 1 in CP3, since they are very
tied to the CP2 internals. Would it be out of line to provide some
deprecation warnings for create_request and call in order to encourage
people to migrate to a testing tool that will work in 1.1 and 2.0 ?
--
Mark Ramm-Christensen
email: mark at compoundthinking dot com
blog: www.compoundthinking.com/blog
Christopher Arndt
> We are working (really Luke Macken is working) to integrate WebTest
> into 1.0 and 1.1
> deprecation warnings for create_request and call in order to encourage
> people to migrate to a testing tool that will work in 1.1 and 2.0 ?
We could consider this for TG 1.1. We consider TG 1.0 stable and in
maintenance mode, so no new features or deprecations of existing ones
should go into the core.
Just my 2 cents.
Chris
Mark Ramm
> maintenance mode, so no new features or deprecations of existing ones
> should go into the core.
The problem is that migrations to TG 1.1 will require breaking all
your tests unless we do the crazy work to make create_request work
with cherrypy 3. So it would be nicer if we could tell people to
update their tests in place in TG1 -- and then move their app to TG
1.1, where they can use those tests to verify that their app continues
to work.
As I see it the problem with making create_request work is that it
does not -- return a response object, but instead it just ends up
generating a response object on cherrypy.response as a side effect --
and I don't see a way to do that without monkeying with the cherrypy
internals in some way.
--Mark Ramm
Florent Aide
>
> > We could consider this for TG 1.1. We consider TG 1.0 stable and in
> > maintenance mode, so no new features or deprecations of existing ones
> > should go into the core.
>
> The problem is that migrations to TG 1.1 will require breaking all
> your tests unless we do the crazy work to make create_request work
> with cherrypy 3. So it would be nicer if we could tell people to
> update their tests in place in TG1 -- and then move their app to TG
> 1.1, where they can use those tests to verify that their app continues
> to work.
Mark,
1.0 is closed. It took time and effort, we now intend to move to 1.1
but this enhancement in testing you are talking about is interesting.
I am working on TG 1.1 / CP3 right now and I've broken quite a few
things already so I would be interesting in helping establish this new
testing framework so that I can have the internal TG test suite
guarantee that I have the same functionality in 1.1 once I finished
moving to CP3. To be honest I think that this work will go in 1.2 if I
break to much things (for the moment I did not break the config
system)
My problem is that I am quite resource stretched and will not have
much time to give on each subject so if someone can help on this
testing subject I would be grateful. What would be the level of
changes/breakage induced by this new testing method?
Florent.
Ken Kuhlman
I've also been playing in this space. I agree with Mark that the
webtest API is nicer than the tg1.0 method, but I'm slightly more
hopeful of being able to maintain a good degree of backwards
compatibility during the transition.
The approach I've taken is to have create_request return a response
object, whose body can be checked instead of
cherrypy.response.body[0]. The response that's returned is massaged
slightly to look more like a webtest response. When the project is
ready to make the switch, a config option can be enabled to start
using webtest directly.
Other issues that come up include the fact that webtest automatically
checks the response's status code, and different handling of cookies.
I've attached a patch to #1466 to show where I'm at. I've got a few
tweaks I want to make yet, but I've got all the tests passing. I had
hoped to be in position to start comiting changes in the next couple
of days, but I can pull back if anything thinks I'm seriously
off-track. I'd also like to see Luke's code if it's available.
Note that I have the same problem as Mark, in that this really takes
two releases to roll out fully: one to put the new method in place &
deprecate the old way, and another to drop the old stuff completely.
I had been working under the assumption that 1.1 would still be
cp2-based, and that there will be later 1.x releases which would
target cp3.
-ken
Mark Ramm
> much time to give on each subject so if someone can help on this
> testing subject I would be grateful. What would be the level of
> changes/breakage induced by this new testing method?
We don't need to break anything. The changes needed are centered on
a small test for the presence of a testing environment specific
variable in the wsgi environ. Other than that, everything should
just work.
Seems like we could add this as a small bugfix (it's like 10 lines of
code) in 1.1 tomorrow. We'll talk more about backporting it to 1.0.x
when you have time.... it's pretty simple, and would make migrations
easier. Perhaps we could slip it in since it doesn't break anything,
but skip the deprecation warnings...
lmacken
> We are working (really Luke Macken is working) to integrate WebTest
> into 1.0 and 1.1 so that you can write application tests that are
> "framework independent." WebTest is the tool we use for testing in
> TG2 and I think it has a much nicer API than TG1.
I threw together a proof-of-concept patch that you can find here:
http://trac.turbogears.org/ticket/1762
The current implementation allows you to do something like this
(pretend there is a 'hot_action' method exposed that disallows
anonymous users):
class TestPages(testutil.DBWebTest):
def test_forbidden(self):
# Hot action is forbidden. WebTest ensures that this call
returns a 403,
# thus we don't have to make any assertions.
self.app.get('/hot_action', status=403)
def test_webob_response(self):
user = User(user_name=u"test", password=u"test")
self.login_user(user)
res = self.app.get('/hot_action')
assert "Hot WSGI action" in res
assert res.namespace['tg_flash'] == u'Hot WSGI action'
So my patch creates two unit testing parent classes: TGWebTest and
DBWebTest. TGWebTest sets up and tears down a WebTest instance
attached to a CherryPy wsgi app mounted as 'self.app'. The DBWebTest
does the same, but also is a subclass of DBTest, so it will
automatically setup and tear down your database tables as well. I
also threw together a patch (http://trac.turbogears.org/ticket/1764)
that makes the testutil.DBTest work with SQLAlchemy as well (I'm new
to alchemy, so suggestions for how to do this better would be
appreciated).
In order to retain similar functionality to
testutil.set_identity_user, I implemented a login_user method (which
we could rename to set_identity_user if we want), that makes a WebTest
call to our login method with the credentials of a given user. Since
WebTest handles cookies and everything for us, this method turns out
to be trivial.
As for testutil.{call,create_request}, those would be consolidated
into a single 'app.get' call. As Mark mentioned, the
response.namespace will contain the dictionary values that are
returned from the controller (which my patch sets up) Altering
testutil.call to use WebTest is not very difficult (I have a patch
locally), but create_request will require many more nasty CherryPy
hacks. So I'm wondering if it is really worth it to hack this stuff
into the current API, or to simply default to the new WebTest stuff in
newer versions?
Being new to TurboGears hacking, I'm most likely overlooking many
things here, so your comments and suggestions would be appreciated :)
Cheers!
luke