Unit testing in web2py : Some thoughts

[Jon Romero]

I've read about web2py and unit testing and I tried many things.
These are my thoughts (please correct me If I am wrong)

1. web2py uses doctests. doctests must be inside the controller
(increasing the noise) and are not flexible. Also, is not trivial to
use them in another automated built/test system
2. The only official example online is here http://www.web2py.com/AlterEgo/default/show/213
where you
cannot import models. I think that you have to load the environment
first (any code snippet how to do that)?

So, any decent solution that you guys know?

Thanks

[Thadeus Burgess]

Are we looking at the same thing?

The link you provided gives an example of using the unittest classes
(not doctest), in which you can import anything you want since it is
just a python file.

In the example is an example of executing your controllers in a web2py
environment.

Also the doctest example
http://www.web2py.com/examples/default/examples#testing_examples
anything in the >>> is python code so you can import modules if you
want.

Also, the doctests have access to the web2py environment, so they have
access to db.

-Thadeus

> --
> You received this message because you are subscribed to the Google Groups "web2py-users" group.
> To post to this group, send email to web...@googlegroups.com.
> To unsubscribe from this group, send email to web2py+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/web2py?hl=en.
>
>

[spiffytech]

> The link you provided gives an example of using the unittest classes
> (not doctest), in which you can import anything you want since it is
> just a python file.

AlterEgo 213 (which Jon linked to) does not show how to get your
controllers to see your database models- the example code doesn't
interact with a database at all. I've tried every snippet of promising
code on the mailing list and haven't been successful. I, too, would
appreciate an example of exposing models to a controller in an
external unit test

<A rather lengthy aside to counter the large pile of "just use
doctests" posts in the group archives>
The biggest downside to doctests is that they do a poor job of testing
functions that rely heavily on a database. Since doctests don't
automatically run setUp() and tearDown() functions before/after every
test (as external unit tests do) you have to devise clunky,
inconvenient ways to bring the database back to a known state for each
test.

Another downside to doctests is they are harder to organize; you can't
cleanly separate different sorts of tests that all relate to the same
function (i.e., when a function is very sensitive to initial
conditions). With external unit tests each logically distinct test can
be represented with a distinct function, and that test can be run on
it's own, without the overhead of any other tests (unlike doctests,
where everything that tests a function must be run in order to test
the part you're interested in).
</aside>

As the AlterEgo example shows, web2py already supports external unit
testing. All that's missing is an example demonstrating how to test a
database-driven function.

[Thadeus Burgess]

python web2py.py --help

-S APPNAME, --shell=APPNAME
run web2py in interactive shell or IPython (if
installed) with specified appname
-T TEST_PATH, --test=TEST_PATH
run doctests in web2py environment; TEST_PATH like
a/c/f (c,f optiona
-R PYTHON_FILE, --run=PYTHON_FILE
run PYTHON_FILE in web2py environment; should be used
with --shell o
-M, --import_models auto import model files; default is False; should be
used with --shell option

python web2py.py -T <yourapp>/<controller>/<function>

It executes all doctests in environment. Have a look at sql.py for
example doctesting with a database (using a sqlite database)

And if you want to use unittest

python web2py.py -S <yourapp> -M -R
/path/to/web2py/applications/yourapp/tests/testControllerA.py

Will execute the file inside of the web2py environment (with access to
database models as denoted by the -M option)'

Now since we had such a difficult time finding these options, please
list links as to where the documentation on this subject can be
updated and explained further.

-Thadeus

[Jon Romero]

Thanks! That was what I was looking for!
The problem now is that is you try to run the unit test the way it is
described here (http://www.web2py.com/AlterEgo/default/show/213 ), I
get this error:
> python web2py.py -S init -M -R applications/myapp/tests/default.py

option -S not recognized
Usage: web2py.py [options] [test] [...]

Options:
-h, --help Show this message
-v, --verbose Verbose output
-q, --quiet Minimal output

Examples:
web2py.py - run default set of tests
web2py.py MyTestSuite - run suite 'MyTestSuite'
web2py.py MyTestCase.testSomething - run
MyTestCase.testSomething
web2py.py MyTestCase - run all 'test*' test
methods
in MyTestCase

If I just put a prin "Hello world!" inside the test file, everything
works ok. So, I suppose that this file tries to load AGAIN the web2py
environment and there is
some sort of mix up?

Any idea?

[spiffytech]

That is unittest.main() trying to parse the arguments on the command
line. Web2py is passing the original command line (sys.argv) to the
unit test script, which sees the arguments for web2py.py and says "I
don't know what these mean". A quick hack to gluon/shell.py ("sys.argv
= sys.argv[4:]" on line 165 in web2py version 1.74.6) got me past
this. Is there a clean, reliable way to not pass scripts all of the
web2py.py CLI arguments?

Once I hacked shell.py I ran into another problem: the unittest module
has the scope of web2py.py and not my script. Stepping through the
unittest module code reveals that whenever unittest.main() tries to
find the default test, or a test you specify, or tries to run all
tests, it sees itself as being in "module '__main__' from
'web2py.py'". since web2py.py doesn't contain any tests the unittest
sees nothing to do, and quits. If you named a specific test to run,
the unittest module throws an exception and complains that such a test
does not exist. How could this scoping issue be solved?
unittest.main() takes a 'module' argument but I don't know what to put
there to make it work.

Thadeus, I found the web2py.py help string for "-R" misleading; when I
run unit tests I don't want to launch a shell, and since the help
string says "-R" requires the shell option, I concluded that "-R" was
not what I needed.

Also, AlterEgo 213 gives no indication that the sample unit test
script should be run with web2py.py. Rather, it looks like a perfectly
ordinary script that would be run with "python test.py". AlterEgo 213
further misleads by showing the user how to set up the web2py
environment inside the unit test script, rather than relying on
web2py.py to set it up. Once I get unit tests working I'll gladly
create a wiki entry that could replace AlterEgo 213, demonstrating the
'correct' way to go about this.

On Feb 23, 6:23 am, Jon Romero <darks...@gmail.com> wrote:
> Thanks! That was what I was looking for!
> The problem now is that is you try to run the unit test the way it is

> described here (http://www.web2py.com/AlterEgo/default/show/213), I

[Thadeus Burgess]

What version of web2py are you running?

What is the output of ``python web2py.py --help``

-Thadeus

[spiffytech]

I'm running the latest version, 1.75.4.

On Feb 23, 11:35 am, Thadeus Burgess <thade...@thadeusb.com> wrote:
> What version of web2py are you running?
>
> What is the output of ``python web2py.py --help``
>
> -Thadeus
>

[Jon Romero]

So, any idea/hint how to do it? Do we need a patch?

[Thadeus Burgess]

consider controllers/default.py

def index():
"""
>>> db(db.people.id > 0).count()
'35'
"""
return db(db.people.id > 0).count()

now when I run

`python web2py.py -T init/default/index`

produces my doctest.

-Thadeus

[Thadeus Burgess]

If I create a unittest

init/tests/testControllerDefault.py

and run the following command line

python web2py.py -S <yourapp> -M -R

applications/init/tests/testControllerDefault.py

My test is ran.

I don't see what is so confusing.

-Thadeus

[spiffytech]

The confusion is not with doctests, but with external unit tests
created with the 'unittest' module. Could you please post your
testControllerDefault.py so I can see what you're doing different from
me?

Please allow me to elaborate on my problems running unit tests with
web2py. Here is my (very minimal) example unit test file:
=================
#!/usr/bin/python
import unittest

class TestListActiveGames(unittest.TestCase):
def testListActiveGames(self):
print "This line should print out"

if __name__ == "__main__":
unittest.main()
=================

If the test is actually run, the command line will show "This line
should print out".

The file is stored at <web2py_root>/applications/api/tests/test.py. My
current working directory is <web2py_root>.
I run the following command:

=======
python web2py.py -S api -M -R applications/api/tests/test.py
=======

And I get this output:

================
web2py Enterprise Web Framework
Created by Massimo Di Pierro, Copyright 2007-2010
Version 1.75.4 (2010-02-18 20:57:56)
Database drivers available: pysqlite
================

The output does not contain the line, "This line should print out",
indicating the unit tests are not being run.

I did some debugging and found that the last two lines, which are
typical for unittest files, do not work when the file is executed by
web2py. Rather than being equal to "__main__", which indicates the
file is being executed from the command line, __name__ is equal to
"__builtin__". I modified the code to work anyway:

================
#!/usr/bin/python
import unittest

class TestListActiveGames(unittest.TestCase):
def testListActiveGames(self):
print "This line should print out"

unittest.main()
================

Now, I receive this output:

=======
web2py Enterprise Web Framework
Created by Massimo Di Pierro, Copyright 2007-2010
Version 1.75.4 (2010-02-18 20:57:56)
Database drivers available: pysqlite2

option -S not recognized
Usage: web2py.py [options] [test] [...]

Options:
-h, --help Show this message
-v, --verbose Verbose output
-q, --quiet Minimal output

Examples:
web2py.py - run default set of tests
web2py.py MyTestSuite - run suite 'MyTestSuite'
web2py.py MyTestCase.testSomething - run
MyTestCase.testSomething
web2py.py MyTestCase - run all 'test*' test
methods
in MyTestCase

=======

More debugging revealed that this is output by unittest.main(). The
unittest module tries to parse arguments on the command line, sees the
"-S" option that was passed to web2py.py, says "I don't know what '-S'
means!", and exits.

To coerce unittest.main() to continue running, I made a quick hack to
shell.py to strip out the extra command line arguments in sys.argv
before test.py is run:

==== gluon/shell.py, insert line 165 ====
sys.argv = sys.argv[5:]
===================

I call the test file again, with the same command as before, and get
this:

================
web2py Enterprise Web Framework
Created by Massimo Di Pierro, Copyright 2007-2010
Version 1.75.4 (2010-02-18 20:57:56)
Database drivers available: pysqlite2

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
================

The line, "This line should be printed out", is still not printed.
Stepping through the code with Python's debugging module, pdb, shows
that unittest.main() thinks it's being called from web2py.py, and not
from test.py. unittest.main() looks for tests to run in web2py.py,
finds none, and quits.

So now I'm stuck: the unittest module expects test.py to be run as a
standalone script, but instead test.py is being executed within the
scope of web2py.py, which is confusing the unittest module in several
ways.

On Feb 24, 1:02 pm, Thadeus Burgess <thade...@thadeusb.com> wrote:
> If I create a unittest
>
> init/tests/testControllerDefault.py
>
> and run the following command line
>
> python web2py.py -S <yourapp> -M -R
> applications/init/tests/testControllerDefault.py
>
> My test is ran.
>
> I don't see what is so confusing.
>
> -Thadeus
>

> On Wed, Feb 24, 2010 at 12:01 PM, Thadeus Burgess <thade...@thadeusb.com> wrote:
> > consider controllers/default.py
>
> > def index():
> > """
> >>>> db(db.people.id > 0).count()
> > '35'
> > """
> >    return db(db.people.id > 0).count()
>
> > now when I run
>
> > `python web2py.py -T init/default/index`
>
> > produces my doctest.
>
> > -Thadeus
>

[Thadeus Burgess]

Replacing the way you run test suites helps. Instead of using .main()
add them manually.

I would suggest reading the following article, as it includes methods
to aggregate your test suites together.

http://agiletesting.blogspot.com/2005/01/python-unit-testing-part-1-unittest.html

import sys
sys.argv = sys.argv[5:]

import unittest

class TestDefaultController(unittest.TestCase):

def testPrintStatement(self):

print "This line should print"

def testDatabaseRecordCount(self):
print "Records in database --- ", db(db.auth_user.id>0).count()

suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(TestDefaultController))
unittest.TextTestRunner(verbosity=2).run(suite)

python web2py.py -S pms -M -R applications/pms/test/testDefaultController.py

web2py Enterprise Web Framework
Created by Massimo Di Pierro, Copyright 2007-2010

Version 1.75.4 (2010-02-18 14:55:03)
Database drivers available: SQLite3
/home/thadeusb/web2py/applications/pms/modules/utils.py:16:
DeprecationWarning: the md5 module is deprecated; use hashlib instead
import md5
testDatabaseRecordCount (__builtin__.TestDefaultController) ...
Records in database --- 4052
ok
testPrintStatement (__builtin__.TestDefaultController) ... This line
should print
ok

----------------------------------------------------------------------
Ran 2 tests in 0.006s

OK

-Thadeus

[spiffytech]

Thanks! Interesting article! My test cases now execute. However, I
have a couple new questions, including a problem accessing the db in
my controller.

I modified my test file as AlterEgo 213 indicates so my unit tests can
access the controller's functions. Here is my updated test file:

==============
#!/usr/bin/python
import sys
import unittest

from gluon.shell import exec_environment
from gluon.globals import Request, Response, Session
from gluon.storage import Storage

sys.arvg = sys.argv[5:] # web2py.py passes the whole command line to
this script

class TestListActiveGames(unittest.TestCase):
def setUp(self):
self.request = Request() # Use a clean Request
self.controller = exec_environment('applications/api/
controllers/10.py', request=self.request)

def testListActiveGames(self):
self.request.post_vars["game_id"] = 1
self.request.post_vars["username"] = "spiffytech"
self.controller.list_active_games()

suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(TestListActiveGames))
unittest.TextTestRunner(verbosity=2).run(suite)
==============

It is called with the command:

==============

python web2py.py -S api -M -R applications/api/tests/test.py

==============

The output is this:

==============

web2py Enterprise Web Framework
Created by Massimo Di Pierro, Copyright 2007-2010
Version 1.75.4 (2010-02-18 20:57:56)
Database drivers available: pysqlite2

testListActiveGames (__builtin__.TestListActiveGames) ... ERROR

======================================================================
ERROR: testListActiveGames (__builtin__.TestListActiveGames)
----------------------------------------------------------------------
Traceback (most recent call last):
File "applications/api/tests/test.py", line 19, in
testListActiveGames
self.controller.list_active_games()
File "applications/api/controllers/10.py", line 47, in
list_active_games
games = db(((db.game.user1==username)|(db.game.user2==username)) &
(db.game.victory==-2)).select(db.game.user1, db.game.user2,
db.game.id, db.game.turn_number).as_list()
NameError: global name 'db' is not defined

----------------------------------------------------------------------
Ran 1 test in 0.008s

FAILED (errors=1)
==============

Questions:
1) How can I get my controller to see the database?
2) Am I simply doing something very wrong? I would expect the web2py "-
S" option to set up the environment, complete with my controller's
functions, and Request/Storage/Response objects available for
instantiation. However, several posts on the mailing list indicate
that I need to run exec_enviroment() for access to my controllers.
Also, the Request/Storage/Response objects don't seem to exist in the
shell.

On Feb 24, 2:52 pm, Thadeus Burgess <thade...@thadeusb.com> wrote:
> Replacing the way you run test suites helps. Instead of using .main()
> add them manually.
>
> I would suggest reading the following article, as it includes methods
> to aggregate your test suites together.
>

> http://agiletesting.blogspot.com/2005/01/python-unit-testing-part-1-u...

[Thadeus Burgess]

The unit test already has access to web2py environment (and db or
anything else in your models). ``exec_environment`` just recreates a
blank environment, when executing your controller it executes it
outside of the current global scope.

Instead of running exec_environment on your controller, run
``execfile('/path/to/file', globals())``. This way the controller is
executed in the current environment instead of a new one.

The objects for Request Storage and Response don't exist even to your
web2py app, you always have to import them if you are using them. So
if you wanted to use Storage even in your models you would need an
"from gluon.storage import Storage" somewhere.

Do you want your unit tests to work on your actual database? Usually I
define a test.sqlite database that has
nothing in it, and during my tests gets data inserted, altered, and
then removed at the end of the tests.

-Thadeus

[Nicol van der Merwe]

Hi guys

This stuff is very interesting. I would like to request, if possible, that someone makes a web2pyslice or proper AlterEgo entry on how to setup and run these kinds of tests for web2py. I am very interested in setting up tests for my application but I'm a bit lost as I've never done so before (plus I'm unfortunately just too busy to research all this and make my own slice).

It would be very much appreciated if this can be done :)

Nicolaas

[Jon Romero]

Yeap, spiffytech where you able to run tests (no doctests) using the
database?

On Feb 25, 11:20 am, Nicol van der Merwe <aspersie...@gmail.com>
wrote:

> Hi guys
>
> This stuff is very interesting. I would like to request, if possible, that
> someone makes a web2pyslice or proper AlterEgo entry on how to setup and run
> these kinds of tests for web2py. I am very interested in setting up tests
> for my application but I'm a bit lost as I've never done so before (plus I'm
> unfortunately just too busy to research all this and make my own slice).
>
> It would be very much appreciated if this can be done :)
>
> Nicolaas
>

> ...
>
> read more »

[spiffytech]

I'm going to write up a nice, clear wiki article on unit testing with
the unittest module based on what I learned in this discussion. I'll
be sure to link to it here when it's done.

On Feb 25, 4:20 am, Nicol van der Merwe <aspersie...@gmail.com> wrote:
> Hi guys
>
> This stuff is very interesting. I would like to request, if possible, that
> someone makes a web2pyslice or proper AlterEgo entry on how to setup and run
> these kinds of tests for web2py. I am very interested in setting up tests
> for my application but I'm a bit lost as I've never done so before (plus I'm
> unfortunately just too busy to research all this and make my own slice).
>
> It would be very much appreciated if this can be done :)
>
> Nicolaas
>

> ...
>
> read more »

[Nicol van der Merwe]

Super awesome, thanks!

--

You received this message because you are subscribed to the Google Groups "web2py-users" group.
To post to this group, send email to web...@googlegroups.com.
To unsubscribe from this group, send email to web2py+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/web2py?hl=en.

--
If we hit that bullseye, the rest of the dominoes should fall like a house of cards. Checkmate.  -Z. Brannigan

[spiffytech]

execfile() did the trick! Things are working nicely now.

Using a test database would be very helpful. How do you recommend
setting one up? Do I need to copy/paste the table/field definitions
from the 'db' object, or is there a way to make a copy of the 'db'
object and point it to a different SQLite file?

On Feb 24, 10:54 pm, Thadeus Burgess <thade...@thadeusb.com> wrote:
> The unit test already has access to web2py environment (and db or
> anything else in your models). ``exec_environment`` just recreates a
> blank environment, when executing your controller it executes it
> outside of the current global scope.
>
> Instead of running exec_environment on your controller, run
> ``execfile('/path/to/file', globals())``. This way the controller is
> executed in the current environment instead of a new one.
>
> The objects for Request Storage and Response don't exist even to your
> web2py app, you always have to import them if you are using them. So
> if you wanted to use Storage even in your models you would need an
> "from gluon.storage import Storage" somewhere.
>
> Do you want your unit tests to work on your actual database? Usually I
> define a test.sqlite database that has
> nothing in it, and during my tests gets data inserted, altered, and
> then removed at the end of the tests.
>
> -Thadeus
>

> ...
>
> read more »

[Tiago Almeida]

I concur. Thanks :)

[Thadeus Burgess]

So the easiest way to use a testing db and your existing tables is to
automatically recreate them.

So assuming you are in the web2py environment and have access to ``db``

>>> test_db = DAL('testing.sqlite')

>>> for tablename in db.tables:
>>> table_copy = [copy.copy(f) for f in db[tablename]]
>>> test_db.define_table(tablename, *table_copy)

This will create a new testing sqlite database. Then it will go
through all tables defined in db, then copy their fields to a list,
then it will define a new table in the sqlite with the copied fields.

Now any functions that you might want to unit test might rely on your
``db`` object, which could be an issue depending on how you have your
code structured.

-Thadeus

[spiffytech]

Last question: Is there any value in having web2py.py pass scripts the
full command line (sys.argv)? If not, would you be open to a patch to
gluon/shell.py so that it only passes the script's filename and any
CLI variables shell.py didn't use? I'd also appreciate suggestions on
how to do that more reliably than sys.argv[5:].

On Feb 25, 11:36 am, Thadeus Burgess <thade...@thadeusb.com> wrote:
> So the easiest way to use a testing db and your existing tables is to
> automatically recreate them.
>
> So assuming you are in the web2py environment and have access to ``db``
>
> >>> test_db = DAL('testing.sqlite')
> >>> for tablename in db.tables:
> >>>   table_copy = [copy.copy(f) for f in db[tablename]]
> >>>   test_db.define_table(tablename, *table_copy)
>
> This will create a new testing sqlite database. Then it will go
> through all tables defined in db, then copy their fields to a list,
> then it will define a new table in the sqlite with the copied fields.
>
> Now any functions that you might want to unit test might rely on your
> ``db`` object, which could be an issue depending on how you have your
> code structured.
>
> -Thadeus
>
> On Thu, Feb 25, 2010 at 8:02 AM, Tiago Almeida
>

> <tiago.b.alme...@gmail.com> wrote:
> > I concur. Thanks :)
>

> > On Thu, Feb 25, 2010 at 1:52 PM, Nicol van der Merwe <aspersie...@gmail.com>
> > wrote:
>
> >> Super awesome, thanks!

> ...
>
> read more »

[spiffytech]

Works great! I added an "import copy" in db.py, and added a line in my
unit test to rename "test_db" to "db" so that functions in the test
script will use the test DB.

For posterity, here is a complete working example of unit tests with
web2py, with access to the database, using test database. The wiki
article I create will explain each piece and why it's needed.

====== To run the unit tests, type this on the command line: ======
python web2py.py -S api -M -R applications/api/controllers/test.py #
Fill in your own values for the test file and controller name
================================================

====== applications/api/controllers/test.py ======

#!/usr/bin/python
import sys
import unittest

from gluon.globals import Request # So we can reset the request for
each test

sys.arvg = sys.argv[5:] # web2py.py passes the whole command line to
this script

db = test_db # Rename the test database so that functions will use it
instead of the real database
execfile("applications/api/controllers/10.py", globals()) # Brings
the controller's functions into this script's scope

class TestListActiveGames(unittest.TestCase):
def setUp(self):
request = Request() # Use a clean request

def testListActiveGames(self):
# Set variables for the test function
request.post_vars["game_id"] = 1
request.post_vars["username"] = "spiffytech"

# Call a function from the controller "10.py" and print the
dictionary it returns
resp = list_active_games()
print resp
self.assertEquals(0, len(resp["games"]))

# Manually specify tests to run; the web2py environment breaks
unittest.main()
# Taken from here: http://agiletesting.blogspot.com/2005/01/python-unit-testing-part-1-unittest.html

suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(TestListActiveGames))
unittest.TextTestRunner(verbosity=2).run(suite)

db(db.game.id>0).delete() # Empty out the test database so it's fresh
for next time
======================================

====== Added to the bottom of applications/api/models/db.py ======
# Create a test database that's laid out just like the "real" database
import copy
test_db = DAL('sqlite://testing.sqlite') # DB name and location
for tablename in db.tables: # Copy tables!

table_copy = [copy.copy(f) for f in db[tablename]]
test_db.define_table(tablename, *table_copy)

===================================================

On Feb 25, 11:36 am, Thadeus Burgess <thade...@thadeusb.com> wrote:
> So the easiest way to use a testing db and your existing tables is to
> automatically recreate them.
>
> So assuming you are in the web2py environment and have access to ``db``
>
> >>> test_db = DAL('testing.sqlite')
> >>> for tablename in db.tables:
> >>>   table_copy = [copy.copy(f) for f in db[tablename]]
> >>>   test_db.define_table(tablename, *table_copy)
>
> This will create a new testing sqlite database. Then it will go
> through all tables defined in db, then copy their fields to a list,
> then it will define a new table in the sqlite with the copied fields.
>
> Now any functions that you might want to unit test might rely on your
> ``db`` object, which could be an issue depending on how you have your
> code structured.
>
> -Thadeus
>
> On Thu, Feb 25, 2010 at 8:02 AM, Tiago Almeida
>

> <tiago.b.alme...@gmail.com> wrote:
> > I concur. Thanks :)
>

> > On Thu, Feb 25, 2010 at 1:52 PM, Nicol van der Merwe <aspersie...@gmail.com>
> > wrote:
>
> >> Super awesome, thanks!

> ...
>
> read more »

[spiffytech]

*addendum: at the bottom of applications/api/controllers/test.py add
the line "db.commit()" to force web2py to clear the database. web2py
automatically uses database transactions. They are executed
automatically after a controller's function is called, which is why
you don't normally have to call commit yourself. In external scripts,
though, all inserts/updates/deletes in the database have to be
manually committed.)

> # Taken from here:http://agiletesting.blogspot.com/2005/01/python-unit-testing-part-1-u...

> ...
>
> read more »

[Thadeus Burgess]

Yes, I forgot to mention the db.commit().

In fact, I would db.commit() after every successful
insertion/delete/update this way your tests will be more in line?

I do not know any reason as to why web2py passes along args that it is
not using. However I think this could be beneficial in certain scripts
that need to rely on certain parameters, like I have a backup script
that relies on the app name, I can look at the -S arg to determine
what app I am currently working on for this script.

There should be a way though to easily filter out web2py args from
sys.argv in a script.... perhaps a helper function in shell.py that
can do this for you when you call it.

-Thadeus

[spiffytech]

What about adding to the shell environment a variable containing the
name of the current app? Then scripts like yours could still get the
information they need, and the extra values could be removed from
sys.argv without a script having to import a helper.

Does this count as breaking backward-compatibility in web2py, or does
web2py only claim compatibility for models/views/controller code?

On Feb 25, 1:42 pm, Thadeus Burgess <thade...@thadeusb.com> wrote:
> Yes, I forgot to mention the db.commit().
>
> In fact, I would db.commit() after every successful
> insertion/delete/update this way your tests will be more in line?
>
> I do not know any reason as to why web2py passes along args that it is
> not using. However I think this could be beneficial in certain scripts
> that need to rely on certain parameters, like I have a backup script
> that relies on the app name, I can look at the -S arg to determine
> what app I am currently working on for this script.
>
> There should be a way though to easily filter out web2py args from
> sys.argv in a script.... perhaps a helper function in shell.py that
> can do this for you when you call it.
>
> -Thadeus
>

[Thadeus Burgess]

I do not like the idea of having to specify the name of the app twice,
it breaks DRY.

What about an additional command line flag that can tell whether to
strip sys.argv or not. This will keep backwards compatibility, and
also allow for scripts to have a simple sys.argv if required.

web2py.py --clean-argv

On this note, have you attempted to use unittest.TextTestRunner(...)
without the sys.argv=sys.argv[5:]

I believe that TextTestRunner should work since it shouldn't look at
command line args.

-Thadeus

[spiffytech]

I had not tried taking the sys.argv statement out of my code. You are
correct: when using TestTextRunner, sys.argv does not need to be
cleaned. I like the idea of a command line flag. It's probably a good
idea to offer such a flag, even though it turns out it's not needed to
run unit tests.

On Feb 25, 2:49 pm, Thadeus Burgess <thade...@thadeusb.com> wrote:
> I do not like the idea of having to specify the name of the app twice,
> it breaks DRY.
>
> What about an additional command line flag that can tell whether to
> strip sys.argv or not. This will keep backwards compatibility, and
> also allow for scripts to have a simple sys.argv if required.
>
> web2py.py --clean-argv
>
> On this note, have you attempted to use unittest.TextTestRunner(...)
> without the sys.argv=sys.argv[5:]
>
> I believe that TextTestRunner should work since it shouldn't look at
> command line args.
>
> -Thadeus
>

[Jonathan Lundell]

I'm really glad to see this work happening, and I look forward to the slice.

Perhaps we could eventually have unit tests as part of the welcome app, or indeed all three standard apps, both for documentation purposes and to set a good example.

[Jon Romero]

Good job!

I am still having problem with unit testing forms. It seems that I
cannot do a post.

# Set variables for the test function

request.env.request_method = 'POST'
request.post_vars["body"] = "body"
#request.vars = Storage(body = "bodyyyy")

resp = index()
print "vars", resp['form']

#self.assertTrue(resp['form'].errors) # do we have errors?
#self.assertTrue(resp['form'].errors.has_key('name')) # is
something missing?

Any ideas?

On Feb 25, 11:15 pm, Jonathan Lundell <jlund...@pobox.com> wrote:
> I'm really glad to see this work happening, and I look forward to the slice.
>

> Perhaps we could eventually haveunittests as part of the welcome app, or indeed all three standard apps, both for documentation purposes and to set a good example.

[Jon Romero]

ok I managed to do unit testing on forms:

def testForm(self):
request.vars["username"] = "great-guy"
request.vars["email"] = "b...@bla.com"

request.vars["_formname"] = "bla_create" #this is the form
name

resp = index()

self.assertTrue(resp['form'].errors) # do we have errors?

self.assertEquals(1, len(resp["all_users"]))

On Mar 1, 2:20 pm, Jon Romero <darks...@gmail.com> wrote:
> Good job!
>

> I am still having problem withunittestingforms. It seems that I

[mdipierro]

Would you post a web2pyslice?

[Jon Romero]

I'll post a full example in bitbucket and then a slice :)

[spiffytech]

I finally finished the article I promised. I couldn't post to the
wiki, so I posted to AlterEgo instead:

http://web2py.com/AlterEgo/default/show/260

Since I've done little with web2py forms and auth, they're not
covered. Feel free to change the article to include how to test them.

[Nicol van der Merwe]

Very cool SpiffyTech!

It is much appreciated :)

I would like to see (at some point) documentation with a sample application and its respective tests - so one can see how it all fits together.

Still very informative.

Nicolaas

--
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To post to this group, send email to web...@googlegroups.com.
To unsubscribe from this group, send email to web2py+un...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/web2py?hl=en.

[Tiago Almeida]

Thank you very much for this. I've been wanting to integrate unit tests in my project but didn't found the time to investigate this issue.

Best regards,
Tiago
---

[Jon Romero]

I made a slice:
http://www.web2pyslices.com/main/slices/take_slice/67

I'll post a project also to bitbucket.

I've created a unit system but I made a lot of assumptions and I think
that we should talk about it:
First, I created a testRunner.py file (in the same directory level
with web2py.py) that runs ALL tests that exist in the tests directory
of an app.
Don't know whether it should stay there (or go to gluon) but it's
application-agnostic (reads the args when it's called and runs
only the tests of the specific application). And with one command, you
run ALL your tests (without needing any configuration). Just add
tests, inside the tests directory.

Then, I copied a file that is need for posting to forms to
gluon.contrib.

Now, the structure is this:
applications/appname/tests/controllers/controller_name.py
or
applications/appname/tests/models/model_name.py
or
whatever

The controller_name should be the same with the controller we are
unittesting (it's "imported" automatically).

The class name should be ControllernameController (feels like Rails, I
know). It's automatically inserted to unittesting also.
Have in mind that it's Controller NOT Controllers.

So, any ideas?

PS: Does forms, auth and database testing

On Mar 2, 4:41 pm, spiffytech <spiffyt...@gmail.com> wrote:
> I finally finished the article I promised. I couldn't post to the
> wiki, so I posted to AlterEgo instead:
>
> http://web2py.com/AlterEgo/default/show/260
>
> Since I've done little with web2py forms and auth, they're not
> covered. Feel free to change the article to include how to test them.
>
> On Mar 2, 8:59 am, Jon Romero <darks...@gmail.com> wrote:
>
> > I'll post a full example in bitbucket and then a slice :)
>
> > On Mar 2, 2:30 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:
>
> > > Would you post a web2pyslice?
>
> > > On Mar 2, 4:39 am, Jon Romero <darks...@gmail.com> wrote:
>

> > > > ok I managed to dounittestingon forms:

[eddie]

Hi all,

I'm looking at adding unit tests to a project I am working on. I had a couple of questions.

First of all, is the testRunner.py approach, as documented in slice 67, still a reasonable way to go, or has all the effort switched over to the test_runner tool in the web2py_utils package?

I was initially starting with the basic slice example, as I had this running a year or two back on another little project. I'm finding that when sys.argv[2] is accessed in the sample testRunner.py file (line 33), it is only of length 1, the contents is ['testRunner.py'].

I can see there's a fair bit of discussion around the use of sys.argv in this thread. I wasn't sure if something has been dropped from web2py that now means sys.argv is not populated. Or was this always relying on some other hack that copied it across?

If the example on slice 67 is now out of date, I'm happy to find that out and cut over to a newer style. I couldn't see any discussion on here suggesting either way.

Cheers,

Eddie

[eddie]

I should add I'm on web2py version 1.92.1.

And I should also add I'm extremely supportive of the efforts to build some better unit testing infrastructure like this, these examples are fantastic.

[Thadeus Burgess]

test_runner in web2py_utils was designed around slice 67. test_runner lets you do a little more, and handles more of the state management for you.

--
Thadeus