Testing recommendations

[Bill]

I've been impressed with what I've seen and am working on my first
python web app. Are there recommendations for testing that fits in
with GAE, e.g. local testing through the simulated datastore? I see
that python has doctest and unit tests and Django runs tests through
either of those frameworks.
Thanks,
Bill

[ryan]

Sure! Great question. Here's a snippet that will work with Python's
built-in unittest module. It should be easy to port to other unit test
frameworks. (Note the overlap with the SetupStubs() method in google/
appengine/tools/dev_appserver.py in the SDK.)

from google.appengine.api import apiproxy_stub_map
from google.appengine.api import datastore_file_stub
from google.appengine.api import mail_stub
from google3.apphosting.api import urlfetch_stub
from google3.apphosting.api import user_service_stub

APP_ID = u'test_app'
AUTH_DOMAIN = 'gmail.com'
LOGGED_IN_USER = 'te...@example.com' # set to '' for no logged in user

class MyTest(unittest.TestCase):

def setUp(self):
# Ensure we're in UTC.
os.environ['TZ'] = 'UTC'
time.tzset()

# Start with a fresh api proxy.
apiproxy_stub_map.apiproxy = apiproxy_stub_map.APIProxyStubMap()

# Use a fresh stub datastore.
stub = datastore_file_stub.DatastoreFileStub(APP_ID, '/dev/null', '/
dev/null')
apiproxy_stub_map.apiproxy.RegisterStub('datastore_v3', stub)

# Use a fresh stub UserService.
apiproxy_stub_map.apiproxy.RegisterStub(
'user', user_service_stub.UserServiceStub())
os.environ['AUTH_DOMAIN'] = AUTH_DOMAIN
os.environ['USER_EMAIL'] = LOGGED_IN_USER

# Use a fresh urlfetch stub.
apiproxy_stub_map.apiproxy.RegisterStub(
'urlfetch', urlfetch_stub.URLFetchServiceStub())

# Use a fresh mail stub.
apiproxy_stub_map.apiproxy.RegisterStub(
'mail', mail_stub.MailServiceStub())

[mrshoe]

Bill,
I would also recommend Selenium for scripted functional testing
(http://selenium.openqa.org/). The IDE FireFox plug-in makes it easy
to create the scripts.

shoe

On Apr 9, 4:17 pm, ryan <ryanb+appeng...@google.com> wrote:
> Sure! Great question. Here's a snippet that will work with Python's
> built-in unittest module. It should be easy to port to other unit test
> frameworks. (Note the overlap with the SetupStubs() method in google/
> appengine/tools/dev_appserver.py in the SDK.)
>
> from google.appengine.api import apiproxy_stub_map
> from google.appengine.api import datastore_file_stub
> from google.appengine.api import mail_stub
> from google3.apphosting.api import urlfetch_stub
> from google3.apphosting.api import user_service_stub
>
> APP_ID = u'test_app'
> AUTH_DOMAIN = 'gmail.com'

> LOGGED_IN_USER = 't...@example.com' # set to '' for no logged in user

[Andy]

I wonder if Selenium-RC could be made to work in the App Engine system
so one could drive tests with Python code.

On Apr 9, 4:29 pm, mrshoe <mrs...@gmail.com> wrote:
> Bill,
> I would also recommendSeleniumfor scripted functional testing

[Brett Morgan]

Bring me up to speed here. My recollections of Selenium are that it is
a tool that initially sits between a browser and a webserver under
test, and observes an interaction. There after it interacts directly
with the webserver in replaying the initial interaction stream that it
was trained with. All interactions are by way of HTTP.

If my understanding is correct, i'm not sure how Selenium couldn't be
used against GAE. What parts of selenium's featureset are you worried
about?

[Andy]

To start with, I'm still looking for where to get the Selenium Python
module referenced in Selenium's generated Python.
But I'm interested in any other good suggestions as well. I'm finding
Selenium capable but a little hard to automate into larger tests.

On Apr 14, 3:13 am, "Brett Morgan" <brett.mor...@gmail.com> wrote:

> If my understanding is correct, i'm not sure howSeleniumcouldn't be

[Brett Morgan]

On Tue, Apr 15, 2008 at 5:56 PM, Andy <andy.lis...@gmail.com> wrote:
>
> To start with, I'm still looking for where to get the Selenium Python
> module referenced in Selenium's generated Python.

I'm not quite following you here. Can you expand?

> But I'm interested in any other good suggestions as well. I'm finding
> Selenium capable but a little hard to automate into larger tests.

Ah. It's been a couple of years since I last looked at Selenium. I'm
guessing my knowledge on how to automate selenium tests is seriously
out of date. =)

[Andy]

Selenium-RC is not necessary since Selenium core can be added to your
AppEngine project directory and run from there. Collected steps below
to save you time:

1. Download the core:
http://selenium-core.openqa.org/download.jsp
2. unzip selenium core zip file
3. copy this selenium core directory into your App path somewhere.
4. edit your app.yaml file to permit static access (static because it
doesn't need a sever interpreter. selenium is written in javascript,
so it runs in your browser.)
Under "handlers:", add something like this:
- url: /selenium
static_dir: ./tests/SeleniumTests/selenium-core-0.8.3

static_dir is where I copied my selenium-core directory.
To access selenium, I now open the app engine url, http:localhost:8080/
selenium/index.html and run the TestRunner. From there, open your test
suite and go. I had trouble loading individual tests for some reason,
but suites workes.

I could have of course done all of this using the Firefox Selenium-IDE
plugin with fewer steps, but using the core approach does make it
cross browser supporting.

I'm still trying to figure out how to use Python for this. I'll
continue that in a separate message.

[Andy]

On Apr 15, 1:00 am, "Brett Morgan" <brett.mor...@gmail.com> wrote:
> >  To start with, I'm still looking for where to get theSeleniumPython

> >  module referenced inSelenium'sgenerated Python.

>
> I'm not quite following you here. Can you expand?

Selenium-IDE can export test scripts into Python unit test form. But
those tests try to "import selenium", which fails, and I have yet to
locate the selenium module anywhere on the web. Still looking.

[Brett Morgan]

Ahh. Mmm. Tried selium mailing lists?

[Andy]

On Apr 15, 2:16 am, "Brett Morgan" <brett.mor...@gmail.com> wrote:
> >  Selenium-IDE can exporttestscripts into Python unittestform. But

> >  those tests try to "import selenium", which fails, and I have yet to
> >  locate the selenium module anywhere on the web. Still looking.
>
> Ahh. Mmm. Tried selium mailing lists?

Good point. I did finally find it. I was looking in the wrong place.
It was bundled with Selenium-RC which runs as a service. The Python
module is meant to run with that.

So for the record, I was able to run the selenium-RC server on my Mac
and direct it to drive my AppEngine website by way of Firefox. RC is
not limited to only Firefox, but that's all I tested.

It was a little confusing since I wasn't already familiar with how the
selenium system was supposed to work. Here's what I figured out:

The Python Selenium test scripts are -not- run within the AppEngine,
nor do they need to be. The Selenium-RC server is a (Java based)
server that acts as a proxy of sorts. Your Python test script (located
in your local environment, not in the app engine directory) can
control the selenium java server which then controls your browser
which then drives your website. That website-under-test can be
anywhere locally, in the Google app engine, or on the internet.

In my case, both the Selenium server and Google App Engine were
serving on my localhost. Since they are on different ports (by
default), they don't conflict at all. Python connects to the selenium
port and selenium drives firefox to the AppEngine port.

A nice bonus is you can use Selenium-IDE GUI to create your tests in
Selenium native format initially, if you want. It has an export
feature that can convert it and save in Python form. They -almost-
work immediately. I couldn't figure out how to set the $baseURL
variable, so I manually edited the Python export template (Options-
>Options...->Formats). I also added a #!/usr/bin/python at the top.
Having the template right is valuable since it is overwritten evertime
you export an update.

My unit tests work! I am looking forward to tweaking in Python from
now on. Flow control within Selenium is frustratingly primitive.

[Brett Morgan]

So they are actually jython scripts instead of python?

[Andy]

On Apr 15, 4:05 pm, "Brett Morgan" <brett.mor...@gmail.com> wrote:
> So they are actually jython scripts instead of python?

I don't believe so (but I'm sketchy on Jython). The Selenium-RC
service can be controlled by CLI, returning a session id that is used
to control it. I was assuming the Python is driving it that way. The
script controls are available in a number of languages: Ruby, Perl,
PHP, C# and Java. So I'm reasoning the server isn't language dependent.

[Brett Morgan]

Curious. I'll probably get around to having a look at selenium again soon.

In my copious spare time.

Heh. I kill me. =)

[Andy]

On Apr 9, 4:17 pm, ryan <ryanb+appeng...@google.com> wrote:

> from google3.apphosting.api import urlfetch_stub
> from google3.apphosting.api import user_service_stub

Where does one get that google3 package? I have been searching but yet
to find it (with Google search engine even.)

Is it installed with the AppEngine SDK? If so, where? My python isn't
finding it.

[Bill]

Andy,

If you look at the dev_appserver.py, you'll see the import references
from google.appengine.api. Ryan might be accessing an internal Google
library.
So:
from google.appengine.api import urlfetch_stub
from google.appengine.api import user_service_stub

-Bill