Using TDD, what are best practices for testing the front end?

[Andrew Bialecki]

I thought I'd throw this out there even though it's not necessarily an
MXUnit topic. How do you guys typically handle unit testing your
front-end and/or what are best practices? For instance, let's say I
want to change my view, how do I handle the testing of this?

This is important to my application because very often that display is
database driven, so making sure it is grabbing the right data and
displaying it correctly is important.

[Marc Esher]

Andrew, are you talking about "recorder" style tests ala selenium? Or
strictly programmatic options? Or both?

I've got a fair amount of experience using httpunit to do front end
testing. it's not for the faint of heart, and with more and more ajax,
frankly it's just not really a viable option anymore as its javascript
support seems not to be up to the challenge. I don't believe httpunit
is being supported anymore and so I don't think that problem is going
to improve. I really, really like it, but I fear its time may have
passed.

At my former job, we had a small group whose sole purpose was "testing
technologies". I know they settled on mercury test director and astra
quicktest, and those worked well for them, although that was 4 or 5
years ago.

Seems like the bigguns now are selenium, canoo, and watir. at least,
those are the ones i hear a lot about. Bill and Rankin, you guys have
played pretty extensively with canoo, right?

I'd really love to see this discussion go somewhere because word on
the street where i work is that, finally, after years of ignoring the
need for it, it looks like we might finally start doing some kind of
automated front-end testing and I want to be sure that the group doing
this knows all their options.

great topic, Andrew.

marc

[Alan Livie]

I think this raises 2 issues.

1) I think rather than thinking about testing the UI you are concerned with the database data a particular method returns that eventually drives the UI.
So I think this is still a job for the unit test. You will want to assert the method returns a query and that data in the returned rows is what you expect.

2) As its data you're really testing you want data you can trust for the tests. And you either set up a test database that has static reliable data for testing or you add some stuff to the test cases that loads the data before doing the tests on it.
But then maybe this is a bad idea if the purpose of your test is to check the live data :-)

I haven't looked into DBunit yet but would it be a big job to have a dbunit-esque extension to mxUnit?

Alan
________________________________________
From: mxu...@googlegroups.com [mxu...@googlegroups.com] On Behalf Of Andrew Bialecki [andrew....@gmail.com]
Sent: 28 April 2008 00:37
To: mxunit
Subject: [mxunit:333] Using TDD, what are best practices for testing the front end?

[Alan Livie]

I thought I would clarify when I said 'You will want to assert the method returns a query' ... this of course may not be the case. It could be an array of beans, and IBO etc but you get my point!
________________________________________
From: mxu...@googlegroups.com [mxu...@googlegroups.com] On Behalf Of Alan Livie [ALi...@efinancialcareers.com]
Sent: 28 April 2008 09:01
To: mxu...@googlegroups.com
Subject: [mxunit:335] Re: Using TDD, what are best practices for testing the front end?

[Andrew Bialecki]

Marc and Alan, thanks for the input. I think you both make some good
points. Let me explain how we currently handle front end testing and
see if you guys have any thoughts on how to improve this.

First, the framework we use for our website uses CFCs to "render" each
HTML page. So in our framework for each page on our site, we have a
CFC that has a "render" method that is called to handle all the logic
of the view for that page. We pass into that method any relevant, raw
(i.e. queries, structs, etc.) information that is going to be used to
figure out what to display on that page. I think this does a pretty
good job separating the model and the view from each other.

So at this point you can imagine if all a page shows is a select list
of names, I would pass some object -- let's say a query -- into this
"render" method and then loop over the entries to create the select
list. To further complicate things, let's say I want to add a generic
option to the top of my select list, like "Look at all names", as long
as the query we pass in contains at least two names.

So the output I would want for multiple names is:

Names to filter on:
<select>
<option value ="-1">Look at all names</option>
<option value ="1">Billy</option>
<option value ="2">Bobby</option>
</select>

or for a query with only one name:
<select>
<option value ="1">Billy</option>
</select>

Make sense?

The question I've been trying to figure out is where is the right
place to check that the select list I am creating is correct. I think
both of you guys have identified that there are sort of two issues
here. One question is, is the query I'm passing in to loop over
correct, and I should probably check that by unit testing whatever
method I am using to return that. The second question is, given that
query, is the output right?

This second question is, in my opinion, harder. Right now we actually
are using Mercury to drive some automated front-end testing. This
testing usually involves two phases. The first is checking that
navigation works correctly (just clicking links and making sure it can
get to different pages) and then checking values (for instance,
looking at values in tables to see if they are correct by comparing
them to baselines we set on previous runs). This approach kind of
gets at the second issue. I'm kind of wondering whether it's better
to write unit tests that check the HTML output by comparing it with a
string or whether it's better to try to set up some Mercury tests to
do this by testing if it can click around. I'm also interested in
what other front end frameworks are best -- so Marc, if you have any
favorites out of your list that you recommend I try, let me know.

Alan, I think I should definitely be doing what you mentioned and unit
testing the data that I am building HTML out of. That will definitely
help solve a lot of problems.

Let me know if you have any other thoughts on what might be best
practices or ideas in general for what I'm trying to test.

- Andrew

On Apr 28, 4:03 am, Alan Livie <ALi...@efinancialcareers.com> wrote:
> I thought I would clarify when I said 'You will want to assert the method returns a query' ... this of course may not be the case. It could be an array of beans, and IBO etc but you get my point!
> ________________________________________
> From: mxu...@googlegroups.com [mxu...@googlegroups.com] On Behalf Of Alan Livie [ALi...@efinancialcareers.com]
> Sent: 28 April 2008 09:01
> To: mxu...@googlegroups.com
> Subject: [mxunit:335] Re: Using TDD, what are best practices for testing the front end?
>
> I think this raises 2 issues.
>
> 1) I think rather than thinking about testing the UI you are concerned with the database data a particular method returns that eventually drives the UI.
> So I think this is still a job for the unit test. You will want to assert the method returns a query and that data in the returned rows is what you expect.
>
> 2) As its data you're really testing you want data you can trust for the tests. And you either set up a test database that has static reliable data for testing or you add some stuff to the test cases that loads the data before doing the tests on it.
> But then maybe this is a bad idea if the purpose of your test is to check the live data :-)
>
> I haven't looked into DBunit yet but would it be a big job to have a dbunit-esque extension to mxUnit?
>
> Alan
> ________________________________________

> From: mxu...@googlegroups.com [mxu...@googlegroups.com] On Behalf Of Andrew Bialecki [andrew.biale...@gmail.com]

[Peter Bell]

Just to add to what Alan said:

Tests should be repeatable and independent, so for any test that
depends on the db state, you're going to have to have an automated
tool for putting the db into a known state. I'm working on something
like dbUnit (it isn't a port as I don't love their API, but solves
similar class of problems) and I'll throw it over to Marc and Bill
once I'm done with it for their input. I'm not convinced it should be
part of MX Unit in the same way as DB Unit is a separate project to
JUnit, but I don't really mind either way. I need the code to scratch
my own itch, but I don't need another OSS project to manage, so no
doubt if anyone finds value in it a home will be found for the code
somewhere.

Additional thoughts re: the testing:
- Acceptance test via wattir/selenium. You'll need to put the db in a
known state before running the tests.
- Unit test the model API, mocking out db access for quick tests that
it does the right stuff with data without having to call the db (which
is slow)
- Unit test the db access code (again, you'll need something like db
Unit) - these tests will run slow so may be run less often
- Unit test controller, mocking out the model to ensure it handles its
responsibilities correctly.
- Consider using a View/Presenter pattern so if there is anything non-
trivial in the view that needs tested it can be unit tested via
testing view helpers/presenters.
- There are also various frameworks for making assertions against the
DOM which you may want to check out, but I've not had a chance to play
with them yet.

Best Wishes,
Peter

[Peter Bell]

Consider refactoring render to use some smaller, more discrete,
testable methods. Then you can write unit tests for them. If you want
to "do the right thing", you might choose to break the helper methods
into a composed object with public methods, if not, just make the
helper methods public (or package protected) for testing purposes - it
isn't perfect, but may well be a good next step.

Best Wishes,
Peter

[Marc Esher]

In general I'm not a big fan of testing private methods, but sometimes
you gotta do what you gotta do. To that end, mxunit does have a
facility for doing so: http://mxunit.org/doc/index.cfm?doc=testprivate

The upside is that it lets you keep private stuff private that should
be private. The downside is that you create the potential for a bit of
brittleness in your tests because you now have a test that would break
should you decided to refactor out that private method in question.
My opinion on that reservation is "who cares, i'll just delete the
test".

anyway, it's there if you need it.

marc

[Peter Bell]

Nice to know! I think it's appropriate for evolutionary refactoring
where you can't quite bring yourself to create another class file just
yet to encapsulate the functionality. I may become more opposed over
time, but for now I like having the crutch!

Best Wishes,
Peter

[billy]

Andrew, et al,

We built a new assertion mechanism into mxunit (rc1-0.9.90) that
directly addresses your question. Though its use admittedly blurs that
unit test line. This is not necessarily a bad thing, but something to
consider ... Anyway, if you want to test generated html content either
by passing in a string or a url you can use assertXPath():

assertXpath(String xpath, any data, [string text], [string message])

Parameters
String xpath string representing an xpath expression
any data String or URL to search
[string text] The text to match against the xpath expression. If
omitted, this assertion returns true if any elements of the xpath
epxression are found. Not required. Defaults to ""
[string message] The mssage to display when this assertion fails
Not required. Defaults to The XPath expression, #arguments.xpath#,
did not match the data.

Note, that this is not part of the core and needs to be loaded in
setUp() or in your test like this:

addAssertDecorator("mxunit.framework.XPathAssert");

Example usage:

domNodes = assertXpath("/html/body/form/option[@value='-1']", "http://
localhost/apps/thingundertest.cfm");

or

domNodes = assertXpath("/html/body/form/option[@value='-1']",
generatedHtmlContentVariable);

debug(domNodes);

assertXPath() is odd in that it returns data, specifically, xml nodes
matching the xpath expression. Note that this uses TagSoup and XOM to
parse and convert the html to an xml dom. It will insert missing tags,
including html, body, etc. This may be a bit confusing as your cfc
that generates content may only generate <select> ...</select>.
assertXpath() will expect /html/body/form/select as a "correct" xpath,
as <select> by itself is not valid xml or xhtml. We've lately been
keeping these tests in an integration directory as they may touch
system boundaries and can be expensive.

I promise to have this and all the other assertions documented very
soon.

Peter: can't wait to see the db stuff!

I'm also hearing a need for functional testing ... We've tossed around
ideas. Folks are doing some interesting things:
https://lift.dev.java.net/

public void testGoogleImageSearch() throws Exception {
goTo("http://www.google.com/");
assertThat(page, has(title("Google")));
clickOn(link("Images"));
enter("kizoom", into(textField()));
clickOn(button("Search Images"));
assertThat(page, has(text("Kizoom summer party")));
assertThat(it, has(image().withUrlThat(endsWith("summer04.jpg"))));
}

Might be something good forthe cf world. Combine that with a good db
wrapper maybe ...

best,
bill

[Peter Bell]

Sure I'm missing something simple here - just not sure what :-<

I have the following test method:

 <cffunction name="testShouldThrowAnErrorWhenWeAskForAValidLoadedPropertyThatIsNotGettable">

<cfscript>

BaseObject = createObject("component","ibo.com.BaseObject").init(GettablePropertyNameList="FirstName");

BaseObject.load( FormData );

try {

ReturnValue = BaseObject.get( "LastName" );

fail();

}

catch(Exception e){

};

assertTrue( true );

</cfscript>

</cffunction> 

(FormData is a struct created in the setup() and works just fine in other test methods). FirstName is the only gettable property, so when I try to BaseObject.get("LastName") I expect the object under test to throw an error telling me that LastName is not a gettable property. So if the error is throw I don't want an error - I want a green bar. I should only get an error if it hits the fail() which it shouldn't do.

I actually get:

Application: LastName is not a gettable property. The gettable properties are FirstName.

So it's going red because of the throw running (which is what I want to do and test).

FWIW, below is the method under test.

<cffunction name="get" output="false" returntype="any" access="public" hint="I return the value of a property name providing it is gettable.">

<cfargument name="PropertyName" type="string" required="true" hint="Name of the property to return." />

<cfargument name="DefaultPropertyValue" type="any" required="false" hint="Default value to be returned if given property name is gettable but has not been set." />

<cfscript>

if (isGettable( arguments.PropertyName ))

{

if (StructKeyExists( variables, "Get#arguments.PropertyName#" )){

return Evaluate( "variables.Get#arguments.PropertyName#()" );

} 

else {

return "";

};

}

else {

throw("#arguments.PropertyName# is not a gettable property. The gettable properties are #variables.GettablePropertyNameList#.");

}

</cfscript>

</cffunction>

I'm trying to exercise the throw clause in the final else block.

Any input appreciated!

Best Wishes,

Peter

[Marc Esher]

Hey Peter,
here's how i currently do it:

<cffunction name="demoFail2">
<cfset obj = createObject("component","myObject")>
<cftry>
<cfset IWishThisWouldPassButItWont =
obj.PayMarc5MillionToDrinkDalwhinnie15AllDayLong()>
<cfset fail("Should not get here. Function under test should've
thrown an error because I can't find someone to pay me lots of money
to drink great scotch")>
<cfcatch type="mxunit.exception.AssertionFailedError">
<cfrethrow>
</cfcatch>
<cfcatch type="any"><!--- expect to get here ---></cfcatch>
</cftry>


</cffunction>

this kind of verbosity, although currently required, drives me nuts.
it will be dealt with after we release 1.0 (shortly).

so, bottom line, have a catch that looks for an
mxunit.exception,AssertionFailedError and rethrow if it gets there.

just in case you didnt' know, on cfsnippets.org there's an mxunit
package that contains this as an example. i personally have a snippet
in my snip tree that has a stub for this, with a keyboard trigger of
"testerror", and it creates the boilerplate junk for me so I can focus
on the stuff between the cftry and the first catch block. not ideal,
i know, but it saves a few keystrokes.