Testing under the skin of the UI - controller or service?

[Miguel Almeida]

Dear all,

Following our discussion on tests in/under the UI and what you lose the lower you go (see https://groups.google.com/forum/#!topic/specificationbyexample/WDl3l8MBC8s ), I have been thinking about different places you can plug your specification to, and what the best place should be.

I started thinking about this when reading the Iowa Student loan case study, where Tim Anderson says "Our test was green, but when we actually opened a browser and tried it in development, it didn't work. Even though we coded the validation rule, it wasn't hooked up in the right place."

The first way I thought of plugging my specification was to the service layer directly. On a fictitious example  "find the oldest person of given nationality", I can plug it directly to a service method that, given a country, finds that person. Because I'll probably be using a database and might want to take advantage of the fact, the test will use a real database.

This is peaceful, but I worry about Tim's issue. Coding the rule correctly doesn't mean I will create my MVC plugging correctly - hell, I could even have that correct but forget to add the spring IoC configuration in a xml file. So an alternative might be:
- test it through the MVC.  As some (Spring, Struts in Java world) have some testing helpers to mock http requests and session, it's probably still "under the skin", albeit one level up

What you add in confidence (correct configuration, essentially adding integration checks to your test) you lose in independence (if you change your MVC, your tests will fail. But how likely is it that you change your mvc anyway?). On the other hand, you wouldn't have tests for every configuration - I think it would be cumbersome to have a test for, say, every spring-managed class to ensure configuration is as expected).

I was hoping to get other people's feedback on this. Did you ever go through the same dilemma? Do you have a standard way of plugging your acceptance tests, does it depend or do you think working directly under the UI (at the mock-http-but-with-your-mvc level) is not a good idea (and why)?


Cheers,

Miguel Almeida

[timander]

Hi Miguel, thanks for mentioning part of my story. We quickly learned from our mistakes and I would say that we were testing deep in the guts instead of "just beneath the skin" in the example you mentioned.

My current favorite way to test "just beneath the skin" (in Spring MVC) is by using mock http requests
Here's a (very rough) example:
https://github.com/timander/hungryhawk/blob/master/hungryhawk/src/test/java/net/timandersen/util/MockDispatcherServlet.java
https://github.com/timander/hungryhawk/blob/master/hungryhawk/src/test/java/net/timandersen/hungryhawk/fixtures/PersonaLisaFixture.java

Testing with something like Selenium or HTMLUnit is testing on top of the skin, testing beneath the skin (with no container running) can be done with the MockDispatcherServlet.

I hope this is helpful.

Regards,
Tim Andersen

[Miguel Almeida]

Thanks Tim,

Did you ever consider testing lower - eg, at the service level directly - like I mentioned? This would ignore everything about http altogether.

Miguel

[Tim Andersen]

Hi Miguel,

Our unit tests are usually written with JUnit, and our acceptance
tests are usually written with FitNesse. One of the 'light bulb'
moments we had a few years back is that unit tests and acceptance
tests that test the same code are not duplication. They have different
purposes and different scopes, with different levels of abstraction.

Making our FitNesse Fixtures instantiate a new Controller class or
find the Controller bean in the Spring Context is considered (to us)
an anti-pattern because it is going too deep and produces false
green-bar (this symptom may have been magnified because of the
flow-based nature of our application).

A better approach is to make the FitNesse Fixture produce a
MockHttpServletRequest and pass it to the correct URL with the
intended HttpMethod.

Sorry this explanation is implementation specific, but I hope I
understood your question enough to make a useful delineation. If I'm
way off please let me know.

Tim Andersen

> --
> You received this message because you are subscribed to the Google Groups
> "Specification by Example" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/specificationbyexample/-/yulJdClIK7EJ.
> To post to this group, send email to
> specificati...@googlegroups.com.
> To unsubscribe from this group, send email to
> specificationbyex...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/specificationbyexample?hl=en.
>

[Ürgo Ringo]

Hi Miguel,

In the past we have had some projects where we wrote automated tests
on UI level. We found that such test suite soon becomes quite big
burden for maintenance. Only benefit of going through UI is that in
case of web app you will include JavaScript code into your coverage.

However, this client side logic is also quite unstable so we had to
rewrite tests several times. Finally we decided to put main effort on
testing the server side only. At least in our case we didn't share
much client side code between different pages of the site so we could
quite easily identify the potential area that might be affected by one
or other change in JavaScript. So the amount of regression testing
needed was relatively tolerable for manual UI testing.


Ürgo Ringo

[Miguel Almeida]

Hi Tim,

You're right spot-on in what I am trying to discuss.
We're just starting with this kinds of tests and I understand that "false-green" feeling: it's exactly my worry when I instantiate the service directly (which is even more deep than the controller).

On the other hand, don't you get too implementation-specific by putting your tests right on the mockHttp layer? I imagine a few overheads like:

- assuming, say your application needs authentication, you'll have to add an authenticated user to every test: did you ever get either code repetition or too complicated setups (eg: start with tests extending from a abstractAuthenticatedTest where you put the "authenticateUser" setup code and end up with lots of code in that/those abstract class(es). This might even lead to "false-reds": your test fails because, say, you didn't setUp correctly or the setUp needed to change, even though the business rule is correct

- Too implementation-specific: things like needing to define the url to pass into the mockRequest (and they can change more frequently than the business rule) or changing the request elements (because you decide name and surname aren't good and change them to doctor.name and doctor.surname), that lead to overhead in test maintenance

Cheers,

Miguel Almeida

[Tim Andersen]

Hi Miguel,

In practice, testing from the mockHttp layer without starting the
container has been a good technique for us. We haven't had difficulty
with test maintenance or being too implementation specific. When a URL
changes, there's one place to change it in test code. When we refactor
a controller, the mockHttp request often doesn't need to change
(because it doesn't care about implementation). When request
parameters change, then we need to also add them in the fixture that
creates the mockHttp request, but that is a contract change. As far as
the user authentication piece, you can add that to your mockHttp
request and assume it as part of the persona generating those
requests.

Please try it. For us, we have had to experiment with many different
levels of abstraction before we found the "sweet spot".

Regards,
Tim Andersen

> --
> You received this message because you are subscribed to the Google Groups
> "Specification by Example" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/specificationbyexample/-/nKF4oMTtNE8J.