Tautologous Testing
Steve Bryant
make sure that it goes to the correct users.
The easiest way that I can think to test this is to pass the criteria
into my Users service and see that I get the same users back as the
messages that I will send.
The problem with this approach is that I know I will do exactly that
in the component that will send the email. At which point, it seems
like I have a tautology (given that the test and the function are
doing the same thing as it relates to getting users).
The other option that I can think of is to write a manual query to
check the results. Beyond mere duplication of work, this query itself
could quickly become outdated (part of the reason why I have things
wrapped up in services instead of having ad hoc queries all over the
pace).
Any thoughts or suggestions?
Bob Silverberg
Which component is it in the scenario that you are trying to test,
and what does the method that you're trying to test do?
If you are testing a service method, and it turns around and calls a
composed Emailer component, then creating a mock for the Emailer might
do the trick. You could then interrogate the mock to see if it was in
fact called with the correct list of email addresses. Or maybe that's
way off base. I think more detail of what the components are and
which methods are involved might make it easier to understand.
Cheers,
Bob
Bob Silverberg
www.silverwareconsulting.com
Steve Bryant
Sorry I should have provided better information and probably I should
not have mentioned email at all as it isn't really relevant.
I have a Mailings.cfc service (Mailings.cfc) with a
getMailingMessages() method which returns an array of structs. Each
struct represents an email messages. It has a sendMailing() message
which just calls getMailingMessages() and sends on email for each
struct. So, I really only need to test getMailingMessages().
I also have a Users server (Users.cfc) with a getUsers() method that
returns a query of users based on given criteria.
I want to use Users.getUsers() to check the users that are returned by
Mailings.getMailingMessages(). The problem with this is that I know
that Mailings.getMailingMessages() will use Users.getUsers()
internally.
At this point, I have a tautology. It is impossible for my test to
fail because it effectively does the same thing as the method itself.
The tautology seems useless as a test, but avoiding it means
recreating the logic of Users.getUsers() and I see only drawbacks to
doing that.
Thanks,
Steve
Peter Boughton
> Mailings.getMailingMessages().
Why?
You've got a test that confirms Users.getUsers works, right?
Your test for Mailings.getMailingMessages should confirm it returns an
appropriate array of structs for the known set of users - you don't
need to re-test the users stuff, because you've got a simple and
independant Users.getUsers test already.
Peter Boughton
call and breaks it then it is flagged up?
(i.e. you're not so much testing the function itself, you're testing
the *use* of the function)
Steve Bryant
I wondered if that wouldn't be the answer. As it happens, I wrote
getUsers before I started unit testing so I don't have any yet. I do
plan to add some there in the future and I'm confident enough in the
present that it is working correctly.
I'm not sure I follow what is left to test for getMailingMessages()
then. I was going to have two tests that it was returning the correct
users, but that seems to be a redundancy so I am not sure what else I
need to test there (except, perhaps, that the structure has the
correct keys, but that seems a bit too granular to me).
Am I missing something obvious? (it wouldn't be the first time, I'm sure)
Thanks,
Steve
Bob Silverberg
>
> Bob,
>
> Sorry I should have provided better information and probably I should
> not have mentioned email at all as it isn't really relevant.
>
> I have a Mailings.cfc service (Mailings.cfc) with a
> getMailingMessages() method which returns an array of structs. Each
> struct represents an email messages. It has a sendMailing() message
> which just calls getMailingMessages() and sends on email for each
> struct. So, I really only need to test getMailingMessages().
>
OK, so what is getMailingMessages() supposed to do? What is its
purpose? What exactly are you testing?
> I also have a Users server (Users.cfc) with a getUsers() method that
> returns a query of users based on given criteria.
>
> I want to use Users.getUsers() to check the users that are returned by
> Mailings.getMailingMessages(). The problem with this is that I know
> that Mailings.getMailingMessages() will use Users.getUsers()
> internally.
>
I think perhaps that's where the problem lies. Because you're trying
to test Mailings.getMailingMessages(), but it is using
Users.getUsers(), you need to mock Users, otherwise you're doing an
integration test. It seems like you need to mock Users and have
getUsers() return some data in whatever format getUsers() actually
returns data. You will create that data in your test, so it will
always be the same, which makes it easy to do your asserts. Your test
will then call getMailingMessages(), and you can then check that the
results look the way they're supposed to, based on the mocked input
from getUsers().
--
Bob Silverberg
www.silverwareconsulting.com
Marc Esher
Here's my follow-up question: What is likely to change? What is likely to break? If you could look forward a year and say "how did my test NOT catch [this new bug]", what do you predict that bug would be?
Or, to put it another way, what about these components worries you? What don't you trust?
You've said you're not worried about the mailing itself. Good. So are you worried that the query you get isn't going to be right? or that you'll loop over the query wrong? Or that since Mailings relies on users.getUsers, that if something in users.getUsers changes then your Mailings component might get inadvertently hosed, or worse, start sending out bad messages?
Marc
Steve Bryant
That is a great question! I hadn't considered it that way, but it
really does seem like the right way to think about it.
I don't have an immediate answer for the question itself, but I think
you have set me up with my answer. As soon as I can answer that
question, then I think I am set to know what tests to write.
I just want to add that I am really excited about this line of
thought. It isn't a perspective I had taken before exactly, but I
really see a lot of value in it.
Thanks!
Steve
Marc Esher
Awesome. I look forward to hearing about how you proceed. this is definitely getting into the hairy parts of testing, but also the wonderful parts IMHO. This I think is where you get into the oft-touted but hard-to-nail-down benefit of testing where "testing guides design".
marc