Unit-Testing Dilemma

[Nan]

In most situations, my app, upon receiving an HTTP request, sends data
to a third-party API, and returns an empty HttpResponse. I need to
test that the correct data is sent to the third-party API based on
internal application state. I'm perplexed as to how to intercept this
data in a unit test.

Here's a sort of dummy version of the code in question. What I need
to be able to test is that depending on the data in the CurrentState
object, the value of the "message" parameter sent by the API call is
correct. Any suggestions?

# myapp.views
from myapp.models import CurrentState
from myapp.exceptions import *
from third_party.api import theAPI

def my_view(request):
state = CurrentState.get(pk=request.session.get('state'))
new_data = request.POST.get('new_data')
state.update(new_data)
if state.x() and state.y():
result = theAPI.call(msg='%s/%s' % (state.a(), new_data))
elif state.z():
result = theAPI.call(msg='%s/%s' % (state.b(), new_data))
else:
result = theAPI.call(msg=state.c())
if not result:
raise ApiFailureException()
return HttpResponse('success')


# third_party.api
import urllib, urllib2
from django.conf import settings

class theAPI(object):
def call(self, msg=''):
data = urllib.urlencode({'message': msg})
req = urllib2.Request(url=settings.THIRD_PARTY_API_URL, data=data)
resp = urllib2.urlopen(req)
return resp.status == '200'

Thanks, folks

[Daniel Roseman]

On Monday, June 20, 2011 6:07:59 PM UTC+1, Nan wrote:

In most situations, my app, upon receiving an HTTP request, sends data
to a third-party API, and returns an empty HttpResponse.  I need to
test that the correct data is sent to the third-party API based on
internal application state.  I'm perplexed as to how to intercept this
data in a unit test.

Here's a sort of dummy version of the code in question.  What I need
to be able to test is that depending on the data in the CurrentState
object, the value of the "message" parameter sent by the API call is
correct.  Any suggestions?

 

Don't test the third-party service. You don't have any control over it - you just have to assume that it works, and always does what it says in the published API.

The only bit you need to test is your interaction with that API. The best way to do that is to use a technique called "mocking", where you replace the part of your code which calls the API with a dummy object which always returns what you tell it. Of course, there are plenty of libraries to enable this, but my favourite is Michael Foord's one, simply called "mock" - see http://www.voidspace.org.uk/python/mock/

Using mock, you would patch your "theAPI" class to use a mock function in place of the "call" method, which returns the right data depending on the parameters. Then assert that your code does the correct things in response.

--

DR.

[Nan]

I'm not testing the third-party service. I need to test *what I send
to them*. I.e. that the output of my_view is correct. The trouble is
that neither my_view nor the API call actually returns the output that
I need to check.

Does that make sense?

On Jun 20, 1:59 pm, Daniel Roseman <dan...@roseman.org.uk> wrote:
> On Monday, June 20, 2011 6:07:59 PM UTC+1, Nan wrote:
>
> > In most situations, my app, upon receiving an HTTP request, sends data
> > to a third-party API, and returns an empty HttpResponse.  I need to
> > test that the correct data is sent to the third-party API based on
> > internal application state.  I'm perplexed as to how to intercept this
> > data in a unit test.
>
> > Here's a sort of dummy version of the code in question.  What I need
> > to be able to test is that depending on the data in the CurrentState
> > object, the value of the "message" parameter sent by the API call is
> > correct.  Any suggestions?
>
> Don't test the third-party service. You don't have any control over it - you
> just have to assume that it works, and always does what it says in the
> published API.
>
> The only bit you need to test is your interaction with that API. The best
> way to do that is to use a technique called "mocking", where you replace the
> part of your code which calls the API with a dummy object which always
> returns what you tell it. Of course, there are plenty of libraries to enable
> this, but my favourite is Michael Foord's one, simply called "mock" -

> seehttp://www.voidspace.org.uk/python/mock/

[DrBloodmoney]

On Mon, Jun 20, 2011 at 3:52 PM, Nan <ring...@gmail.com> wrote:
> I'm not testing the third-party service.  I need to test *what I send
> to them*.  I.e. that the output of my_view is correct.  The trouble is
> that neither my_view nor the API call actually returns the output that
> I need to check.
>
> Does that make sense?

Mock is one good solution. Here's what I've done in the past
(basically half-assed mock):

1. Have representative data sets that are good for the service (eg.
whatever you send to them, and whatever they send you in return).
2. Monkey patch the call:

def hackety_patch():
from StringIO import StringIO
data = StringIO(testdata_response_from_API)
data.seek(0)
return data.read()

# in TestCase subclass
def setUp(self):
third_party.api.urllib2.urlopen = hackety_patch

def tearDown(self):
third_party.api.urllib2.urlopen = urllib2.urlopen

3. Break up your API calling code into more testable units to truly
isolate your independent code from the API calling code. It'll be much
easier to catch problems in the API integration code.

[Nan]

Hm, I'm not worried about receiving a valid response from the third-
party API, just about testing the value of the "msg" parameter that's
passed into it. I need to test the msg parameter because it is in
turn essentially a proxy for which state was reached in my_view.

my_view is actually a great deal more complex than in the example, and
is indeed broken into many smaller function calls. I need to unit
test to make sure that the logic is correct -- and since all those
function calls return empty HttpResponse objects, I can't use their
return values to test the correctness of their logic.

Just brainstorming here, could there be a way around this by placing a
logging call of some sort in theAPI.call() that would only be executed
during unit testing, and then to test the contents of the log?


On Jun 20, 6:20 pm, DrBloodmoney <drbloodmo...@gmail.com> wrote:

[Ian Clelland]

On Mon, Jun 20, 2011 at 4:25 PM, Nan <ring...@gmail.com> wrote:

Hm, I'm not worried about receiving a valid response from the third-
party API, just about testing the value of the "msg" parameter that's
passed into it.  I need to test the msg parameter because it is in
turn essentially a proxy for which state was reached in my_view.

my_view is actually a great deal more complex than in the example, and
is indeed broken into many smaller function calls.  I need to unit
test to make sure that the logic is correct -- and since all those
function calls return empty HttpResponse objects, I can't use their
return values to test the correctness of their logic.

Your view function may indeed be too complex to test properly, and it sounds like it is too tightly coupled with with the API call -- there is no way to call the view without having it call the actual 3rd-party API as imported at the top of the file.

There are a couple of ways out of this -- one is to use mocks, as suggested before. Coupled with the dependency injection pattern, you would normally do something to have the testing framework pass the mock API to the view, and then you would query the mock at the end of the test to see what was passed to it. Since Django's test framework is based around end-to-end testing (making requests through TestCase.client and examining the response returned,) this can be difficult to achieve.

The other way is to make your view function as simple as possible -- simple enough that it is obviously correct, without any unit tests. Take *everything* that can affect the message which is passed to the API, and abstract that into another function that the view can call. Then test *that* function as a unit.

 

Just brainstorming here, could there be a way around this by placing a
logging call of some sort in theAPI.call() that would only be executed
during unit testing, and then to test the contents of the log?

This sounds like turning your API wrapper into a mock object at test time -- it might be the easiest way to do it, if you go the mock route.

If theAPI is a class, then give it methods like setTestMode() and resetTestMode(), and use them in your setUp and tearDown methods. Then, if test mode is enabled, don't actually make the third party call, just store the passed-in message in a module-level or class-level list and return success. After your view returns, check the list to see that the message was correct.

Depending on the tests you have, you may want to have a way to tell the API to return success or various kinds of failure. Most mocking frameworks have ways of telling the mock objects what to return before making the method calls.

At some point, though, I would start to become wary of putting too much testing code in the real classes. (I'm sure there's an unpleasant name for that kind of pattern)

 

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django...@googlegroups.com.
To unsubscribe from this group, send email to django-users...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

--
Regards,
Ian Clelland
<clel...@gmail.com>

[Nan]

> Your view function may indeed be too complex to test properly, and it sounds
> like it is too tightly coupled with with the API call -- there is no way to
> call the view without having it call the actual 3rd-party API as imported at
> the top of the file.

I'd be a little confused as to how to factor that out. I mean, in the
actual app that call is refactored behind a function that wraps the
third-party API, and I could theoretically monkey-patch something over
that function call for unit testing. But the view still has to return
an HttpResponse, and a blank one.

> The other way is to make your view function as simple as possible -- simple
> enough that it is obviously correct, without any unit tests. Take
> *everything* that can affect the message which is passed to the API, and
> abstract that into another function that the view can call. Then test *that*
> function as a unit.

Yes, it is very complex: it has to assess about 30 different potential
states via nested conditionals. It's actually broken down into a
number of sub-functions that each deal with a few sub-states; and
while refactoring might be a good idea, I wouldn't want to even
attempt that before having some good unit tests in place!

I think what you're suggesting (correct me if I'm wrong) is to have
the top-level view function be the only place the API is called, and
to use only a single call; every other sub-function that reaches an
exit state should instead return a string containing the message to
send?

Thinking aloud, my concern then becomes that some of the sub-functions
must actually make multiple API calls. So we'd have to be returning
tuples or something instead.

> If theAPI is a class, then give it methods like setTestMode() and
> resetTestMode(), and use them in your setUp and tearDown methods. Then, if
> test mode is enabled, don't actually make the third party call, just store
> the passed-in message in a module-level or class-level list and return
> success. After your view returns, check the list to see that the message was
> correct.

Oh! Thank you! I didn't understand before how a mock API might make
that possible -- it makes a lot more sense now. That soudns like the
perfect way to do it.

At least until we've got enough tests to make sure the refactoring is
correct, I need to be able to create the tests without touching the
actual view logic, so testing what the API receives makes the most
sense to me.

[Ian Clelland]

On Tue, Jun 21, 2011 at 7:30 AM, Nan <ring...@gmail.com> wrote:

> Your view function may indeed be too complex to test properly, and it sounds
> like it is too tightly coupled with with the API call -- there is no way to
> call the view without having it call the actual 3rd-party API as imported at
> the top of the file.

I'd be a little confused as to how to factor that out.  I mean, in the
actual app that call is refactored behind a function that wraps the
third-party API, and I could theoretically monkey-patch something over
that function call for unit testing.  But the view still has to return
an HttpResponse, and a blank one.

You're right, the view does need to return an HttpResponse; there's nothing you can do about that. My point about tight coupling was that your view has also been made responsible for instantiating the API wrapper that it thinks it needs, and there's no way to tell it not to. In other frameworks, I would immediately suggest using dependency injection -- the view depends on an API wrapper instance, so it shouldn't be creating that instance; that should be provided to it:

def my_view(request, api):

    ...

    result = api.call(msg)

because then you could call my_view(request, theAPI()) in most circumstances, but my_view(request, fake_api) in test code. Unfortunately, the view is almost always called directly by Django, and you don't have much control over its parameters.

> The other way is to make your view function as simple as possible -- simple
> enough that it is obviously correct, without any unit tests. Take
> *everything* that can affect the message which is passed to the API, and
> abstract that into another function that the view can call. Then test *that*
> function as a unit.

Yes, it is very complex: it has to assess about 30 different potential
states via nested conditionals.  It's actually broken down into a
number of sub-functions that each deal with a few sub-states; and
while refactoring might be a good idea, I wouldn't want to even
attempt that before having some good unit tests in place!

I think what you're suggesting (correct me if I'm wrong) is to have
the top-level view function be the only place the API is called, and
to use only a single call; every other sub-function that reaches an
exit state should instead return a string containing the message to
send?

That's what I was suggesting; that way the view becomes simple enough that anyone looking at it can be assured of its correctness, without a host of unit tests. Those tests can be applied to the functions that actually construct the messages. But, see below -- I didn't realise that this effort was to aid in refactoring the code.

 

Thinking aloud, my concern then becomes that some of the sub-functions
must actually make multiple API calls.  So we'd have to be returning
tuples or something instead.

> If theAPI is a class, then give it methods like setTestMode() and
> resetTestMode(), and use them in your setUp and tearDown methods. Then, if
> test mode is enabled, don't actually make the third party call, just store
> the passed-in message in a module-level or class-level list and return
> success. After your view returns, check the list to see that the message was
> correct.

Oh!  Thank you!  I didn't understand before how a mock API might make
that possible -- it makes a lot more sense now.  That soudns like the
perfect way to do it.

At least until we've got enough tests to make sure the refactoring is
correct, I need to be able to create the tests without touching the
actual view logic, so testing what the API receives makes the most
sense to me.

If this is all for a refactoring, then you're probably on the right track there -- instrument the existing object for testing, rather than restructuring the view first. Get the code into a state where you can trust it, and then you can start restructuring it to make it more easily testable in the future.

Good luck! :)

[Nan]

> That's what I was suggesting; that way the view becomes simple enough that
> anyone looking at it can be assured of its correctness, without a host of
> unit tests. Those tests can be applied to the functions that actually
> construct the messages.

Right, it's really those supporting functions that I need to test, but
they currently don't return anything more informative than the view
does (i.e. blank HttpResponse)

> If this is all for a refactoring, then you're probably on the right track
> there -- instrument the existing object for testing, rather than
> restructuring the view first. Get the code into a state where you can trust
> it, and then you can start restructuring it to make it more easily testable
> in the future.

Well, it's not just for refactoring -- we're planning to add
additional features that will make the state logic even more complex.
But if more testable code would require refactoring, I'd like to
already have some form of tests in place anyway because there's a
decent chance that refactoring without them would break something.

[Andy McKay]

On 2011-06-20, at 12:52 PM, Nan wrote:

> I'm not testing the third-party service. I need to test *what I send
> to them*.

Use Mock and assert_called_with:

http://www.voidspace.org.uk/python/mock/mock.html#mock.Mock.assert_called_with

In this case you'd set theAPI.call as your mock and check that under different conditions it is called correctly.
--
Andy McKay
an...@clearwind.ca
twitter: @andymckay

[Andrew Brookins]

You don't need mocks or dependency injection in this case. Just separate the message construction code, so you can test it in isolation:

# myapp.views
from django.http import HttpResponse
from myapp.models import CurrentState
from myapp.exceptions import ApiFailureException

from third_party.api import theAPI


def my_view(request):

    state = CurrentState.get(pk=request. session.get('state'))

    new_data = request.POST.get('new_data')
    state.update(new_data)

    msg = _get_message(state, new_data)
    result = theAPI.call(msg)
    if result is None:
        return ApiFailureException()
    return HttpResponse('success')

def _get_message(state, new_data):
    message = None
    if state.x() and state.y():
        message = '%s/%s' % (state.a(), new_data)
    elif state.z():
        message = '%s/%s' % (state.b(), new_data)
    else:
        message = state.c()
    return message

[Nan]

> Use Mock and assert_called_with:
> http://www.voidspace.org.uk/python/mock/mock.html#mock.Mock.assert_ca...

> In this case you'd set theAPI.call as your mock and check that under different conditions it is called correctly.

Oh, perfect -- thank you, that will help a lot!

> You don't need mocks or dependency injection in this case. Just separate the
> message construction code, so you can test it in isolation:

Yeah, it looks like that may be a next step, but I'm hesitant to do
even that much refactoring without tests already in place, due to the
complexity and fragility of the logic.