Unit test inner procedures

[Zelphir Kaltstahl]

Sometimes I find myself thinking: "I should really write some tests for all of this!"

But then I ask myself: "Uhm, how can I test some of the inner procedures of this procedure?"

I sometimes use inner procedures when no other part of the code needs access to some procedure and it fits purpose-wise into that wrapping procedure.

It is also helpful to wrap things, which I want to be exchangeable. For example for some xexpr rendering on a website, I could make a renderer for the whole website, which then internally is broken down into parts, which are all implemented by their own procedures, which are inner procedures to the all-wrapping renderer. This way I can return some procedure which uses these inner procedures (its in the closure's environment). This seems very useful to me and I would like to keep some code that way. It also keeps namespaces cleaner and makes naming easier, because a procedure inside a wrapping procedure can have simpler names than outside of it in some cases.

However I have this problem of "How to unit test these inner procedures?" Ideally I would not need to put tests into the wrapping procedure, but could keep the tests separate in another file.

Here is some code example:

~~~

(define (modulator clazz)
  (define (modulo a-number)
    (remainder a-number clazz))
  modulo)
(let ([my-modulator (modulator 7)])
  (displayln "My modulator will do the job!")
  (my-modulator 50))

~~~

(OK this is a very artificial example.)

How would I unit test the `modulo` procedure, without taking it outside of its wrapping procedure? Is there an easy way this can be done? (or maybe inner procedure unit testing is a big no-no? If so, why?)

[Jack Firth]

I don't think you can directly test an inner procedure while keeping your test code separately loadable (e.g. different file or module). It doesn't seem like a good idea to me, personally. Inner procedures communicate to me that I can change, reorganize, delete, and otherwise do whatever I want to them without breaking any code outside the definition of the outer procedure. Breaking tests in a different file with a refactoring of an inner procedure would be very surprising to me.

Instead, I recommend not using inner procedures so extensively. Instead define functions within modules (or possibly submodules) and use `provide` with `contract-out` to declare which functions make the public API of your module. You can then add a test submodule which has access to the inner workings of the outer module and test "private" helper functions that way. Here's an example:

#lang racket;; note that using #lang implicitly creates a module around the whole file

(provide
  (contract-out
    [my-public-function (-> input? output?)]))

(define (my-public-function input)
  (helper2 (helper1 input)))

(define (helper1 input) ...)
(define (helper2 input) ...)

(module+ test ;; inside this submodule we can see helper1 and helper2, even though they're not provided
  (require rackunit)
  (check-equal? (helper1 test-input) test-output)
  ... more tests here ...)

[John Clements]

+1. Totally agree.

> --
> You received this message because you are subscribed to the Google Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Zelphir Kaltstahl]

Huh, that looks reasonable.

So far I've not used modules much, except the implicit one by:

~~~

#lang racket

~~~

So the example is helpful.

The tests are not in another file, but at least they are not inside a wrapping procedure and are in a way separate.

Maybe I should have a look at the different ways of defining modules again and use them more.

[Matthias Felleisen]

+2    (that’s what the style guide says, too)

BUT, one could easily imagine an extension to the unit testing framework where inner tests work, too. With a combination of coverage and unit testing, you can usually get these inner unit tests to run and record their status the same way outer ones do in module+. Pyret, for example, does exactly this, so we should be abel to do it too. 

[Jack Firth]

BUT, one could easily imagine an extension to the unit testing framework where inner tests work, too. With a combination of coverage and unit testing, you can usually get these inner unit tests to run and record their status the same way outer ones do in module+. Pyret, for example, does exactly this, so we should be able to do it too.

Looking at Pyret, you're referring to the "where" syntax right? So this:

fun sum(l):

  cases (List) l:

    | empty => 0

    | link(first, rest) => first + sum(rest)

  end

where:

  sum([list: ]) is 0

  sum([list: 1, 2, 3]) is 6

end

...means that the "where" body is composed of tests of the `sum` function. I like this a lot and want it for Racket (in a way that's more direct than submodules). But I have no idea how it should work for nested functions that close over variables of the outer function. Would the tests specify the closure bindings maybe?

[Benjamin Lerner]

(Pyret co-lead dev here.)

The way nested tests work for us in Pyret is actually simpler than that: As a dummy example, consider a curried addition function

fun make-adder(num1 :: Number):
  fun result(num2 :: Number):
    num1 + num2
  where:
    result(5) is num1 + 5
    result(10) is num1 + 10
  end
  result
where:
  make-adder(3)(6) is 9
  make-adder(4)(2) is 6
end

This definition will run six test cases — the two test cases for make-adder will each run the two nested test cases for result. These will be reported as three blocks of test cases: one block for make-adder and two blocks for result. The test cases for result run in the lexical scope of the body of make-adder, so they have closed over num1 as part of their environment.

(In practice, this can lead to many, many test cases, obviously. So when running a program in Pyret, by default we only run the test cases lexically present in the main module of the program.)

~ben

​

[Jack Firth]

I like it. Added an issue to figure out a Rackety way to do this; feedback strongly encouraged.

[Sean Kanaley]

Regarding submod testing, would it make sense to have a submod* form that recursively imports? I've recently become a fan of this approach to testing inner procedures but it seems to require both (submod "." <name>) (submod "." <name> test), or maybe a version that specifically imports all tests recursively.