Unit tests for functions with varying output

[Merle Sterling]

Some of the XPath functions, such as now(), local-date(), local-
dateTime(), random(), etc, will produce a different output each time
they are run and don't seem suited to having a unit test. How would
you write a test condition to verify it is correct? Those types of
functions need a human to verify that it worked.

So do we simply omit unit tests for those types of functions or write
only the simple 'exists' test?

[Mark Birbeck]

Hi Merle,

> Some of the XPath functions, such as now(), local-date(), local-
> dateTime(), random(), etc, will produce a different output each time
> they are run and don't seem suited to having a unit test.

Just about everything can be tested. :) But I agree that some things
will take a it more working out than others, and will require a bit of
lateral thinking.

> How would
> you write a test condition to verify it is correct? Those types of
> functions need a human to verify that it worked.

There will of course be some functions that we cannot test across
their whole range of applicability, that's true. So you are right that
it's going to be difficult to test every possible date (although as it
happens we could do this, but I'll leave that for another day).

But the key thing is that we don't want humans to have to test
anything! All we want to do is run a test and see either a green bar
or a red one.

So we need to ensure that we test as much as we can.

For example, in the spec, the description of random() begins:

This function generates and returns a uniformly distributed random or
pseudorandom number in the range from 0.0 up to but excluding 1.0.

So that's immediately two conditions that you can check for.

Also, the function takes an optional boolean parameter, so you should
add tests for passing null, passing no parameters, passing two
parameters, and so on.

Those are the easy ones, but thinking through a little more, we
realise that we can also test that two successive calls to random()
are not equal:

Assert.areNotEqual(evalXPath('random()').stringValue(),
evalXPath('random()').stringValue());

And so on.

I agree that it's very difficult to *perfectly* test that the number
itself is random, but as you can see, there is still a lot more we can
do than just test that the function exists.

On now(), the spec say that it should return a value formatted as follows:

2006-10-14T01:04:17Z

You can test all of this, from really basic tests like "is the string
20 characters long" (we'll need to clarify whether it can be 19, i.e.,
whether the 'Z' is optional, but assume for the sake of this
discussion that it is not):

var xpDate = evalXPath('now()').stringValue();
Assert.areEqual(20, xpDate.length);
Assert.areEqual('Z', xpDate.charAt(19));

to "is the 11th character a 'T'":

var xpDate = evalXPath('now()').stringValue();
Assert.areEqual('T', xpDate.charAt(10));

through to tests on the date:

var jsDate = new Date();
var xpDate = evalXPath('now()').stringValue();
Assert.areEqual(jsDate.getYear(), xpDate.substr(0, 4));

Assert.areEqual('-', xpDate.charAt(4));

var m = jsDate.getMonth() + 1;
Assert.areEqual((m < 10) ? "0" + m : m, xpDate.substr(5, 2));

Assert.areEqual('-', xpDate.charAt(6));

var d = jsDate.getDay() + 1;
Assert.areEqual((d < 10) ? "0" + d : d, xpDate.substr(6, 2));

And so on, testing the time in the same way.

(I've not tried any of the script...I'm just showing you the direction.)

I realise that this doesn't test for the whole range of possible
dates, which is probably why you thought it was not possible to test
now(). But any problems with the now() function would have to be
pretty subtle to fall through this set of tests, and anyway, they
should eventually get caught.

(Agile has the idea of 'exploratory testing', where periodically
someone would do some 'what if scenarios'. So they might set their
computer's date and time to Xmas 2020, or to the last day of February
in a leap-year, or change the time zone to Australia...etc.)

> So do we simply omit unit tests for those types of functions or write
> only the simple 'exists' test?

No...definitely not! :) We have to have tests, so no matter what, we
have to find a way to create them. Hopefully these examples show that
even when we can't test everything, we can often test an enormous
amount.

Regards,

Mark

[Merle Sterling]

I was thinking along those lines but was more concerned about the fact
that we can't test if the result is actually correct. We can go along
ways toward verifying that it is at least formatted correctly and can
have tests for boundary conditions or invalid input but will have to
live with the fact that the automated unit tests can't truly verify
correctness for some of these functions.

But thanks for giving me lots of ideas for what is acceptable for
these cases.