Test kinds (was [GOOS] Presenter-first design)

[Kim Gräsman]

I think terminology gets more useful it it's defined in context.

I read about Google's naming too, and one thing they focused on was
defining exactly what constitutes a smal/medium/large test, what
characteristics they have, what you can and can't to in terms of
dependencies, etc.

This year I'm going to go through our test stack internally and
produce guidelines for developers for when to pick what kind of test,
what relative pros/cons they have, what technologies they involve,
etc.

I'm consciously staying away from terms like "unit", "integration",
"system" and Google's sizing terms to find words that reflect our
architecture, which would hopefully make it more intuitive to devs
where a given test fits.

Here's to hoping everybody interprets my terms the same way :-)

- Kim

On Thu, May 16, 2013 at 1:23 AM, Steve Freeman <st...@m3p.co.uk> wrote:
> Apparently Google talk in terms of Small (isolated, very fast), Medium
> (integration, touches some external resources), and Large (system-level).
> S.
>
> Sent from a device without a keyboard. Please excuse brevity, errors, and
> embarrassing autocorrections.
>
> On 15 May 2013, at 21:03, Rick Pingry <rpi...@gmail.com> wrote:
>
> yeah, I think it is just because the term "Unit" is suspect. Unit is such a
> vague word that implies different things in different contexts. That is a
> tough battle to wage, but I understand the need to have a common vernacular.
>
> Who is it that defines what a "Unit Test" really means? Is there a
> standards committee?
>
> On Tuesday, May 14, 2013 2:35:23 AM UTC-6, Colin Vipurs wrote:
>>
>>
>>> This time I wanted to focus purely on unit tests, or what we like to call
>>> "class isolation tests" (testing a single class and mocking all
>>> collaborators) because the word Unit is defined so differently by different
>>> people.
>>
>>
>> The phrase "unit test" has become massive overloaded, something I've seen
>> get worse as the years go on and more people do it (I once saw a test launch
>> MS Word and the developer still insisted it was a "unit" test). Rather than
>> rename them, I fight tooth and nail with everyone I meet that misuses the
>> phrase and try to educate them as to what a "unit" test really is.

[Torbjörn Kalin]

+1

We'll never find consensus on the definition of a unit test or an acceptance test. It's difficult enough to define categories that work in only one system.

Instead, let's talk about why we write the test, what we want to test, and how we do it (in that order):

- What is the purpose of the test?

Is it intended to help the programmer progress? Should it work as a safety net for future changes? Or as documentation? Is the intention to gain trust from testers? Or to gain trust/acceptance from other parts of the organisation? Or are we writing tests to learn how some new framework behaves? Or how a legacy system works? Do we want to make sure that an interface doesn't break compatibility?

Do we just want to exercise some code to see if it crashes (smoke test)?

- What is it that we want to test?

Are we verifying some internal logic (and, if so, what kind) or an algorithm? Or do we want to verify the interaction with the database (e.g. ORM mapping/SPs)? Or the interaction with another system?

Do we want to verify the state of a component, or is it the interaction between components that is interesting?

- How do we test it?

Are we executing nothing but code? Do we touch the file system? The database? An external system owned by us? An external system owned by someone else? A fake implementation of an external system?

Are we executing use case flows, or just checking the transition from one state to another?

/T

--

---
You received this message because you are subscribed to the Google Groups "Growing Object-Oriented Software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to growing-object-oriente...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.