Micro Testing (Again)
13 views
Skip to first unread message
Charlie Poole
Jul 16, 2020, 10:44:49 AM7/16/20
to lonely-coac...@googlegroups.com
Hi All,
It has been a few years since I asked for and received lots of
feedback here in answer to the question "What would a test framework
tailored specifically for micro-testing look like."
I did a little work, set it aside, travelled for a year, came home and
finally began to work seriously on it. The pandemic helped me
concentrate and I now have something that somewhat works.
It's called TC-Lite and is aimed at .NET Core testing. A few main
points, many of which connect back to that earlier discussion...
* It uses self-executing test assemblies that only need to reference
the SUT and the framework.
* It creates a separate fixture instance, intended to be extremely
lightweight, for each test case
* It uses attributes to designate tests, most commonly `[TestCase]`.
* It has no setup or teardown, at least for now. That was something
folks were a bit divided on.
This is still under development on github and has not yet had a binary
release. There's a sort of specification at
https://github.com/TestCentric/tc-lite/blob/master/README.md (Sort of
because it was written after the code and merely expresses my current
direction)
Although this targets a particular platform - as do all test
frameworks - I think some of the notions would translate well to other
languages and technology. In particular, it has been very handy that I
don't have to read and load assemblies (due to the self-execution) and
I'm finding parallel test execution much simpler without the need to
coordinate setting up and tearing down shared instances of the
fixture.
Thanks for the feedback I got here way back in 2017!
Charlie
It has been a few years since I asked for and received lots of
feedback here in answer to the question "What would a test framework
tailored specifically for micro-testing look like."
I did a little work, set it aside, travelled for a year, came home and
finally began to work seriously on it. The pandemic helped me
concentrate and I now have something that somewhat works.
It's called TC-Lite and is aimed at .NET Core testing. A few main
points, many of which connect back to that earlier discussion...
* It uses self-executing test assemblies that only need to reference
the SUT and the framework.
* It creates a separate fixture instance, intended to be extremely
lightweight, for each test case
* It uses attributes to designate tests, most commonly `[TestCase]`.
* It has no setup or teardown, at least for now. That was something
folks were a bit divided on.
This is still under development on github and has not yet had a binary
release. There's a sort of specification at
https://github.com/TestCentric/tc-lite/blob/master/README.md (Sort of
because it was written after the code and merely expresses my current
direction)
Although this targets a particular platform - as do all test
frameworks - I think some of the notions would translate well to other
languages and technology. In particular, it has been very handy that I
don't have to read and load assemblies (due to the self-execution) and
I'm finding parallel test execution much simpler without the need to
coordinate setting up and tearing down shared instances of the
fixture.
Thanks for the feedback I got here way back in 2017!
Charlie
Gerard Meszaros
Dec 31, 2020, 6:02:35 PM12/31/20
to lonely-coac...@googlegroups.com
This group has been very quiet for the last few months so I'll stir things up a
bit.
I'm wondering what people are using for ATDD tools these days. A client has a
need to provide examples of desired behaviour using spreadsheets showing input
data and expected outputs. Normally, I would recommend using Fitnesse for this
but I since the spreadsheets contain formulas that show how the expected results
are derived, pasting them into a Fitnesse wiki page would lose this information.
So ideally something that can read examples/testcases directly from the
spreadsheet.
Also, it seems that most of the ATDD tools I can find mentions of online seem to
peter out around 2010. Have they gone out of style? Have we given up on ATDD and
Example-Driven-Development (or Specification-by-Example)?
Thoughts?
Happy New Year everyone!
Gerard
--
Gerard Meszaros
Lean/Agile Coach/Mentor/Trainer
http://www.gerardmeszaros.com
http://xunittraining.com
1-403-827-2967
Twitter: @gerardmes
LinkedIn: gerardmeszaros
Author of the Jolt Productivity Award winning book "xUnit Test Patterns - Refactoring Test Code." Learn more at http://xunitpatterns.com/index.html
bit.
I'm wondering what people are using for ATDD tools these days. A client has a
need to provide examples of desired behaviour using spreadsheets showing input
data and expected outputs. Normally, I would recommend using Fitnesse for this
but I since the spreadsheets contain formulas that show how the expected results
are derived, pasting them into a Fitnesse wiki page would lose this information.
So ideally something that can read examples/testcases directly from the
spreadsheet.
Also, it seems that most of the ATDD tools I can find mentions of online seem to
peter out around 2010. Have they gone out of style? Have we given up on ATDD and
Example-Driven-Development (or Specification-by-Example)?
Thoughts?
Happy New Year everyone!
Gerard
Gerard Meszaros
Lean/Agile Coach/Mentor/Trainer
http://www.gerardmeszaros.com
http://xunittraining.com
1-403-827-2967
Twitter: @gerardmes
LinkedIn: gerardmeszaros
Author of the Jolt Productivity Award winning book "xUnit Test Patterns - Refactoring Test Code." Learn more at http://xunitpatterns.com/index.html
Jon Kern
Dec 31, 2020, 9:14:22 PM12/31/20
to lonely-coac...@googlegroups.com
Cucumber…
--
--- You received this message because you are subscribed to the Google Groups "Lonely Coaches Sodality" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lonely-coaches-so...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lonely-coaches-sodality/fa1cea84-179d-8729-ee67-3eaa3fa98471%40gerardm.com.
Charlie Poole
Jan 1, 2021, 1:46:50 AM1/1/21
to lonely-coac...@googlegroups.com
I've always been a big fan of FIT itself, on which Fitnesse was based.
Back in 2004, Ward and I did a series of presentations where we jumped
back and forth between FIT and sunit/junit/nunit depending on whether
the bit we were working on was intended to satisfy the direct intent of
the user (i.e. AT) or of the programmer (UT).
We were using html then, with tables that expressed the test along with
explanatory text. Depending on the level of effort put into it, this gave
very nice documentation, particularly when working in a domain that
required a lot of expert input. IIRC some folks had written adapters so
that excel tables could be used but I never did that.
I wonder if anyone still maintains FIT?
BTW I've always found it unfortunate that so much of the development
of AT tools ended up focusing on eliminating the need for the programmer
to be involved. While I like the idea of the user being able to experiment
and add cases, I think the collaboration between programmer and user
in designing the fixtures gives benefit far beyond merely creating the tests.
If ATs no longer required collaboration, we'd have to invent some other
excuse to do it. :-)
Charlie
Chris Pitts
Jan 2, 2021, 5:29:31 AM1/2/21
to lonely-coac...@googlegroups.com
Hi Gerard
But as Charlie has already pointed out, don’t use this as an excuse to bypass the conversation! If a spreadsheet is a convenient way of presenting test data then that’s cool, but it still needs to be generated as a collaboration between the customer and the dev team. That’s where the value lies.
HTH
Chris
--
Chris Pitts
Thirsty Bear Software
http://www.thirstybear.co.uk/
Company Number 5442181. Registered in England & Wales.
Registered office:Arnold House, 2 New Road, Brading, Isle of Wight, PO36 0DT
Thirsty Bear Software
http://www.thirstybear.co.uk/
Company Number 5442181. Registered in England & Wales.
Registered office:Arnold House, 2 New Road, Brading, Isle of Wight, PO36 0DT
st...@m3p.co.uk
Jan 2, 2021, 6:45:06 AM1/2/21
to Lonely Coaches Sodality
There's a lot to be said for Excel with conventions as the medium, although it doesn't diff well. It's certainly a useful output format.
So, to me a lot more teams should consider writing their own. It's really not that much effort, and the team should be growing its own language.
My concern with cucumber is that it can make people stop too early in developing their own language and that for many teams, they're not doing anything adventurous enough to compensate for the impedance mismatch between languages.
Original FIT was wonderfully minimal, very Wardish. Given it's only 12 classes or so, it wouldn't take long to re-implement (which I think was part of Ward's point). My favourite extension was FitLibrary, but Rick never figured out how to do open source properly. On my canonical project, we made a lot of use of HTML formatting and nested tables to represent complicated data.
So, to me a lot more teams should consider writing their own. It's really not that much effort, and the team should be growing its own language.
The hard part is remembering to keep it minimal.
S
Jeff Langr
Jan 2, 2021, 10:51:06 AM1/2/21
to lonely-coac...@googlegroups.com, Gerard Meszaros
Hi Gerard,
As Jon suggests, Cucumber has become the de facto standard here, and I still get a couple customers a year wanting to ramp up on it.
I did have a large US customer that had me in a few times a year up until about 2 years ago to teach new groups of folks on Fitnesse, but that's been the only one I've encountered, or even heard about, for about 8 years. Still, a quick look suggests that the tool continues to be updated regularly.
IIRC you could look at Fitnesse page source to divulge underlying calculations, but it sounds like that's not quite what you want.
Have we given up on ATDD / BDD? I met a lot of teams who burned themselves with it, and talked to many who said "we tried that and it sucked." As with most things agile, you can find piles of dysfunctional attempts that directly counter what the experts (Cucumber docs themselves, e.g.) tell you to avoid. (As one example, if you search about for codebases to see what others have done, you'll find a large amount of folks whose tests drive a GUI, despite that usually being a path to hell.)
I remember c. 2000 when we were first starting to help Object Mentor customers with ATDD; we had a lot of fun putting together our own little tools, each tailored to the customer's interests. Most of them had some basis in spreadsheets.
Cheers,
Jeff
Gerard Meszaros wrote on 12/31/20 4:02 PM:
As Jon suggests, Cucumber has become the de facto standard here, and I still get a couple customers a year wanting to ramp up on it.
I did have a large US customer that had me in a few times a year up until about 2 years ago to teach new groups of folks on Fitnesse, but that's been the only one I've encountered, or even heard about, for about 8 years. Still, a quick look suggests that the tool continues to be updated regularly.
IIRC you could look at Fitnesse page source to divulge underlying calculations, but it sounds like that's not quite what you want.
Have we given up on ATDD / BDD? I met a lot of teams who burned themselves with it, and talked to many who said "we tried that and it sucked." As with most things agile, you can find piles of dysfunctional attempts that directly counter what the experts (Cucumber docs themselves, e.g.) tell you to avoid. (As one example, if you search about for codebases to see what others have done, you'll find a large amount of folks whose tests drive a GUI, despite that usually being a path to hell.)
I remember c. 2000 when we were first starting to help Object Mentor customers with ATDD; we had a lot of fun putting together our own little tools, each tailored to the customer's interests. Most of them had some basis in spreadsheets.
Cheers,
Jeff
Gerard Meszaros wrote on 12/31/20 4:02 PM:
Charlie Poole
Jan 2, 2021, 11:04:15 AM1/2/21
to lonely-coac...@googlegroups.com
Good point about the size of FIT. In many cases, my fixtures added up to more than the size of the framework.
--
---
You received this message because you are subscribed to the Google Groups "Lonely Coaches Sodality" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lonely-coaches-so...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lonely-coaches-sodality/ea5d0500-1acc-4e63-b461-84f69d4c1a5fn%40googlegroups.com.
J. B. Rainsberger
Jan 3, 2021, 2:20:02 PM1/3/21
to lonely-coac...@googlegroups.com
On Thu, Dec 31, 2020 at 7:02 PM Gerard Meszaros <gmka...@gmail.com> wrote:
I'm wondering what people are using for ATDD tools these days.
xUnit/xSpec, mostly. I haven't found the "friendly to non-programmers" format helpful in industrial-strength situations. We write examples together, then I transcribe them into code, and thereafter customers don't care. They prefer demos to seeing cells turn green. I have a style for writing examples which is not limited in the same way that Gherkin is.
Jason Yip
Jan 3, 2021, 3:59:48 PM1/3/21
to Lonely Coaches Sodality
+1 to what J.B. said.
Charlie Poole
Jan 3, 2021, 4:21:53 PM1/3/21
to lonely-coac...@googlegroups.com
Joe's comment points out something quite important that hadn't been said before.
When you write customer tests, they have to be written for _your_actual_customers_ not for some mental construct you hold about what the customer _ought_ to want. The FIT approach was conceived in the first place because of customers who liked defining and seeing things in tables. If that's not what _your_ customer likes, do something else!
This post is in reaction to comments I've seen elsewhere - not on this thread - that purport to describe Gherkin or Fit or some other approach as the best one "for customers."
--
---
You received this message because you are subscribed to the Google Groups "Lonely Coaches Sodality" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lonely-coaches-so...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lonely-coaches-sodality/21c35f8f-2583-4d4f-b60c-1e393137dbf2n%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages