I've uploaded a patch with a WIP-version of the sphinx extension I'm
hacking on. It's about doctests with fixtures similar to unittests. So
far it generates html output and allows you to run your documentation
as a normal unittest testsuite. I've written some documentation and
added it to the sphinx documentation.
I'm thinking same thing and just started implementing.
Behavior Driven Development(BDD) tries to get closer to documentation
rather than TDD.
So I think BDD tools is more friendly to Sphinx. Techinically, BDD and
TDD are very close.
These are just different in method's names.
And more, I'm interested in cucumber's test. It is implemented on Ruby/Rspec.
> I've uploaded a patch with a WIP-version of the sphinx extension I'm
> hacking on. It's about doctests with fixtures similar to unittests. So
> far it generates html output and allows you to run your documentation
> as a normal unittest testsuite. I've written some documentation and
> added it to the sphinx documentation.
Am Samstag, 7. November 2009 11:40:11 schrieb Chris Withers:
> What is a WIP-version?
A Work in progress version ;) That version is far from perfect and I just wanted to get general feedback on the idea.
> Have you looked at Manuel?
No, not yet. Thanks for the hint.
From a first glance there is some functionality I was looking for (running the documentation as testsuites to embed them in continuous testing utilities like bitten).
The syntax I realized is more testsuite centric. For example a testsuite in your documentation will look like this:
This test doesn't fail because ``a`` has been assigned in the setup code.
Also take a look at the teardown code. As that is a doctest too, failures will be reported as they would in a normal unittest. ------------------------>8--------------------
Note that the above two paragraphs are inlined documentation for the testcase.
> Why are you looking for html output?
This will result in html output as in the attached screenshot. I hope that the integration of testsuite in your documentation will lead to uncryptic and understandable testcode as I've seen it in my own and other projects ;-)
Also note that each testcase is run in its own environment to prevent aftereffects. I know that this can be done with doctest blocks as well, but they don't integrate well with testsuites as far as I know.
So, do you think that there's the need for something like this?
> Am Samstag, 7. November 2009 11:40:11 schrieb Chris Withers: >> What is a WIP-version? > A Work in progress version ;) That version is far from perfect and I just > wanted to get general feedback on the idea.
>> Have you looked at Manuel? > No, not yet. Thanks for the hint.
>>From a first glance there is some functionality I was looking for (running the > documentation as testsuites to embed them in continuous testing utilities like > bitten).
> The syntax I realized is more testsuite centric. For example a testsuite in > your documentation will look like this:
> This test doesn't fail because ``a`` has been assigned in the setup code.
> Also take a look at the teardown code. As that is a doctest too, failures > will be reported as they would in a normal unittest. > ------------------------>8--------------------
> Note that the above two paragraphs are inlined documentation for the testcase.
>> Why are you looking for html output? > This will result in html output as in the attached screenshot. I hope that the > integration of testsuite in your documentation will lead to uncryptic and > understandable testcode as I've seen it in my own and other projects ;-)
> Also note that each testcase is run in its own environment to prevent > aftereffects. I know that this can be done with doctest blocks as well, but > they don't integrate well with testsuites as far as I know.
> So, do you think that there's the need for something like this?
Are you aware of the existing doctest extension (if not, see http://sphinx.pocoo.org/ext/doctest.html)? It already does something similar; but not as sophisticated. It is more oriented towards making sure that the examples you give in the documentation don't stop working.
Georg -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.13 (GNU/Linux)
Am Mittwoch 11 November 2009 10:47:27 schrieb Georg Brandl:
> Are you aware of the existing doctest extension (if not, see > http://sphinx.pocoo.org/ext/doctest.html)? It already does something > similar; but not as sophisticated. It is more oriented towards making > sure that the examples you give in the documentation don't stop working.
Yes, in fact I've used it extensively before. It's very nice but there were at least two drawbacks which I tried to circumvent with sphinxtest:
Doctests in a documentation file are run as a singular test. But you often want to test your code in different use-cases which you don't want to scatter over multiple files. If all tests are run in a singular environment it may get messy to find the root cause of a failure.
The sphinx doctest builder is difficult to integrate into other utilities like continuous testing frameworks (these usually expect unittests).
I will experiment a bit more with doctest blocks and see if I can get comfortable with those :)
> Am Mittwoch 11 November 2009 10:47:27 schrieb Georg Brandl:> Are you aware of the existing doctest extension (if not, see
> >http://sphinx.pocoo.org/ext/doctest.html)? It already does something
> > similar; but not as sophisticated. It is more oriented towards making
> > sure that the examples you give in the documentation don't stop working.
> Yes, in fact I've used it extensively before. It's very nice but there were at
> least two drawbacks which I tried to circumvent with sphinxtest:
> Doctests in a documentation file are run as a singular test. But you often want
> to test your code in different use-cases which you don't want to scatter over
> multiple files. If all tests are run in a singular environment it may get messy
> to find the root cause of a failure.
> The sphinx doctest builder is difficult to integrate into other utilities like
> continuous testing frameworks (these usually expect unittests).
> I will experiment a bit more with doctest blocks and see if I can get
> comfortable with those :)
I'm not saying you have to use the doctest extension :) As I said,
it's not a
serious unittesting tool, but to make sure that snippets continue to
work.
(In docs, you often use objects created earlier in later snippets, so
the shared
environment makes sense.) I just wanted to make sure you know that it
exists.
What you're developing looks very interesting, because it could at one
point
combine documentation and tests -- kind of "literate testing" :)
Ontje Lünsdorf wrote: > Doctests in a documentation file are run as a singular test. But you often want > to test your code in different use-cases which you don't want to scatter over > multiple files. If all tests are run in a singular environment it may get messy > to find the root cause of a failure.
