changes to Amara test suite

[Andrew Dalke]

Hello everyone,

I've just started working on Amara and Akara. Uche suggested that
my first project be to look at fixing and improving the Amara test
suite, inside of "test/"

The suite was derived from Python's unittest framework, with a
driver program called testall.sh. There were several non-trivial
changes to the framework which made it hard to work with. These
included:
- a declarative internal DSL for test specifications, build on
metaclasses
- a single metaclass which had hooks for all possible derived test
cases
- adding options for capturing stdout and for dry runs of the tests

The metaclass created new functions on the fly, which made it hard to
figure out even which data element caused the problem. The metaclass
was shared by all the test classes, making it hard to introduce
debugging code without debugging everything. Because the metaclass
was generic, the declaration language had many repeated data fields
to fit into the generic system.

I have been working to get rid of the metaclass test objects and to
replace the local extensions to unittest with nose. Nose implements
nearly all of the features that amara had, plus many more, such as
code coverage.

For those interested in it, I've pushed my current working branch to:

ssh://hg.4suite.org//home/dalke/amara-test

I'm still working on getting everything moved over. Once that's done
I'll be able to strip out a lot of test support code from the test/
subdirectory.

Andrew Kuchling has also started with the project. He will be looking
into fixing the test failures (many of the tests do fail), among
other tasks.

Here's the current status of my changes, on a per-subdirectory basis.

bindery/
* Conversion complete *
Ran 63 tests in 8.100s
FAILED (errors=3, failures=2)

demo/ (there is only test_atomtools.py)
* Conversion complete *
This requires amara.bindery which in turn requires the 3rd party
package "html5lib"
The page at http://code.google.com/p/html5lib/ say it is not a
"strict prerequisite"
However, this seems like a prerequisite to me.

Ran 2 tests in 0.732s
Note: there is some code in one of the test functions which is not
used.
I don't know why it's present but ignored.

docs/
* Conversion complete *
Ran 33 tests in 0.192s
FAILED (failures=1)

dom/
* Conversion complete *
Ran 3 tests in 0.239s

lib/
* Conversion complete *
Ran 36 tests in 0.200s

sevendays/
* Conversion complete *
Ran 9 tests in 2.489s
FAILED (failures=3)

tree/
* Conversion complete *
Ran 11 tests in 59.631s
(but there are SyntaxWarnings)

writers
* Conversion complete *
Ran 7 tests in 0.026s
FAILED (errors=4, failures=1)

xmlcore
* Conversion complete *
Ran 3 tests in 0.003s
FAILED (errors=3)

xpath
*** Still working in this one to remove the metaclass usage ***
Ran 56 tests in 0.159s
FAILED (errors=1, failures=1)

xslt:
*** Still working in this one to remove the metaclass usage ***
Ran 227 tests in 11.895s
FAILED (errors=36, failures=38)

xupdate:
*** This seems very out-of-date! ****
Ran 1 test in 0.176s
FAILED (errors=1)

Andrew
da...@dalkescientific.com

[Uche Ogbuji]

Andrew Dalke wrote:

> xupdate:
> *** This seems very out-of-date! ****
> Ran 1 test in 0.176s
> FAILED (errors=1)

Yeah, the XUpdate test suite was ported from 4Suite before the code was.
Working on that should go somewhere on our roadmap. I don't expect it
to be a straightforward port, though; It will probably not be a
straightforward port, but should operate in streaming fashion, as much
as possible, which means it will depend on planned changes to the parser
infrastructure.

--
Uche Ogbuji http://uche.ogbuji.net
Founding Partner, Zepheira http://zepheira.com
Linked-in profile: http://www.linkedin.com/in/ucheogbuji
Articles: http://uche.ogbuji.net/tech/publications/
friendfeed: http://friendfeed.com/uche
Join me at STC09:
* http://www.semantic-conference.com
* http://friendfeed.com/stc09

[Uche Ogbuji]

I have merged the amara-test branch to trunk. I'm sorry for the delay.
If the amara-test branch is to continue, I think it might be best to
re-clone it from trunk?

I did leave notes on the merge process, as an aid to other developers:

http://wiki.xml3k.org/Amara2/Developer_notes#Samplebranchmergesession

Thanks to both Andrews for their contributions.

--Uche

[Uche Ogbuji]

I moved this to

http://wiki.xml3k.org/Amara2/Developer_notes/Mercurial

--Uche

Join me at Balisage:
* http://www.balisage.net/