Having an agreed upon set of GEDCOM files for testing would indeed be
great! Some of Tamura's tests would be a good start, but like any good
set of unit tests, it would be a large undertaking to get full
coverage. I see that we could break down testing into Import
compatibility and Export compatibility. For Import, it would be nice
to have small and large test GEDCOMs, as those test two very different
things.
For testing Import, it would be nice to see:
* messages produced during import
* evidence that the data was correctly parsed (maybe use Export to
test, but that introduces additional possible errors)
* time it took to load
* assumptions made for unspecified data
* how it deals with non-official extensions (should not crash, but just ignore)
* overall score
For testing Export, it would be nice to see:
* differences between resulting GEDCOM and original (maintain original facts)
* listing non-official extensions (perhaps producing extra information
should not be penalized?)
* time it took to save
* overall score
Users could self-report running the tests on different products:
* test file, version
* product, version
* operating system, version
* other relevant software versions
* computer specs (RAM, CPU, etc)
I think someone could put together a Ruby on Rails or Django site
fairly quickly...
-Doug