Re: test suites for Sage
Tim Daly
One involves the Rubi test suite using patterns for integration.
This is a very large test suite, well over 9000 integrals.
I have created test cases for Axiom but not yet written the
rule-based integration patterns. I'll move them into the CATS
area in the near future. This establishes a baseline against
which the new rule-based integration patterns can test. I have
to add special functions like polylogs which Axiom does not yet
recognize before I can go further and then write the pattern rules.
Piezas's work is also pattern matching. The difficulty for
computational mathematics is that some of the patterns might
not hold true across all domains. They are identities so this
should not be a problem but I'd have to look at them in more
detail to be convinced. I've done no more than glance at the pages.
The second test suite involves graphing. I am using the
CRC Standard Curves and Surfaces (David von Seggern) book
which covers a huge range of graphs, from random (fractal,
gaussian, non-guassian) to determinate (algebraic, integral,
transcendental, piecewise continuous). This is moving, but
slowly, and will probably not see the light of day until 2011.
I tried to write a test suite based on Yudell Luke's
The Special Functions and Their Approximations that involved
expansions of MeijerG function but I don't know that I've got
the chops to do it. Maybe in my next life :-)
Tim (somebody, I can't find his name) was going to try the
CATS integration test suite in Sage. It should be rather easy
as Sage uses Maxima and the syntax is very close. I don't
know what became of that effort.
If you do a test suite on graphs I'd be happy to try to
make it work in Axiom. This would likely involve writing new
algebra code so it would be helpful if you included citations
of algorithms (if you know them) as you develop it. It looks
like Gross and Yellin includes high level algorithms so that
would be exceptionally helpful. I strongly favor basing test
suites on books containing algorithms.
A test suite for sequences and series is on my wish list but
I have not found a good reference for it.
I think it would be useful to collect good reference books for
any of the mathematics we want to test. This would give other
people a good starting point for developing test suites.
For the CATS test suite each file contains the problem, the
expected answer, and Axiom's answer. The format of the files
makes it possible to run them automatically which I do with
every system build. Does Sage have such a file format?
Tim Daly
Minh Nguyen wrote:
> This is a split off from the thread "Random banter about Sage
> standards" at http://groups.google.com/group/sage-devel/browse_thread/thread/c80b87648c213c5
>
>
> Hi Tim,
>
> On Fri, Aug 27, 2010 at 9:59 PM, Tim Daly <da...@axiom-developer.org> wrote:
>
>> For *standard test suites*, I believe we should have these in many
>> different areas. We should collect a set of reference books that are
>> fairly comprehensive and jointly develop tests that run on all systems.
>> Rubi is doing this with rule based integration.
>> http://www.apmaths.uwo.ca/~arich/
>> Axiom is doing this with the Computer Algebra Test Suite
>> http://axiom-developer.org/axiom-website/CATS/index.html
>>
>
> This is a point I at least agree on. I think it would be a moderately
> difficult project to write a Sage counterpart to the Axiom CATS. It
> would be an interesting project to see how one could develop a similar
> test suite based on T. Piezas III's book "A Collection of Algebraic
> Identities" available at
>
> https://sites.google.com/site/tpiezas/Home
>
> I've have been thinking about writing a test suite for Sage's graph
> theory functionalities. The following comprehensive book provides a
> good starting point on what to include:
>
> * Jonathan L. Gross and Jay Yellen (eds). Handbook of Graph Theory.
> CRC Press, 2004.
>
> All of these items are on my todo list [1], but I haven't made any
> headway on any of them in the last few months.
>
> [1] http://sage.math.washington.edu/home/mvngu/todo.html
>
>
Tim Lahey
> Hi Tim,
>
> On Mon, Aug 30, 2010 at 1:47 PM, Tim Daly <da...@axiom-developer.org> wrote:
>> Tim (somebody, I can't find his name) was going to try the
>> CATS integration test suite in Sage. It should be rather easy
>> as Sage uses Maxima and the syntax is very close. I don't
>> know what became of that effort.
>
>
I was working on it back in version 3, but got caught up in thesis stuff and
then basically stopped using Sage in version 4 since the partial derivatives
aren't presented in a useful form for me. What I've done is located at,
http://github.com/tjl/sage_int_testing
Once my thesis is done, I'll probably look at adapting it to Sympy/FEMhub.
Cheers,
Tim.
---
Tim Lahey
PhD Candidate, Systems Design Engineering
University of Waterloo
http://www.linkedin.com/in/timlahey
Nicolas M. Thiery
> A few months ago I experimented with writing test cases in LaTeX as
> part of a detailed discussion of underlying relevant theory. So for
> example you want to write some test cases for graph theory. You start
> by writing a short document explaining any relevant theory and runtime
> complexity of algorithms you want to cover, with references to any
> literature where appropriate. Next write down properties you're
> interested in together with any expected results, followed by a
> transcript of a Sage session that verify the stated properties. All of
> these are written up as a LaTeX document just like your normal LaTeX
> document. To run the test cases through Sage, I used a little tool [1]
> to extract the test cases and run them through the Sage doctesting
> framework. As an example of what I mean, refer to this document [2]
> which is a work in progress.
For the record: the (relatively new) sageexample environment of
sagetex allows for including doctest like examples in the latex file:
\begin{sageexample}
sage: 1 + 1
2
\end{sageexample}
Upon compilation, those examples are all extracted to a separate file
which you can test using sage -t. That's what we use to check the
examples in our French introductory Sage book. This still has a couple
glitches, but the more people use it, the better it will become :-)
Best,
Nicolas
--
Nicolas M. Thi�ry "Isil" <nth...@users.sf.net>
http://Nicolas.Thiery.name/
Tim Daly
This is similar to the CATS pamphlet files.
They use noweb format but they could use the \begin...\end
latex environment. I plan to make that change anyway.
If we look at one of the files, e.g.
http://axiom-developer.org/axiom-website/CATS/schaum1.input.pamphlet
it is a sequence of blocks. The Axiom comment character is "--" so
any line beginning with "--" is ignored by Axiom.
Each block is of the form
--S (number)
An Axiom Command Line
--R
--R The expected output from Axiom
--R
--E (number)
So when this file is run Axiom sees a command line followed by
a set of comments. The comments contain the expected output.
A post-process reads the output from Axiom and compares it
to the comments. The --S/--E numbering scheme allow the compare
function to check that all tests run and list the number of the
failing function.
This allows us to automate the testing.
Tim Daly