Test file development

[Tim Daly]

I spent a lot of time creating test files such as mapleok.

There have been many new ones since the fork, see

axiom/src/input/*

It would be simple to ask an AI system to develop a suite of

test files for each of the various domains. It would also be

possible to ask the AI to use current textbooks, e.g. Calculus

to develop more comprehensive test suites.

Even better, the latest AI systems could probably tell you why

a test failed, pointing at lines of code.

Tim

[Qian Yun]

Thanks, I actually plan to use some of that tests,
as a wider coverage test.

The problem is, running time. I plan to mark/filter-out
the long running tests, so that I can choose to
run a quick test, or a full long test.

- Qian

> --
> You received this message because you are subscribed to the Google
> Groups "FriCAS - computer algebra system" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to fricas-devel...@googlegroups.com <mailto:fricas-
> devel+un...@googlegroups.com>.
> To view this discussion visit https://groups.google.com/d/msgid/fricas-
> devel/f44779a7-9939-44b4-b8a0-c875eeb0e41en%40googlegroups.com <https://
> groups.google.com/d/msgid/fricas-devel/f44779a7-9939-44b4-b8a0-
> c875eeb0e41en%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Tim Daly]

Which AI are you using?

Could you talk a bit about your setup? Perhaps others can help.

Tim

> You received this message because you are subscribed to a topic in the Google Groups "FriCAS - computer algebra system" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/fricas-devel/mNZF51awCpY/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to fricas-devel...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/fricas-devel/7ca7629b-0a60-4d60-b612-0f38436a439d%40gmail.com.

[Qian Yun]

I'm using gemini-cli.

The problem with tests is that, it's tricky to know the
expected result, especially because same expression can
have different equivalent forms.

Currently I have to rely on the textual output,
to catch difference/regression between commits.

- Qian

[Tim Daly]

Yes. That's why my version of test suites stores the output
of the system in the tests rather than trying to difference the
equivalent forms hoping for a zero. Even if the original output
was incorrect it can be compared with the new output to
generate an error.

This was especially hard in the integration test suites.
Fortunately my "AI" was Barry Trager. :-)

Perhaps giving the AI both the stored form and the new form
would allow it to explain the differences.

Gemini is good but the new systems are astonishing.
Unfortunately I don't have professional level of access on a
regular basis.

Tim

> To view this discussion visit https://groups.google.com/d/msgid/fricas-devel/86f36a9f-6fcc-4eea-8413-39c5fd974d6d%40gmail.com.