Multiple test files run into the interpreter

[Grégory Vanuxem]

Hello,

I am adding some input test files using the FriCAS testing framework
from M. Rubey for manual testing. During the coding test and running
test files sequentially in the interpreter, I noticed that previous
test files seem to be re-run. To be more explicit, here are the tail
of two interpreter sessions, one where I check only jws.input from a
fresh FriCAS session, the second I check jws.input but, after a
jnemo.input test.

**************************************************************************
)read src/input/jws
[snip]
statistics()

=============================================================================
General WARNINGS:
* do not use ')clear completely' before having used 'statistics()'
It clears the statistics without warning!
* do not forget to pass the arguments of the testXxxx functions as Strings!
Otherwise, the test will fail and statistics() will not notice!

=============================================================================
Testsuite: Julia WS Object Domains
failed (total): 0 (8)

=============================================================================
testsuite | testcases: failed (total) | tests: failed (total)
Julia WS Object Domains 0 (8) 0 (41)
=============================================================================
File summary.
unexpected failures: 0
expected failures: 0
unexpected passes: 0
total tests: 41


New FriCAS session:
*****************************************************************************************
)read src/input/jnemo
[snip]
statistics()

=============================================================================
General WARNINGS:
* do not use ')clear completely' before having used 'statistics()'
It clears the statistics without warning!
* do not forget to pass the arguments of the testXxxx functions as Strings!
Otherwise, the test will fail and statistics() will not notice!
=============================================================================
WARNINGS:
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber Extensions
test: 1
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber Extensions
test: 2
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber Arithmetic
test: 1
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber Arithmetic
test: 2
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber Arithmetic
test: 3
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber Arithmetic
test: 4
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber Arithmetic
test: 5
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber Arithmetic
test: 6
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber Arithmetic
test: 7
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber minimal polynomial
test: 1
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: Optimized norm
test: 1
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: Optimized norm
test: 2
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: Optimized norm
test: 3
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField arithmetic
test: 1
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField arithmetic
test: 2
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField arithmetic
test: 3
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField arithmetic
test: 4
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField arithmetic
test: 5
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField arithmetic
test: 6
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField arithmetic
test: 7
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField trigonometric functions
test: 1
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField trigonometric functions
test: 2
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField trigonometric functions
test: 3
testing in a domain without InputForm!

=============================================================================
Testsuite: Julia Nemo
failed (total): 0 (16)

=============================================================================
testsuite | testcases: failed (total) | tests: failed (total)
Julia Nemo 0 (16) 0 (66)
=============================================================================
File summary.
unexpected failures: 0
expected failures: 0
unexpected passes: 0
total tests: 66

)read src/input/jws
[snip]
statistics()

=============================================================================
General WARNINGS:
* do not use ')clear completely' before having used 'statistics()'
It clears the statistics without warning!
* do not forget to pass the arguments of the testXxxx functions as Strings!
Otherwise, the test will fail and statistics() will not notice!
=============================================================================
WARNINGS:
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber Extensions
test: 1
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber Extensions
test: 2
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber Arithmetic
test: 1
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber Arithmetic
test: 2
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber Arithmetic
test: 3
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber Arithmetic
test: 4
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber Arithmetic
test: 5
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber Arithmetic
test: 6
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber Arithmetic
test: 7
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMAlgebraicNumber minimal polynomial
test: 1
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: Optimized norm
test: 1
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: Optimized norm
test: 2
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: Optimized norm
test: 3
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField arithmetic
test: 1
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField arithmetic
test: 2
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField arithmetic
test: 3
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField arithmetic
test: 4
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField arithmetic
test: 5
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField arithmetic
test: 6
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField arithmetic
test: 7
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField trigonometric functions
test: 1
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField trigonometric functions
test: 2
testing in a domain without InputForm!
Testsuite: Julia Nemo
Testcase: NMExactCalciumField trigonometric functions
test: 3
testing in a domain without InputForm!

=============================================================================
Testsuite: Julia Nemo
failed (total): 0 (16)
=============================================================================
Testsuite: Julia WS Object Domains
failed (total): 0 (8)

=============================================================================
testsuite | testcases: failed (total) | tests: failed (total)
Julia Nemo 0 (16) 0 (66)
Julia WS Object Domains 0 (8) 0 (41)
=============================================================================
File summary.
unexpected failures: 0
expected failures: 0
unexpected passes: 0
total tests: 107
******************************************************************************************


Is it what is expected? It seems the previous test (jnemo.input) is
re-run when checking jws.input. Wouldn't it be preferable to reset the
testing framework? Something like after statistics() for example,
reset the test framework?

Regards,

Greg

[Waldek Hebisch]

On Sun, Dec 28, 2025 at 03:30:37PM +0100, Grégory Vanuxem wrote:
> Hello,
>
> I am adding some input test files using the FriCAS testing framework
> from M. Rubey for manual testing. During the coding test and running
> test files sequentially in the interpreter, I noticed that previous
> test files seem to be re-run. To be more explicit, here are the tail
> of two interpreter sessions, one where I check only jws.input from a
> fresh FriCAS session, the second I check jws.input but, after a
> jnemo.input test.

<snip<

> Is it what is expected? It seems the previous test (jnemo.input) is
> re-run when checking jws.input. Wouldn't it be preferable to reset the
> testing framework? Something like after statistics() for example,
> reset the test framework?

Testing framework keeps current score. Running a test adds to
the score. So, tests are run just when you run corresponding
input file, but reported score is cumulative (that is old score
is added to the current score). From docstring of 'statistics':

++ statistics() prints out a summary of the outcome of the testcases so
++ far. Use \spad{clear completely} to reset the statistics.

So, you are supposed to use ')clear completely' to reset the
statistics and start with zero score. In principle we could
provide something like 'clear_statistics', but ')clear completely'
have an advantage of clearing various caches. Note that
compiling a Spad domain has similar efect to ')clear completely',
so in normal use there is no much difference.

--
Waldek Hebisch

[Grégory Vanuxem]

Le dim. 28 déc. 2025 à 18:42, Waldek Hebisch <de...@fricas.org> a écrit :
>
> On Sun, Dec 28, 2025 at 03:30:37PM +0100, Grégory Vanuxem wrote:
> > Hello,
> >
> > I am adding some input test files using the FriCAS testing framework
> > from M. Rubey for manual testing. During the coding test and running
> > test files sequentially in the interpreter, I noticed that previous
> > test files seem to be re-run. To be more explicit, here are the tail
> > of two interpreter sessions, one where I check only jws.input from a
> > fresh FriCAS session, the second I check jws.input but, after a
> > jnemo.input test.
> <snip<
> > Is it what is expected? It seems the previous test (jnemo.input) is
> > re-run when checking jws.input. Wouldn't it be preferable to reset the
> > testing framework? Something like after statistics() for example,
> > reset the test framework?
>
> Testing framework keeps current score. Running a test adds to
> the score. So, tests are run just when you run corresponding
> input file, but reported score is cumulative (that is old score
> is added to the current score). From docstring of 'statistics':
>
> ++ statistics() prints out a summary of the outcome of the testcases so
> ++ far. Use \spad{clear completely} to reset the statistics.

Shame on me. I should have read the docstring. Since
testsuite/testcase clear all variables and function definitions I
thought previous statistics should have been deleted too. But
statistics are incrementally added. That makes sense in fact. Thanks
for this clarification.

Greg

PS: BTW 'clear completely' could be more explicit in the docstring I
think. Specified as a system command or using \spad{)clear
completely}.