fyi, list of exceptions for Fricas, summer 2024 edition of independent CAS integration tests

[Nasser M. Abbasi]

FYI, 

This page   

Has now updated list of all exceptions generated by FriCAS 1.3.10 built with sbcl 2.3.11 on Linux via sagemath 10.3. 

Percentage of integrals which generated an exception is 2.19% 

Total number of integrals tested is now 106,812

The page above shows the commands and the exceptions generated. 

Individual tests Reports are not yet completed, they are building now from Latex and it will take may be 3-4 weeks to finish. But the main top level report is completed now.

Fricas did well. 4th place out of 9 CAS systems with 77.117% solved.  

The SQL database is also fully build and contains all the result of all tests. Link to the database page to download is here the database can be used to obtain more information about each test made. Some examples

Find number of integrals failed by Fricas with exception

sqlite> select count(*) from main where fricas_grade="F(-2)";
2339

Get list of all exceptions by Fricas. The sagemath-Fricas interface is much better now than before (I am using sagemath 10.3). 

I see very few exceptions due to sagemath interface issues now. (such as exp_integral_e)

sqlite> select DISTINCT(fricas_anti) from main where fricas_grade="F(-2)";

Exception raised: TypeError >>  Error detected within library code:   integrate: implementation incomplete (constant residues)

Exception raised: TypeError >>  Error detected within library code:   integrate: implementation incomplete (has polynomial part)

Exception raised: TypeError >>  Error detected within library code:   not invertible

Exception raised: TypeError >>  Error detected within library code:   Curve not irreducible after change of variable 0 -> infinity

Exception raised: TypeError >>  Error detected within library code:   integrate: 

implementation incomplete (residue poly has multiple non-linear factors)

Exception raised: TypeError >>  Error detected within library code:   integrate: implementation incomplete (trace 0)

Exception raised: TypeError >>  Error detected within library code:   catdef: division by zero

Exception raised: RecursionError >> maximum recursion depth exceeded

Exception raised: RuntimeError >> no explicit roots found

Exception raised: TypeError >>  Error detected within library code:   alglogextint: unimplemented

Exception raised: TypeError >>  Error detected within library code:   Not integrable (provided residues have no relations)

Exception raised: TypeError >>  Error detected within library code:   do_alg_rde: unimplemented kernel

Exception raised: TypeError >> An error occurred when FriCAS evaluated (x)*(exp((x)*(((-0.10000000000000001)::EXPR INT)))):    Cannot convert the value from type Float to Expression(Integer) .

Exception raised: TypeError >>  Error detected within library code:   failed of mode Union(SparseUnivariatePolynomial(Expression(Complex(Integer))),failed) cannot be coerced to mode SparseUnivariatePolynomial(Expression(Complex(Int

Exception raised: TypeError >>  Error detected within library code:   failed of mode Union(SparseUnivariatePolynomial(Expression(Integer)),failed) cannot be coerced to mode SparseUnivariatePolynomial(Expression(Integer))

Exception raised: TypeError >> An error occurred when FriCAS evaluated ((x)^(((100)::EXPR INT)))*(exp_integral_e(((1)::EXPR INT),(a)*(x))):   There are no library operations named exp_integral_e       Use HyperDoc Browse or issue

Exception raised: TypeError >> An error occurred when FriCAS evaluated ((x)^(((2)::EXPR INT)))*(exp_integral_e(((1)::EXPR INT),(a)*(x))):   There are no library operations named exp_integral_e       Use HyperDoc Browse or issue  

Exception raised: TypeError >> An error occurred when FriCAS evaluated (x)*(exp_integral_e(((1)::EXPR INT),(a)*(x))):   There are no library operations named exp_integral_e       Use HyperDoc Browse or issue                      

Exception raised: TypeError >> An error occurred when FriCAS evaluated exp_integral_e(((1)::EXPR INT),(a)*(x)):   There are no library operations named exp_integral_e       Use HyperDoc Browse or issue                            

Exception raised: TypeError >> An error occurred when FriCAS evaluated ((x)^(((-1)::EXPR INT)))*(exp_integral_e(((1)::EXPR INT),(a)*(x))):   There are no library operations named exp_integral_e       Use HyperDoc Browse or issue

Exception raised: TypeError >> An error occurred when FriCAS evaluated ((x)^(((-2)::EXPR INT)))*(exp_integral_e(((1)::EXPR INT),(a)*(x))):   There are no library operations named exp_integral_e       Use HyperDoc Browse or issue

Exception raised: TypeError >> An error occurred when FriCAS evaluated ((x)^(((-3)::EXPR INT)))*(exp_integral_e(((1)::EXPR INT),(a)*(x))):   There are no library operations named exp_integral_e       Use HyperDoc Browse or issue

Exception raised: TypeError >> An error occurred when FriCAS evaluated ((x)^(((-4)::EXPR INT)))*(exp_integral_e(((1)::EXPR INT),(a)*(x))):   There are no library operations named exp_integral_e       Use HyperDoc Browse or issue

Exception raised: TypeError >> An error occurred when FriCAS evaluated (((d)*(x))^(m))*(exp_integral_e(((1)::EXPR INT),(b)*(x))):   There are no library operations named exp_integral_e       Use HyperDoc Browse or issue          

Exception raised: TypeError >> An error occurred when FriCAS evaluated ((((d)*(x))+(c))^(((3)::EXPR INT)))*(exp_integral_e(((1)::EXPR INT),((b)*(x))+(a))):   There are no library operations named exp_integral_e       Use HyperDoc

Exception raised: TypeError >> An error occurred when FriCAS evaluated ((((d)*(x))+(c))^(((2)::EXPR INT)))*(exp_integral_e(((1)::EXPR INT),((b)*(x))+(a))):   There are no library operations named exp_integral_e       Use HyperDoc

Exception raised: TypeError >> An error occurred when FriCAS evaluated (((d)*(x))+(c))*(exp_integral_e(((1)::EXPR INT),((b)*(x))+(a))):   There are no library operations named exp_integral_e       Use HyperDoc Browse or issue    

Exception raised: TypeError >> An error occurred when FriCAS evaluated exp_integral_e(((1)::EXPR INT),((b)*(x))+(a)):   There are no library operations named exp_integral_e       Use HyperDoc Browse or issue                      

Exception raised: TypeError >> An error occurred when FriCAS evaluated ((((d)*(x))+(c))^(((-1)::EXPR INT)))*(exp_integral_e(((1)::EXPR INT),((b)*(x))+(a))):   There are no library operations named exp_integral_e       Use HyperDo

Exception raised: TypeError >>  Error detected within library code:   logextint: unhandled kernel

Exception raised: AttributeError >> NoneType object has no attribute replace

Exception raised: AttributeError

Exception raised: TypeError >>  Error detected within library code:   integrate: implementation incomplete (irrational residues)

 

 --Nasser

[Waldek Hebisch]

On Tue, Apr 16, 2024 at 03:17:18AM -0700, 'Nasser M. Abbasi' via FriCAS - computer algebra system wrote:
> FYI,
>
> This page
> <https://12000.org/my_notes/CAS_integration_tests/reports/summer_2024/indexchapter2.htm>

>
>
> Has now updated list of all exceptions generated by FriCAS 1.3.10 built
> with sbcl 2.3.11 on Linux via sagemath 10.3.
>
> Percentage of integrals which generated an exception is 2.19%
>
> Total number of integrals tested is now 106,812
>
> The page above shows the commands and the exceptions generated.
>
> Individual tests Reports are not yet completed, they are building now from
> Latex and it will take may be 3-4 weeks to finish. But the main top level
> report

> <https://12000.org/my_notes/CAS_integration_tests/reports/summer_2024/index.htm>

> is completed now.
>
> Fricas did well. 4th place out of 9 CAS systems with 77.117% solved.
>
> The SQL database is also fully build and contains all the result of all
> tests. Link to the database page to download is here

> <https://12000.org/my_notes/CAS_integration_tests/reports/summer_2024/DATA_BASE/index.htm> the

> database can be used to obtain more information about each test made. Some
> examples
>
> Find number of integrals failed by Fricas with exception
>
> sqlite> select count(*) from main where fricas_grade="F(-2)";
> 2339
>
> Get list of all exceptions by Fricas. The sagemath-Fricas interface is much
> better now than before (I am using sagemath 10.3).
>
> I see very few exceptions due to sagemath interface issues now. (such
> as exp_integral_e)

Thanks for information. The exceptions below look like Sage
errors:

Exception raised: RecursionError >> maximum recursion depth exceeded

Exception raised: RuntimeError >> no explicit roots found

Exception raised: AttributeError >> NoneType object has no attribute replace

Exception raised: AttributeError

and (as you mentioned) a bunch of errors like:

Exception raised: TypeError >> An error occurred when FriCAS evaluated
((x)^(((100)::EXPR INT)))*(exp_integral_e(((1)::EXPR INT),(a)*(x))):
There are no library operations named exp_integral_e Use HyperDoc
Browse or issue

FriCAS does not allows floating point numbers in integrands. This
causes errors like:

Exception raised: TypeError >> An error occurred when FriCAS evaluated
(x)*(exp((x)*(((-0.10000000000000001)::EXPR INT)))): Cannot convert the
value from type Float to Expression(Integer) .

The one below look as borderline case: here "failed" comes from FriCAS,
but fishy type, that is:

SparseUnivariatePolynomial(Expression(Complex(Integer)))

suggests that there is something wrong with "user input", that is
with Sage interface.

--
Waldek Hebisch

[Nasser M. Abbasi]

" riCAS does not allows floating point numbers in integrands. This
causes errors like:

Exception raised: TypeError >> An error occurred when FriCAS evaluated
(x)*(exp((x)*(((-0.10000000000000001)::EXPR INT)))): Cannot convert the
value from type Float to Expression(Integer) ."

I found there is only one such input

sqlite> select count(*) from main where fricas_anti LIKE '%Cannot convert the value from type Float%';
1

It turns out this is how it was in the original Mathematica Rubi input file:

{x/E^(0.1*x), x, 3, -100/E^(x/10) - (10*x)/E^(x/10), 1}

May be Albert Rich had typo when he wrote this one. I will fix it in a future update. I did not

see this myself before.

For the other ones you say they look like interface errors, will look at these as well and see what I can find.

If they are sagemath interface issues, will report then to sagemath,

--Nasser

[Nasser M. Abbasi]

FYI;

I entered few issues at sagemath related to these exception. Here is summary and links to sagemath tracking

sqlite3 cas_integration_tests.db
SQLite version 3.45.0 2024-01-15 17:01:13

sqlite> select count(*) from main where fricas_anti LIKE '%maximum recursion depth exceeded%';
3

sqlite> select command_in_fricas from main where fricas_anti LIKE '%maximum recursion depth exceeded%';
integrate((1-x)^2014*x,x, algorithm="fricas")
integrate((1-x)^2014*x,x, algorithm="fricas")
integrate((1-x)^2020*x,x, algorithm="fricas")

Bug entered at sagemath

https://github.com/sagemath/sage/issues/37812
----------------------------------------------------------
sqlite> select count(*) from main where fricas_anti LIKE '%NoneType object has no attribute replace%';
6


sqlite> select command_in_fricas from main where fricas_anti LIKE '%NoneType object has no attribute replace%';

integrate((x^2+1)^(1/2)/(x^2+(x+(x^2+1)^(1/2))^(1/2)),x, algorithm="fricas")
integrate((x^2+1)^(1/2)/(x^2+(x+(x^2+1)^(1/2))^(1/2)),x, algorithm="fricas")
integrate((x^2-(x^2+1)^(1/2))/(x^2+(x+(x^2+1)^(1/2))^(1/2)),x, algorithm="fricas")
integrate((x^2-(x^2+1)^(1/2))/(x^2+(x+(x^2+1)^(1/2))^(1/2)),x, algorithm="fricas")
integrate((1-(x-(x^2+1)^(1/2))^(1/2))/(x^4-2*x^2*(x^2+1)^(1/2)),x, algorithm="fricas")
integrate(x^4/(1-x*(x^2+1)^(1/2)*(x-(x^2+1)^(1/2))^(1/2)),x, algorithm="fricas")

Bug entered at sagemath

https://github.com/sagemath/sage/issues/37813

-------------------------------------------------------------------

sqlite> select count(*) from main where fricas_anti LIKE '%no explicit roots found%';
16

sqlite> select command_in_fricas from main where fricas_anti LIKE '%no explicit roots found%';
integrate(1/(b*x^7+a),x, algorithm="fricas")
integrate(1/(-b*x^7+a),x, algorithm="fricas")
integrate(x^4/(b^3*x^6+9*a*b^2*x^4+27*a^2*c*x^3+27*a^2*b*x^2+27*a^3),x, algorithm="fricas")
integrate(x^3/(b^3*x^6+9*a*b^2*x^4+27*a^2*c*x^3+27*a^2*b*x^2+27*a^3),x, algorithm="fricas")
integrate(x/(b^3*x^6+9*a*b^2*x^4+27*a^2*c*x^3+27*a^2*b*x^2+27*a^3),x, algorithm="fricas")
integrate(1/(b^3*x^6+9*a*b^2*x^4+27*a^2*c*x^3+27*a^2*b*x^2+27*a^3),x, algorithm="fricas")
integrate(1/(a-b*sin(x)^5),x, algorithm="fricas")
integrate(1/(a+b*sin(x)^5),x, algorithm="fricas")
integrate(1/(a-b*cos(x)^5),x, algorithm="fricas")
integrate(1/(a+b*cos(x)^5),x, algorithm="fricas")
integrate((e*x+d)^2*(a+b*arctan(c*x^3)),x, algorithm="fricas")
integrate((e*x+d)*(a+b*arctan(c*x^3)),x, algorithm="fricas")
integrate(1/(a+b*sinh(x)^5),x, algorithm="fricas")
integrate(1/(a+b*cosh(x)^5),x, algorithm="fricas")
integrate(1/(a-b*cosh(x)^5),x, algorithm="fricas")
integrate((-a*x^5+b)/(b*x+a)^(1/2)/(x^5+a*b),x, algorithm="fricas")


https://github.com/sagemath/sage/issues/37814

--------------------------------------------------------------------------------------

sqlite> select count(*) from main where fricas_anti LIKE '%Exception raised: AttributeError%';
12

sqlite> select command_in_fricas from main where fricas_anti LIKE '%Exception raised: AttributeError%';

integrate((x^2+1)^(1/2)/(x^2+(x+(x^2+1)^(1/2))^(1/2)),x, algorithm="fricas")
integrate((x^2+1)^(1/2)/(x^2+(x+(x^2+1)^(1/2))^(1/2)),x, algorithm="fricas")
integrate(1/(x-(a*x+b)^(1/2)*(c+(a*x+b)^(1/2))^(1/2)),x, algorithm="fricas")
integrate(1/(x-(a*x+b)^(1/2)*(c+(a*x+b)^(1/2))^(1/2)),x, algorithm="fricas")
integrate((x^2-(x^2+1)^(1/2))/(x^2+(x+(x^2+1)^(1/2))^(1/2)),x, algorithm="fricas")
integrate((x^2-(x^2+1)^(1/2))/(x^2+(x+(x^2+1)^(1/2))^(1/2)),x, algorithm="fricas")
integrate(x/(x+(c+(a*x+b)^(1/2))^(1/2)),x, algorithm="fricas")
integrate(x/(x+(c+(a*x+b)^(1/2))^(1/2)),x, algorithm="fricas")
integrate((1-(x-(x^2+1)^(1/2))^(1/2))/(x^4-2*x^2*(x^2+1)^(1/2)),x, algorithm="fricas")
integrate(x^4/(1-x*(x^2+1)^(1/2)*(x-(x^2+1)^(1/2))^(1/2)),x, algorithm="fricas")
integrate(x/(x-(a*x+b)^(1/2)*(c+(a*x+b)^(1/2))^(1/2)),x, algorithm="fricas")
integrate(x/(x-(a*x+b)^(1/2)*(c+(a*x+b)^(1/2))^(1/2)),x, algorithm="fricas")



https://github.com/sagemath/sage/issues/37816


--------------------------------------------------

sqlite> select count(*) from main where fricas_anti LIKE '%SparseUnivariatePolynomial(Expression(Complex(Integer)))%';
2

sqlite> select command_in_fricas from main where fricas_anti LIKE '%SparseUnivariatePolynomial(Expression(Complex(Integer)))%';
integrate(cot(x)/(a+b*sin(x)^3)^(1/2),x, algorithm="fricas")
integrate(tan(x)/(a+b*cos(x)^3)^(1/2),x, algorithm="fricas")

These come from Fricas itself

5) -> integrate(cot(x)/(a+b*sin(x)^3)^(1/2),x)                    

 
   >> Error detected within library code:
   "failed" of mode Union(SparseUnivariatePolynomial(Expression(Complex(Integer))),"failed")

   cannot be coerced to mode SparseUnivariatePolynomial(Expression(Complex(Integer)))


(5) -> integrate(tan(x)/(a+b*cos(x)^3)^(1/2),x)

 
   >> Error detected within library code:
   "failed" of mode Union(SparseUnivariatePolynomial(Expression(Complex(Integer))),"failed")

   cannot be coerced to mode SparseUnivariatePolynomial(Expression(Complex(Integer)))

------------------------------------------------
sqlite> select count(*) from main where fricas_anti LIKE '%exp_integral_e%';
18
sqlite> select command_in_fricas from main where fricas_anti LIKE '%exp_integral_e%';


integrate(x^100*exp_integral_e(1,a*x),x, algorithm="fricas")
integrate(x^2*exp_integral_e(1,a*x),x, algorithm="fricas")
integrate(x*exp_integral_e(1,a*x),x, algorithm="fricas")
integrate(exp_integral_e(1,a*x),x, algorithm="fricas")
integrate(exp_integral_e(1,a*x)/x,x, algorithm="fricas")
integrate(exp_integral_e(1,a*x)/x^2,x, algorithm="fricas")
integrate(exp_integral_e(1,a*x)/x^3,x, algorithm="fricas")
integrate(exp_integral_e(1,a*x)/x^4,x, algorithm="fricas")
integrate((d*x)^m*exp_integral_e(1,b*x),x, algorithm="fricas")
integrate((d*x+c)^3*exp_integral_e(1,b*x+a),x, algorithm="fricas")
integrate((d*x+c)^2*exp_integral_e(1,b*x+a),x, algorithm="fricas")
integrate((d*x+c)*exp_integral_e(1,b*x+a),x, algorithm="fricas")
integrate(exp_integral_e(1,b*x+a),x, algorithm="fricas")
integrate(exp_integral_e(1,b*x+a)/(d*x+c),x, algorithm="fricas")
integrate(exp_integral_e(1,b*x+a)/(d*x+c)^2,x, algorithm="fricas")
integrate(exp_integral_e(1,b*x+a)/(d*x+c)^3,x, algorithm="fricas")
integrate(exp_integral_e(1,b*x+a)/(d*x+c)^4,x, algorithm="fricas")
integrate((d*x+c)^m*exp_integral_e(1,b*x+a),x, algorithm="fricas")

This error happens since Fricas has no exp_integral_e. This needs to be
translated correctly to Fricas. See discussion at

https://groups.google.com/g/fricas-devel/c/A9f4li7gq84


Added bug report

https://github.com/sagemath/sage/issues/37817

--Nasser

[Martin R]

I started to look into one of the problems (https://github.com/sagemath/sage/issues/37813):

res := integrate((x^2+1)^(1/2)/(x^2+(x+(x^2+1)^(1/2))^(1/2)), x);

works nicely, but converting to InputForm (which I use to do the translation to sage) fails.  Is there a good reason for that - i.e., is this a bug, or just a problem with memory?

Best wishes,

Martin

(2) -> inform := res :: INFORM

Heap exhausted during garbage collection: 0 bytes available, 16 requested.
Gen  Boxed   Code    Raw  LgBox LgCode  LgRaw  Pin       Alloc     Waste        Trig      WP GCs Mem-age
 1   16426      0      0      0      0      0   12   269054560     69024   172047802       0   1  1.3993
 2   46668      0    145      0      0      0   68   766811856    172336     2000000   30615   0  0.9840
 3       0      0      0      0      0      0    0           0         0     2000000       0   0  0.0000
 4       0      0      0      0      0      0    0           0         0     2000000       0   0  0.0000
 5       0      0      0      0      0      0    0           0         0     2000000       0   0  0.0000
 6    1430      6    631    172      0     58    0    36868464    765584     2000000    2156   0  0.0000
           Total bytes allocated    =    1072734880
           Dynamic-space-size bytes =    1073741824
GC control variables:
   *GC-INHIBIT* = true
   *GC-PENDING* = true
   *STOP-FOR-GC-PENDING* = false
fatal error encountered in SBCL pid 829857 tid 829857:
Heap exhausted, game over.

Welcome to LDB, a low-level debugger for the Lisp runtime environment.

[Waldek Hebisch]

On Thu, Apr 18, 2024 at 08:45:53AM -0700, 'Martin R' via FriCAS - computer algebra system wrote:
> I started to look into one of the problems
> (https://github.com/sagemath/sage/issues/37813):
>
> res := integrate((x^2+1)^(1/2)/(x^2+(x+(x^2+1)^(1/2))^(1/2)), x);
>
> works nicely, but converting to InputForm (which I use to do the
> translation to sage) fails. Is there a good reason for that - i.e., is
> this a bug, or just a problem with memory?
>
> Best wishes,
>
> Martin
>
> (2) -> inform := res :: INFORM
>
> Heap exhausted during garbage collection: 0 bytes available, 16 requested.

<snip>

> Total bytes allocated = 1072734880
> Dynamic-space-size bytes = 1073741824

For me it works. The result is big for humans, but should be no
problem for modern computers. I am using FriCAS trunk build
using sbcl-1.2.4 (currently with 3Gb limit). Tried also version
with 2Gb limit and sbcl-2.2.9 with 1Gb limit. Note
I did:

res := integrate((x^2+1)^(1/2)/(x^2+(x+(x^2+1)^(1/2))^(1/2)), x);
ii := res::InputForm;

that is I am _not_ printing resulting InputForm. But I also
separately printed the InputForm, it works, just is slow when
printing to terminal and useless because the result is much
bigger than terminal scrollback buffer.

--
Waldek Hebisch

[Martin R]

Hi Waldek!

Thanks for the rapid answer!

I have:

)lisp (lisp-implementation-version)

2.1.11.debian
)lisp (sb-ext:dynamic-space-size)

1073741824

)version

FriCAS 2022-07-16 compiled at Fr 12 Aug 2022 15:17:27 CEST

I'm currently compiling the ECL version.

Unfortunately, because of the MacOS problem (https://github.com/sagemath/sage/pull/37041) most sage users won't use the newest FriCAS.  So I'll first check whether that makes a difference.

Martin

[Nasser M. Abbasi]

These are useful lisp commands, I did not know about them. This is what I get for my Fricas installation

                      FriCAS Computer Algebra System
                Version: FriCAS 1.3.10 built with sbcl 2.3.11
                 Timestamp: Wed Jan 10 09:37:52 PM CST 2024 

(1) -> )lisp (lisp-implementation-version)
Value = "2.3.11"
(1) -> )lisp (sb-ext:dynamic-space-size)

Value = 4294967296

I am also running Fricas and sagemath on VBox under windows 10. The OS is Linux Manjaro

>fricas --version
FriCAS 1.3.10
based on sbcl 2.3.11
>sage --version
SageMath version 10.3, Release Date: 2024-03-19
>

[Martin R]

I have now FriCAS with ECL, but I now realize that I am doing very silly things in the interface between FriCAS to sage:

* I do an unnecessary unparse of the InputForm (this runs forever on ECL, and crashes sbcl)

* I throw the result away

* I convert the InputForm into a string using a customized printer

* I parse the result

Oh dear, what did I do!

I guess that I was scared of creating a very long history in the FriCAS process if I transmit the InputForm atom by atom.  I guess I should cook up a simple protocol to transmit an ordered tree, maybe as a Stream.

Martin

[Martin R]

OK, I think I have to give up.  The InputForm consists of 23 964 324 atoms.  I guess that there is no sensible way to transmit this, right?

Martin

[Dima Pasechnik]

On 18 April 2024 21:51:34 BST, 'Martin R' via FriCAS - computer algebra system <fricas...@googlegroups.com> wrote:
>OK, I think I have to give up. The InputForm consists of 23 964 324
>atoms. I guess that there is no sensible way to transmit this, right?

In-memory - just how Maxima library interface is operating.
No need for pexpect interface then.
Create a FAS module loadable into libecl,
and, well, you have a huge increase in speed of the interface.

[Waldek Hebisch]

On Thu, Apr 18, 2024 at 01:51:34PM -0700, 'Martin R' via FriCAS - computer algebra system wrote:
> OK, I think I have to give up. The InputForm consists of 23 964 324
> atoms. I guess that there is no sensible way to transmit this, right?

Well, compress it. FriCAS internal representation is designed
to share subexpressions. Things like %%Q0 take place for
definition and than every use just references definition.
Convertion to InputForm essentially forces expansion and
"unsharing" of subexpression.

In principle we could restore sharing using "lambda expression",
that is apply anonymous function to arguments, like:

(x, y) +-> x^2 + x*y + y^3)(x+1, sqrt(x - 1) + x)

2 +-----+ 3 2
(1) (3 x + 2 x)\|x - 1 + x + 5 x + 1

This is small expression, so anonymous function just adds bulk,
but for bigger expressions such form could be much smaller,
because arguments are just described once instead of repeating
description multiple times. In principle convertion to
InputForm could use this, but clearly that would require
extra code in Expression and some logic to decide when the
result is smaller than more strightforward approach. And
InputForm is supposed to rebuild back given value, so there
would be need for testing and in principle we may need to
fix some interpreter bugs. And there is significant potential
for breakage in sbcl: by default FriCAS functions are compiled
and sbcl compiler tends to blow up on largish functions which
contain no conditionals (I am not sure if lack of conditionals
is essential, just my guess).

As a safer alternative we could implement 'where' in intepreter
(there is no function involved, so no compilation to blow up).

Or you could implement a "compress" pass in your interface
that walks the tree and identifies all shared subtrees.
I mean something like 'minimalise' in 'src/interp/compress.boot'.

In somewhat different spirit, interface should allow transmiting
large values when needed. And 23 964 324 atoms means few hundreds
megabytes. Modern machines can move it from place to place in tens
of milliseconds. Pointer walking takes more time, but should be
managable, on fast machine sbcl can produce this InputForm in about
50 milliseconds. Of course, the silly 1Gb limit in sbcl means that
default version can not do it. With enough memory convertion of
input form to string (unparse) takes 9.84s and resulting
string has 67Mb. And of course InputForm was not designed
for heavy use..

--
Waldek Hebisch

[Waldek Hebisch]

On Thu, Apr 18, 2024 at 01:51:34PM -0700, 'Martin R' via FriCAS - computer algebra system wrote:
> OK, I think I have to give up. The InputForm consists of 23 964 324
> atoms. I guess that there is no sensible way to transmit this, right?

Some more data. Using ECL on fast machine with plenty of memory:

(1) -> )set messages time on
(1) -> res := integrate((x^2+1)^(1/2)/(x^2+(x+(x^2+1)^(1/2))^(1/2)), x);

Type: Union(Expression(Integer),...)
Time: 0.02 (IN) + 71.12 (EV) + 0.48 (OT) = 71.62 sec
(2) -> ii := res::InputForm;

Type: InputForm
Time: 0.44 (IN) = 0.44 sec
(3) -> ss := unparse(ii);

Type: String
Time: 149.74 (EV) = 149.74 sec
(4) -> #ss

(4) 67845674
Type: PositiveInteger
Time: 0 sec
(5) -> ss2 := copy(ss);

Type: String
Time: 0.33 (EV) = 0.33 sec

'unparse' is slow, but this seems to be mainly general ECL slowness
(on the same machine 'unparse' in sbcl-compiled FriCAS needs 9.84s).
As you see copy of resulting string takes 0.33, so this is realtively
easy to transmit. _Parsing_ this is harder, FriCAS parser in sbcl
compiled FriCAS needs 55.53s, up to now ECL used more than 14 CPU
hours (probably about 1.5 hours in real time) and did not finish.

BTW: FriCAS parser is rather slow, better parser should be
able to parse this 10-20 times faster.
--
Waldek Hebisch

[Martin R]

I don't know how to do it.  Note that this should really work with any lisp implementation to make sense, because some people (eg., me) will have fricas installed with sbcl (because this is fastest), and the interface shouldn't insist on an ECL installation.

Do you know how to do it?

Martin

[dim...@gmail.com]

On Fri, Apr 19, 2024 at 01:26:27AM -0700, 'Martin R' via FriCAS - computer algebra system wrote:
> I don't know how to do it. Note that this should really work with any lisp
> implementation to make sense, because some people (eg., me) will have
> fricas installed with sbcl (because this is fastest), and the interface
> shouldn't insist on an ECL installation.
>
> Do you know how to do it?

There is https://github.com/quil-lang/sbcl-librarian
which relies on a recent https://www.sbcl.org/manual/#Calling-Lisp-From-C

If SBCL is the preferred Lisp then this route can be pursued -
I don't however know enough about FriCAS to see how to use
sbcl-librarian to call FriCAS functions from Python.
(There doesn't even seem to be an example on calling FriCAS from its
underlyng Lisp available anywhere - from that it should be doable)

I don't think it can be totally Lisp-agnostic:
SBCL does not allow embedding in the way ECL does (It's Embeddable
Common Lisp for a reason...)

Dima

> --
> You received this message because you are subscribed to the Google Groups "FriCAS - computer algebra system" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to fricas-devel...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/fricas-devel/49a360d1-f38a-4477-bd90-43be950ecd16n%40googlegroups.com.

[Martin R]

Hi Nasser,

could you try the branch at https://github.com/sagemath/sage/pull/37836?  It should give significant performance gains for your testsuite.

Best wishes,

Martin

[Nasser M. Abbasi]

Hello Matrin,

Sure, will try your version on a large integration test file and see if it is faster than the what I had with 10.3 sage.

The problem is that I know nothing about github.  I just know how to login to github and enter bug reports on the CAS systems I use at the issues page. That is all.

When I build sagemath, I download the sagemath latest zip file from https://mirrors.mit.edu/sage/devel/index.html 

I have no idea how to do what you said "try the branch at" . I do not know what this means. 

I see on the link you gave no zip file for sagemath to download.

If you could give me a link to sagemath zip file with your fixes in it, will be happy to download it, build it like I did for 10.3 and try it.

One day I want to take course at school how to use github if I can find one.

--Nasser

[Dima Pasechnik]

On Sat, Apr 20, 2024 at 7:31 PM 'Nasser M. Abbasi' via FriCAS - computer algebra system <fricas...@googlegroups.com> wrote:

Hello Matrin,

Sure, will try your version on a large integration test file and see if it is faster than the what I had with 10.3 sage.

The problem is that I know nothing about github.  I just know how to login to github and enter bug reports on the CAS systems I use at the issues page. That is all.

When I build sagemath, I download the sagemath latest zip file from https://mirrors.mit.edu/sage/devel/index.html 

I have no idea how to do what you said "try the branch at" . I do not know what this means. 

I see on the link you gave no zip file for sagemath to download.

If you could give me a link to sagemath zip file with your fixes in it, will be happy to download it, build it like I did for 10.3 and try it.

Do you know how to apply patches? If so, you just need to apply the patch corresponding to the PR Martin mentioned. It's here:

https://patch-diff.githubusercontent.com/raw/sagemath/sage/pull/37836.diff

 

One day I want to take course at school how to use github if I can find one.

It doesn't have much to do with GitHub, it's git that you need to know a bit about.

GitHub is just a hosting platform for git repositories.

HTH

Dima

To view this discussion on the web visit https://groups.google.com/d/msgid/fricas-devel/d27c981d-32e7-4743-b1af-00dabdb6384en%40googlegroups.com.

[Nasser M. Abbasi]

"Do you know how to apply patches? If so, you just need to apply the patch corresponding to the PR Martin mentioned. It's here:

https://patch-diff.githubusercontent.com/raw/sagemath/sage/pull/37836.diff" 

I did the following

----------------------------------------------

pwd
    /home/me/TMP/sage-10.3/src/sage/interfaces

wget https://patch-diff.githubusercontent.com/raw/sagemath/sage/pull/37836.diff

patch fricas.py 37836.diff

patching file fricas.py
Hunk #1 succeeded at 802 (offset -1 lines).
Hunk #2 succeeded at 1715 (offset -1 lines).
Hunk #3 succeeded at 1993 (offset -1 lines).
Hunk #4 succeeded at 2018 (offset -1 lines).
Hunk #5 succeeded at 2049 (offset -1 lines).
Hunk #6 succeeded at 2066 (offset -1 lines).

---------------------------------------

Re-run 2 test files., One had 700 integrals and one had 3,000 integrals.

The one with 3,000 integrals had improved mean time from 0.09 seconds per integral

to 0.07 seconds per integral. The one with 700 integrals improved the time from 0.23 seconds to

0.21 seconds. 

Will wait for official sage 10.4 to be out and may be next version of Fricas and Maxima

to do a full rebuild as it takes me 2-3 months to redo everything. But it seems the patch

did speed things from this limited test.

--Nasser

[Martin R]

Thank you so much for trying!  I admit that I'm slightly disappointed by this improvement, because it really was a very silly mistake.

Anyway, for things like integrate((1-x)^1300*x, x, algorithm="fricas") with large output, on my computer, it takes

* 3.5 seconds to transfer the string produced in FriCAS to sage.

* 4.2 seconds in total.

Note that the 3.5 seconds do *not* include producing the string in FriCAS.

Martin

[Dima Pasechnik]

On Sat, Apr 20, 2024 at 7:45 PM Dima Pasechnik <dim...@gmail.com> wrote:

On Sat, Apr 20, 2024 at 7:31 PM 'Nasser M. Abbasi' via FriCAS - computer algebra system <fricas...@googlegroups.com> wrote:

Hello Matrin,

Sure, will try your version on a large integration test file and see if it is faster than the what I had with 10.3 sage.

The problem is that I know nothing about github.  I just know how to login to github and enter bug reports on the CAS systems I use at the issues page. That is all.

When I build sagemath, I download the sagemath latest zip file from https://mirrors.mit.edu/sage/devel/index.html 

I have no idea how to do what you said "try the branch at" . I do not know what this means. 

I see on the link you gave no zip file for sagemath to download.

If you could give me a link to sagemath zip file with your fixes in it, will be happy to download it, build it like I did for 10.3 and try it.

Do you know how to apply patches? If so, you just need to apply the patch corresponding to the PR Martin mentioned. It's here:

https://patch-diff.githubusercontent.com/raw/sagemath/sage/pull/37836.diff

Just to mention that this URL is equivalent to

   https://github.com/sagemath/sage/pull/37836.diff

which tells one how to get the patch from a GitHub PR (just add the magical .diff)

There is also a variation (.patch) which, I think, produces a series of diffs, one per each commit.

[Grégory Vanuxem]

Le dim. 21 avr. 2024 à 13:07, Dima Pasechnik <dim...@gmail.com> a écrit :

[SNIP]

Do you know how to apply patches? If so, you just need to apply the patch corresponding to the PR Martin mentioned. It's here:

https://patch-diff.githubusercontent.com/raw/sagemath/sage/pull/37836.diff

Just to mention that this URL is equivalent to

   https://github.com/sagemath/sage/pull/37836.diff

which tells one how to get the patch from a GitHub PR (just add the magical .diff)

There is also a variation (.patch) which, I think, produces a series of diffs, one per each commit.

Thanks a lot ! I was looking for that ;-) There are sometimes numerous commits also in the FriCAS GitHub repository. It can be adapted for commits I guess. Otherwise, I have not found a simple and easy way. 

- Greg