the issue of %iint generated by Fricas

[Nasser M. Abbasi]

Hello. I use sagemath to call fricas integrate as part of build the independent CAS integration tests.

I am not running the tests using sagemath 9.2 against Fricas 1.3.6

There are antiderivatives generated by Fricas which uses %iint.

This cause a problem when converted to Latex for the report.

Here is a bug report on this against sagemath 

https://ask.sagemath.org/question/48409/possible-invalid-latex-translation-from-fricas-result/

https://trac.sagemath.org/ticket/28630  

But not fixed yet and do not know when it will be fixed.

Is it possible that fricas not use %iint as that causes problem converting to Latex. 

Here is an example

sage: x,a,b,n=var('x a b n')
sage: integrate(x^2*(a+b*log(c*x^n))*polylog(3,e*x),x, algorithm="fricas")
-1/972*(4*(4*b*n - 3*a)*x^3*e^3 + 9*(3*b*n - 2*a)*x^2*e^2 + 36*(2*b*n - a)*x*e + 36*(3*b*n*x^3*e^3*log(x) + 3*b*x^3*e^3*log(c) - (2*b*n - 3*a)*x^3*e^3 - b*n)*%iint(x, -log(-x*e + 1)/x) - 36*((b*n - a)*x^3*e^3 - b*n + a)*log(-x*e + 1) - 6*(2*b*x^3*e^3 + 3*b*x^2*e^2 + 6*b*x*e - 6*(b*x^3*e^3 - b)*log(-x*e + 1))*log(c) - 6*(2*b*n*x^3*e^3 + 3*b*n*x^2*e^2 + 6*b*n*x*e - 6*(b*n*x^3*e^3 - b*n)*log(-x*e + 1))*log(x) - 108*(3*b*n*x^3*e^3*log(x) + 3*b*x^3*e^3*log(c) - (b*n - 3*a)*x^3*e^3)*polylog(3, x*e))*e^(-3)

After converting to Latex it gives (using sagemath latex command)

-\frac{4 \, {\left(4 \, b e^{3} n - 3 \, a e^{3}\right)} x^{3} + 9 \, {\left(3 \, b e^{2} n - 2 \, a e^{2}\right)} x^{2} + 36 \, {\left(2 \, b e n - a e\right)} x + 36 \, {\left(3 \, b e^{3} n x^{3} \log\left(x\right) + 3 \, b e^{3} x^{3} \log\left(c\right) - {\left(2 \, b e^{3} n - 3 \, a e^{3}\right)} x^{3} - b n\right)} {\rm %iint}\left(e, x, -\frac{\log\left(-e x + 1\right)}{e}, -\frac{\log\left(-e x + 1\right)}{x}\right) - 36 \, {\left({\left(b e^{3} n - a e^{3}\right)} x^{3} - b n + a\right)} \log\left(-e x + 1\right) - 6 \, {\left(2 \, b e^{3} x^{3} + 3 \, b e^{2} x^{2} + 6 \, b e x - 6 \, {\left(b e^{3} x^{3} - b\right)} \log\left(-e x + 1\right)\right)} \log\left(c\right) - 6 \, {\left(2 \, b e^{3} n x^{3} + 3 \, b e^{2} n x^{2} + 6 \, b e n x - 6 \, {\left(b e^{3} n x^{3} - b n\right)} \log\left(-e x + 1\right)\right)} \log\left(x\right) - 108 \, {\left(3 \, b e^{3} n x^{3} \log\left(x\right) + 3 \, b e^{3} x^{3} \log\left(c\right) - {\left(b e^{3} n - 3 \, a e^{3}\right)} x^{3}\right)} {\rm polylog}\left(3, e x\right)}{972 \, e^{3}}

And the above do not compile due to %iint, as % is treated as comment and all the latex after that is lost giving latex error.

Is there a way for FriCAS not to use %  in the output it generates?  Each time I have to edit the latex by hand and add \% everywhere they show

May be this is not possible, But I thought to just ask.

Thanks

--Nasser

[Nasser M. Abbasi]

opps, sorry, typo

                   I am not running the tests using sagemath 9.2 against Fricas 1.3.6  

Meant to say:

                   I am NOW running the tests using sagemath 9.2 against Fricas 1.3.6    

(Earlier build used sagemath 9.0) 

--Nasser

[Ralf Hemmecke]

Waldek,

you have introduced %iint here:

https://github.com/fricas/fricas/commit/3b129a3ba619d8a8825cce132170d75a236b3e57

but probably never meant it to be visible as an output form.

Ralf

[Waldek Hebisch]

On Fri, Jan 08, 2021 at 01:11:41PM -0800, 'Nasser M. Abbasi' via FriCAS - computer algebra system wrote:
>
> There are antiderivatives generated by Fricas which uses %iint.
>

<snip>

>
> Is it possible that fricas not use %iint as that causes problem converting
> to Latex.
>
> Here is an example
>
> sage: x,a,b,n=var('x a b n')
> sage: integrate(x^2*(a+b*log(c*x^n))*polylog(3,e*x),x, algorithm="fricas")
> -1/972*(4*(4*b*n - 3*a)*x^3*e^3 + 9*(3*b*n - 2*a)*x^2*e^2 + 36*(2*b*n -
> a)*x*e + 36*(3*b*n*x^3*e^3*log(x) + 3*b*x^3*e^3*log(c) - (2*b*n -
> 3*a)*x^3*e^3 - b*n)*%iint(x, -log(-x*e + 1)/x) - 36*((b*n - a)*x^3*e^3 -
> b*n + a)*log(-x*e + 1) - 6*(2*b*x^3*e^3 + 3*b*x^2*e^2 + 6*b*x*e -
> 6*(b*x^3*e^3 - b)*log(-x*e + 1))*log(c) - 6*(2*b*n*x^3*e^3 + 3*b*n*x^2*e^2
> + 6*b*n*x*e - 6*(b*n*x^3*e^3 - b*n)*log(-x*e + 1))*log(x) -
> 108*(3*b*n*x^3*e^3*log(x) + 3*b*x^3*e^3*log(c) - (b*n -
> 3*a)*x^3*e^3)*polylog(3, x*e))*e^(-3)

<snip>

> Is there a way for FriCAS not to use % in the output it generates?

This is a bug. '%iint' should be internal only and should not
appear in the output of integrator.

BTW. This issue was already reported by Martin Rubey.

--
Waldek Hebisch

[oldk1331]

Hi, my thoughts are:

1. Maybe Sage should be able to automatically escape '%' in its 'latex'
command.

When using FriCAS to generate latex, it returns '\%' directly.

2. As Waldek said, '%iint" should be used only internally. But the
following workaround prints '%iint' as 'iint':

diff --git a/src/algebra/liouv.spad b/src/algebra/liouv.spad
index 420845b9..c070ca28 100644
--- a/src/algebra/liouv.spad
+++ b/src/algebra/liouv.spad
@@ -221,6 +221,7 @@
derivative(opli2, (z1 : F) : F +-> log(z1) / (1 - z1))
derivative(opfis, (z1 : F) : F +-> sin(pi()*z1^2/(2::F)))
derivative(opfic, (z1 : F) : F +-> cos(pi()*z1^2/(2::F)))
+ display(opiint, (z1 : List O) : O +-> prefix('iint::O, z1))
setProperty(opint, SPECIALEQUAL, eqint@((K, K) -> Boolean) pretend
None)
setProperty(opint, SPECIALDIFF, dvint@((List F, SE) -> F) pretend None)
setProperty(opiint, SPECIALDIFF, dviint@((List F, SE) -> F) pretend
None)


On 1/9/21 5:11 AM, 'Nasser M. Abbasi' via FriCAS - computer algebra

[Ralf Hemmecke]

Hi Qian,

> 1. Maybe Sage should be able to automatically escape '%' in its 'latex'
> command.

I don't think that is an option if %iint was not meant to be visible to
an enduser.

> 2. As Waldek said, '%iint" should be used only internally. But the
> following workaround prints '%iint' as 'iint':

What would that help? Whether it prints as %iint or iint, I still would
not know how to interpret that FriCAS is returning. What does '%iint'
stand for?

Ralf

[Dima Pasechnik]

we can fix Sage interface only if we know what %iint stands for. Is it
a kind of polylogarithm?

> --
> You received this message because you are subscribed to the Google Groups "FriCAS - computer algebra system" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to fricas-devel...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/fricas-devel/2266c422-5328-3c65-44f1-437fe7b22bca%40gmail.com.

[oldk1331]

On Sat, Jan 9, 2021, 7:24 PM Ralf Hemmecke <ra...@hemmecke.org> wrote:

Hi Qian,

> 1. Maybe Sage should be able to automatically escape '%' in its 'latex'
> command.

I don't think that is an option if %iint was not meant to be visible to
an enduser.

I mean it's a Sage bug that it can't translate for example variable 'a%b' to latex output.

> 2. As Waldek said, '%iint" should be used only internally.  But the
> following workaround prints '%iint' as 'iint':

What would that help? Whether it prints as %iint or iint, I still would
not know how to interpret that FriCAS is returning. What does '%iint'
stand for?

Ralf

It solves Nasser's problem at hand...

[Waldek Hebisch]

On Sat, Jan 09, 2021 at 12:24:47PM +0100, Ralf Hemmecke wrote:
> > 2. As Waldek said, '%iint" should be used only internally. But the
> > following workaround prints '%iint' as 'iint':
>
> What would that help? Whether it prints as %iint or iint, I still would
> not know how to interpret that FriCAS is returning. What does '%iint'
> stand for?

Indefinite integral (== general Liouvillian function). Main reason
of using '%' is to avoid clashes with user-defined things.
Uninterned symbols would be better to avoid clashes, but would
complicate debugging...

--
Waldek Hebisch

[Ralf Hemmecke]

Hi Waldek,

The % sign is only a minor problem.
But what is the meaning of

log(- e x + 1) log(- e x + 1)
%iint(e,x,- --------------,- --------------)
e x

? It would be good to know (and best specified in the code) even if it
is not intended to be visible to an enduser.

Ralf

[oldk1331]

On 1/9/21 8:24 PM, Ralf Hemmecke wrote:
> But what is the meaning of
>
> log(- e x + 1) log(- e x + 1)
> %iint(e,x,- --------------,- --------------)
> e x
>
> ? It would be good to know (and best specified in the code) even if it
> is not intended to be visible to an enduser.
>
> Ralf
>

Hi Ralf,

Let the value of %iint(v1,v2,...,vn,f1,f2,...,fn) be f,
then we have

D(f,v1)=f1
D(f,v2)=f2
...
D(f,vn)=fn

This can be seen from 'dviint' from liouv.spad.

If you trace 'postSubst' in 'integrat.spad', you can find
that these '%iint' are nested and can be solved recursively.

I already have some code to solve
integrate(polylog(3,x),x)
and
integrate(x^2*(a+b*log(c*x^n))*polylog(3,e*x),x)
But missing other corner cases which I don't know if we'll encounter later.

I'll post my code when I polish it more.

- Qian

[Waldek Hebisch]

To specify Liouvillian function essentially is the same as specifying
derivative. General Liouvillian function may be multivariate
and for technical reasons it is convenient to treat all
Liouvillian functions as multivariate, for example 'Ei(x*y)'
is treated as finction of two variables: 'x' and 'y'.
Transcenedntal constants are treated as variables, so
above means that we have two variables: 'e' and 'x'.
The other two argument to '%iint' give corresponding derivatives.

So, oldk is right. I would like to stress that there is
nothing more: the rest like nesting is handled by general
machinery and in principle all Liouvillian functions are
legal. This means that all derivatives (as long as they
satisfy consistency condition for cross derivatives)
are legal.

BTW1: The is a problem with current code. I wrote "essentially"
above because there is possible constant (constant of integration).
When dealing with single Liouvillian function constant of
integration may be ignored. Current code assumes that
constant of integration is always zero. This problem
is really independent of representation of Liouvillian
function that is of meaning of '%iint'.

BTW2: Such details as internal representation of Liouvillian
functions are normally left undocumented, because
documentation encourages users to depend on such details
and they are subject to change. In fact, '%iint' may
be gone in few years (or even months if you are optimistic):
ATM is seems that different representation of functions
in integrator would work better. By "different
representation" I mean different than Expression.

--
Waldek Hebisch

[Ralf Hemmecke]

>> It would be good to know (and best specified in the code) even if
>> it is not intended to be visible to an enduser.

> BTW2: Such details as internal representation of Liouvillian
> functions are normally left undocumented, because documentation
> encourages users to depend on such details and they are subject to
> change.

I usually distinguish between documentation for users (that's
essentially what should go into the ++ docstrings) and for programmers.
Both are important and useful. Of course, everything that is not in
explicitly part of the API (i.e. the ++ docstring) is subject to change
any time.

> In fact, '%iint' may be gone in few years (or even months if you are
> optimistic):

I have certainly nothing against removing %iint from FriCAS users, but
currently, without any documentation, it basically says that FriCAS
cannot integrate that expression.

So as a short term fix we have two options:
(1) before returning a result, FriCAS checks whether it contains %iint
and if yes, then returns "not integrable" in elementary functions.
(2) Add a "where" expression to the result that says what the %iint
expression stands for or (until %iint disappears in a few months)
explain in in ++ docstrings of "integrate" (with a note that %iint
might be gone in the future).

I somewhat would prefer (2). But we should do something about %iint now.

Ralf

[oldk1331]

The patch is as following or see the attachment.

This code only solves the case like

%iint(x,c1*%iint(c2*%iint(c3*%iint(...)/f3(x))/f2(x))/f1(x))

Which is the case we encountered.

I'm not aware about other integrals it can't handle, yet.

- Qian

diff --git a/src/algebra/integrat.spad b/src/algebra/integrat.spad
index bfcc8e0a..7d5e68d8 100644
--- a/src/algebra/integrat.spad
+++ b/src/algebra/integrat.spad
@@ -245,6 +245,33 @@
not(member?(k, (tf := tower f))) then
k := find_int(k, tf)
f := eval(f, [k], [v])
+ -- liint is a list of untranslated '%iint kernels. If they are
+ -- parts of ekers, then we may solve them by recursion.
+ liint : List K := [k for k in otf | is?(operator k, '%iint) and
not member?(k, ekers)]
+ for k in reverse liint repeat
+ pos := position(ek +-> member?(k, tower(ek::F)), ekers)
+ if zero? pos then
+ error "integrate: untranslated %iint kernels"
+ ek := ekers.pos
+ ev := evals.pos
+ repeat
+ dek := D(ek::F, x)
+ -- assert the numerator of dek is a single kernel of '%iint
+ numKers := kernels numerator dek
+ if # numKers = 1 and is?(operator first numKers, '%iint) then
+ dev := D(ev, x)
+ coeff := 1/(first coefficients numer dek)::F
+ v := coeff * dev * denominator dek
+ if k::F = coeff * numerator dek then
+ f := eval(f, [k], [v])
+ break
+ else
+ ek := first numKers
+ ev := v
+ if not member?(k, tower(ek::F)) then break
+ else
+ -- print("currently can't handle this '%iint
kernel"::OutputForm)
+ break
if not(empty? ltan) then
ltemp := [tan2temp k for k in ltan]
f := eval(f, ltan, [k::F for k in ltemp])

[Waldek Hebisch]

I think both proposals are wrong. Namely, they try to hide the
bug. Hiding bugs makes sense only if bug is very disruptive
and very hard to fix. Neither is true in this case.
Otherwise hiding bugs is counterproductive: you want bugs
to be visible so that they can be found and fixed.

Variation of (1), that is signalinig internal error when
unexpected '%iint' survives 'postSubst' makes some sense.
But it adds little value (this whole discussion started
because Sage signals error) and there is some subtlety.
Namely integrator may be called from some other routine.
It is possible that caller introduces '%iint', such
'%iint' should survive integration and routine that
introduced it is supposed to remove it from final
result.

--
Waldek Hebisch

[Waldek Hebisch]

On Sun, Jan 10, 2021 at 08:22:25PM +0800, oldk1331 wrote:
> The patch is as following or see the attachment.
>
> This code only solves the case like
>
> %iint(x,c1*%iint(c2*%iint(c3*%iint(...)/f3(x))/f2(x))/f1(x))
>
> Which is the case we encountered.
>
> I'm not aware about other integrals it can't handle, yet.

First general comment: idea behind 'integrat.spad' is rather
simple: build isomorphism from input to differential field
expected by integrator. Isomorphism includes inverse mapping.
This simple idea is spoiled but some difficulties. First,
input may be written in redundant way, so we get map back
but it is not inverse on datastructure level. Second,
'integrat.spad' tries to do its work as composition of
helper functions. Unfortunately, composition introduces
troubles. At core of current bug is fact that single
kernel like 'polylog(3, x)' generates differential
field of transcendence degree 3 over Q(x), so needs
3 kernels. One of kernels is 'log' which causes no
trouble. But 'realLiovillian' produces two '%iint'-s
while there is only single 'polylog(3, x)' in the input.

Proper solution would note that input differential field
contains 'polylog(2, x)'. It is problematic to do this
in clean way, as we should simultaneously build mapping
in one direction and the inverse. However, the attached
patch is doing that in moderately hacky way.

AFAICS your patch tries to cure symptoms. It probably
would work, but I think it is better to get closer
to source of the problem.

BTW: 'integrat.spad' currently contains a pile of hacks,
for that reason alone I would like to restructure it.
There are also problems that I do not know how to fix
by simple hacks. In this case I thought that fix would
be more complicated, but writing explanation above
I realised that relatively simple hack may work.

--
Waldek Hebisch

[Ralf Hemmecke]

Waldek,

your commit

commit 240e2641928fab6f3bd2d638c5d5c437a8f68d7a
Author: Waldek Hebisch <heb...@math.uni.wroc.pl>
Date:   Thu Jan 21 21:09:35 2021 +0100

    Better handle '%iint'

cures the initial issue, but does it also prevent %iint from ever appearing in user output?

Ralf

[Waldek Hebisch]

Well, I fixed one bug. It is unrealistic to expect that single
fix will fix all possible bugs involving %iint. As I wrote,
when there is a bug we want to see it (to be able to fix it).

To say this differently: when you have data structure like
a stack than pushes and pops should be balanced. Fixing
one issue with lack of balance does not mean the problem
will not appear in different place. Better code structure
(possible after rewrite) can signifcantly reduce risk,
but ATM eliminating all bugs is unrealistic.

--
Waldek Hebisch

[Ralf Hemmecke]

I was not asking whether you removed all bugs from FriCAS, but rather
whether you intended to fix the "%iint in output" issue with this commit
and believe that you succeeded or whether you are *sure* that you left
some cases.

It is about conveying some information with the commit message. "Better"
tells almost nothing.

Anyway, I am happy that you fixed at least this one issue. Thanks.

Ralf