communication between fricas and sage

[Martin R]

Dear all!

given the continued interest in some features of fricas from some sage users, I had a look at the fricas interface in sage, and I'm thinking of improving it.

The main question is: how can sage receive the output from fricas?

currently, sage essentially sends "unparse(result:InputForm)" to fricas, and then parses the output: it strips away the prompt and the step number and it looks for the type information, all using regular expressions.

a first improvement I was thinking of is to use the ioHook to separate the algebra output from the remaining information.  however, at least the step number appearing just before the algebra output is currently not decorated.

is there a better way to get the results of computations - and also to send instructions to fricas?

I'm afraid, python cannot call lisp directly, can it?

All the best and thanks for including the fix so quickly,

Martin

[Bill Page]

On 11 August 2016 at 20:31, 'Martin R' via FriCAS - computer algebra

system <fricas...@googlegroups.com> wrote:
> Dear all!
>
> given the continued interest in some features of fricas from some sage
> users, I had a look at the fricas interface in sage, and I'm thinking of
> improving it.
>

Great. I would be very happy to see a better Sage interface for FriCAS
and would be willing to help with such an effort.

> The main question is: how can sage receive the output from fricas?
>
> currently, sage essentially sends "unparse(result:InputForm)" to fricas, and
> then parses the output: it strips away the prompt and the step number and it
> looks for the type information, all using regular expressions.
>

If I recall correctly there is also a peculiar "synchronization"
procedure that tries to overcome synchronization problems that were
probably due to the old version of pexpect that was in use in Sage
until a few releases ago. One side-effect of this is that it is not
convenient to use the %%() history function in the Sage interface. To
the best of my knowledge this procedure can now be eliminated.

> a first improvement I was thinking of is to use the ioHook to separate the
> algebra output from the remaining information. however, at least the step
> number appearing just before the algebra output is currently not decorated.
>

I think using ioHook would definitely be an improvement.

> is there a better way to get the results of computations - and also to send
> instructions to fricas?
>

It seems to me that the best way to call FriCAS would be via the
socket interface that is used by Hyperdoc. Unfortunately it is not
documented anywhere. The only source of information that I know of is
the source code for Hyperdoc.

> I'm afraid, python cannot call lisp directly, can it?
>

In principle Python can call Lisp if Lisp is implemented as a library.
This is supposed to be possible with ECL. But building FriCAS as a
library might be a significant challenge. It is not clear if there
would be a significant gain for the Sage-Fricas interface since most
communication would still have to take place as exchange of command
and results strings in any case - unless you want to tackle direct
conversion of data types.

Bill Page.

[Ralf Hemmecke]

>> I'm afraid, python cannot call lisp directly, can it?

> In principle Python can call Lisp if Lisp is implemented as a library.
> This is supposed to be possible with ECL. But building FriCAS as a
> library might be a significant challenge. It is not clear if there
> would be a significant gain for the Sage-Fricas interface since most
> communication would still have to take place as exchange of command
> and results strings in any case - unless you want to tackle direct
> conversion of data types.

FriCAS as a lisp-library? Insn't that what Kurt Pagani has done when he
made the fricas-jupyter connection available?

It's https://github.com/nilqed/fricas_jupyter if I am not mistaken.

Ralf

[Dima Pasechnik]

from reading https://github.com/fredokun/cl-jupyter/blob/master/about-cl-jupyter.pdf

one sees that the interaction between CL and jupyter goes via sockets.

This is close to how the current Sage interface works, and quite far from embedding FriCAS as a C-library.

The latter would greatly improve stability and speed of communication between Sage and FriCAS; it's probably

not too much work, after all there is such a C-library interface for Maxima used in Sage.

(it's another story how to make it even more efficient by doing "proper", non-string interface; not sure this is done in Maxima's case)

Dima

  

Ralf

[Martin R]

After some sleep I decided to go for the simple thing first.  In particular, I want to restrict myself to fricas.  Here is a list of things todo on the sage side:

1) implement a method that (reliably!) yields a tuple (type, 1d algebra output, 2d algebra output, error message, etc) as strings.  1d algebra output should come in "unparsed inputform" as one very long string, I'd say.  2d algebra output should be empty if 1d output is available.

It should be able to deal with very long lines, too, possibly be using file io.  This involves use of ioHook and some regexps.

2) implement a method that translates fricas output into sage types.

I think this could work as follows: we want to map fricas types to sage constructors, possibly recursively.  Recall that what's really sent to fricas is something like "sage23 := [x^n/y^n for n in 1..3]", so that sage can access the result using the variable "sage23".  This string is referred to as self._name

Now what I propose is a method which takes a parsed type, e.g., "Integer" or ("List", "Integer") or 

("UnivariatePolynomial", "x", ("Fraction", "Integer"))

def to_sage(type):

    if type == "Integer":

         return to_sage_integer()

    elif type == "String":

         return to_sage_string()

    ...

    elif isinstance(type, tuple):

        if type[0] == "List":

            return to_sage_list(type[1])

        elif type[0] == "Fraction":

            return to_sage_fraction(type[1])

        ....

and so on.

So, if the type is ("List" ("Fraction" ("UnivariatePolynomial" "x" "Integer"))), this method would call

    to_sage_list(("Fraction" ("UnivariatePolynomial" "x" "Integer"))

which might be something like

def to_sage_list(self, type):

    n = fricas.eval("#" + self._name).to_sage_integer()

    return [fricas.eval(self._name + ".%n").to_sage(type) for n in range(1,n+1)]

I'm not sure whether this is clever.  What do you think?

[Bill Page]

On 12 August 2016 at 06:54, 'Martin R' via FriCAS - computer algebra

system <fricas...@googlegroups.com> wrote:
> After some sleep I decided to go for the simple thing first. In particular,
> I want to restrict myself to fricas. Here is a list of things todo on the
> sage side:
>
> 1) implement a method that (reliably!) yields a tuple (type, 1d algebra
> output, 2d algebra output, error message, etc) as strings.

> ...

> This involves use of ioHook and some regexps.
>
> 2) implement a method that translates fricas output into sage types.

> ...

>
> I'm not sure whether this is clever. What do you think?
>

Except for the use of ioHook that sounds almost like what the
interface does now.

[Martin R]

Except for the use of ioHook that sounds almost like what the
interface does now.

Yes, it's quite close.  It's mainly an idea for organising the code.  Note that, for example, fricas(1/2).sage() is currently not working.

I have already written a little.  Currently I'm trying to find out whether fricas.eval is supposed to return a string - I'd like it to return my tuple...

Martin 

[Bill Page]

Martin,

Maybe this is already clear to you but I think it is important to get
one's mind around the fact that the Sage interface treats things in
FriCAS like objects whose methods are implemented by FriCAS. So for
example

sage: x=fricas('x')
sage: y=fricas('y')
sage: z=x+y
sage: type(z)
<class 'sage.interfaces.fricas.FriCASElement'>

even though the third line above makes no mention of FriCAS explicitly. Then

sage: zs = z.sage()
sage: type(zs)

<type 'sage.rings.polynomial.multi_polynomial_libsingular.MPolynomial_libsingular'>

And

sage: zf=fricas(zs)
sage: type(zf)
<class 'sage.interfaces.fricas.FriCASElement'>

Things like 'fricas.eval()' are low-level functions which operate on
strings and not normally used explicitly when using FriCAS through the
Sage interface.

On 12 August 2016 at 09:38, 'Martin R' via FriCAS - computer algebra

fricas.eval() is supposed to return a string. That is fairly
fundamental to the external interface design in Sage.

Bill Page.

[Waldek Hebisch]

Martin R wrote:
>
> given the continued interest in some features of fricas from some sage
> users, I had a look at the fricas interface in sage, and I'm thinking of
> improving it.
>
> The main question is: how can sage receive the output from fricas?
>
> currently, sage essentially sends "unparse(result:InputForm)" to fricas,
> and then parses the output: it strips away the prompt and the step number
> and it looks for the type information, all using regular expressions.
>
> a first improvement I was thinking of is to use the ioHook to separate the
> algebra output from the remaining information. however, at least the step
> number appearing just before the algebra output is currently not decorated.
>
> is there a better way to get the results of computations - and also to send
> instructions to fricas?

There are many ways. Some issues were discussed few years
ago when Serge D. Mechveliani tried to create interface
between FriCAS and Haskell.

I suspect that you want to do little improvement as opposed
to major rewrite, so some ideas that _may_ be easy:

- using S-expressions to communicate with FriCAS. On FriCAS
side it is quite easy to generate textual form of S-expressions.
S-expressions were designed to be easy to parse, so handling
them on Python side should be easy too
- as Dima wrote Sage interface to Maxima works by putting Maxima
in the same process as Sage. This should be possible with
FriCAS too. I just checked that using Maxima build on to
of sbcl and 'load-fricas.lisp' both FriCAS and Maxima seem
to work in single process. So it seems that small adjustment
to build system should be enough

--
Waldek Hebisch

[Dima Pasechnik]

as far as the latter is concerned, most of the needed code is already in Sage;

namely, src/sage/interfaces/maxima_lib.py is basically an example of running Maxima in ECL, 

which is embedded in Python; for the latter the code is in 

src/sage/libs/ecl.pyx

In the latter one sees an example of calling CL functions from Sage:

        sage: from sage.libs.ecl import *

        sage: ecl_eval("(defun fibo (n)(cond((= n 0) 0)((= n 1) 1)(T (+ (fibo (- n 1)) (fibo (- n 2))))))")

        <ECL: FIBO>

        sage: ecl_eval("(mapcar 'fibo '(1 2 3 4 5 6 7))")

        <ECL: (1 1 2 3 5 8 13)>

That is, one would just need to adapt src/sage/interfaces/maxima_lib.py to loading/running FriCAS instead.

For this one needs to know some parts of FriCAS, that is, how to load it in a running CL instance, etc etc...

Dima

 

--
                              Waldek Hebisch

[Martin R]

There is however one thing to keep in mind: FriCAS is (at least: used to be) MUCH faster when compiled with sbcl.  Thus, I'd like to be able to use the interface also with an FriCAS compiled with sbcl, when this is provided by the system.

(For the moment I'm doing the absolutely straightforward and making good progress.)

Martin

[Waldek Hebisch]

Martin R wrote:
>
> There is however one thing to keep in mind: FriCAS is (at least: used to
> be) MUCH faster when compiled with sbcl. Thus, I'd like to be able to use
> the interface also with an FriCAS compiled with sbcl, when this is provided
> by the system.

Well, the difference between sbcl and ecl is quite significant.
On the same 2.3 GHz machine, testsuite takes:

real 5m5.469s
user 4m47.746s
sys 0m17.570s

with sbcl and

real 17m2.270s
user 18m28.927s
sys 1m13.511s

I do not know how ecl managed to get sum of user and system times
to be bigger than real time (the testsuite is single threaded).
Maybe this is due to some costly multithreaded initialization...

--
Waldek Hebisch

[Martin R]

Hi there, especially Bill!

I won't work on the interface the next few days, so maybe you want to have a look.

https://trac.sagemath.org/ticket/21231

Please do not be offended that I renamed some of your methods, that was just for debugging purposes.

There are a few important tasks left:

* implement error and message handling in the method eval of the class FriCAS in fricas.py.  This might require playing with the ioHook in the last element of the tuple FRICAS_INIT_CODE and some pattern matching.

I think we should distinguish between FriCAS output which are just messages (like when executing a command such as )show etc.), parser errors and other errors.

* implement translation of more domains.  This requires modification of the methods _get_sage_type and _sage_ in class FriCASElement.  In particular, all the matrix domains are missing, stream and series domains are missing, Complex is missing,... Here FriCAS can also show off some of it's capabilities speed wise!  For every domain you add, you have to include a (small) testcase in the docstring of _sage_.

* adapting the doctests in other files and fricas_integrator in src/sage/symbolic/integration/ (which possibly should only use fricas("...").sage() and none of eval etc., but for this ._sage_ must be made more robust.

All the best,

Martin

[Waldek Hebisch]

With the following patch applied to FriCAS trunk:

------------------<cut here>---------------------------

Index: src/interp/util.lisp
===================================================================
--- src/interp/util.lisp (revision 2024)
+++ src/interp/util.lisp (working copy)
@@ -392,6 +392,7 @@
(push (list 'defparameter el (symbol-value el))
initforms)))
(push `(interpsys-ecl-image-init ,spad) initforms)
+ (push `(fricas-restart) initforms)
(setf initforms (reverse initforms))
(push `progn initforms)
(setf FRICAS-LISP::*fricas-initial-lisp-forms* initforms)
@@ -421,7 +422,7 @@
spad)
(format *standard-output* "before fricas-restart~%")
(force-output *standard-output*)
- (fricas-restart))
+)

(defun interpsys-image-init (parse-files comp-files browse-files
asauto-files spad)

------------------------<cut here>------------------


I can use the following to produce fricas as shared library:

--------------<cut here>----------------------------

;;; load compiler
(defun foo () nil)
(compile 'foo)

(let ((*default-pathname-defaults*
#P"//sklad/hebisch/fricas/axp7/ax-build87/src/interp/"))
(load "../lisp/fricas-package.lisp")
)
(let ((*default-pathname-defaults*
#P"//sklad/hebisch/fricas/axp7/ax-build87/src/interp/"))
(load "../lisp/fricas-config.lisp")
(load "../lisp/fricas-lisp")
(load "../lisp/primitives.lisp")
(load "../lisp/fricas-ecl.lisp")
(load "makeint.lisp")
(let ((initforms (reverse FRICAS-LISP::*fricas-initial-lisp-forms*)))
(setf initforms (reverse (cdr initforms)))
(c:build-fasl "fricas_lib"
:lisp-files FRICAS-LISP::*fricas-initial-lisp-objects*
:ld-flags FRICAS-LISP::*fricas-extra-c-files*
:epilogue-code initforms)
)
)

--------------------<cut here>--------------------------

This needs to be run once after FriCAS build. It creates
"fricas_lib.fas" in the 'src/interp' subdirectory of FriCAS
build tree.

Note: the paths above are from my machine. Adjust to your
envirinment.

Once FriCAS shared library is created in ECL you can do:

(load "src/interp/fricas_lib.fas")
(in-package "BOOT")
(fricas-init)

and then use FriCAS functions. Presumably in Sage it will look
like:

ecl_eval("(load \"src/interp/fricas_lib.fas\")")
ecl_eval("(in-package \"BOOT\")")
ecl_eval("(fricas-init)")

At some moment you need to set AXIOM environment variable
to inform FriCAS where it can find other files it needs.
In Sage it is probably most convenient to do from Python
code.

--
Waldek Hebisch

[Martin R]

Would this (or something similar) also work for sbcl builds?

Martin

[Waldek Hebisch]

Martin R wrote:
>
> Would this (or something similar) also work for sbcl builds?

There are two aspects here. One is running in the same process.
This is not going to work with sbcl (more precisely, there are
hard probles to resolve). Other is logical structure
of interface. You could use the same logical structure in
sbcl interface, just passing arguments and results via sockets.
Of course even that would require some support code, but
should be not that hard. Possibly fricas_jupyter code
could be reused.

Now, of course there is question of possible benefits.
IMO avoiding interpreter and (p)expect is a big
win. Namely, both are preformance sinks and add
complexity to the interface. Comparatively, replacing
sockect by in-process calls is smaller gain.
More precisely, for C programs difference between
in-process calls (essentialy two clocks that is of
order 1ns) and socket comunication (few us) is
quite significant. But Python calls have large
overhead (of order of hundreds clocks) so
by using in-process calls you get something like
10 times speedup. OTOH (p)expect and interpreter
seem to have overhead of order of 1ms, much larger
than difference between in-process communication
and sockets.

Also, getting rid of regex based parser would be
a win. I love regexes and use them when I feel
they are appropriate. However, simple regexes
usually give you 99% correct parser. When you
try to cover remaining cases complexity grows
and in Dijkstra classification you get
program "complex enough to have no obvious flaws".
In other words, when it passes test you still
can not be confident that it works correctly.
Performancewise, regex based parsers tend to
be slow.

--
Waldek Hebisch

[Bill Page]

Martin,

On 13 August 2016 at 04:36, 'Martin R' via FriCAS - computer algebra

system <fricas...@googlegroups.com> wrote:
> Hi there, especially Bill!
>
> I won't work on the interface the next few days, so maybe you want to have
> a look.
>
> https://trac.sagemath.org/ticket/21231
>

Thanks. I am building the development version of Sage now so I can
checkout your ticket. I really appreciate your work on this!

> Please do not be offended that I renamed some of your methods, that was
> just for debugging purposes.
>

Quite the opposite - I am greatly encouraged that someone took the
time to work on this. A quick review of your changes from the patch
looks like a very good start to me.

> There are a few important tasks left:
>
> * implement error and message handling in the method eval of the class
> FriCAS in fricas.py. This might require playing with the ioHook in the last
> element of the tuple FRICAS_INIT_CODE and some pattern matching.
>
> I think we should distinguish between FriCAS output which are just messages
> (like when executing a command such as )show etc.), parser errors and other
> errors.
>

Yes. Doing this with regex pattern matching is a bit error prone. This
is attempted in the old Axiom wiki code. It would be nice if FriCAS
could help with some minimal markup that would be easier to parse.

As you know, there are two "modes" in which to use FriCAS in Sage and
especially in the notebook interface. It is possible to just use the
notebook interface without any of the Sage interface as such. In that
case what you type in input cells goes to FriCAS and what FriCAS
outputs is returned directly to the notebook interface. On SMC I wrote
a simple mode (magic) that passes FriCAS output (with a few minor
tweaks) to markdown for post-processing. I wonder how hard it would be
to get this to work on with the Sage notebook and/or Jupyter?

> * implement translation of more domains. This requires modification of the
> methods _get_sage_type and _sage_ in class FriCASElement. In particular,
> all the matrix domains are missing, stream and series domains are missing,
> Complex is missing,... Here FriCAS can also show off some of it's
> capabilities speed wise! For every domain you add, you have to include a
> (small) testcase in the docstring of _sage_.
>

OK.

> * adapting the doctests in other files and fricas_integrator in
> src/sage/symbolic/integration/ (which possibly should only use
> fricas("...").sage() and none of eval etc., but for this ._sage_ must
> be made more robust.
>

I think it would be excellent if FriCAS was easily available for Sage
users. Please continue your work on this interface.

Bill Page.

[Bill Page]

> Dima Pasechnik wrote:
> ...

>> running Maxima in ECL,
>> which is embedded in Python; for the latter the code is in
>> src/sage/libs/ecl.pyx
>>
>> In the latter one sees an example of calling CL functions from Sage:
>>
>> sage: from sage.libs.ecl import *
>> sage: ecl_eval("(defun fibo (n)(cond((= n 0) 0)((= n 1) 1)(T (+
>> (fibo (- n 1)) (fibo (- n 2))))))")
>> <ECL: FIBO>
>> sage: ecl_eval("(mapcar 'fibo '(1 2 3 4 5 6 7))")
>> <ECL: (1 1 2 3 5 8 13)>
>>
>> That is, one would just need to adapt src/sage/interfaces/maxima_lib.py to
>> loading/running FriCAS instead.
>> For this one needs to know some parts of FriCAS, that is, how to load it in
>> a running CL instance, etc etc...

On 13 August 2016 at 07:08, Waldek Hebisch <heb...@math.uni.wroc.pl> wrote:
>
> With the following patch applied to FriCAS trunk:

> ...

> I can use the following to produce fricas as shared library:

> ...

That is excellent.

> Once FriCAS shared library is created in ECL you can do:
>
> (load "src/interp/fricas_lib.fas")
> (in-package "BOOT")
> (fricas-init)
>
> and then use FriCAS functions. Presumably in Sage it will look
> like:
>
> ecl_eval("(load \"src/interp/fricas_lib.fas\")")
> ecl_eval("(in-package \"BOOT\")")
> ecl_eval("(fricas-init)")
>

What is the best way to interact with FriCAS once the shared library
is loaded and FriCAS in initialized? For example, what would be
required to call FriCAS with a command string from Python and return
the result as a string?

I think that the gains from running FriCAS as a library will out
weight the difference in performance between ECL and SBCL for most
Sage users.

Bill Page.

[Waldek Hebisch]

Well, what is best depends on your criteria. From performance point
of view best is accessing the same data and that is what interface
between ECL and C can do. I do not know what Sage interface to
ECL is doing, but I suspect that it essentially provides Python
wrapper of ECL data. Of course, FriCAS and Sage are using different
data structures and you need some translation layer. If
you insist on using strings, than extracting string from
result given by ecl_eval should be easy. But one point about
library interface is that there is no need to go via strings.
For example FriCAS can directly fill C array with data or
Python can directly read data from FriCAS data structures.

> I think that the gains from running FriCAS as a library will out
> weight the difference in performance between ECL and SBCL for most
> Sage users.

At functionality level socket interface and in-process calls can
have almost the same functionality. Main difference is performance.
If you are using strings for data exchange, then you decrease
performance advantage. I supect that for most users performance
difference between in-process calls and socket interface
will not matter. Of course, I mean here socket interface
with proper translation layer, not current regex + pexpect
hack.

--
Waldek Hebisch

[Martin R]

I think that the gains from running FriCAS as a library will out
weight the difference in performance between ECL and SBCL for most
Sage users.

I doubt that this is the case.  Just consider why would you call FriCAS from sage for a computation?  I can only think of two reasons:

1) the functionality is not available in sage.  (quite unlikely, meanwhile, although there are some bits)

2) FriCAS is much faster.  In this case, you probably won't care about a second lost because of a slow interface.

for both reasons, I also believe that the most important part of the interface is to highlight features in FriCAS in the docstrings.

Martin

[Bill Page]

Martin,

Ok, perhaps I am not a "typical" Sage user but the reasons I want to
call FriCAS from Sage is to provide the things that are missing from
FriCAS - especially the graphics and the notebook interface.
Personally I would much rather do mathematical coding in FriCAS/SPAD
than in Sage. In spite of all the nasty things that SPAD can do to
you, Sage is a worse "can-of-worms" when it comes to development.

Statistics over the last few years seem to show that there are not
many Sage users in the opposite situation, i.e wanting to use FriCAS
for something that is missing in Sage. Can you give some examples?

Bill.

On 13 August 2016 at 14:21, 'Martin R' via FriCAS - computer algebra

[Waldek Hebisch]

Martin R wrote:
>
> > I doubt that this is the case. Just consider why would you call FriCAS
> from sage for a computation? I can only think of two reasons:
>
> 1) the functionality is not available in sage. (quite unlikely, meanwhile,
> although there are some bits)

Well, Sage uses Maxima as its default integrator. There are whole
classes of functions that FriCAS can integrate and Maxima can not
(the opposite happens, but is rare). Also, it is not hard
to find examples where Maxima gives nonelemetary answer when
elementary integral exists. FriCAS answers are irredundant:
nonelementary parts are necessary to express the answer.

FriCAS has solver for differential linear ODE-s of higher
order and for systems. IIUC Sage (via Maxima) is limited to
order 2.

I belive that FriCAS limit command is stronger than Maxima
and Sympy. The difference here is probably smaller than in
case of integrator, but still there is reason to call
FriCAS limit.

I wonder if Sage has symbolic Jordan decomposition? FriCAS
has (under name generalizedEigenvectors). Given activity
of combinat group Sage probably has support for formal
power series. But I wonder how it compares to FriCAS
support?

FriCAS has various noncommutative stuff. IIUC physicists
are interested in shuffle and related algebras and computation
in them is related to Hall bases. While we do not have
ready shuffle algebra needed ingerdients are present in
FriCAS.

As a little curiosity, from 2011 we have domain for ordinals.
At ISSAC 2015 support for ordinals was prominently present
among new things freshly added to Maple. I guess here
FriCAS is ahead of Maple and Maple is ahead of Sage.

>
> 2) FriCAS is much faster. In this case, you probably won't care about a
> second lost because of a slow interface.

It depends. If you want to call routine thousends of times
per second than even if base routine is significantly faster
the effective speed may be lower due to interface overheads.

--
Waldek Hebisch

[Ralf Hemmecke]

On 08/13/2016 09:16 PM, Bill Page wrote:
> Ok, perhaps I am not a "typical" Sage user but the reasons I want to
> call FriCAS from Sage is to provide the things that are missing from
> FriCAS - especially the graphics and the notebook interface.

Don't you use FriCAS_jupyter from Kurt Pagani?
But true, graphics is not the best in FriCAS.

Ralf

[Bill Page]

On 13 August 2016 at 17:08, Ralf Hemmecke <ra...@hemmecke.org> wrote:
> On 08/13/2016 09:16 PM, Bill Page wrote:
>> Ok, perhaps I am not a "typical" Sage user but the reasons I want to
>> call FriCAS from Sage is to provide the things that are missing from
>> FriCAS - especially the graphics and the notebook interface.
>
> Don't you use FriCAS_jupyter from Kurt Pagani?

Yes occasionally, perhaps more in the future as I change-over from
using Sage worksheets in Sage to Jupyter worksheets. But at least on
SMC I still prefer to use the custom fricas/markdown mode that I wrote
about a year ago. Jupyter still seems "foreign" to me.

> But true, graphics is not the best in FriCAS.
>

There is no FriCAS graphics that works on SMC and in particular no
animation graphics in FriCAS at all.

Bill Page.

[Bill Page]

Martin,

On 13 August 2016 at 09:44, Bill Page <bill...@newsynthesis.org> wrote:
>
> On 13 August 2016 at 04:36, 'Martin R' via FriCAS - computer algebra
> system <fricas...@googlegroups.com> wrote:
>> Hi there, especially Bill!
>>
>> I won't work on the interface the next few days, so maybe you want to have
>> a look.
>>
>> https://trac.sagemath.org/ticket/21231
>>
> Thanks. I am building the development version of Sage now so I can
> checkout your ticket. I really appreciate your work on this!
>

It took a little longer than I expected to navigate Sage trac etc. but
I have a test system now.

Here is the first problem I found:

sage: fricas('1$Polynomial(Fraction(Integer))').sage()
...
/home/wspage/sage/local/lib/python2.7/site-packages/sage/interfaces/fricas.pyc
in _sage_(self)
638 base_ring = self._get_sage_type(type[1])
639 vars = self.variables()._get_1d_output()[1:-1]
--> 640 R = PolynomialRing(base_ring, vars)
641 return R(self._get_1d_output())
642
...
ValueError: variable name must be nonempty

BTW, how do I turn off the horribly excessive code coloration that
seems to have crept in to Sage command line?

Bill.

[Martin R]

Am Samstag, 13. August 2016 22:08:16 UTC+2 schrieb Waldek Hebisch:

Martin R wrote:
>
> > I doubt that this is the case.  Just consider why would you call FriCAS
> from sage for a computation?  I can only think of two reasons:
>
> 1) the functionality is not available in sage.  (quite unlikely, meanwhile,
> although there are some bits)  

[snipped several examples]

Thank you for this wonderful list.  (that's exactly what I had in mind with "some bits")

 I think that the ore domain in FriCAS may also be stronger currently (sage is just about to get more here), guessing is the reason why I started all this, series expansion is also much better in FriCAS

 

> 2) FriCAS is much faster.  In this case, you probably won't care about a
> second lost because of a slow interface.

It depends.  If you want to call routine thousends of times
per second than even if base routine is significantly faster
the effective speed may be lower due to interface overheads.

I am certainly not against a fast interface! (only against putting in more than a week myself)

So: if someone helps (as you did already), I'm all for it!

I am already very careful to write the "fricas to sage" translation part without parsing of strings.

However, in the case you sketch, I'd write a little FriCAS routine and fetch only the end result.

Martin

[Martin R]

great, please add a test in the docstring! (the branch name will have to change to "/public/ticket-number" (or something like this), I don't know how to do this right now.)

Martin

[Martin R]

Hi Waldek!  (and all others of course, too!)

Am Samstag, 13. August 2016 22:08:16 UTC+2 schrieb Waldek Hebisch:

Well, Sage uses Maxima as its default integrator.  There are whole
classes of functions that FriCAS can integrate and Maxima can not
(the opposite happens, but is rare).  Also, it is not hard
to find examples where Maxima gives nonelemetary answer when
elementary integral exists.  FriCAS answers are irredundant:
nonelementary parts are necessary to express the answer. 

integration is one (and so far the only) part of sage which actually uses FriCAS (optionally).

 

FriCAS has solver for differential linear ODE-s of higher
order and for systems.  IIUC Sage (via Maxima) is limited to
order 2.

Great, I added an example from one of the input files.  (I know nothing hardly anything about ODE's.)

I belive that FriCAS limit command is stronger than Maxima
and Sympy.  The difference here is probably smaller than in
case of integrator, but still there is reason to call
FriCAS limit. 

OK, I'll check!

 

I wonder if Sage has symbolic Jordan decomposition?  FriCAS
has (under name generalizedEigenvectors).

I don't know what you mean here.  Sage has Jordan decomposition over algebraic numbers.

I checked generalizedEigenvectors matrix [[1, x], [0, 1]] but this gives a wrong result:

(5) -> m := matrix([[1,x],[0,1]])

        +1  x+

   (5)  |    |

        +0  1+

                                            Type: Matrix(Polynomial(Integer))

(6) -> generalizedEigenvectors m

                                +0+ +1+

   (6)  [[eigval= 1,geneigvec= [| |,| |]]]

                                +1+ +0+

Type: List(Record(eigval: Union(Fraction(Polynomial(Integer)),SuchThat(Symbol,Polynomial(Integer))),geneigvec: List(Matrix(Fraction(Polynomial(Integer))))))

 

Given activity
of combinat group Sage probably has support for formal
power series.  But I wonder how it compares to FriCAS
support?

This is another area where FriCAS is far ahead of sage, especially concerning expansion of expressions. 

FriCAS has various noncommutative stuff.  IIUC physicists
are interested in shuffle and related algebras and computation
in them is related to Hall bases.  While we do not have
ready shuffle algebra needed ingerdients are present in
FriCAS.

I think the only way to compete with sage in the territory of algebras is speed.  In particular, the shuffle algebra is in  sage and its quite easy to add new algebras.

 

As a little curiosity, from 2011 we have domain for ordinals.
At ISSAC 2015 support for ordinals was prominently present
among new things freshly added to Maple.  I guess here
FriCAS is ahead of Maple and Maple is ahead of Sage.

OK, that's another area I know nothing about and which apparently sage doesn't have.

Thanks for your support!  Besides, the interface is now mostly ready!

Martin

[Waldek Hebisch]

Why do you think it is wrong? The first vector is an eigenvector,
the second is in kernel of (A - 1)^2 and linearly independent of the
first. This gives Jordan form:

[ 1 lambda ]
[ ]
[ 0 1 ]

Matrices with different lambda are equivalent, so this is
Jordan form of input matrix. More generally, generalized
eigenvectors corresponding to lambda gives you basis of
subspace where (A - lambda) is nilpotent. Since in
your case m - 1 is nilpotent the result is rather trivial.

--
Waldek Hebisch

[Martin R]

I didn't understand the output, now I do, thanks!  It does appear similar to sage:

sage: R.<x> = PolynomialRing(QQ)

sage: m = matrix([[1, x],[0,1]])

sage: m.jordan_form(transformation =true)

(

[1 1]  [x 0]

[0 1], [0 1]

)

Do you have an example of a limit where maxima and sympy fail?

Martin

[Waldek Hebisch]

Martin R wrote:

> Do you have an example of a limit where maxima and sympy fail?

I do not have such example handy. However, I quickly tried
few examples and the third is:

(5) -> limit(x^2*exp(-x)*Ei(x) - x, x=%plusInfinity)

(5) 1
Type: Union(OrderedCompletion(Expression(Integer)),...)

AFIACS this is correct. OTOH:

Maxima 5.38.1 http://maxima.sourceforge.net
using Lisp SBCL 1.2.4.debian
Distributed under the GNU Public License. See the file COPYING.
Dedicated to the memory of William Schelter.
The function bug_report() provides bug reporting information.
(%i1) limit(x^2*exp(-x)*expintegral_ei(x) - x, x, inf);
(%o1) minf

which is wrong.

BTW: This is easy for Gruntz method so Sympy probably gets it right.

--
Waldek Hebisch

[Dima Pasechnik]

I recommend this be crossposted to sage-devel and perhaps to sage-combinat

There are people who know all about formal power series in Sage...

[Martin R]

Hi Waldek!

Once FriCAS shared library is created in ECL you can do:

(load "src/interp/fricas_lib.fas")
(in-package "BOOT")
(fricas-init)

and then use FriCAS functions.  Presumably in Sage it will look
like:

I was trying this, and now have a prompt "BOOT>", but don't know how to continue.  How exactly can I call a FriCAS function now?

Also, will a variation of this work with sbcl?

Thanks,

Martin

[Martin R]

Hi again!

I found |parseAndInterpret|, and |getFunctionFromDomain|, but still need a little help.  In particular, I'd like to create a list of integers without using strings.

Martin

[Waldek Hebisch]

See:

https://common-lisp.net/project/ecl/manual/re25.html
https://common-lisp.net/project/ecl/manual/re34.html

How to call those from Python is another story...

--
Waldek Hebisch

[Martin R]

I know how to make a lisp list.  But I don't know how to make a FriCAS list!  In other words, what do I have to type after

BOOT>

to get an element of List(Integer).

Related: I managed to do

BOOT> (|getFunctionFromDomain| '|sin| '(|Expression| (|Integer|)) '((|Expression| (|Integer|)))

but so far, I did not manage to get the Expression Integer "sin x".  Also,

BOOT> (|parseAndInterpret| "integrate(sin x,x)")

tells me that sin is not exposed...

Thanks for your patience!

Martin

[Waldek Hebisch]

> I know how to make a lisp list. But I don't know how to make a FriCAS
> list! In other words, what do I have to type after
>
> BOOT>
>
> to get an element of List(Integer).

Nothing. Lisp list == FriCAS list.

>
> Related: I managed to do
>
> BOOT> (|getFunctionFromDomain| '|sin| '(|Expression| (|Integer|))
> '((|Expression| (|Integer|)))
>
> but so far, I did not manage to get the Expression Integer "sin x". Also,
>
> BOOT> (|parseAndInterpret| "integrate(sin x,x)")
>
> tells me that sin is not exposed...

Have you set AXIOM environment variable? Without it FriCAS
only simplest things work.

--
Waldek Hebisch

[Martin R]

OK, setting $AXIOM helped to make (|parseAndInterpret|  "integrate(sin(x), x)") work.  Thanks!

But I'm still having a very hard time to do even simple stuff without parsing a string.  For example, if you could show me how to call

guessADE([1,2,3,4,5,6,7])$GUESSINT

it would be really helpful.  So far, I was successful with

(|getFunctionFromDomain| '|guessADE| '(|GuessInteger|) '((|List| (|Fraction| (|Integer|))))))

but I cannot apply it to a list :-(

Martin

[Waldek Hebisch]

Martin R wrote:
>
> But I'm still having a very hard time to do even simple stuff without
> parsing a string. For example, if you could show me how to call
>
> guessADE([1,2,3,4,5,6,7])$GUESSINT
>
> it would be really helpful. So far, I was successful with
>
> (|getFunctionFromDomain| '|guessADE| '(|GuessInteger|) '((|List|
> (|Fraction| (|Integer|))))))
>
> but I cannot apply it to a list :-(

Applying is easy. You probably mean that you do not know
how to build list of fractions? In case of fractions
one can cheat: FriCAS Fraction is just a Lisp cons.
In general most of FriCAS types provide "constructor
functions", that is finctions which build elements of
given domain from appropriate data. In case of
Fraction(Integer) you would use '/:(Integer, Integer) -> %'.
So use 'getFunctionFromDomain' to get constructor
function and then use is to build fractions. Collect
fractions into a list and call 'guessADE'. As inline
code this would be quite long, so I do not write it.
It is probably better from the beginning to write
appropriate convertion routines. Once you handle
few basic constructors like Fraction, Polynomial,
Expression, Matrix, Vector, List you will cover
a lot of FriCAS functionality.

--
Waldek Hebisch

[Martin R]

OK, got it!  Thank you!

BOOT> (setf l '((1 . 1) (2 . 1) (3 . 1) (4 . 1) (5 . 1)))

((1 . 1) (2 . 1) (3 . 1) (4 . 1) (5 . 1))

BOOT> (setf g (|getFunctionFromDomain| '|guessADE| '(|GuessInteger|) '((|List| (|Fraction| (|Integer|))))))

(#<compiled-function |GUESS;guessADE;LL;101|> . #<vector 0000000004f790c0>)

BOOT> (SPADCALL l g)

NIL

BOOT> (setf l '((1 . 1) (2 . 1) (3 . 1) (4 . 1) (5 . 1) (6 . 1)))

((1 . 1) (2 . 1) (3 . 1) (4 . 1) (5 . 1) (6 . 1))

BOOT> (SPADCALL l g)

(((1

   #S(SPAD-KERNEL :OP #<vector 000000000529b9c0> :ARG

      (((1 #S(SPAD-KERNEL :OP #<vector 0000000004f487b0> :ARG NIL :NEST 1)

         (1 0 . 1))

        0 . 1)

       ((1 #S(SPAD-KERNEL :OP #<vector 0000000004e559f0> :ARG NIL :NEST 1)

         (1 0 . 1))

        0 . 1))

      :NEST 2)

   (1 0 . 1))

  0 . 1))

BOOT> (SPADCALL l g)