SAGE and FriCAS
Martin Rubey
currently, there is a discussion on SAGE about GiNaC going to be a partial
replacement for Maxima in SAGE, if I understand correctly.
Thus, if anybody else but me wants to see a similar role played by FriCAS, I
think it's high time to act. I must admit, however, that I have been unable to
do anything useful with respect to that, so I beg you *not* to regard this mail
as a complaint.
I'm very unsure on how to proceed. Just about *all* math colleagues I know are
starting to use SAGE now, and actually, that system covers just about
everything, meanwhile.
If I just liked Python better...
Martin
Bill Page
On Fri, Aug 15, 2008 at 12:49 PM, you wrote:
>
> This is just a heads up:
>
> currently, there is a discussion on SAGE about GiNaC going to be a
> partial replacement for Maxima in SAGE, if I understand correctly.
>
No, I think the discussion is whether to implement more symbolic
functionality in native Sage or whether to try to get that
functionality (and faster performance) by linking to an existing
library for symbolic programming. I expect Maxima will continue to be
a core Sage package. It is just that some of things in Sage that are
currently done by passing these expressions to Maxima might be done by
calling some external subroutine.
One of the alternatives to GiNaC that is being considered is SymPy - a
native Python library for symbolic computation.
> Thus, if anybody else but me wants to see a similar role played by FriCAS,
> I think it's high time to act.
I am not so sure that a similar role can be played by FriCAS. The
FriCAS/Axiom libraries are not so easily called by an external
program. Instead we have the same option to interface with FriCAS as
we have with Maxima - via the pseudo-terminal (pty) serial interface.
This has the same drawbacks in both Maxima and FriCAS.
With some significant work, and using ECL, there is an opportunity in
both Maxima and FriCAS to interface with Sage via a program library
API, i.e. calling Lisp as a library. There is also with FriCAS an
opportunity to better represent the algebraic data structures that are
passed from Sage and for the FriCAS type system to map directly into
the Sage object hierarchy. This is something that is not possible with
Maxima but extending the existing 'axiom.py' interface to handle this
would be a lot of work.
For the most part I think the Sage developers have a rather narrow
definition of what they think the need/want in a symbolic package.
From what I understand, that means that they would think of FriCAS as
providing just one or two domains - Expression Integer, Expression
Float. All the rest of FriCAS is not so much of interest to them since
(from their point of view) they have already re-implemented
(re-invented? ;-) it in the main body of Sage and through interfaces
with other more specialized packages.
> I must admit, however, that I have been unable to do anything useful with
> respect to that, so I beg you *not* to regard this mail as a complaint.
>
> I'm very unsure on how to proceed. Just about *all* math colleagues
> I know are starting to use SAGE now, and actually, that system covers
> just about everything, meanwhile.
>
For some reason Sage always reminds me of the "Borg" on Startrek. ;-)
> If I just liked Python better...
>
What's wrong with Python? What's better in Spad?
I think we (you?) need to spend some more time trying to answer these questions.
Regards,
Bill Page.
William stein
On Aug 15, 11:51 am, "Bill Page" <bill.p...@newsynthesis.org> wrote:
> Martin,
>
> On Fri, Aug 15, 2008 at 12:49 PM, you wrote:
>
> > This is just a heads up:
>
> > currently, there is a discussion on SAGE about GiNaC going to be a
> > partial replacement for Maxima in SAGE, if I understand correctly.
*partial* replacement for Maxima in Sage.
> No, I think the discussion is whether to implement more symbolic
> functionality in native Sage or whether to try to get that
> functionality (and faster performance) by linking to an existing
> library for symbolic programming.
> I expect Maxima will continue to be
> a core Sage package.
> It is just that some of things in Sage that are
> currently done by passing these expressions to Maxima might be done by
> calling some external subroutine.
>
> One of the alternatives to GiNaC that is being considered is SymPy - a
> native Python library for symbolic computation.
many Sage users use SymPY from sage. I hope that SymPy
will someday provide things like symbolic integration and
maybe symbolic system solving to Sage, among other things.
I also hope SymPy is (re-)designed to have a "pluggable
symbolic core", which could be implemented using native
Cython, pure Python, Ginac, Fricas, or something else.
> > Thus, if anybody else but me wants to see a similar role played by FriCAS,
> > I think it's high time to act.
>
> I am not so sure that a similar role can be played by FriCAS. The
> FriCAS/Axiom libraries are not so easily called by an external
> program. Instead we have the same option to interface with FriCAS as
> we have with Maxima - via the pseudo-terminal (pty) serial interface.
> This has the same drawbacks in both Maxima and FriCAS.
>
> With some significant work, and using ECL, there is an opportunity in
> both Maxima and FriCAS to interface with Sage via a program library
> API, i.e. calling Lisp as a library. There is also with FriCAS an
> opportunity to better represent the algebraic data structures that are
> passed from Sage and for the FriCAS type system to map directly into
> the Sage object hierarchy. This is something that is not possible with
> Maxima but extending the existing 'axiom.py' interface to handle this
> would be a lot of work.
> For the most part I think the Sage developers have a rather narrow
> definition of what they think the need/want in a symbolic package.
and Mathematica. That's it.
> From what I understand, that means that they would think of FriCAS as
> providing just one or two domains - Expression Integer, Expression
> Float. All the rest of FriCAS is not so much of interest to them since
> (from their point of view) they have already re-implemented
> (re-invented? ;-) it in the main body of Sage and through interfaces
> with other more specialized packages.
>
> > I must admit, however, that I have been unable to do anything useful with
> > respect to that, so I beg you *not* to regard this mail as a complaint.
>
> > I'm very unsure on how to proceed. Just about *all* math colleagues
> > I know are starting to use SAGE now, and actually, that system covers
> > just about everything, meanwhile.
>
> For some reason Sage always reminds me of the "Borg" on Startrek. ;-)
FriCAS
and Spad would have already been assimilated.
Martin Rubey
> I am not so sure that a similar role can be played by FriCAS. The
> FriCAS/Axiom libraries are not so easily called by an external
> program. Instead we have the same option to interface with FriCAS as
> we have with Maxima - via the pseudo-terminal (pty) serial interface.
> This has the same drawbacks in both Maxima and FriCAS.
I recently thought that it might be quite easy to have a domain SAGEForm,
similar to OutputForm or --
actually, doesn't OpenMath provide what SAGE needs?
Martin
Bill Page
On Sat, Aug 16, 2008 at 1:59 AM, Martin Rubey wrote:
OpenMath was an early attempt to do something much more ambitious than
what Sage needs. *If* the OpenMath option in Axiom was working today,
then it would indeed be a good way to pass output from Axiom back to
Sage while retaining as much of the Axiom type information as
possible. The necessary hooks for OpenMath generation are present in
the current panAxiom code, but OpenMath as implemented in Axiom
depends on an external library that is not currently included with any
panAxiom version.
But as I said, OpenMath attempts to solve the general problem of
exporting full semantic information from a computer algebra system.
This is more than what is required for a good interface to Sage. In
panAxiom we also have the MathML output. Currently only presentation
MathML is generated but there is an extension of MathML that include
much more content (semantic) information without getting into the
generalities of OpenMath. Perhaps a 'SAGEform' domain could be based
on an extension of the MathML output option.
But still misses one of the hardest parts - mapping Axiom type
structures back to Sage in a way that other parts of Sage understand
these structures. At a mathematical level (e.g. various kinds of
polynomials) this is probably not too hard, but at a more fundamental
level things get more difficult. For example, Sage (actually
originating with Python) has not data structure directly corresponding
to 'Record' or 'Union' in Axiom. This can cause a lot of trouble when
trying to access the results of those computations in Axiom that
return complex results. You will no doubt be aware that this occurs
when accessing the output of GUESS from within Sage.
The "dumb" mode of calling an external program like Axiom just passes
a string, e.g.
sage: axiom('x^2+1')
then the parsing of 'x^2+1' occurs entirely within Axiom. But it is
not so interesting to always be creating and passing strings.
There is very a little documentation available in Sage about how the
conversion from Sage internal format to some external format (such as
used when calling Axiom). This happens automatically when you write
something like
sage: axiom(x^2+1)
without including the 'quotes'. In this case Sage parses the expression
x^2+1
and creates a native polynomial object. Then the 'axiom' function must
now recursively process this Sage expression until it finds objects
and operations near the bottom of the tree that it knows how to
interpret as Axiom objects and operations. Most of the coding that
accomplishes this is outside of the 'axiom.py' interface itself,
distributed over several other Python classes. As far as I know
William Stein is still the best source for how this conversion works.
He has said that "really it is very simple" and I guess I do
understand parts of it, but you should expect to spend some time to
re-discover how it works sufficiently well in order to improve it.
In order to pass Axiom expressions back to Sage, it seems to me that
we might have to implement something like this only working in
reverse.
Anyway, I think there is a lot of potential for improvement in the
existing 'axiom.py' interface even without considering how to solve
the problem of an efficient application program interface. One thing
that I miss a lot when using Axiom from the Sage notebook is the
ability to embed Spad code and compile it. This should be quite
straightforward to add to 'axiom.py'.
I would be glad to work with anyone else interested in improving the
Axiom/FriCAS interface for Sage.
Regards,
Bill Page.
Bill Page
>
> On Mon, Aug 18, 2008 at 5:10 PM, Bill Page wrote:
>> ...
>> Anyway, I think there is a lot of potential for improvement in the
>> existing 'axiom.py' interface even without considering how to
>> solve the problem of an efficient application program interface.
>> One thing that I miss a lot when using Axiom from the Sage
>> notebook is the ability to embed Spad code and compile it. This
>> should be quite straightforward to add to 'axiom.py'.
>>
>> I would be glad to work with anyone else interested in improving
>> the Axiom/FriCAS interface for Sage.
>
> Python directly. Then all the problems above could be solved
> much more easily. If the only way of interacting with lisp is over a
> pexpect interface, then something is very wrong.
>
I agree that an efficient API is desirable - especially if some
external program is to be called very often to perform some small
operation that is part of some larger calculation. That certainly is
the case for the way Sage uses Maxima now. In this case the lack of an
efficient API is a big limitation. But I do not see how this is the
case with Axiom. It is not clear to me that any of "the problems above
could be solved much more easily" given such an API. From my point of
view the problems are at a different conceptual level.
There is supposed to be an "easy" way to call programs compiled with
ECL from C (and thus via some suitable wrapper from Python or from
Cython), but I have yet to see this demonstrated. Even in this case
there remains the problem of conversion of data structures which is
likely to add back some (hopefully small part) of the overhead that
you removed by eliminating pexpect.
Regards,
Bill Page.
Bill Page
>
> On Mon, Aug 18, 2008 at 6:24 PM, Bill Page wrote:
>>
>> ... It is not so easy to call Python code from a C mainline
>> program either, is it?
>
> Thanks to Cython, it is very easy to call my Python implementation
> of something from pure C. And it's fast, for example:
>
> http://freehg.sympy.org/u/certik/csympy/file/991e40db913e/basic.c
>
> line 841:
>
> 841 import_csympy();
> 842 coeff = multi(m, n, &len);
>
> multi is implemented here
> http://freehg.sympy.org/u/certik/csympy/file/991e40db913e/csympy.pyx:
>
> 146 from multinomial import multinomial_coefficients
> 147
> 148
> 149 cdef api sympyint *multi(int m, int n, int *list_len):
> 150 r = multinomial_coefficients(m, n)
> 151 cdef int len2 = len(r)
> 152 list_len[0] = len2
> 153 l = []
> 154 for k, v in r.iteritems():
> 155 l.extend(k)
> 156 l.append(v)
> 157 cdef sympyint *cl = <sympyint*> malloc(len(l)*sizeof(sympyint))
> 158 for i in range(len(l)):
> 159 try:
> 160 cl[i] = l[i]
> 161 except OverflowError:
> 162 print "Overflow occured, using 'cl[i] = 2'."
> 163 print l[i]
> 164 cl[i] = 2
> 165 return cl
>
> So you see it calls pure Python function and just converts the
> dictionary it returns to a C array of integers.
>
Ok, thanks for the example. Yes I admit that using Cython this way is
quite nice. It makes me think that maybe it would be interesting to
write such Cython wrappers for calling larger parts of Sage from some
external program (such as FriCAS) this way.
But these sort of conversions are exactly the kind of thing to which I
was referring. In your example, the conversion is quite simple but it
even looks a little clumsy in this case. My main point in these
comments, is that when converting between two different high-level
representations of some more complex mathematical object, e.g. a
matrix of p-adic polynomials, etc. these conversions can potentially
dominate the interface.
Regards,
Bill Page.
Ralf Hemmecke
> comments, is that when converting between two different high-level
> representations of some more complex mathematical object, e.g. a
> matrix of p-adic polynomials, etc. these conversions can potentially
> dominate the interface.
I guess, the problem that Bill was talking about, can be a little backed
by the following page:
http://axiom-wiki.newsynthesis.org/TypeTowerDemo
Try to convert back and forth to Sage. I don't care how fast that would
be, but you must get the types right.
Additionally, could somebody produce a similar notebook in Sage... (or
SymPy) to do the same thing. That would help me in seeing how these
things are done in Sage. Thanks.
Ralf
Bill Page
>
> On Mon, Aug 18, 2008 at 7:08 PM, Bill Page wrote:
>>
>> Ok, thanks for the example. Yes I admit that using Cython this way
>> is quite nice. It makes me think that maybe it would be interesting
>> to write such Cython wrappers for calling larger parts of Sage from
>> some external program (such as FriCAS) this way.
>
>
>> But these sort of conversions are exactly the kind of thing to which
>> I was referring. In your example, the conversion is quite simple but
>> it even looks a little clumsy in this case. My main point in these
>> comments, is that when converting between two different high-level
>> representations of some more complex mathematical object, e.g.
>> a matrix of p-adic polynomials, etc. these conversions can
>> potentially dominate the interface.
>
>
The only "other option" that I can think of is something like Martin
Rubey suggested near the start of this thread: a new domain in FriCAS
named 'SAGEForm'. The representations used in 'SAGEForm' would
correspond directly to actual Sage data structures. In order to do
this in a general manner it would be very convenient for FriCAS to be
able to call Sage functions via Cython wrappers in the manner you have
described in this thread. The 'SAGEForm' domain would provide an
application interface to Sage for the rest of the FriCAS system. This
API would have to provide something like a shared buffer in which to
make the native Sage representation that it constructs available to
the rest of Sage.
Each FriCAS domain that you want to be able to represent directly in
Sage would have to provide an operation such as:
coerce: % -> SAGEForm
(similar to 'coerce: % -> OutputForm' and 'coerce: % -> InputForm').
This operation would convert values of that domain to it's
corresponding representation in Sage and in the case of complex "type
towers" in Axiom, these conversions could be called recursively. It
also should be possible to write a default coercion from 'InputForm'
to 'SAGEForm' that would work in some cases. 'InputForm' is the
existing domain in FriCAS that represents input to the Axiom
interpreter.
In fact using 'InputForm' in this way is exactly what the current
'axiom.py' interface does now. Computed FriCAS/Axiom values are
converted to 'InputForm' (which is just a Lisp 'SExpression') and then
from that form to the unparsed linear string form which is returned to
Sage (over the pty serial interface). The representation in
'InputForm' eliminates almost all type information but in a lot of
cases this is sufficient for Sage to convert to its native form since
when converting this representation backwards, Sage makes many of the
same type inferences that are made by the Axiom interpreter. But
rather than converting 'InputForm' to a linear string representation
and back, we want to go directly to a Sage data structure. This means
that in many domains a simple generic conversion (e.g. integer
literals to integer values) might not be the best one can do.
Important semantic information relevant to the representation might be
lost. In this case the domain could provide it's own direct coercion
to 'SAGEForm'.
In this proposed solution, writing SAGEForm in FriCAS is where most of
the work would be done.
Regards,
Bill Page.