Is there a time frame for the next version of Fricas?

[Nasser M. Abbasi]

I am planning to make new version of [cas integration tests](https://www.12000.org/my_notes/CAS_integration_tests/index.htm) when sagemath 9.6 is released which should happen in the next few weeks.

I was wondering if Fricas next version will be released any time this summer, so I can hold on and wait for that, or if I should use 1.3.7 from last year.

I know it is not possible to predict a release time, just asking for possible time frame.

Thanks

--Nasser

[Ralf Hemmecke]

I am currently working on the jfricas interface.
Unfortunately, there are some rather unsafe elements in it,
I must patch the functions |mathprintWithNumber| and OBEY

https://github.com/hemmecke/jfricas/blob/master/jfricas/webspad.lisp#L84

;;; Override a FriCAS function in order to remove the equation number.
(DEFUN |mathprintWithNumber| (|x|)
(PROG ()
(RETURN
(PROGN
;(|ioHook| '|startAlgebraOutput|)
(|saySpadMsg| '|--FORMAT:BEG:Algebra|)
(|maprin| (|outputTran2| |x|))
(|saySpadMsg| '|--FORMAT:END:Algebra|)
;(|ioHook| '|endOfAlgebraOutput|)
))))

(defun OBEY (S)
(sb-ext::process-exit-code
(sb-ext::run-program "/bin/sh" (list "-c" S) :input t
:output *standard-output* :error *standard-output*)))

and would be somehow happy if I wouldn't have to do this.

Overriding OBEY is for catching the output from the external process in
order to be able to show it in a jupyter cell. Maybe my Lisp knowledge
is too weak and it would also somehow work without changing OBEY.

https://github.com/fricas/fricas/blob/master/src/interp/vmlisp.lisp#L743

#+:sbcl
(defun OBEY (S)
#-:win32 (sb-ext::process-exit-code
(sb-ext::run-program "/bin/sh"
(list "-c" S) :input t :output t :error t))
#+:win32 (sb-ext::process-exit-code
(sb-ext::run-program "sh"
(list "-c" S) :input t :output t :error t :search t)))

The (boot) function mathprintWithNumber is perhaps a bit harder to make
it working in jfricas and in vanilla fricas.

I do not know whether it's a good idea to integrate jfricas (in for
sbcl) into the FriCAS sources. That would require to ask to include the
hunchentoot webserver (or a nice and simple way possible without root
access) to install hunchentoot at the start of jfricas.

Other than this there should be a little more stuff for fricas.github.io
-- but that can be done in a somewhat independent way from the next
release and wouldn't count as a blocker.

Ralf

[Waldek Hebisch]

New release is likely, but ATM there are many unknowns. Compared
to 1.3.7 current developement version has two releated changes
to integration code. Developement version can handle few more
integrals expressible in terms of polylog. 1.3.7 used pattern
matching as "last chance integrator", in developement version
pattern matching is not used (removed). They are related:
after improvement to algorithmic code it is supposed to handle
everything that was handled by FriCAS patern matcher. The
changes are unlikely to significantly change test results.

There are much bigger changes in queue, but ATM it is premature
to predict when they are ready.
--
Waldek Hebisch

[Waldek Hebisch]

On Tue, May 03, 2022 at 11:28:58PM +0200, Ralf Hemmecke wrote:
> I am currently working on the jfricas interface.
> Unfortunately, there are some rather unsafe elements in it,
> I must patch the functions |mathprintWithNumber| and OBEY
>
> https://github.com/hemmecke/jfricas/blob/master/jfricas/webspad.lisp#L84
>
> ;;; Override a FriCAS function in order to remove the equation number.
> (DEFUN |mathprintWithNumber| (|x|)
> (PROG ()
> (RETURN
> (PROGN
> ;(|ioHook| '|startAlgebraOutput|)
> (|saySpadMsg| '|--FORMAT:BEG:Algebra|)
> (|maprin| (|outputTran2| |x|))
> (|saySpadMsg| '|--FORMAT:END:Algebra|)
> ;(|ioHook| '|endOfAlgebraOutput|)
> ))))

I do not see why you are doing this. If you want to call
'saySpadMsg' appropriate io hook should be enough. For
skipping line numbers setting '$IOindex' to NIL should
be enough.

>
> (defun OBEY (S)
> (sb-ext::process-exit-code
> (sb-ext::run-program "/bin/sh" (list "-c" S) :input t
> :output *standard-output* :error *standard-output*)))
>
> and would be somehow happy if I wouldn't have to do this.

That looks reasonable. Currently we have 't' which means
'*termianl-io*'. Usualy termial IO is the same as standard
output, but sometimes sbcl is doing crazy things with it.
AFAICS we _never_ want sbcl '*termianl-io*'.

> I do not know whether it's a good idea to integrate jfricas (in for sbcl)
> into the FriCAS sources. That would require to ask to include the
> hunchentoot webserver (or a nice and simple way possible without root
> access) to install hunchentoot at the start of jfricas.

If we can:
- integrate it into build system so that FriCAS builds when using
other lisp (so hunchentoot would be built conditionaly)
- actually works on reasonably large subset of Linux distributions
- does not break/interfere with users Lisp setup,
then this would be reasonable.

OTOH it is not clear what you _really_ need. Simple web server
is few hundred lines. Do you need any fancy features? I would
guess that you do not need TLS/SSL (for local use this adds essentially
no extra security). Can hunchentoot be stripped of unneeded
parts?

--
Waldek Hebisch

[Ralf Hemmecke]

>> (defun OBEY (S)
>> (sb-ext::process-exit-code
>> (sb-ext::run-program "/bin/sh" (list "-c" S) :input t
>> :output *standard-output* :error *standard-output*)))
>>
>> and would be somehow happy if I wouldn't have to do this.
>
> That looks reasonable. Currently we have 't' which means
> '*termianl-io*'. Usualy termial IO is the same as standard
> output, but sometimes sbcl is doing crazy things with it.
> AFAICS we _never_ want sbcl '*termianl-io*'.

Thank you. Would that mean that I could prepare a patch to change OBEY
in vmlisp.lisp to

#+:sbcl
(defun OBEY (S)

#-:win32 (sb-ext::process-exit-code

(sb-ext::run-program "/bin/sh"
(list "-c" S) :input t :output *standard-output*

:error *error-output*))

#+:win32 (sb-ext::process-exit-code
(sb-ext::run-program "sh"

(list "-c" S) :input t :output *standard-output*

:error *error-output*)))

Or rather as I suggestet to ":error *standard-output*"?

For jfricas it wouldn't probably matter since I anyway set
*error-output* to *standard-output*.

In fact, browsing through the FriCAS code, it seems that *error-output*
is only used in the lisp optimizer.

Ralf

[Waldek Hebisch]

On Wed, May 04, 2022 at 10:50:50AM +0200, Ralf Hemmecke wrote:
> >>(defun OBEY (S)
> >> (sb-ext::process-exit-code
> >> (sb-ext::run-program "/bin/sh" (list "-c" S) :input t
> >> :output *standard-output* :error *standard-output*)))
> >>
> >>and would be somehow happy if I wouldn't have to do this.
> >
> >That looks reasonable. Currently we have 't' which means
> >'*termianl-io*'. Usualy termial IO is the same as standard
> >output, but sometimes sbcl is doing crazy things with it.
> >AFAICS we _never_ want sbcl '*termianl-io*'.
>
> Thank you. Would that mean that I could prepare a patch to change OBEY in
> vmlisp.lisp to
>
> #+:sbcl
> (defun OBEY (S)
> #-:win32 (sb-ext::process-exit-code
> (sb-ext::run-program "/bin/sh"
> (list "-c" S) :input t :output *standard-output*
> :error *error-output*))
> #+:win32 (sb-ext::process-exit-code
> (sb-ext::run-program "sh"
> (list "-c" S) :input t :output *standard-output*
> :error *error-output*)))
>
> Or rather as I suggestet to ":error *standard-output*"?

Sorry for confusion. I wrongly assumed that 'sb-ext::run-program'
uses standard Lisp convention about stream use. Looking into
'sb-ext::run-program' docs shows that 't' means inheriting
standard input, output and error output from current process,
which at least in some cases seem to be right thing.

You say that you want to catch error output. But what errors?
A little grepping of sources shows that currently one main use case
for OBEY is starting external editor, where inherited file descriptors
seem to be correct (and in jFriCAS you probably do not want to support
this or offer replacement compatible with Web interface). Other use
case are calls to few system commands where errors are _not_ expected.
If some unexpected error pops up it is debatable where message should go,
but resonable convention is that those errors should go to FRICASsys
standard error.

In particular, I not sure what you are doing with OBEY that you
need to catch standard output?

To put it differently, ATM you can launch text editor or a shell
via ')system' command and it almost works (there is some breakage
due to clef/rlwrap interfering, AFAICS without sman it works
fine). This is not very important use case, but it is nice
to keep what works working. OTOH in jFriCAS I would _not_
expect current ')system' to work, at least not without some
alternative implementation.

>
> For jfricas it wouldn't probably matter since I anyway set *error-output* to
> *standard-output*.
>
> In fact, browsing through the FriCAS code, it seems that *error-output* is
> only used in the lisp optimizer.

Well, currently when we generate Lisp code Lisp '*standard-output*' is
redirected to file. But we definitely do not want error messages going
to '*standard-output*': some messages are just warning, but if
emmited to file would make it syntactiacally incorrect. User normally
does not look at generated file so error messages emmited to file
are effectively lost. We should be more consistent about this in
other places, error messages should consistenly go to '*error-output*'
but since in many cases '*standard-output*' and '*error-output*'
are the same wrong code may "work" for long time.

BTW: As I wrote Saturn (Texexplorer) had notebook interface. While
code supporting Saturn is removed now you can look at past versions
to see how they handled tricky problems. In particular i-output.boot
was modified to behave as needed by Saturn. IIRC this included
TeX output of types and handling of line numbers.

BTW2: My impression is that in Saturn "unexpected" error messages
went to AXIOMsys console.

--
Waldek Hebisch

[Ralf Hemmecke]

> Looking into 'sb-ext::run-program' docs shows that 't' means
> inheriting standard input, output and error output from current
> process, which at least in some cases seem to be right thing.

The current process is FRICASsys. So it goes to the stdout and stderr of
whatever FRICASsys gets. Do I understand this correctly?
At least that would be confirmed by the fact that when I start fricas as
follows

https://github.com/fricas/jfricas/blob/master/jfricas/fricaskernel.py#L518

pid = Popen(['gnome-terminal', '--title=jfricas', '--'] +
['fricas','-nosman','-eval',prereq,'-eval',start])

instead of the default

pid = Popen(['fricas','-eval',prereq,'-eval',start]) (*)

then typing ")sys ls" in a jfricas notebook cell outputs into the
gnome-terminal.

However, this is, of course not what I want. In particular, when a user
types ")sys pwd" he wouldn't get that information at all (with (*)),
because then the process is whatever Jupyter does internally. It does
certainly NOT appear on the console from where I start "jupyter notebook".

This problem with stdout of )sys commands is one of the reasons why we
added such an option with gnome-terminal into the fricaskernel.py.

> You say that you want to catch error output. But what errors?

(1) -> )sys foo
/bin/sh: 1: foo: not found

That is visible in a ordinary fricas session but not in jfricas unless I
start OBEY with the ":error *standard-output" (or *error-output*) option.

> A little grepping of sources shows that currently one main use case
> for OBEY is starting external editor, where inherited file
> descriptors seem to be correct (and in jFriCAS you probably do not
> want to support this or offer replacement compatible with Web
> interface).

Hmmm... maybe I should even disallow ")edit". Oh... no need... even in
the console fricas it says.

(1) -> )edit "foo.input"

Warning: environment variable FRICASEDITOR not set.
Launching 'more' in an 'xterm'.
Press space to continue, 'q' to quit.

)edit is nowadays only necessary for people working completely in
ASCII-mode and do not now about Ctrl-Z to stop a process, do something
else (like editing) and come back into the fricas console. That pretty
much sounds like ")edit" can be removed, because I do not see a use case
anymore.

Anyway, with

(sb-ext::process-exit-code
(sb-ext::run-program "/bin/sh"
(list "-c" S) :input t :output *standard-output*
:error *error-output*))

From "FRICASEDITOR=emacs jfricas" I can perfectly fine issue the command
)edit "foo.input" in a jfricas notebook cell.

> Other use case are calls to few system commands where errors are
> _not_ expected. If some unexpected error pops up it is debatable
> where message should go, but resonable convention is that those
> errors should go to FRICASsys standard error.

Not in the case of jfricas. The jupyter notebook is the only visible
form of communication. I have no control over how fricaskernel.py (and
therefore FRICASsys) is started. No idea what jupyter does with stdout
and stderr and from where I could fetch that output.

> In particular, I not sure what you are doing with OBEY that you need
> to catch standard output?

I want ")sys pwd" to show me something. We have included ")!
some-shell-command" in jfricas, just because we didn't know how to catch
)sys output", but ")! pwd" and ")sys pwd" do not give the same result,
because ")!" always starts a new shell. Maybe with now having a way that
)sys can output something, ")!" should probably go away or be synonymous
to )sys.

> To put it differently, ATM you can launch text editor or a shell via
> ')system' command and it almost works (there is some breakage due to
> clef/rlwrap interfering, AFAICS without sman it works fine). This
> is not very important use case, but it is nice to keep what works
> working. OTOH in jFriCAS I would _not_ expect current ')system' to
> work, at least not without some alternative implementation.

With the modified OBEY, it works fine at least for what I have seen so
far. I do not claim that I have tested it well enough, but I am happy
that ")sys pwd" works. I do not want to do fancy stuff and when
eventually jFriCAS is mature enough and we can offer a server and a
webinterface to access fricas from the internet, we probably want to
turn off )sys and other dangerous things completely. But that shouldn't
concern us now in the local use case.

>> For jfricas it wouldn't probably matter since I anyway set
>> *error-output* to *standard-output*.
>>
>> In fact, browsing through the FriCAS code, it seems that
>> *error-output* is only used in the lisp optimizer.
>
> Well, currently when we generate Lisp code Lisp '*standard-output*'
> is redirected to file. But we definitely do not want error messages
> going to '*standard-output*': some messages are just warning, but
> if emmited to file would make it syntactiacally incorrect.

Hmmm... here the question is when does the optimizer jump in? Only at
the time when fricas is build or also when a user defines a function and
that function is compiled at its first use? Only the second case would
be problematic. For the first we could let OBEY be as it is now.

> User normally does not look at generated file so error messages
> emmited to file are effectively lost.

In the beginning jfricas collected output from different streams in
different stores. But that was a problem with the order in which jfricas
would show the total output, because a cell can potentially contain
multiline input like

foo1
)set output algebra off
)set output tex on
foo2
)sys pwd
)set output algebra on
)set output tex off
foo3

I wouldn't want to see the output of foo1 and foo3 before the output of
foo2 in tex. So everything in jfricas goes to *standard-output*, but
comes with begin and end markers that tell fricaskernel.py in which
format it should be presented. Maybe not perfect, but works.

> BTW: As I wrote Saturn (Texexplorer) had notebook interface. While
> code supporting Saturn is removed now you can look at past versions
> to see how they handled tricky problems. In particular
> i-output.boot was modified to behave as needed by Saturn. IIRC this
> included TeX output of types and handling of line numbers.

Tex output is completely uninteresting for jfricas. We have
FormatMathJax for that.

> BTW2: My impression is that in Saturn "unexpected" error messages
> went to AXIOMsys console.

The question is what is the "AXIOMsys console"? Other question is when I
type ")sys foo" and get an error message like "/bin/sh: 1: foo: not
found", then I wouldn't count that as an error of the ")sys" command,
but rather a valid output of what I asked for. If ")sys" itself or any
other comman finds weird behaviour and wants to report (to FriCAS
developers), then it is questionable where this should go. But I would
even argue that it should be visible to the user with clear message that
he/she should report to the fricas-devel list.

Ralf

[Waldek Hebisch]

On Thu, May 05, 2022 at 08:37:27AM +0200, Ralf Hemmecke wrote:
> >Looking into 'sb-ext::run-program' docs shows that 't' means inheriting
> >standard input, output and error output from current process, which at
> >least in some cases seem to be right thing.
>
> The current process is FRICASsys. So it goes to the stdout and stderr of
> whatever FRICASsys gets. Do I understand this correctly?

Yes. On my machine current FRICASsys has process number 7985.
Doing 'ls -l /proc/7985/fd' I see:

lrwx------ 1 hebisch hebisch 64 May 5 11:04 0 -> /dev/pts/2
lrwx------ 1 hebisch hebisch 64 May 5 11:04 1 -> /dev/pts/2
lrwx------ 1 hebisch hebisch 64 May 5 11:04 10 -> socket:[22234]
lrwx------ 1 hebisch hebisch 64 May 5 11:04 11 -> socket:[22235]
lrwx------ 1 hebisch hebisch 64 May 5 11:04 12 -> socket:[24271]
lrwx------ 1 hebisch hebisch 64 May 5 11:03 2 -> /dev/pts/2
lrwx------ 1 hebisch hebisch 64 May 5 11:04 3 -> /dev/tty

So standard error, that is descriptor 2 is connected to pseudotrminal
/dev/pts/2. You can also see that on file descriptor 3 sbcl tries
do dig real terminal. Preferably nothing should do to file
descriptor 3, while 2 is fine.

I make almost no use of ')system' (and the same for ')edit'). However
this is no reason to break them.

> >Other use case are calls to few system commands where errors are
> >_not_ expected. If some unexpected error pops up it is debatable
> >where message should go, but resonable convention is that those
> >errors should go to FRICASsys standard error.
>
> Not in the case of jfricas. The jupyter notebook is the only visible
> form of communication. I have no control over how fricaskernel.py (and
> therefore FRICASsys) is started. No idea what jupyter does with stdout
> and stderr and from where I could fetch that output.

I would hope that there is some way, at least for debugging. But
I am not going to search for this.

> >In particular, I not sure what you are doing with OBEY that you need
> > to catch standard output?
>
> I want ")sys pwd" to show me something. We have included ")!
> some-shell-command" in jfricas, just because we didn't know how to catch
> )sys output", but ")! pwd" and ")sys pwd" do not give the same result,
> because ")!" always starts a new shell. Maybe with now having a way that
> )sys can output something, ")!" should probably go away or be synonymous
> to )sys.
>
> >To put it differently, ATM you can launch text editor or a shell via
> > ')system' command and it almost works (there is some breakage due to
> > clef/rlwrap interfering, AFAICS without sman it works fine). This is not
> >very important use case, but it is nice to keep what works working. OTOH
> >in jFriCAS I would _not_ expect current ')system' to work, at least not
> >without some alternative implementation.
>
> With the modified OBEY, it works fine at least for what I have seen so
> far. I do not claim that I have tested it well enough, but I am happy
> that ")sys pwd" works. I do not want to do fancy stuff and when
> eventually jFriCAS is mature enough and we can offer a server and a
> webinterface to access fricas from the internet, we probably want to
> turn off )sys and other dangerous things completely. But that shouldn't
> concern us now in the local use case.

For the record: you can do whatever dangerous you like just at
Spad/Lisp level. You would have to disable quite a lot of things
and do seriouos security audit to allow untrusted user at FriCAS
command prompt. OTOH I do not see if ')system' makes sense via
web. And clearly in remote case ')edit' needs different
implementation.

> >>For jfricas it wouldn't probably matter since I anyway set
> >>*error-output* to *standard-output*.
> >>
> >>In fact, browsing through the FriCAS code, it seems that *error-output*
> >>is only used in the lisp optimizer.
> >
> >Well, currently when we generate Lisp code Lisp '*standard-output*' is
> >redirected to file. But we definitely do not want error messages
> > going to '*standard-output*': some messages are just warning, but
> >if emmited to file would make it syntactiacally incorrect.
>
> Hmmm... here the question is when does the optimizer jump in? Only at
> the time when fricas is build or also when a user defines a function and
> that function is compiled at its first use? Only the second case would
> be problematic. For the first we could let OBEY be as it is now.

Main case is Spad compilation (which somewhat advanced user is
supposed to do). The same my apply to situations when w generate
output to files. Arguably we should not do this, but ATM main
way to get output in desired place (including files) is via
redirection. Fortunately, OBEY is now used less than in the
past so normally there should be no OBEY in middle of compilaton".
OTOH uncheckable conditions like "you should not call OBEY in
middle of compilaton" are not nice, so using *error-output*
for OBEY _normally_ is better.

> >User normally does not look at generated file so error messages
> >emmited to file are effectively lost.
>
> In the beginning jfricas collected output from different streams in
> different stores. But that was a problem with the order in which jfricas
> would show the total output, because a cell can potentially contain
> multiline input like
>
> foo1
> )set output algebra off
> )set output tex on
> foo2
> )sys pwd
> )set output algebra on
> )set output tex off
> foo3
>
> I wouldn't want to see the output of foo1 and foo3 before the output of foo2
> in tex. So everything in jfricas goes to *standard-output*, but comes with
> begin and end markers that tell fricaskernel.py in which format it should be
> presented. Maybe not perfect, but works.

OK.

> >BTW: As I wrote Saturn (Texexplorer) had notebook interface. While code
> >supporting Saturn is removed now you can look at past versions to see how
> >they handled tricky problems. In particular
> >i-output.boot was modified to behave as needed by Saturn. IIRC this
> >included TeX output of types and handling of line numbers.
>
> Tex output is completely uninteresting for jfricas. We have FormatMathJax
> for that.

Well, one who does not read code written by others will repeat their
mistakes.

On most crude level if you looked at Saturn code you will see that
there were a lot of conditionals, code would do sligthly different
thing for Saturn. In particular you should have flag signaling
other part of FriCAS that it runs for jFriCAS (probably you already
have it). If you want different behaviour from OBEY for jFriCAS,
than test flag in OBEY and do what needed.

> >BTW2: My impression is that in Saturn "unexpected" error messages went to
> >AXIOMsys console.
>
> The question is what is the "AXIOMsys console"? Other question is when I
> type ")sys foo" and get an error message like "/bin/sh: 1: foo: not found",
> then I wouldn't count that as an error of the ")sys" command, but rather a
> valid output of what I asked for. If ")sys" itself or any other comman finds
> weird behaviour and wants to report (to FriCAS developers), then it is
> questionable where this should go. But I would even argue that it should be
> visible to the user with clear message that he/she should report to the
> fricas-devel list.

As I wrote, ')system' is somewhat special. Arguably tying its output
to Lisp *standard-output* is wrong. There is more subtle question of
tying error output of ')system' to FRICASsys standard output, but
Lisp *standard-output* can point to quite different place than
FRICASsys standard output (or place where you are trying to redirect
*standard-output*). But if jFriCAS can not implement 'OBEY' as
intended, than your approach is reasonable to get going. However
this should be condidtional to jFriCAS.

Sorry that my message got so long. Main point is that IMO redirection
that you want for jFriCAS ATM should be conditional.

--
Waldek Hebisch

[Ralf Hemmecke]

> Main point is that IMO redirection that you want for jFriCAS ATM
> should be conditional.

Just very short answer. This is in the current state the case, via
patching OBEY at the time a jfricas notebook is opened (i.e. a new
FRICASsys process is started).

I can perfectly live with having OBEY redirecting only conditionally if
I prepare a patch for inclusion of the jfricas part into the FriCAS repo.

Thank you for your opinions and help.

Ralf

[Qian Yun]

Hi Nasser,

I just remembered that there is a change that may have huge effects
on your CAS integration tests.

That is about kernel caching.

There is a change after 1.3.7 that fixes a bug, that improves accuracy
but deceases performance.
[commit 56a678d4c43f88bade75d0072f8f44487adb2206]

Also on the same subject, this kernel caching mechanism may heavily
affect benchmark test -- simply put, run a test that computes 5000
integrals will take much longer time than computes each integral
individually. Because of kernel caching, if you compute 5000 integrals,
in the end there might be hundreds of thousands kernels sitting in
cache, making integration (and all other expression operation) very
slow (because of most time spent on kernel look up, which requires
expression comparison, which are expensive and unnecessary in this case)

So, if in your test, FriCAS needs to compute many integrals in one
batch, I strongly advise you to clear the kernel cache after each
integration or 10 integrations. This may very likely gives a huge
performance boost to the result. (The accuracy percentage may also
improve very very slightly as well.)

So although FriCAS will not have a release soon, I hope you can
use FriCAS from git or just with this kernel cache patch, and together
with "clear kernel cache after a few integrals" advice, in your
next CAS integration tests, I'm sure the results will be interesting.

You can contact me or let's discuss here for technical details to make
this happen.

- Qian

On 5/4/22 03:49, 'Nasser M. Abbasi' via FriCAS - computer algebra system
wrote:

> --
> You received this message because you are subscribed to the Google
> Groups "FriCAS - computer algebra system" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to fricas-devel...@googlegroups.com
> <mailto:fricas-devel...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/fricas-devel/56017115-cacf-4079-9454-5b2b75d6352fn%40googlegroups.com
> <https://groups.google.com/d/msgid/fricas-devel/56017115-cacf-4079-9454-5b2b75d6352fn%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Waldek Hebisch]

On Thu, May 12, 2022 at 11:13:45AM +0800, Qian Yun wrote:
> Hi Nasser,
>
> I just remembered that there is a change that may have huge effects
> on your CAS integration tests.
>
> That is about kernel caching.
>
> There is a change after 1.3.7 that fixes a bug, that improves accuracy
> but deceases performance.
> [commit 56a678d4c43f88bade75d0072f8f44487adb2206]

I expect this to have modest impact of test results, almost
surely less than 1 per hundred and most likely less than one
in thoushends. One reason that this bug was unfixed
for so long is due to difficulty of repreducing it...

> Also on the same subject, this kernel caching mechanism may heavily
> affect benchmark test -- simply put, run a test that computes 5000
> integrals will take much longer time than computes each integral
> individually.

IIUC currently Nasser is running each test is separate copy
of FriCAS. This causes longer time because each integral
pays startup cost.

> Because of kernel caching, if you compute 5000 integrals,
> in the end there might be hundreds of thousands kernels sitting in
> cache, making integration (and all other expression operation) very
> slow (because of most time spent on kernel look up, which requires
> expression comparison, which are expensive and unnecessary in this case)

IME main bottlenecks are series expansion and limits: they use
a lot of kernel lookups and create a lot of kernels. This
affects definite integration, but impact on indefinit integration
seem to be smaller. Also, Rubi testsuite have quite limited
coverage when viewed in terms of kernes: it repeats the same
kernels in many examples. It is not clear to me how many
kernels would be created if one tried to run the whole testsuite
as a single run.

> So, if in your test, FriCAS needs to compute many integrals in one
> batch, I strongly advise you to clear the kernel cache after each
> integration or 10 integrations. This may very likely gives a huge
> performance boost to the result. (The accuracy percentage may also
> improve very very slightly as well.)

It certainly would be intersting to run testsuite in this way.
In the past clearing cache exposed a bug (fixed now). Pushing
harder may uncover more bugs.

--
Waldek Hebisch

[Nasser M. Abbasi]

Hello   Qian;

If I understand you, you are recommending that each call to integrate 

to Fricas from sagemath be done by first clearing all kernel cache in Fricas first?

Well, it happens that this is what CAS integration test does. Because, sagemath

does not have a direct way to do a timeout on an integrate call when calling

external CAS process (I need to look at their pexpect interface to see if

that works with Fricas. I never used it before.  

So what CAS integration test does now is spawn a new process for each call to integrate

and sets a timeout on the subprocess itself. This works.

Here I show one sagemath script to test one integral. I also post the code for the script

below and a link to it as well. If you have Sagemath and Fricas installed on your PC,

then you can do

```

>cd  CAS_integration_tests/
>cd reports/
>cd summer_2022/

>sage ./test_one_integral_in_sage_direct.sage
Test starting.....
after process start()..waiting to finish
inside doTheIntegration integrand= sin(x)
inside doTheIntegration x= x
inside subprocess. integrate returned  -cos(x)
process.join completed....
process completed in time. Read the result
['-cos(x)']
finished, closing file....
```

The script test_one_integral_in_sage_direct.sage is

```

#!/usr/bin/env sage

#script to test one integral for Fricas, giac and maxima
#spawns a process for each integral call, in order to do
#timeout since sagemath does not directly supports putting
#a timeout on a call to integrate.

import os, sys, time, datetime, ntpath
import multiprocessing as mp
from sage.all import *
           
def doTheIntegration(THE_INPUT):
   
    problem = THE_INPUT.get()        
    try:        
        print("inside doTheIntegration integrand=", problem[0])
        print("inside doTheIntegration x=", problem[1])

        #change algorithm to "giac" or "maxima" to test others

        anti = integrate(problem[0],problem[1],algorithm="fricas")

        print("inside subprocess. integrate returned ", anti)
        resultBack = [str(anti)]                

    except Exception as ee:
        anti = "Exception raised: " + type(ee).__name__ +  " >> " +ee.args[0]
        print(anti)
        resultBack = [anti]                            
       
    THE_INPUT.put(resultBack)

if __name__ == "__main__":

    #in actual program, this is read from a file by main()
    #which sets the variables. Then it calls subprocess to do
    #the integration.
    x = var('x')
    variable = x
    integrand = "sin(x)"
   
    mp.set_start_method('spawn')    
    print("Test starting.....")
   
    theQueue = mp.Queue()
    theQueue.put([integrand,str(variable)]) #integrand, variable, optimal
    process = mp.Process(group=None,target=doTheIntegration, args=(theQueue,))  
    process.start()                            
    print("after process start()..waiting to finish")                      

    #changed to 4 minutes to compensate for extra time
    #needed to start subprocess for each call. Keep all other
    #CAS systems (Maple, Mathematica, mupad, Rubi, Sympy) at
    #3 minutes since those are called directly.
                                   
    process.join(4*60)

    print("process.join completed....")  

    if process.exitcode == None or process.is_alive():
        print ("process did not terminate. Kill it. Timed out")
        process.terminate()
    else:
        print ("process completed in time. Read the result")
        anti = theQueue.get()                                                        
        print(anti)

    del(theQueue)              
    print ("finished, closing file....")
 ```

Here is a link to the above. test_one_integral_in_sage_direct.sage

So based on what you said, to clear the cache before each call, the above does this

already. So it is OK to test Fricas like this?

In the next test (summer 2022) I added one minute to Fricas, Maxima and Giac to 

compensate for the extra overheads of spawning new process each time to be fair

with other CAS systems which are called directly,

--Nasser

[Nasser M. Abbasi]

Fyi,

Using just released sagemath 9.6 RC 4,  the sagemath  pexpect interface seems 

to allow timeout and it works on initial tests.

Since it is much simpler than using spawn and multiprocessing, then CAS integration

test will use it for summer 2022 test report.

Here is an example show it works

```

#!/usr/bin/env sage

import os, sys, time, datetime, ntpath

from sage.all import *
           
if __name__ == "__main__":

    x = var('x')

    integrand = sin(x)
    fricas(2) #just to start fricas and attach to it

    fricas._expect.timeout = 0.0000001 #set time out
    
    try:                
        anti = integrate(integrand,x,algorithm="fricas")
        print("did not time out. Result is ",anti)
    except Exception as ee:
        print("Exception raised! " + type(ee).__name__)

```

>sage ./test_one_integral_in_sage_direct_not_using_spawn.sage
Exception raised! TIMEOUT
>

Changing the timeout to 60 now gives

>sage ./test_one_integral_in_sage_direct_not_using_spawn.sage
did not time out. Result is  -cos(x)

Using the above makes the rest script simpler (this will be done in a loop).

But now the issue is with clearing the fricas kernel cache which you should

will produce better result.

How would this be done from sagemath script above? I now do not know.

I could ask the sagemth folks if they know and if it is possible to do this

via from sagemath. There might be a way to kill the attached fricas process

and start new one before each call, all from within the script. This should have

the effect of clearing the cache ofcourse since new Fricas process is

started for each call.

If there is no need to clear the Fricas cache repeatedly then it will make

things simpler but you are implying that one will be get better result if

this is done.

--Nasser

[Nasser M. Abbasi]

An update. .After testing sagemath  pexpect more for timeout with giac and maxima, I found it does not work with these CAS systems.

It worked with Fricas only.  But since I want to use same script for all, will stick to current

method I am using, which is to use spawn and multiprocessing for timeout. Even though it 

is more complicated, but it works for all of them.

This also has the side advantage that it starts new Fricas process for each problem, so

do not have to worry about clearing the cache.

--Nasser

[Waldek Hebisch]

On Thu, May 12, 2022 at 07:52:25PM -0700, 'Nasser M. Abbasi' via FriCAS - computer algebra system wrote:
> Fyi,
>
> Using just released sagemath 9.6 RC 4, the sagemath pexpect interface
> seems
> to allow timeout and it works on initial tests.

BTW: Developement version of FriCAS has preliminary implementation
of timeouts. It works only with sbcl, but IIUC you use sbcl
and sbcl is best for performance (and extra features).

There are FriCAS commands:

)clear scaches
)clear all

First clears kernel cache and few other caches. The second primarlly
is intended "refersh" FriCAS, that is forget various things like
variable settings but also clear caches.

OTOH it is not clear for me if you need clearing. On 10000 random
exp-log examples run as single file FriCAS needs to create a
lot of kernels and can do the integrals in 3 minutes (on a fast machine).

IIUC you run each file separately and other files have
smaller nuber of example than 10000 file.

--
Waldek Hebisch

[Qian Yun]

Hi Nasser,

First, so you are already starting a new process for each integral.
Then there's no problem of slow down caused by big kernel cache.

Second, about the new patch. I wonder if you are running Sage
on Windows or Linux? Is it easy to replace the FriCAS binary?
Also, IIRC, the Sage compiles FriCAS with ecl instead of sbcl.
Using sbcl may give a good performance boost. Also building
with GMP may increase performance slightly. With better
performance, we can expect less timeout results.

- Qian

On 5/13/22 08:09, 'Nasser M. Abbasi' via FriCAS - computer algebra

system wrote:
> Hello  Qian;
>
> If I understand you, you are recommending that each call to integrate
> to Fricas from sagemath be done by first clearing all kernel cache in
> Fricas first?
>
> Well, it happens that this is what CAS integration test does. Because,
> sagemath
> does not have a direct way to do a timeout on an integrate call when calling
> external CAS process (I need to look at their pexpect interface

> <https://groups.google.com/g/sage-devel/c/9_VdKmiu5Ts/m/mcUtUWOKAAAJ> to

> <https://12000.org/tmp/sagemath_test_script/test_one_integral_in_sage_direct.sage>

> tests](https://www.12000.org/my_notes/CAS_integration_tests/index.htm <https://www.12000.org/my_notes/CAS_integration_tests/index.htm>)

>
> > when sagemath 9.6 is released which should happen in the next few
> weeks.
> >
> > I was wondering if Fricas next version will be released any time
> this
> > summer, so I can hold on and wait for that, or if I should use 1.3.7
> > from last year.
> >
> > I know it is not possible to predict a release time, just asking for
> > possible time frame.
> >
> > Thanks
> > --Nasser
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "FriCAS - computer algebra system" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send
> > an email to fricas-devel...@googlegroups.com
> > <mailto:fricas-devel...@googlegroups.com>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/fricas-devel/56017115-cacf-4079-9454-5b2b75d6352fn%40googlegroups.com
> <https://groups.google.com/d/msgid/fricas-devel/56017115-cacf-4079-9454-5b2b75d6352fn%40googlegroups.com>
>
> >

> <https://groups.google.com/d/msgid/fricas-devel/56017115-cacf-4079-9454-5b2b75d6352fn%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/fricas-devel/56017115-cacf-4079-9454-5b2b75d6352fn%40googlegroups.com?utm_medium=email&utm_source=footer>>.

>
>
> --
> You received this message because you are subscribed to the Google
> Groups "FriCAS - computer algebra system" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to fricas-devel...@googlegroups.com
> <mailto:fricas-devel...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/fricas-devel/7b770128-5553-407a-8e60-dc2f1f028f5fn%40googlegroups.com
> <https://groups.google.com/d/msgid/fricas-devel/7b770128-5553-407a-8e60-dc2f1f028f5fn%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Qian Yun]

Hi Nasser,

From your reply, seems you are using pexpect (aka one session for
all integrals) instead of multiprocessing (aka one session for one
integral).

So the way to clear kernel cache is to just pass ")clear all" to
FriCAS. If the unbalanced ")" causes trouble, use "clearCmdAll()$Lisp"
instead. Run this after each integral or a few dozen integrals.

- Qian


On 5/13/22 10:52, 'Nasser M. Abbasi' via FriCAS - computer algebra

> <https://groups.google.com/g/sage-devel/c/9_VdKmiu5Ts/m/mcUtUWOKAAAJ> to

> <https://12000.org/tmp/sagemath_test_script/test_one_integral_in_sage_direct.sage>

> <https://www.12000.org/my_notes/CAS_integration_tests/index.htm>)

> > when sagemath 9.6 is released which should happen in the next
> few weeks.
> >
> > I was wondering if Fricas next version will be released any
> time this
> > summer, so I can hold on and wait for that, or if I should
> use 1.3.7
> > from last year.
> >
> > I know it is not possible to predict a release time, just
> asking for
> > possible time frame.
> >
> > Thanks
> > --Nasser
> >
> > --
> > You received this message because you are subscribed to the
> Google
> > Groups "FriCAS - computer algebra system" group.
> > To unsubscribe from this group and stop receiving emails from
> it, send
> > an email to fricas-devel...@googlegroups.com
> > <mailto:fricas-devel...@googlegroups.com>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/fricas-devel/56017115-cacf-4079-9454-5b2b75d6352fn%40googlegroups.com

> <https://groups.google.com/d/msgid/fricas-devel/56017115-cacf-4079-9454-5b2b75d6352fn%40googlegroups.com>
>
> >
> <https://groups.google.com/d/msgid/fricas-devel/56017115-cacf-4079-9454-5b2b75d6352fn%40googlegroups.com?utm_medium=email&utm_source=footer
> <https://groups.google.com/d/msgid/fricas-devel/56017115-cacf-4079-9454-5b2b75d6352fn%40googlegroups.com?utm_medium=email&utm_source=footer>>.
>
>

> --
> You received this message because you are subscribed to the Google
> Groups "FriCAS - computer algebra system" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to fricas-devel...@googlegroups.com
> <mailto:fricas-devel...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/fricas-devel/02d35b66-02ae-492e-bef4-ff63979cb916n%40googlegroups.com
> <https://groups.google.com/d/msgid/fricas-devel/02d35b66-02ae-492e-bef4-ff63979cb916n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Ralf Hemmecke]

> So the way to clear kernel cache is to just pass ")clear all" to
> FriCAS.  If the unbalanced ")" causes trouble, use "clearCmdAll()$Lisp"
> instead.

https://fricas.github.io/api/MoreSystemCommands.html?highlight=moresystemcommand

I thought the preferred non-lispy way is

systemCommand("clear all")$MoreSystemCommands

Ralf

[Ralf Hemmecke]

On 13.05.22 09:06, Qian Yun wrote:

> Also, IIRC, the Sage compiles FriCAS with ecl instead of sbcl.

Well, Sage seems to be smart. I have installed sage via apt-get in
ubuntu 18.04 and no sage package for fricas is installed.

When I type

integrate(sin(x),x,algorithm='fricas')

it gives an error if the fricas script was not in the PATH when I
started sage. Otherwise after calling "integrate" there is then a fricas
process running which points to the path where I installed an sbcl fricas.

Ralf

[Dima Pasechnik]

yes, this is as expected - Sage's fricas package builds fricas using
ecl, and installs it somewhere.
The interface is already there, regardless of fricas package being
installed, or not.

Dima
>
> Ralf

>
> --
> You received this message because you are subscribed to the Google Groups "FriCAS - computer algebra system" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to fricas-devel...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/fricas-devel/f4f7f6d6-f07c-a9ed-d247-acc431656ad8%40hemmecke.org.

[Nasser M. Abbasi]

>>>>

Second, about the new patch. I wonder if you are running Sage
on Windows or Linux? Is it easy to replace the FriCAS binary?
Also, IIRC, the Sage compiles FriCAS with ecl instead of sbcl.
Using sbcl may give a good performance boost. Also building
with GMP may increase performance slightly. With better
performance, we can expect less timeout results.

>>>>>

Hello Qian;

When running CAS integration tests, I use Fricas build directly from sources outside of sagemath. 

Sources from 

https://sourceforge.net/projects/fricas/files/fricas/1.3.7/fricas-1.3.7-full.tar.bz2/download

I run everything on Linux. Now using WSL 2.0 under windows 10 Linux subsystem

```

>which fricas
/usr/local/bin/fricas
>fricas --version
FriCAS 1.3.7
based on sbcl 2.0.1.debian
```

If I wanted to use Fricas that builds by sagemath, I would do, inside sagemath

./configure --enable-fricas

And this gives when done the following

 ```

>which fricas
/home/me/TMP/sage-9.6.rc4/local/bin/fricas
>fricas --version
FriCAS 1.3.7
based on ecl 21.2.1
```

So it is the first one above I will use for CAS integration tests. Not the version that comes with sagemath.

I have to check if Fricas was build with GMP or not. I just followed the instructions to build fricas

from source, and do not know now how to check if Fricas was build with GMP or not. But next

time I build Fricas from source I look to see if there is an option for this.

Thanks.

--Nasser

[Ralf Hemmecke]

Use ./configure --with-gmp ...

https://github.com/fricas/fricas/blob/master/configure.ac#L321

See also the installation guide

https://fricas.github.io/install.html#gmp-optional
https://fricas.github.io/install.html#using-gmp-with-sbcl-or-clozure-cl

Ralf

[oldk1331]

Hi Nasser:

Since you are building FriCAS from source,

so for next CAS integration tests, will you

use git version instead of 1.3.7?

- Qian

To unsubscribe from this group and stop receiving emails from it, send an email to fricas-devel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fricas-devel/ef9198fc-422b-4066-9040-3825e02cd5dbn%40googlegroups.com.

[Ralf Hemmecke]

On 04.05.22 03:23, Waldek Hebisch wrote:
> On Tue, May 03, 2022 at 11:28:58PM +0200, Ralf Hemmecke wrote:
>> I am currently working on the jfricas interface.
>> Unfortunately, there are some rather unsafe elements in it,
>> I must patch the functions |mathprintWithNumber| and OBEY
>>
>> https://github.com/hemmecke/jfricas/blob/master/jfricas/webspad.lisp#L84
>>
>> ;;; Override a FriCAS function in order to remove the equation number.
>> (DEFUN |mathprintWithNumber| (|x|)
>> (PROG ()
>> (RETURN
>> (PROGN
>> ;(|ioHook| '|startAlgebraOutput|)
>> (|saySpadMsg| '|--FORMAT:BEG:Algebra|)
>> (|maprin| (|outputTran2| |x|))
>> (|saySpadMsg| '|--FORMAT:END:Algebra|)
>> ;(|ioHook| '|endOfAlgebraOutput|)
>> ))))
>
> I do not see why you are doing this. If you want to call
> 'saySpadMsg' appropriate io hook should be enough. For
> skipping line numbers setting '$IOindex' to NIL should
> be enough.

Well, not so easy. ;-) I could probably do this with the ioHook, but
the Jupyter interface also has numbers for input/output. I use $IOindex
to syncronize FriCAS step numbers with Jupyter numbers. Setting $IOindex
to nil is thus counterproductive.

Maybe I could change the |mathprintWithNumber| in FriCAS to depend on a
flag that if not set prevents outputting the equation number.
Sounds a bit strang, given the name ...withNumber.

mathprintWithNumber x ==
ioHook("startAlgebraOutput")
x:= outputTran2 x
maprin
$IOindex => ['EQUATNUM,$IOindex,x]
x
ioHook("endOfAlgebraOutput")

Must probably look deeper for an appropriate solution.

Ralf

[Waldek Hebisch]

You mean you read $IOindex?

> Maybe I could change the |mathprintWithNumber| in FriCAS to depend on a flag
> that if not set prevents outputting the equation number.

Using flag is quite reasonable.

> Sounds a bit strang, given the name ...withNumber.
>
> mathprintWithNumber x ==
> ioHook("startAlgebraOutput")
> x:= outputTran2 x
> maprin
> $IOindex => ['EQUATNUM,$IOindex,x]
> x
> ioHook("endOfAlgebraOutput")

Well, as you see _not_ printing number is already part of behaviour.
More accurate name probaby would be 'mathprintMaybeWithNumber', but
in several other places we also do some actions conditionally,
without spelling out disclaimers inside name.

--
Waldek Hebisch

[Nasser M. Abbasi]

I will wait until an official Fricas release is made. This way I know what version I am actually using. 

It is no problem for CAS integration tests as I can always update the result when Fricas is released even if it take 6 months from now or a year. The tests take very long time to do anyway.

   

I am still struggling now to figure how to tell sagemath 9.6 to use latest versions of other CAS systems (giac and maxima) and so far, not getting any help how to do that. I need to resolve that issue first.

regards

--Nasser

[Ralf Hemmecke]

>>> I do not see why you are doing this. If you want to call
>>> 'saySpadMsg' appropriate io hook should be enough. For
>>> skipping line numbers setting '$IOindex' to NIL should
>>> be enough.
>>
>> Well, not so easy. ;-) I could probably do this with the ioHook, but
>> the Jupyter interface also has numbers for input/output. I use $IOindex to
>> syncronize FriCAS step numbers with Jupyter numbers. Setting $IOindex to nil
>> is thus counterproductive.
>
> You mean you read $IOindex?

Yes I do.

https://github.com/fricas/jfricas/blob/master/jfricas/webspad.lisp#L63

===================
(defun ws-fmt (marker)
(|sayMSG| (format nil "--FORMAT:~A:~D" marker boot::|$IOindex|)))

(setf |$ioHook|
(lambda (x &optional args)
(cond
; ((eq x '|startAlgebraOutput|) (ws-fmt "BEG:Algebra"))
; ((eq x '|endOfAlgebraOutput|) (ws-fmt "END:Algebra"))
((eq x '|startPatternMsg|) (ws-fmt "BEG:ERROR"))
((eq x '|endPatternMsg|) (ws-fmt "END:ERROR"))
....
===================

As long as nobody changes it inside a jfricas session, everything works
fine. Do you see a problem with that?
Any other option to read FriCAS' step number?

Ralf

[Waldek Hebisch]

Well, the intent clearly was that "printing" produces a bundle
containing step number and value and that separationg step number
and value should be easy. ATM it seems that you prefer separate
code path to transfer step number. I wonder, maybe adding markers
around step number would better.

To say the truth it looks somewhat strange that you want to
suppres printing of step number but want its value.

--
Waldek Hebisch

[Ralf Hemmecke]

> Well, the intent clearly was that "printing" produces a bundle

> containing step number and value,

The time this method was invented for FriCAS, the people that invented
jupyter where probably not yet born. The idea of having possibly another
UI for FriCAS was not on the screen. "Saturn" was an in-house product.

> and that separationg step number and value should be easy.

Can you do it for all cases? Just two are given below. "Easy" is
something else.

> To say the truth it looks somewhat strange that you want to
> suppres printing of step number but want its value.

The printing of the step number in the fricas algebra output consumes n
colums of space where n depends on the number of digits of the step
number. If the step number appeared in the beginning of the output, then
perhaps I could parse the output, extract the value, remove the columns
from the fricas output, and use it for jupyter. But it doesn't always

Reading $IOindex is definitely a simpler solution, given that the step
number cannot always be easily removed, see below. And I do not think
that it makes sense to print the step number in the FriCAS output (and
thus waste some output columns if it there is anyway a "Out[step]" in
jupyter, which should, of course agree with what FriCAS thinks the step
number is to make %%(stepnumber) work as expected.

Ralf

(124) -> i3 := [x^(-1) for x in d3]

+ 0 1+ + 1 1+ + 2 1+ +1 0+
(124) [| |, | |, | |, | |]
+- 1 0+ +- 1 0+ +- 1 0+ +0 1+

(125) -> m3 := [[x*y for y in i3] for x in r3]

(125)
+ 0 1+ + 1 1+ + 2 1+ +1 0+
[[| |, | |, | |, | |],
+- 1 0+ +- 1 0+ +- 1 0+ +0 1+
+ 1 0+ +1 0+ +1 0+ +0 - 1+ + 0 1+ +1 1+ +2 1+ +1 0+
[| |, | |, | |, | |], [| |, | |, | |, |
|],
+- 1 1+ +0 1+ +1 1+ +1 1 + +- 1 2+ +1 2+ +3 2+ +2 1+
+ 1 0+ + 1 0+ + 1 0+ +0 - 1+
[| |, | |, | |, | |]]
+- 3 1+ +- 2 1+ +- 1 1+ +1 3 +

[Waldek Hebisch]

On Thu, May 19, 2022 at 07:18:01AM +0200, Ralf Hemmecke wrote:
> >Well, the intent clearly was that "printing" produces a bundle
> >containing step number and value,
>
> The time this method was invented for FriCAS, the people that invented
> jupyter where probably not yet born. The idea of having possibly another UI
> for FriCAS was not on the screen. "Saturn" was an in-house product.
>
> >and that separationg step number and value should be easy.
>
> Can you do it for all cases? Just two are given below. "Easy" is something
> else.

I wrote "should". If it is not easy we need to change it. That
is why I wrote about possible new io hook (IIUC this is similar
to what you do). The point is that IMO handling of number should
be together with output. So, I prefer changing 'mathprintWithNumber'
to produce somthing acceptable to jFriCAS (as opposed to out of
bound chanell).

> >To say the truth it looks somewhat strange that you want to
> >suppres printing of step number but want its value.
>
> The printing of the step number in the fricas algebra output consumes n
> colums of space where n depends on the number of digits of the step number.
> If the step number appeared in the beginning of the output, then perhaps I
> could parse the output, extract the value, remove the columns from the
> fricas output, and use it for jupyter. But it doesn't always
>
> Reading $IOindex is definitely a simpler solution, given that the step
> number cannot always be easily removed, see below. And I do not think that
> it makes sense to print the step number in the FriCAS output (and thus waste
> some output columns if it there is anyway a "Out[step]" in jupyter, which
> should, of course agree with what FriCAS thinks the step number is to make
> %%(stepnumber) work as expected.

OK, you can live with text formatting, except that you do not like
how line number is formatted (you have jupyter place for line
numbers). Anyway, IIUC you want:
- have line number available to output it in jupyter way
- no line number in text output

AFAICS we can modify 'mathprintWithNumber' to do what you need.
I am not sure how general this should be. One way is to have
a flag signalling to 'mathprintWithNumber' to do what jFriCAS
needs. Another possiblity is to have more general flags like
"suppress step number in output".

--
Waldek Hebisch