Allegro Common Lisp and FriCAS

[Qian Yun]

Allegro Common Lisp 11.0 is released recently. 6 years after previous
release!

There's an evaluation version available for download.

I have played with it for a while, and with a bare minimal 3 lines
tweak, it can run fricas0!

Of course, full support will need a larger patch and require more
time.

- Qian

[Qian Yun]

Problems encountered so far:

Definition of 'suffixed_name' in src/lisp/primitives.lisp need to be
wrapped in (eval-when (:execute :compile-toplevel :load-toplevel) ... )

Need to rename 'exit-with-status' -> '|exit_with_status|' in
src/scripts/build_helper.

^L control character is used by 6 lines in src/interp/define.boot and
src/interp/i-map.boot

grep ^L *boot ## use ctrl-v ctrl-l to type this character

Replace it with another uncommon ascii symbol should be suffice.

- Qian

[Waldek Hebisch]

On Thu, Dec 28, 2023 at 06:22:29PM +0800, Qian Yun wrote:
> Problems encountered so far:
>
> Definition of 'suffixed_name' in src/lisp/primitives.lisp need to be
> wrapped in (eval-when (:execute :compile-toplevel :load-toplevel) ... )
>
> Need to rename 'exit-with-status' -> '|exit_with_status|' in
> src/scripts/build_helper.

Yes.

> ^L control character is used by 6 lines in src/interp/define.boot and
> src/interp/i-map.boot

What exactly is wrong with using ^L?

--
Waldek Hebisch

[Qian Yun]

; While FILE-COMPILING #'"An Anonymous Function" in #P"i-map.clisp"
; between file character positions 26476 (inclusively) and 26479
(exclusively):
Error: Invalid unmatched right parenthesis read.
[condition type: EXTRA-RIGHT-PAREN-ERROR]

Char position 26476~26479 is the last line of the following snippet.

- Qian

(COND ((NULL |newMap|)
(PROGN (|sayRemoveFunctionOrValue| |op|)
(|putHist| |op| '|alias| NIL |$e|)
'^L))

[Waldek Hebisch]

On Thu, Dec 28, 2023 at 09:10:54PM +0800, Qian Yun wrote:
> ; While FILE-COMPILING #'"An Anonymous Function" in #P"i-map.clisp"
> ; between file character positions 26476 (inclusively) and 26479
> (exclusively):
> Error: Invalid unmatched right parenthesis read.
> [condition type: EXTRA-RIGHT-PAREN-ERROR]
>
> Char position 26476~26479 is the last line of the following snippet.
>
> - Qian
>
> (COND ((NULL |newMap|)
> (PROGN (|sayRemoveFunctionOrValue| |op|)
> (|putHist| |op| '|alias| NIL |$e|)
> '^L))

Looks like a bug in Allegro printer: control characters should be
properly escaped.

--
Waldek Hebisch

[Qian Yun]

I can report this issue back to Franz Inc.

But we can workaround it easily, with a global gensym symbol, right?

BTW, about porting status to Allegro CL:

FFI is not done yet.

It now can build with pre-generated Lisp files.

It has trouble when bootstrapping daase, fails at different places
when compiling F*.spad. So I highly suspect that the default
265MB heap is not enough. And the free version apparently can't
change heap size.

It can dump a image, but not an executable (which is a feature for
enterprise version). So I have to use the "write a shell script"
trick of CCL (removed in 4825daa1).


Anyway, although not perfect, I think this would be a useful
addition for FriCAS to include it.

But before committing the changes, I'd like some cleanup first:

Move the definition of OBEY from vmlisp.lisp to fricas-lisp.lisp.
And use OBEY in places like "makedir".

For some short functions, reorganize

#+:LISPA(defun fun ...)
#+:LISPB(defun fun ...)
...

to

(defun fun
#+:LISPA(...)
#+:LISPB(...)
...
)

- Qian

[Waldek Hebisch]

On Fri, Dec 29, 2023 at 07:55:39AM +0800, Qian Yun wrote:
> I can report this issue back to Franz Inc.
>
> But we can workaround it easily, with a global gensym symbol, right?

Yes, using gensym is reasonable.

> BTW, about porting status to Allegro CL:
>
> FFI is not done yet.
>
> It now can build with pre-generated Lisp files.
>
> It has trouble when bootstrapping daase, fails at different places
> when compiling F*.spad. So I highly suspect that the default
> 265MB heap is not enough. And the free version apparently can't
> change heap size.
>
> It can dump a image, but not an executable (which is a feature for
> enterprise version). So I have to use the "write a shell script"
> trick of CCL (removed in 4825daa1).
>
>
> Anyway, although not perfect, I think this would be a useful
> addition for FriCAS to include it.

Well, it is not clear to me if it is useful. To explain a bit:
initial ports, in particular sbcl port discovered several bugs
in FriCAS, so it was worth doing if only to get rid of bugs.

For several years sbcl gives us best performance, ECL gives
best portablility (it runs on a lot of platforms), clisp
also is/was good for portablity but apparently is is not
as well maintained as ECL. GCL was main platform in open
source Axiom era so required only effort to keep it running.
And GCL runs reasonably well on Debian. Closure CL was
developed mainly on/for Macs, so there was potential for
better Mac OS support. Both Closure CL and Poplog have
very fast compilation. With sbcl (and probably also GCL
and ECL) it seems that about half of FriCAS compile time
is spent in Lisp compilation, so even if we made Spad
compiler infinitely fast, we could only double compilation
speed with sbcl. OTOH in Closure CL and Poplog Lisp
compile time is much lower (but Spad compiler runs slower,
so total build time is longer). I also mentioned that
I had/have some specific hope related to Poplog.

Concerning speed of algebra code, one way to gain better
speed is to use C code. In particular we could try to
generate C code from Spad. Here major issue is cooperation
with Lisp runtime system. IIUC it would be easiest using
ECL, harder with other Lisps.

Main thing is that there are benefits or at least hope
for benefits from various ports. It is not clear to
me what we gain from Allegro port. 256 Mb limit on
heap size means that it will be unsuitable for larger
computations. I consider 1Gb default limit of sbcl as
a problem and keep larger versions for serious use.
256 Mb limit will cause much more problem. And lack
of FFI is serious drawback. Of course this is your
time but my personal take was that I do not want to
work with "free" Allegro or Lispworks, simply I see
no gain. If somebody with "professional" licence
want to contribite a port I would accept it on
"as is basis" (that was case with Lispworks). But
working around limitations of "free" versions looks
pointless.

I would be interesting to know how fast Allegro is.
For compile time you could try RDEEFX or LODOF2.
For runtime most interesting tests uses lot of memory,
one could try Ralf example of finite field factorization,
it should have moderate memory use. Or try some
microbenchmarks.

> But before committing the changes, I'd like some cleanup first:
>
> Move the definition of OBEY from vmlisp.lisp to fricas-lisp.lisp.
> And use OBEY in places like "makedir".

Well, I would prefer less use of OBEY. First, to have less
dependence on external programs. Second, OBEY is most
brain damaged interface for running programs. It is convenient
in some simple cases, but it would be better to have something
in 'run-program' style, that is something taking list of arguments.
And if modifying it I would change name: OBEY is rather
uniformative. I would prefer something like 'run_shell_command'.

> For some short functions, reorganize
>
> #+:LISPA(defun fun ...)
> #+:LISPB(defun fun ...)
> ...
>
> to
>
> (defun fun
> #+:LISPA(...)
> #+:LISPB(...)
> ...
> )

That looks good for really short functions like current OBEY.

> - Qian
>
> On 12/28/23 21:39, Waldek Hebisch wrote:
> > On Thu, Dec 28, 2023 at 09:10:54PM +0800, Qian Yun wrote:
> > > ; While FILE-COMPILING #'"An Anonymous Function" in #P"i-map.clisp"
> > > ; between file character positions 26476 (inclusively) and 26479
> > > (exclusively):
> > > Error: Invalid unmatched right parenthesis read.
> > > [condition type: EXTRA-RIGHT-PAREN-ERROR]
> > >
> > > Char position 26476~26479 is the last line of the following snippet.
> > >
> > > - Qian
> > >
> > > (COND ((NULL |newMap|)
> > > (PROGN (|sayRemoveFunctionOrValue| |op|)
> > > (|putHist| |op| '|alias| NIL |$e|)
> > > '^L))
> >
> > Looks like a bug in Allegro printer: control characters should be
> > properly escaped.
> >
>

> --
> You received this message because you are subscribed to the Google Groups "FriCAS - computer algebra system" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to fricas-devel...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/fricas-devel/55886cbe-79e6-4ad3-af0c-b0ca5894457c%40gmail.com.

--
Waldek Hebisch

[Qian Yun]

On 12/29/23 10:41, Waldek Hebisch wrote:
>
> Well, it is not clear to me if it is useful. To explain a bit:
> initial ports, in particular sbcl port discovered several bugs
> in FriCAS, so it was worth doing if only to get rid of bugs.
>
> For several years sbcl gives us best performance, ECL gives
> best portablility (it runs on a lot of platforms), clisp
> also is/was good for portablity but apparently is is not
> as well maintained as ECL. GCL was main platform in open
> source Axiom era so required only effort to keep it running.
> And GCL runs reasonably well on Debian. Closure CL was
> developed mainly on/for Macs, so there was potential for
> better Mac OS support. Both Closure CL and Poplog have
> very fast compilation. With sbcl (and probably also GCL
> and ECL) it seems that about half of FriCAS compile time
> is spent in Lisp compilation, so even if we made Spad
> compiler infinitely fast, we could only double compilation
> speed with sbcl. OTOH in Closure CL and Poplog Lisp
> compile time is much lower (but Spad compiler runs slower,
> so total build time is longer). I also mentioned that
> I had/have some specific hope related to Poplog.

I think Lispworks has the fastest Lisp complication speed.
For the fricas0 usage, to compile interp/*lisp,

SBCL: 7.5s
CCL: 8.0s
CLISP: 8.9s
Allegro: 13.3s
Lispworks: 3.3s

> Concerning speed of algebra code, one way to gain better
> speed is to use C code. In particular we could try to
> generate C code from Spad. Here major issue is cooperation
> with Lisp runtime system. IIUC it would be easiest using
> ECL, harder with other Lisps.
>
> Main thing is that there are benefits or at least hope
> for benefits from various ports. It is not clear to
> me what we gain from Allegro port. 256 Mb limit on
> heap size means that it will be unsuitable for larger
> computations. I consider 1Gb default limit of sbcl as
> a problem and keep larger versions for serious use.
> 256 Mb limit will cause much more problem. And lack
> of FFI is serious drawback. Of course this is your
> time but my personal take was that I do not want to
> work with "free" Allegro or Lispworks, simply I see
> no gain. If somebody with "professional" licence
> want to contribite a port I would accept it on
> "as is basis" (that was case with Lispworks). But
> working around limitations of "free" versions looks
> pointless.
>
> I would be interesting to know how fast Allegro is.
> For compile time you could try RDEEFX or LODOF2.

For RDEEFX.spad:

SBCL: 9.01s
CCL: 21.86s
CLISP: 34.19s
Allegro: 8.64s
LispWorks: 16.40s

> For runtime most interesting tests uses lot of memory,
> one could try Ralf example of finite field factorization,
> it should have moderate memory use. Or try some
> microbenchmarks.
>

For factor p3:

SBCL: 15.92s (no FFI, with FFI, it is 3.25s)
CCL: 6.23s
CLISP: 41.31s
Allegro: 16.86s
Lispworks: 4.71s


There's no clear winner in all 3 tests. Allegro performance is not bad.

- Qian