make build with sbcl-2.4.0 work out of box

[Qian Yun]

With sbcl-2.4.0, we need to use

./configure --with-lisp="sbcl --dynamic-space-size 2048"

which is inconvenient and caused problems/confusion for a few
people already.

I wonder if there's a way to make "./configure" work again.

If we add "--dynamic-space-size 2048" to fricas_quiet_flags,
then we can't overwrite the dynamic-space-size anymore.

(Because we use
LISP_CMD = $(FRICAS_LISP) $(fricas_quiet_flags)
the options in fricas_quiet_flags will overwrite options in FRICAS_LISP)


The only way I can think of is to add a new configure option:
--with-lisp-option.

Any ideas?

- Qian

[Dima Pasechnik]

it should be easy to basically let ./configure convert --with-lisp=sbcl into
--with-lisp="sbcl --dynamic-space-size 2048"

>
> - Qian
>
> --
> You received this message because you are subscribed to the Google Groups "FriCAS - computer algebra system" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to fricas-devel...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/fricas-devel/6a77b463-e750-40f5-bc67-f7237eddee4c%40gmail.com.

[Ralf Hemmecke]

> it should be easy to basically let ./configure convert --with-lisp=sbcl into
> --with-lisp="sbcl --dynamic-space-size 2048"

Unfortunately, Qian is right --dynamic-space-size must be configurable.
In fact, I would even be much more happy, if I could add

--dynamic-space-size SOMETHING

later, namely, when I start a new FriCAS session. I have computations
that fail even with a value of 10Gb.

Ralf

[Qian Yun]

Actually, you can:

./FRICASsys --dynamic-space-size 20000

OR

fricas -nosman --dynamic-space-size 20000

It not possible with sman though.

- Qian

[Ralf Hemmecke]

Qian,

On 1/16/24 13:39, Qian Yun wrote:
> Actually, you can:
>
> ./FRICASsys --dynamic-space-size 20000
>
> OR
>
> fricas -nosman --dynamic-space-size 20000
>
> It not possible with sman though.

That's enough for me.

So the actual problem is calling the sman version of fricas.

Cannot that be solved by enabling a special sbcl option to the "fricas"
script. That would set an environment variable with the
dynamic-space-size that is picked up (if set) by FRICASsys.

Ralf

[Qian Yun]

The problem is to differentiate which options are passed
to the underlying Lisp and which options are passed to FriCAS.

Add an option like "-lisp-args" in "fricas" shell script would work.

- Qian

[Waldek Hebisch]

On Tue, Jan 16, 2024 at 08:58:46PM +0800, Qian Yun wrote:
> The problem is to differentiate which options are passed
> to the underlying Lisp and which options are passed to FriCAS.
>
> Add an option like "-lisp-args" in "fricas" shell script would work.

In the past there were proposal to use '--' in sman to separate
options passed to main executable (that is FRICASsys) and
options handled by sman. I am not sure if this particular way
is the best way, but we can easily add apropriate code to sman so
that it would know which options are meant for FRICASsys.

Then fricas script could pass option in appropriate way.

In the past there were troubles with clearly specifying what
this mechanizm should do. Namely, some options are really
meant for Lisp implementation, the rest would be for FriCAS
itself. So logically we have 3 groups of options:

1 -- options for Lisp
2 -- options for FriCAS
3 -- options handled by sman or fricas script

I bundled options handled by sman and by fricas script into
one group because clearly both must be developed together and
must coordinate use of options.

There is some trouble with the first two groups of options.
Options for Lisp are outside our control and potentially in
the future there can appear ne ones. And each Lisp has its
own peculiar set of options. So it would be nice to treat
this in open ended way, that is allowing arbitrary options.

Similarly, in case of FRICASsys in principle users could take
advantage of options so again it would be nice to allow
arbitrary options.

Currently we fake passing options to FRICASsys by use of environment
variables, but this has some disadvantages compared to options.

Supportiong arbitrary options (and options with arbitrary arguments!)
requires some way of escaping. This could make fricas command
potentially quite complicated.

Anyway, we could do one of the two:
- design escaping rules which "always" work
- limit option handling to single option, that is
'--dynamic-space-size xxx' for sbcl

The second looks easier, but once there is one option it is
likely that somebody will want sencond one and so on. So
we may end up with bunch of special purpose hacks, which
together in the long run may be more problematic than general
solution.

--
Waldek Hebisch

[Qian Yun]

I think Dima gives a good advice, WDYT, Waldek?

- Qian

diff --git a/configure.ac b/configure.ac
index 073f3e96..99426fb1 100644
--- a/configure.ac
+++ b/configure.ac
@@ -190,13 +190,16 @@ if test -z "$fricas_lisp" ; then
use --with-lisp option to tell FriCAS
how to invoke your Lisp])
fi
+ FRICAS_LISP="$FRICAS_LISP --dynamic-space-size 2048"
fricas_lisp=$FRICAS_LISP
else
## Honor use of Lisp image specified on command line
+ if "$fricas_lisp" = "sbcl" ; then
+ fricas_lisp="sbcl --dynamic-space-size 2048"
+ fi
FRICAS_LISP=$fricas_lisp
- AC_SUBST(FRICAS_LISP)
- :
fi
+AC_SUBST(FRICAS_LISP)

AC_ARG_WITH([lisp-flavor],
[AS_HELP_STRING([--with-lisp-flavor=F],

[Waldek Hebisch]

This is problematic. When we dump image the options are
stored in the image. AFAIK in sbcl options stored in image
override (or maybe replace) command line. Consequently,
there is no way to override dynamic space size. To
get changeable behaviour default dynamic space size
must be configured at sbcl build time.

Basically, there are several bad decisions on sbcl side
(and Linux/Unix side):
1) no ability to change sbcl dynamic space size at runtime
2) Linux confuses "memory" with address space, there is
no working limit on real memory
3) too low default setting of sbcl dynamic space size
4) sbcl handling of options stored in images
5) large memory use by sbcl compiler

Within those constraints there is no good solution:
- building our own sbcl to get sane defults would be
disliked by distributions
- setting new size in images means that it is no longer
changable
- propagating size option to all invocations of images
spreads out sbcl specific workarounds troughout makefiles.

Just to explain better technical constrains:
- sbcl needs single continuous region on memory. If it were
possible to allocate address space first and incrementally
allocate memory at prescribed addresses at least in principle
sbcl could use this
- if it were possible to limit used memory sbcl could allocate
big area of memory (utilizing Linux overcommit). Actual
limiting could be in sbcl (I am not aware of any such possiblity)
or via ulimit (but again, AFAIK there is no possiblity to
limit used (as opossed to allocated) memory).
- we could live with 1G default if sbcl compiler could
operate within the limit. But apparently, somewhat longer
linear stretches of code seem to trigger very bad behaviour
of sbcl. In principle such code is easiest to compile and
can be handled by linear time/space method. And I am not
talking about very long sequences.

--
Waldek Hebisch

[Qian Yun]

On 1/23/24 09:10, Waldek Hebisch wrote:
>
> This is problematic. When we dump image the options are
> stored in the image. AFAIK in sbcl options stored in image
> override (or maybe replace) command line. Consequently,
> there is no way to override dynamic space size. To
> get changeable behaviour default dynamic space size
> must be configured at sbcl build time.
>

"FRICASsys --dynamic-space-size" can override the original value
saved in image.

- Qian

[Waldek Hebisch]

IIRC in the past it was not possible to override stored
dynamic space size. Maybe that has changed.

--
Waldek Hebisch