why does make sagemath_doc_html *always* rebuild sagelib

[Marc Culler]

When I build sage with make the last step is to build sagelib.  If I then run

  make sagemath_doc_html

it rebuilds the sagelib that it just finished building before starting to build the documentation.

Why?

- Marc

[Michael Orlitzky]

There is an insane amount of indirection...

make sagemath_doc_html
-> sage-spkg -i sagemath_doc_html
-> build/pkgs/sagemath_doc_html/spkg-install
-> make -C src/doc doc-html
-> meson compile -C build/sage-distro doc-html
-> python src/build-docs.py ...

And the (duct tape & spit) dependency tracking across the levels is
failing. I have been dealing with this problem in

https://github.com/sagemath/sage/pull/41610

where I am trying to skip to the last step (just build the F'ing
docs). But even that will not be a perfect solution, because the
documentation depends on the docbuilder, and the docbuilder is part of
the sagelib that you are trying to build documentation for. So even
just the last step must (to avoid problems) depend on all of sage.

I recall Tobias was working on a better solution in

https://github.com/sagemath/sage/pull/41156

but I don't know its current status.

[Dima Pasechnik]

For me,

    make doc-html

does the right thing, building html docs, without extra work (after  `make` is done).
If one needs the docs, it makes sense to run
the above straight away, it's a bit quicker, as it avoids extra regeneration of docs ninja targets by meson.

Dima

[Marc Culler]

I just tried make doc-html.  I get:

[sagelib-10.9.beta7] Warning: File /var/tmp/sage-10.9/local/var/lib/sage/scripts/sagelib/spkg-requirements.txt not found
[sagelib-10.9.beta7] Warning: Directory /var/tmp/sage-10.9/local/var/lib/sage/scripts/sagelib not found
[sagelib-10.9.beta7] Removing stamp file /var/tmp/sage-10.9/local/var/lib/sage/installed/sagelib-10.9.beta7
[sagelib-10.9.beta7] [spkg-install] Installing sagelib-10.9.beta7
[sagelib-10.9.beta7] [spkg-install] * Building wheel...

Are the missing files in sage/scripts treated as missing dependencies?  Why would they be missing?  I did not delete them.

- Marc

--
You received this message because you are subscribed to a topic in the Google Groups "sage-devel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sage-devel/dVVleHx0xP0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sage-devel+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/sage-devel/FEA68B7E-8D2F-4957-A43A-D2FA9A0263BA%40gmail.com.

[Dima Pasechnik]

some of these are generated by ./bootstrap
You need to run it either directly, or via make (the latter might be flaky)

[Marc Culler]

There is really only one file reported as missing:: sage/local/var/lib/sage/scripts/sagelib/spkg-requirements.txt.

The other one is the directory containing that file.

I did run ./bootstrap, of course.  And it created the directory and the file.  Moreover, both the directory and the file still exist and are both readable.

But if I verify that they exist, and then run make doc-html, it reports them both as missing.  There is something wrong with the test.  I wish I knew where the test was.

- Marc

[Dima Pasechnik]

Besides, sagelib doesn't have spkg-requirements.txt, nor it has any sage/scripts/sagelib/

You're doing something non-standard, and without full details it's hard to say anything meaningful.

sagelib is a not like the rest of the packages, anyway.

In my SAGEROOT I see

 $ find . -name spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/jupyter_events/spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/gnumake_tokenpool/spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/tzdata/spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/nbconvert/spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/overrides/spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/jupyterlab_mathjax2/spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/types_python_dateutil/spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/jupyter_server/spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/pickleshare/spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/mpmath/spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/jupyter_sphinx/spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/sagetex/spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/jupyterlab_server/spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/jupyter_lsp/spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/notebook_shim/spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/pytz_deprecation_shim/spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/jupyterlab/spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/ipympl/spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/notebook/spkg-requirements.txt
./local/var/lib/sage/venv-python3.13/var/lib/sage/scripts/sagenb_export/spkg-requirements.txt

On Sun, Mar 1, 2026 at 11:25 AM Dima Pasechnik <dim...@gmail.com> wrote:

On Sun, Mar 1, 2026 at 11:19 AM Marc Culler <marc....@gmail.com> wrote:

There is really only one file reported as missing:: sage/local/var/lib/sage/scripts/sagelib/spkg-requirements.txt.

The other one is the directory containing that file.

I did run ./bootstrap, of course.  And it created the directory and the file.  Moreover, both the directory and the file still exist and are both readable.

Does /var/tmp/sage-10.9/local/var/lib/sage/scripts/sagelib exist?

(but you see "Directory /var/tmp/sage-10.9/local/var/lib/sage/scripts/sagelib not found" ?)

/var/tmp doesn't sound like something our build system uses normally.

IMHO all happens in SAGEROOT/build/sage-distro/

Dima

You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/sage-devel/48b8a758-b896-40ca-ac56-6fccfcef31fcn%40googlegroups.com.

[Dima Pasechnik]

On Sun, Mar 1, 2026 at 11:19 AM Marc Culler <marc....@gmail.com> wrote:

There is really only one file reported as missing:: sage/local/var/lib/sage/scripts/sagelib/spkg-requirements.txt.

The other one is the directory containing that file.

I did run ./bootstrap, of course.  And it created the directory and the file.  Moreover, both the directory and the file still exist and are both readable.

Does /var/tmp/sage-10.9/local/var/lib/sage/scripts/sagelib exist?

(but you see "Directory /var/tmp/sage-10.9/local/var/lib/sage/scripts/sagelib not found" ?)

/var/tmp doesn't sound like something our build system uses normally.

IMHO all happens in SAGEROOT/build/sage-distro/

Dima

You received this message because you are subscribed to the Google Groups "sage-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/sage-devel/48b8a758-b896-40ca-ac56-6fccfcef31fcn%40googlegroups.com.

[Marc Culler]

I see:

find . -name spkg-requirements.txt
./local/var/lib/sage/scripts/comm/spkg-requirements.txt
./local/var/lib/sage/scripts/pytest/spkg-requirements.txt
./local/var/lib/sage/scripts/distlib/spkg-requirements.txt
./local/var/lib/sage/scripts/ipykernel/spkg-requirements.txt
./local/var/lib/sage/scripts/filelock/spkg-requirements.txt
./local/var/lib/sage/scripts/pybind11/spkg-requirements.txt
./local/var/lib/sage/scripts/pathspec/spkg-requirements.txt
./local/var/lib/sage/scripts/pure_eval/spkg-requirements.txt
./local/var/lib/sage/scripts/tornado/spkg-requirements.txt
./local/var/lib/sage/scripts/pyproject_api/spkg-requirements.txt
./local/var/lib/sage/scripts/requests/spkg-requirements.txt
./local/var/lib/sage/scripts/gmpy2/spkg-requirements.txt
./local/var/lib/sage/scripts/pyparsing/spkg-requirements.txt
./local/var/lib/sage/scripts/conway_polynomials/spkg-requirements.txt
./local/var/lib/sage/scripts/pytz/spkg-requirements.txt
./local/var/lib/sage/scripts/virtualenv/spkg-requirements.txt
./local/var/lib/sage/scripts/primecountpy/spkg-requirements.txt
./local/var/lib/sage/scripts/tzdata/spkg-requirements.txt
./local/var/lib/sage/scripts/wheel/spkg-requirements.txt
./local/var/lib/sage/scripts/asttokens/spkg-requirements.txt
./local/var/lib/sage/scripts/networkx/spkg-requirements.txt
./local/var/lib/sage/scripts/meson/spkg-requirements.txt
./local/var/lib/sage/scripts/ipywidgets/spkg-requirements.txt
./local/var/lib/sage/scripts/pytz_deprecation_shim/spkg-requirements.txt
./local/var/lib/sage/scripts/pexpect/spkg-requirements.txt
./local/var/lib/sage/scripts/memory_allocator/spkg-requirements.txt
./local/var/lib/sage/scripts/pyproject_hooks/spkg-requirements.txt
./local/var/lib/sage/scripts/pygments/spkg-requirements.txt
./local/var/lib/sage/scripts/sphinxcontrib_htmlhelp/spkg-requirements.txt
./local/var/lib/sage/scripts/sage_setup/spkg-requirements.txt
./local/var/lib/sage/scripts/exceptiongroup/spkg-requirements.txt
./local/var/lib/sage/scripts/py/spkg-requirements.txt
./local/var/lib/sage/scripts/cython/spkg-requirements.txt
./local/var/lib/sage/scripts/entrypoints/spkg-requirements.txt
./local/var/lib/sage/scripts/flit_core/spkg-requirements.txt
./local/var/lib/sage/scripts/parso/spkg-requirements.txt
./local/var/lib/sage/scripts/debugpy/spkg-requirements.txt
./local/var/lib/sage/scripts/cachetools/spkg-requirements.txt
./local/var/lib/sage/scripts/tox/spkg-requirements.txt
./local/var/lib/sage/scripts/sphinx_inline_tabs/spkg-requirements.txt
./local/var/lib/sage/scripts/contourpy/spkg-requirements.txt
./local/var/lib/sage/scripts/hatch_vcs/spkg-requirements.txt
./local/var/lib/sage/scripts/sage_numerical_backends_coin/spkg-requirements.txt
./local/var/lib/sage/scripts/sortedcontainers/spkg-requirements.txt
./local/var/lib/sage/scripts/pytest_xdist/spkg-requirements.txt
./local/var/lib/sage/scripts/alabaster/spkg-requirements.txt
./local/var/lib/sage/scripts/roman_numerals/spkg-requirements.txt
./local/var/lib/sage/scripts/pip/spkg-requirements.txt
./local/var/lib/sage/scripts/gast/spkg-requirements.txt
./local/var/lib/sage/scripts/decorator/spkg-requirements.txt
./local/var/lib/sage/scripts/python_build/spkg-requirements.txt
./local/var/lib/sage/scripts/cffi/spkg-requirements.txt
./local/var/lib/sage/scripts/cycler/spkg-requirements.txt
./local/var/lib/sage/scripts/idna/spkg-requirements.txt
./local/var/lib/sage/scripts/beautifulsoup4/spkg-requirements.txt
./local/var/lib/sage/scripts/markupsafe/spkg-requirements.txt
./local/var/lib/sage/scripts/cvxopt/spkg-requirements.txt
./local/var/lib/sage/scripts/pythran/spkg-requirements.txt
./local/var/lib/sage/scripts/attrs/spkg-requirements.txt
./local/var/lib/sage/scripts/jinja2/spkg-requirements.txt
./local/var/lib/sage/scripts/chardet/spkg-requirements.txt
./local/var/lib/sage/scripts/pycparser/spkg-requirements.txt
./local/var/lib/sage/scripts/cypari/spkg-requirements.txt
./local/var/lib/sage/scripts/trove_classifiers/spkg-requirements.txt
./local/var/lib/sage/scripts/snowballstemmer/spkg-requirements.txt
./local/var/lib/sage/scripts/imagesize/spkg-requirements.txt
./local/var/lib/sage/scripts/jedi/spkg-requirements.txt
./local/var/lib/sage/scripts/iniconfig/spkg-requirements.txt
./local/var/lib/sage/scripts/pynormaliz/spkg-requirements.txt
./local/var/lib/sage/scripts/pyzmq/spkg-requirements.txt
./local/var/lib/sage/scripts/docutils/spkg-requirements.txt
./local/var/lib/sage/scripts/pycosat/spkg-requirements.txt
./local/var/lib/sage/scripts/furo/spkg-requirements.txt
./local/var/lib/sage/scripts/numpy/spkg-requirements.txt
./local/var/lib/sage/scripts/pyproject_metadata/spkg-requirements.txt
./local/var/lib/sage/scripts/packaging/spkg-requirements.txt
./local/var/lib/sage/scripts/sphinx_copybutton/spkg-requirements.txt
./local/var/lib/sage/scripts/sagelib/spkg-requirements.txt
./local/var/lib/sage/scripts/charset_normalizer/spkg-requirements.txt
./local/var/lib/sage/scripts/editables/spkg-requirements.txt
./local/var/lib/sage/scripts/pkgconfig/spkg-requirements.txt
./local/var/lib/sage/scripts/sphinxcontrib_jsmath/spkg-requirements.txt
./local/var/lib/sage/scripts/sphinxcontrib_qthelp/spkg-requirements.txt
./local/var/lib/sage/scripts/colorama/spkg-requirements.txt
./local/var/lib/sage/scripts/setuptools/spkg-requirements.txt
./local/var/lib/sage/scripts/dateutil/spkg-requirements.txt
./local/var/lib/sage/scripts/lrcalc_python/spkg-requirements.txt
./local/var/lib/sage/scripts/cppy/spkg-requirements.txt
./local/var/lib/sage/scripts/fpylll/spkg-requirements.txt
./local/var/lib/sage/scripts/urllib3/spkg-requirements.txt
./local/var/lib/sage/scripts/prompt_toolkit/spkg-requirements.txt
./local/var/lib/sage/scripts/kiwisolver/spkg-requirements.txt
./local/var/lib/sage/scripts/sphinxcontrib_websupport/spkg-requirements.txt
./local/var/lib/sage/scripts/meson_python/spkg-requirements.txt
./local/var/lib/sage/scripts/beniget/spkg-requirements.txt
./local/var/lib/sage/scripts/mpmath/spkg-requirements.txt
./local/var/lib/sage/scripts/sage_docbuild/spkg-requirements.txt
./local/var/lib/sage/scripts/setuptools_scm/spkg-requirements.txt
./local/var/lib/sage/scripts/fonttools/spkg-requirements.txt
./local/var/lib/sage/scripts/sphinx/spkg-requirements.txt
./local/var/lib/sage/scripts/six/spkg-requirements.txt
./local/var/lib/sage/scripts/platformdirs/spkg-requirements.txt
./local/var/lib/sage/scripts/gnumake_tokenpool/spkg-requirements.txt
./local/var/lib/sage/scripts/calver/spkg-requirements.txt
./local/var/lib/sage/scripts/stack_data/spkg-requirements.txt
./local/var/lib/sage/scripts/appdirs/spkg-requirements.txt
./local/var/lib/sage/scripts/jupyterlab_widgets/spkg-requirements.txt
./local/var/lib/sage/scripts/wcwidth/spkg-requirements.txt
./local/var/lib/sage/scripts/hypothesis/spkg-requirements.txt
./local/var/lib/sage/scripts/widgetsnbextension/spkg-requirements.txt
./local/var/lib/sage/scripts/matplotlib/spkg-requirements.txt
./local/var/lib/sage/scripts/sagetex/spkg-requirements.txt
./local/var/lib/sage/scripts/cysignals/spkg-requirements.txt
./local/var/lib/sage/scripts/pyrsistent/spkg-requirements.txt
./local/var/lib/sage/scripts/matplotlib_inline/spkg-requirements.txt
./local/var/lib/sage/scripts/scipy/spkg-requirements.txt
./local/var/lib/sage/scripts/soupsieve/spkg-requirements.txt
./local/var/lib/sage/scripts/traitlets/spkg-requirements.txt
./local/var/lib/sage/scripts/tzlocal/spkg-requirements.txt
./local/var/lib/sage/scripts/pillow/spkg-requirements.txt
./local/var/lib/sage/scripts/pplpy/spkg-requirements.txt
./local/var/lib/sage/scripts/sympy/spkg-requirements.txt
./local/var/lib/sage/scripts/sphinxcontrib_devhelp/spkg-requirements.txt
./local/var/lib/sage/scripts/pluggy/spkg-requirements.txt
./local/var/lib/sage/scripts/sphinxcontrib_applehelp/spkg-requirements.txt
./local/var/lib/sage/scripts/babel/spkg-requirements.txt
./local/var/lib/sage/scripts/nest_asyncio/spkg-requirements.txt
./local/var/lib/sage/scripts/ply/spkg-requirements.txt
./local/var/lib/sage/scripts/jupyter_core/spkg-requirements.txt
./local/var/lib/sage/scripts/execnet/spkg-requirements.txt
./local/var/lib/sage/scripts/ptyprocess/spkg-requirements.txt
./local/var/lib/sage/scripts/pysingular/spkg-requirements.txt
./local/var/lib/sage/scripts/pytest_mock/spkg-requirements.txt
./local/var/lib/sage/scripts/certifi/spkg-requirements.txt
./local/var/lib/sage/scripts/psutil/spkg-requirements.txt
./local/var/lib/sage/scripts/executing/spkg-requirements.txt
./local/var/lib/sage/scripts/pickleshare/spkg-requirements.txt
./local/var/lib/sage/scripts/sphinx_basic_ng/spkg-requirements.txt
./local/var/lib/sage/scripts/zipp/spkg-requirements.txt
./local/var/lib/sage/scripts/ipython_genutils/spkg-requirements.txt
./local/var/lib/sage/scripts/hatchling/spkg-requirements.txt
./local/var/lib/sage/scripts/hatch_readme/spkg-requirements.txt
./local/var/lib/sage/scripts/sphinxcontrib_serializinghtml/spkg-requirements.txt
./local/var/lib/sage/scripts/jupyter_client/spkg-requirements.txt
./local/var/lib/sage/scripts/p_group_cohomology/spkg-requirements.txt
./local/var/lib/sage/scripts/ipython/spkg-requirements.txt

[Marc Culler]

On Sunday, March 1, 2026 at 12:00:16 PM UTC-6  Dima Pasechnik wrote:

On Sun, Mar 1, 2026 at 11:19 AM Marc Culler wrote:

There is really only one file reported as missing:: sage/local/var/lib/sage/scripts/sagelib/spkg-requirements.txt.

The other one is the directory containing that file.

I did run ./bootstrap, of course.  And it created the directory and the file.  Moreover, both the directory and the file still exist and are both readable.

Does /var/tmp/sage-10.9/local/var/lib/sage/scripts/sagelib exist?

Yes.

 

(but you see "Directory /var/tmp/sage-10.9/local/var/lib/sage/scripts/sagelib not found" ?)

Correct.

 

/var/tmp doesn't sound like something our build system uses normally.

IMHO all happens in SAGEROOT/build/sage-distro/

I am using /var/tmp/sage-10.9 as my SAGEROOT.  So the build system is not using /var/tmp intentionally.  It just happens to be the directory containing my SAGEROOT.

- Marc 

[Dima Pasechnik]

In my case these are the only packages installed by Sage, the rest come from the "system", via

./configure --enable-system-site-packages

Why is there a red line below, is it a hanging link?

[Dima Pasechnik]

Note that I also have nothing with sage_setup, and the scripts are in the Sage's venv.

You do have sage_setup, and your files are not in a Sage venv.

Seeing the output of ./config.status --config, and the contents of config.log, might help to understand more.

[Marc Culler]

I colored that line red to make it easier for you to see that

 ./local/var/lib/sage/scripts/sagelib/spkg-requirements.txt

is there in the list.

OK.  I will see what happens if I add that.  I think that should only change the pyvenv.cfg file.  But you are probably trying to tell me that the sage build system inspects the pyvenv.cfg file and makes different decisions depending on what it finds there.  

My files are in a sage venv.  However the sage venv is actually the same as sage/local.

Since I have to build my own python now, I simply install it in sage/local before building sage.  That worked fine for Sage 10.7 and 10.8.  I think it works fine with 10.9, as well.  The only problem I am seeing is that I get these weird reports of files not existing when in fact they do exist.  The resulting sage runs fine (unless I try to create a Gap FreeGroup).

- Marc

[Marc Culler]

Using --enable-system-site-packages does change things.  I still see the warning about the missing file.  But now the file is actually missing, so I guess I can ignore the warning. That is progress.

The core problem I am seeing now is that the gap object is not working in 10.9beta7.  This is definitely a regression from 10.8.  With 10.8, using the identical build setup, I can do:

sage: gap.FreeGroup(3)
Group( [ f1, f2, f3 ] )

But with 10.9beta7 I get a RuntimeError:

sage: gap.FreeGroup(3)
--------------------------------------------------------------------------

-

[ ,,, traceback details removed ...]

RuntimeError: Gap produced error output
Error, FreeGroup: function is not yet defined

   executing __SAGE_LAST__:="__SAGE_LAST__";;FreeGroup(\$sage1);;

This is a problem with the GAP interface, not with GAP:

$ sage/sage -gap
 ┌───────┐   GAP 4.14.0 of 2024-12-05
 │  GAP  │   https://www.gap-system.org
 └───────┘   Architecture: x86_64-pc-linux-gnu-default64-kv9
 Configuration:  gmp 6.3.0, GASMAN, readline
 Loading the library and packages ...
 Packages:   GAPDoc 1.6.7, PrimGrp 3.4.4, SmallGrp 1.5.4, TransGrp 3.6.5
 Try '??help' for help. See also '?copyright', '?cite' and '?authors'
gap> F := FreeGroup(3);
<free group on the generators [ f1, f2, f3 ]>
gap>

- Marc

[Dima Pasechnik]

On Mon, Mar 2, 2026 at 9:10 AM Marc Culler <marc....@gmail.com> wrote:

I colored that line red to make it easier for you to see that

 ./local/var/lib/sage/scripts/sagelib/spkg-requirements.txt

is there in the list.

OK.  I will see what happens if I add that.  I think that should only change the pyvenv.cfg file.  But you are probably trying to tell me that the sage build system inspects the pyvenv.cfg file and makes different decisions depending on what it finds there.  

My files are in a sage venv.  However the sage venv is actually the same as sage/local.

Since I have to build my own python now, I simply install it in sage/local before building sage. 

I think you can have it anywhere, as long  PATH points to where it is located 

(along with other binaries that somehow come together with it, e.g. for python 3.14 it appears to be the following list

idle3 pip pip3.14 pydoc3.14 python3 python3.14
idle3.14 pip3 pydoc3 python python3-config python3.14-config)

and this path entry comes before all other PATH entries with python3 in them...

I might have already asked (sorry) - how do you build python - is it somewhere on GitHub?

We'd like to have instructions for it in view of removing python3 spkg in https://github.com/sagemath/sage/pull/41120

We have reasonable success with https://github.com/astral-sh/python-build-standalone - but it would be nice to 

mention an alternative.

Dima

 

To view this discussion visit https://groups.google.com/d/msgid/sage-devel/c04ff902-1da1-422b-a599-2c2f0575a432n%40googlegroups.com.

[Dima Pasechnik]

On Mon, Mar 2, 2026 at 2:20 PM Marc Culler <marc....@gmail.com> wrote:

Using --enable-system-site-packages does change things.  I still see the warning about the missing file.  But now the file is actually missing, so I guess I can ignore the warning. That is progress.

The core problem I am seeing now is that the gap object is not working in 10.9beta7.  This is definitely a regression from 10.8.  With 10.8, using the identical build setup, I can do:

sage: gap.FreeGroup(3)
Group( [ f1, f2, f3 ] )

But with 10.9beta7 I get a RuntimeError:

sage: gap.FreeGroup(3)
--------------------------------------------------------------------------

-

[ ,,, traceback details removed ...]

RuntimeError: Gap produced error output
Error, FreeGroup: function is not yet defined

   executing __SAGE_LAST__:="__SAGE_LAST__";;FreeGroup(\$sage1);;

This is a problem with the GAP interface, not with GAP:

That's a problem with the pexpect GAP. Does 

    sage: libgap.FreeGroup(3) 

work? Does

   ./sage -t src/sage/features/gap.py 

pass? 

 

To view this discussion visit https://groups.google.com/d/msgid/sage-devel/c15ac44f-891f-4ac3-81dc-e723bdcd9293n%40googlegroups.com.

[Marc Culler]

On Monday, March 2, 2026 at 3:32:23 PM UTC-6 dim...@gmail.com wrote:

 

That's a problem with the pexpect GAP. Does 

    sage: libgap.FreeGroup(3) 

work?

No.  I don't get a RuntimeError (which I think is produced by pexpect) but I get:

GAPError: Error, FreeGroup: function is not yet defined

Does

   ./sage -t src/sage/features/gap.py 

pass?

Yes. I get:

Doctesting 1 file.
sage/src/bin/sage -t --warn-long 5.0 --random-seed=206062923921806203470542245493080431476 sage/src/sage/features/gap.py
    [5 tests, 0.01s wall]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: 0.2 seconds
    cpu time: 0.0 seconds
    cumulative wall time: 0.0 seconds
Features detected for doctesting:

- Marc

[Marc Culler]

On Monday, March 2, 2026 at 2:35:29 PM UTC-6 dim...@gmail.com wrote:

I might have already asked (sorry) - how do you build python - is it somewhere on GitHub?

We'd like to have instructions for it in view of removing python3 spkg in https://github.com/sagemath/sage/pull/41120

We have reasonable success with https://github.com/astral-sh/python-build-standalone - but it would be nice to 

mention an alternative.

I would suggest as an alternative just doing a default build of python3.14 and installing it in /usr/local.  I have done that on Ubuntu systems several times over the past few weeks, with no issues whatsoever.  You need a few development packages - I think bzip2, zlib, lzma, xz, ncurses, readline, sqlite3, openssl and libffi will do it. Then the usual ./configure; make; sudo make install will build and install it in /usr/local.  It should already be in your path and you can use it to build a venv and then move on to building sage.

Building python is nothing compared to building sage.  I never understood why maintaining the spkg was so difficult, nor why removing it was so important.

For the appimage I need to work harder because I need everything to be embedded in the appimage and hence to be relocatable.  But people building sage in the expected way don't need that.

- Marc

 

[Dima Pasechnik]

The usual suspects for strange GAP errors are stale GAP workspaces (or related to them errors). Make sure you don't have any (typically in ~/.sage/, although your setup might have a different location of .sage/)

[Marc Culler]

I deleted .sage/gap and tried again.  No change.

I also tried 

   sage: libgap.eval('F:= FreeGroup(3)')

That produces the same GAPError, but there was some other stuff printed before the traceback which might provide a clue:

#I  gapdoc package is not available. Check that the name is correct
#I  and it is present in one of the GAP root directories (see '??RootPaths')
#I  GAP: needed package gapdoc cannot be loaded
Error, failed to load needed package `gapdoc' (version >= 1.2)
Syntax warning: Unbound global variable in /var/tmp/sage-10.9/local/share/gap/l\
ib/init.g:728
    ColorPrompt( UserPreference( "UseColorPrompt" ) );
    ^^^^^^^^^^^
Error, SetGasmanMessageStatus: function is not yet defined
Error, Variable: 'L1_IMMUTABLE_ERROR' must have a value
Error, Variable: 'ColorPrompt' must have a value

I did find  some other suspicious stuff:

sage: libgap.eval('GAPInfo.RootPaths')
[ "/home/culler/.gap/", "\033[?1034h/var/tmp/sage-10.9/local/lib/gap/", "/var/tmp/sage-10.9/local/share/gap/" ]

The last path contains an empty pkg directory.  The real pkg directory is in $SAGE_ROOT/local/lib/gap.  But I doubt that the second path would work correctly when it has all of those ANSI escape sequences in front of it.  Adding a symlink /var/tmp/sage-10.9/local/share/gap/pkg -> ../../lib/gap/pkg did not help.  But maybe putting ANSI escape sequences in a root path makes GAP crash.

- Marc

- Marc

--

You received this message because you are subscribed to a topic in the Google Groups "sage-devel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sage-devel/dVVleHx0xP0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sage-devel+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/sage-devel/B04DFD67-0325-4949-A30C-4D2A1AD9F7EE%40gmail.com.

[Dima Pasechnik]

just in case, ansi.tools say:

\033[?1034h

enable 8-bit meta key

On Mon, Mar 2, 2026 at 10:13 PM Dima Pasechnik <dim...@gmail.com> wrote:

"\033[?1034h/var/tmp/sage-10.9/local/lib/gap/" is certainly looking bad, and probably is the root of the problem.

No idea where it came from.

You received this message because you are subscribed to the Google Groups "sage-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/sage-devel/CALcZXRHNSvqL8Vi8v3fQLoOroBbi%2BXB%2B%2BSMgV8doMzU7DhLUeQ%40mail.gmail.com.

[Dima Pasechnik]

"\033[?1034h/var/tmp/sage-10.9/local/lib/gap/" is certainly looking bad, and probably is the root of the problem.

No idea where it came from.

You received this message because you are subscribed to the Google Groups "sage-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/sage-devel/CALcZXRHNSvqL8Vi8v3fQLoOroBbi%2BXB%2B%2BSMgV8doMzU7DhLUeQ%40mail.gmail.com.

[Marc Culler]

I can tell you something about where it came from.

 When I look at sage/local/lib/python3.14/site-packages/sage/config.py I see:

# The semicolon-separated list of GAP root paths. This is the list of
# locations that are searched for GAP packages. This is passed directly
# to GAP via the -l flag.
GAP_ROOT_PATHS = "ESC[?1034h/var/tmp/sage-10.9/local/lib/gap/;/var/tmp/sage-10.9/local/share/gap/".replace("${prefix}", SAGE_LOCAL)

The file config.py is generated from sage/src/sage/config.py.in, which contains:

# The semicolon-separated list of GAP root paths. This is the list of
# locations that are searched for GAP packages. This is passed directly
# to GAP via the -l flag.
GAP_ROOT_PATHS = "@GAP_ROOT_PATHS@".replace("${prefix}", SAGE_LOCAL)

So it would seem that some m4 file is inserting the "enable 8 bit meta" escape sequence into @GAP_ROOT_PATHS@ for my enjoyment and entertainment.

Also, when I remove that escape sequence from config.py, guess what?

sage: gap.FreeGroup(3)
Group( [ f1, f2, f3 ] )

- Marc

[Marc Culler]

I managed to track this down.

First, I found this in sage/sage-distro/meson-logs/meson-log.txt:

Running command: /var/tmp/sage-10.9/local/bin/gap -r -q --bare --nointeract -c
 'Display(JoinStringsWithSeparator(GAPInfo.RootPaths,";"));'
--- stdout ---
ESC[?1034h/var/tmp/sage-10.9/local/lib/gap/;/var/tmp/sage-10.9/local/share/gap/

--- stderr ---

So,  meson is running sage's gap executable to find its root paths.  But it is getting the "set 8 bit meta" escape sequence at the beginning of the gap output.

 If I run that command in the shell I do not get the  escape sequence:

sage/local/bin/gap -r -q --bare --nointeract -c 'Display(JoinStringsWithSeparator(GAPInfo.RootPaths,";"));' | xxd
00000000: 2f76 6172 2f74 6d70 2f73 6167 652d 3130  /var/tmp/sage-10
00000010: 2e39 2f6c 6f63 616c 2f6c 6962 2f67 6170  .9/local/lib/gap
00000020: 2f3b 2f76 6172 2f74 6d70 2f73 6167 652d  /;/var/tmp/sage-
00000030: 3130 2e39 2f6c 6f63 616c 2f73 6861 7265  10.9/local/share
00000040: 2f67 6170 2f0a                           /gap/.

However, clearly there are times when the escape sequence does get added.

The culprit turns out to be readline-6.2, which is what my docker image happens to be using. Gemini was able to find some information about this:

        Version 6.3 Fix: In earlier versions (notably 6.0 through 6.2), rl_initialize() would output the "Meta mode ON" sequence to stdout

        even if the output was not a terminal, causing issues like corrupted script output and erroneous characters in program pipes.

This problem did not occur with 10.8.  The difference is probably that the rl_initialize was being called at some earlier stage of the build process, so the escape sequence was being emitted somewhere else, not in the step where the GAP root paths are constructed.

Moral: It is always something.

- Marc

[Dima Pasechnik]

As far as m4 goes, probably having

iff --git a/build/pkgs/gap/spkg-configure.m4 b/build/pkgs/gap/spkg-configure.m4
index 937fe400407..f754b29a455 100644
--- a/build/pkgs/gap/spkg-configure.m4
+++ b/build/pkgs/gap/spkg-configure.m4
@@ -13,14 +13,14 @@ SAGE_SPKG_CONFIGURE([gap], [
       dnl so long as we don't initialize GAP with -r in Sage. But we
       dnl don't want to include it in the hard-coded list.
       GAPRUN="${GAP} -r -q --bare --nointeract -c"
-      _cmd='Display(GAPInfo.KernelInfo.KERNEL_VERSION);'
+      _cmd='ColorPrompt(false); Display(GAPInfo.KernelInfo.KERNEL_VERSION);'
       GAP_VERSION=$( ${GAPRUN} "${_cmd}" 2>/dev/null )
       AX_COMPARE_VERSION(["${GAP_VERSION}"], [ge], [GAP_MINVER], [dnl
        AC_MSG_RESULT([yes])
        AX_COMPARE_VERSION(["${GAP_VERSION}"], [lt], [GAP_LTVER], [dnl
         AC_MSG_RESULT([yes])
         AC_MSG_CHECKING([for gap root paths])
-        _cmd='Display(JoinStringsWithSeparator(GAPInfo.RootPaths,";"));'
+        _cmd='ColorPrompt(false); Display(JoinStringsWithSeparator(GAPInfo.RootPaths,";"));'
         SYS_GAP_ROOT_PATHS=$( ${GAPRUN} "${_cmd}" 2>/dev/null )
         AC_MSG_RESULT([$SYS_GAP_ROOT_PATHS])
         AS_IF([test -n "${SYS_GAP_ROOT_PATHS}"], [

would avoid the trouble you have. (after the usual ./bootstrap etc etc)

To view this discussion visit https://groups.google.com/d/msgid/sage-devel/f7733173-3d56-4142-af48-a12efed9b910n%40googlegroups.com.

[Dima Pasechnik]

And for meson, essentially the same treatment

diff --git a/src/sage/meson.build b/src/sage/meson.build
index d43e636a4e8..f7adc6d68ca 100644
--- a/src/sage/meson.build
+++ b/src/sage/meson.build
@@ -43,7 +43,7 @@ if gap_exe.found()
     '--bare',
     '--nointeract',
     '-c',
-    'Display(JoinStringsWithSeparator(GAPInfo.RootPaths,";"));',
+    'ColorPrompt(false); Display(JoinStringsWithSeparator(GAPInfo.RootPaths,";"));',
     check: true,
   )
   gap_root_paths = gaprun.stdout().strip()

[Marc Culler]

Why do you think that disabling ColorPrompt would make a difference, if the escape sequence is being emitted by rl_initialize()?

- Marc

To view this discussion visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq0OQ4PEeW_K-5hFuPc1ZD_-1kp_hWWADYUbT2jB99%3D92A%40mail.gmail.com.

[Dima Pasechnik]

On Tue, Mar 3, 2026 at 11:06 AM Marc Culler <marc....@gmail.com> wrote:

Why do you think that disabling ColorPrompt would make a difference, if the escape sequence is being emitted by rl_initialize()?

Well, just trying to be less "fancy-terminals"-bound. 

As well, instead of Display(...) one should run Print(...,"\n"), as Display might colour, format, etc. things, if allowed.

Or, even safer, use 'PrintTo("/tmp/foobar42.txt",..., "\n") and get the needed values from "/tmp/foobar42".

(by  right, these should be in a temp dir etc etc)

But, perhaps much easier might be to add '-E' switch to these GAP calls. '-E' means "disable readline (if possible)",

according to the output of "gap -h". (no idea what "if possible" means)

Dima

[Marc Culler]

I see.  Indeed, there are other escape sequences that could sneak if if GAP tries to color its output.

The -E option looks like the key.  I tested like this on my docker image:

[root@f52defcaf0a7 sage-10.9]#  local/bin/gap -r -q --bare --nointeract -c 'Display(JoinStringsWithSeparator(GAPInfo.RootPaths,";"));' > output
[root@f52defcaf0a7 sage-10.9]# od -a output
0000000 esc   [   ?   1   0   3   4   h   /   v   a   r   /   t   m   p
0000020   /   s   a   g   e   -   1   0   .   9   /   l   o   c   a   l
0000040   /   l   i   b   /   g   a   p   /   ;   /   v   a   r   /   t
0000060   m   p   /   s   a   g   e   -   1   0   .   9   /   l   o   c
0000100   a   l   /   s   h   a   r   e   /   g   a   p   /  nl
0000116
[root@f52defcaf0a7 sage-10.9]#  local/bin/gap -E -r -q --bare --nointeract -c 'Display(JoinStringsWithSeparator(GAPInfo.RootPaths,";"));' > output
[root@f52defcaf0a7 sage-10.9]# od -a output
0000000   /   v   a   r   /   t   m   p   /   s   a   g   e   -   1   0
0000020   .   9   /   l   o   c   a   l   /   l   i   b   /   g   a   p
0000040   /   ;   /   v   a   r   /   t   m   p   /   s   a   g   e   -
0000060   1   0   .   9   /   l   o   c   a   l   /   s   h   a   r   e
0000100   /   g   a   p   /  nl
0000106

Note that using Print does not help with this issue, but is still probably a good idea to avoid other escape sequences:

[root@f52defcaf0a7 sage-10.9]#  local/bin/gap -r -q --bare --nointeract -c 'Print(JoinStringsWithSeparator(GAPInfo.RootPaths,";"), "\n");' > output
[root@f52defcaf0a7 sage-10.9]# od -a output
0000000 esc   [   ?   1   0   3   4   h   /   v   a   r   /   t   m   p
0000020   /   s   a   g   e   -   1   0   .   9   /   l   o   c   a   l
0000040   /   l   i   b   /   g   a   p   /   ;   /   v   a   r   /   t
0000060   m   p   /   s   a   g   e   -   1   0   .   9   /   l   o   c
0000100   a   l   /   s   h   a   r   e   /   g   a   p   /  nl
0000116
[root@f52defcaf0a7 sage-10.9]#  local/bin/gap -E -r -q --bare --nointeract -c 'Print(JoinStringsWithSeparator(GAPInfo.RootPaths,";"), "\n");' > output
[root@f52defcaf0a7 sage-10.9]# od -a output
0000000   /   v   a   r   /   t   m   p   /   s   a   g   e   -   1   0
0000020   .   9   /   l   o   c   a   l   /   l   i   b   /   g   a   p
0000040   /   ;   /   v   a   r   /   t   m   p   /   s   a   g   e   -
0000060   1   0   .   9   /   l   o   c   a   l   /   s   h   a   r   e
0000100   /   g   a   p   /  nl
0000106

So the "belt and suspenders" solution to this problem is to use both -E and Print in both meson.build and gap/spkg-configure.m4.

- Marc

--
You received this message because you are subscribed to a topic in the Google Groups "sage-devel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sage-devel/dVVleHx0xP0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sage-devel+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq0u5iCbyEZX15Tqos%3D40SehuS5%2BwV%3DM6DzSB-Gq4q_u6w%40mail.gmail.com.

[Michael Orlitzky]

On 2026-03-03 12:08:25, Marc Culler wrote:
> So the "belt and suspenders" solution to this problem is to use both -E and
> Print in both meson.build and gap/spkg-configure.m4.

At the risk of this becoming a theme, Tobias started working on a PR that
would minimize these shenanigans:

https://github.com/tobiasdiez/sage/pull/19

The summary is that we can find the main GAP root on-the-fly by either

1. gap --print-gaproot (very official sounding)
2. pkg-config --variable=libdir libgap (also works, and is faster)
3. Doing either of those at build time, and putting the value
in GAP_ROOT_PATHS like we do now

In all three cases, we would need to append $HOME/.gap to the
list. (The existing trick prints $HOME/.gap, those above do not.)

Except for one corner case that we could quietly un-support, the list
of paths is only needed at startup, to search for GAP files that may
have changed and (if so) would invalidate your cached workspace.

[Marc Culler]

Actually, Print cannot be easily used in meson.build because meson won't allow the string "\n".

[Marc Culler]

Getting rid of these  shenanigans sounds like progress to me.

- Marc

[Marc Culler]

Dima,

Thank you very much for your suggestion of adding -E to the gap command lines where sage queries GAP for its root paths.  That worked perfectly, and I have now posted a pre-release of a SageMath 10.9 AppImage based on 10.9beta7.

- Marc