SAGE_NUM_THREADS

[enriqu...@gmail.com]

If I set the variable SAGE_NUM_THREADS to a number of cores before opening a sage session (command line or jupyterlab) there is a different behaviour in 10.5 and in 10.6beta;

parallel computation can be done in 10.5, there is no effect in the beta version (I do not know when it started to fail). In the beta version parallel computation is possible declaring the number of cores using the following hack (suggested to me at some point by Miguel Marco)

def miscpus():
    return "put here the number of cores"
sage.parallel.ncpus.ncpus=miscpus

Is it possible to know when SAGE_NUM_THREADS lost its effects?

Thanks, Enrique.

[Dima Pasechnik]

what is the output of sage.parallel.ncpus.ncpus() in 10.5 and in
10.6.beta* for you?
(without using the hack above). Are they using the same Python version?

The code for sage.parallel.ncpus.ncpus didn't change for a year, since 10.3.

Dima




> Thanks, Enrique.
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/sage-devel/fe02ae5f-64ab-4fb5-a826-d97f3d418dccn%40googlegroups.com.

[enriqu...@gmail.com]

For 10.5 with python 3.12.5, sage.parallel.ncpus.ncpus() is 1 without giving a value to SAGE_NUM_THREADS and 16 if sage is opened as "SAGE_NUM_THREADS=16 sage". For 10.6.beta9 (python 3.13.2), the value is always one; in another computer 10.6.beta9 is using python 3.12.5 and the value is always one. 

[John H Palmieri]

I see this too (I'm using OS X, don't know the situation for other platforms). I think the problem is somehow with Python3. If I modify the src/bin/sage-env script (which is where SAGE_NUM_THREADS gets read and perhaps modified) to directly run the script sage-num-threads.py, I get an error, and that error means that sage-env uses the default value of "1" for SAGE_NUM_THREADS.

I applied this change:

diff --git a/src/bin/sage-env b/src/bin/sage-env
index 5a53ab1d2c..3000b4d983 100644
--- a/src/bin/sage-env
+++ b/src/bin/sage-env
@@ -580,6 +580,9 @@ case "$SAGE_NUM_THREADS,$SAGE_NUM_THREADS_PARALLEL" in
         # First, figure out the right values for SAGE_NUM_THREADS (default
         # number of threads) and SAGE_NUM_THREADS_PARALLEL (default number of
         # threads when parallel execution is asked explicitly).
+
+        sage-num-threads.py
+
         sage_num_threads_array=$(sage-num-threads.py 2>/dev/null || echo 1 2 1)
         sage_num_threads_array="${sage_num_threads_array% *}" # strip third item
         SAGE_NUM_THREADS="${sage_num_threads_array% *}" # keep first item

If I run "sage" or "sage -sh", I see

/Users/palmieri/Sage/git/sage/local/var/lib/sage/venv-python3.12/bin/sage-num-threads.py:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  __import__('pkg_resources').require('sagemath-standard==10.6b8')
Traceback (most recent call last):
  File "/Users/palmieri/Sage/git/sage/local/var/lib/sage/venv-python3.12/lib/python3.12/site-packages/pkg_resources/__init__.py", line 656, in _build_master
    ws.require(__requires__)
  File "/Users/palmieri/Sage/git/sage/local/var/lib/sage/venv-python3.12/lib/python3.12/site-packages/pkg_resources/__init__.py", line 1063, in require
    needed = self.resolve(parse_requirements(requirements))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/palmieri/Sage/git/sage/local/var/lib/sage/venv-python3.12/lib/python3.12/site-packages/pkg_resources/__init__.py", line 890, in resolve
    dist = self._resolve_dist(
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/palmieri/Sage/git/sage/local/var/lib/sage/venv-python3.12/lib/python3.12/site-packages/pkg_resources/__init__.py", line 936, in _resolve_dist
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (sphinxcontrib-htmlhelp 2.0.5 (/Users/palmieri/Sage/git/sage/local/var/lib/sage/venv-python3.12/lib/python3.12/site-packages), Requirement.parse('sphinxcontrib-htmlhelp>=2.0.6'), {'sphinx'})

Sage provides sphinxcontrib-htmlhelp 2.0.5. I don't know what's looking for 2.0.6; I got no hits when I ran 'git grep "2[.]0[.]6" '.

[John H Palmieri]

I think the problem is Sphinx: my guess is that when the Sphinx version got bumped in https://github.com/sagemath/sage/pull/39577, sphinxcontrib-htmlhelp also needed a bump. If I look at the Sphinx build log, I see:

spkg-pipinst] ERROR: Could not find a version that satisfies the requirement sphinxcontrib-htmlhelp>=2.0.6 (from sphinx) (from versions: 2.0.5)
[spkg-pipinst] ERROR: No matching distribution found for sphinxcontrib-htmlhelp>=2.0.6

I also see this:

[spkg-pipinst] Warning: The installation needed to use "--no-deps --ignore-installed --ignore-requires-python" to succeed. This means that a dependencies file in build/pkgs/ needs to be updated. Please report this to sage-...@googlegroups.com, including the build log of this package.

I don't know why the build was considered successful after this, but apparently it was. Anyway, consider this as having been reported.

[Dima Pasechnik]

By the way, does anyone knows why
src/sage/lfunctions/zero_sums.pyx has it's own ncpus() defined, as
well as a global cdef NCPUS.

This looks like there was more code that was replaced by
sage.parallel.ncpus.ncpus in a very light-touch way.

> To view this discussion visit https://groups.google.com/d/msgid/sage-devel/5f4e3fd4-d49d-44e8-9864-d0e027d61546n%40googlegroups.com.

[Dima Pasechnik]

I just bumped it in https://github.com/sagemath/sage/pull/39680
Please review.

> To view this discussion visit https://groups.google.com/d/msgid/sage-devel/aa44fb52-fec7-4975-a6eb-e291cf22c44cn%40googlegroups.com.

[John H Palmieri]

By the way, we might want to modify sage.parallel.ncpus.ncpus for Python 3.13 to use process_cpu_count instead of cpu_count, since as of Python 3.13, this is what Python's multiprocessing package does.

[Emmanuel Charpentier]

Data point : on 10.6.beta9 with Python 13.2, I get 1 if sage is started without defining SAGE_NUM_THREADS, and n if I define SAGE_NUM_THREADS being n:

charpent@brique:~$ sage -c 'print(sys.version)' 3.13.2 (main, Feb 5 2025, 01:23:35) [GCC 14.2.0] charpent@brique:~$ sage -c 'print(version())' SageMath version 10.6.beta9, Release Date: 2025-03-09 charpent@brique:~$ sage -c 'print(sage.parallel.ncpus.ncpus())' 1 charpent@brique:~$ SAGE_NUM_THREADS=8 sage -c 'print(sage.parallel.ncpus.ncpus())' 8

HTH,

​

[enriqu...@gmail.com]

I tried this patch in 10.6rc0 since the variable has no effect and I got a similar error (the same one making system jupyterlab unusable with this kernel): wher the last line is:

pkg_resources.ContextualVersionConflict: (numpy 2.2.3 (/usr/local/sagedev/local/var/lib/sage/venv-python3.13/lib/python3.13/site-packages), Requirement.parse('numpy<2,>=1.21'), {'matplotlib'})