Tensor Multiprocessing broken. SageMath 9.5 on Apple Mac OSX 12.1 . SageMath-9.5_x86_64

[Alan Stafford]

I get errors in this notebook if I set the variable tpar to any value greater than one. (Number of Tensor processing cores.)

http://alan.atstafford.co.uk/Sage9.5Problem.ipynb

Regards 

Alan

[Dima Pasechnik]

Attached is converted to .rst notebook, just in case. And the
problematic part is here:

var('tpar','lpar')

tpar=1 #I want to set to greater than 1 but it will not work for
Sage 9.5. Works with Sage 9.1 if I hack ncpus to return 4.
lpar=4

Parallelism().set('tensor',tpar)
Parallelism().set('linbox',lpar)
show(Parallelism())
sage.parallel.ncpus.ncpus()

> --
> You received this message because you are subscribed to the Google Groups "sage-support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-support...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/d4e24e6b-0421-4f46-8535-9a493a8fa8d4n%40googlegroups.com.

[Alan Stafford]

Are you saying I should remove the lines above and just use the SAGE_NUM_THREADS environment variable?

Alan

[Dima Pasechnik]

On Tue, Jan 25, 2022 at 7:34 PM 'Alan Stafford' via sage-support
<sage-s...@googlegroups.com> wrote:
>
>
> Are you saying I should remove the lines above and just use the SAGE_NUM_THREADS environment variable?

no, not at all, I was just extracting these from your notebook, so
that a casual reader can see what your have a problem with.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-support/694d5ba3-57ad-4fab-9d1c-59e5345a21a8n%40googlegroups.com.

[Eric Gourgoulhon]

Hi,

Le mardi 25 janvier 2022 à 12:54:57 UTC+1, alan_thoma...@yahoo.co.uk a écrit :

I get errors in this notebook if I set the variable tpar to any value greater than one. (Number of Tensor processing cores.)

http://alan.atstafford.co.uk/Sage9.5Problem.ipynb

Your notebook works well with tpar=4 on my Ubuntu computer running Sage 9.5.rc4.

What kind of error do you get?

Eric.

[Alan Stafford]

Process SpawnPoolWorker-1: Traceback (most recent call last): File "/private/var/tmp/sage-9.5-current/local/var/lib/sage/venv-python3.9.9/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/private/var/tmp/sage-9.5-current/local/var/lib/sage/venv-python3.9.9/lib/python3.9/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/private/var/tmp/sage-9.5-current/local/var/lib/sage/venv-python3.9.9/lib/python3.9/multiprocessing/pool.py", line 114, in worker task = get() File "/private/var/tmp/sage-9.5-current/local/var/lib/sage/venv-python3.9.9/lib/python3.9/multiprocessing/queues.py", line 368, in get return _ForkingPickler.loads(res) File "/private/var/tmp/sage-9.5-current/local/var/lib/sage/venv-python3.9.9/lib/python3.9/site-packages/sage/manifolds/differentiable/chart.py", line 36, in <module>

[John H Palmieri]

Are you using OS X? Could this possibly be due to how Python handles multiprocessing on OS X vs. other platforms? (See

https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods, in particular the comment "On macOS, the spawn start method is now the default" and https://bugs.python.org/issue33725.)

[Alan Stafford]

I am using Mac OS 12.1 Monterey. The notebook works on Sagemath 9.1 if I hack ncpus.py to return 4. 

[Alan Stafford]

SageMath 9.1 works with the 9.5 version of ncpus.py so one can disregard my tinkering with  it. 

[Alan Stafford]

ImportError: cannot import name 'NumberField' from partially initialized module 'sage.rings.number_field.number_field' (most likely due to a circular import) (/private/var/tmp/sage-9.5-current/local/var/lib/sage/venv-python3.9.9/lib/python3.9/site-packages/sage/rings/number_field/number_field.py) Process SpawnPoolWorker-3:

[Eric Gourgoulhon]

Le mardi 25 janvier 2022 à 23:59:20 UTC+1, John H Palmieri a écrit :

Are you using OS X? Could this possibly be due to how Python handles multiprocessing on OS X vs. other platforms? (See

https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods, in particular the comment "On macOS, the spawn start method is now the default" and https://bugs.python.org/issue33725.)

Thank you John for pointing this!

Alan, could you try to run your notebook after having added the following lines at the top of it?

import multiprocessing as mp

mp.set_start_method('fork')

Eric.

[Alan Stafford]

That works. I get all cores used.

[Alan Stafford]

mp.set_start_method('fork',force=True)

lets one rerun the notebook without restarting.

[Eric Gourgoulhon]

Le jeudi 27 janvier 2022 à 18:59:36 UTC+1, alan_thoma...@yahoo.co.uk a écrit :

That works. I get all cores used.

Great!

I've opened the ticket https://trac.sagemath.org/ticket/33241 for that.

Eric. 

[Alan Stafford]

If I restart the kernel multiprocessing gets reset back to spawn. Rather than have to set it back to fork in every cell it should be set to fork by the kernel restart code.