sage, pari and threading

[Martin R]

I ran into a rather strange problem concerning pari and threading.  After my signature is a small working and a small non-working example giving context, and here is a minimal non-working example:

sage: import threading
sage: t = threading.Thread(target=pari.version)
sage: t.start()
Exception in thread Thread-1 (version):
Traceback (most recent call last):
  File "/usr/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
sage:     self.run()
  File "/usr/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "cypari2/pari_instance.pyx", line 759, in cypari2.pari_instance.Pari.version
  File "cypari2/auto_instance.pxi", line 39540, in cypari2.pari_instance.Pari_auto.version
cysignals.signals.SignalError: Segmentation fault

I found https://github.com/sagemath/sage/issues/28800, which seems to say that this is an issue, with the caveat that I don't know anything about threading.  I did not write the code responsible for threading in the application I'm working on (FindStat), but it seems that threading is somewhat unavoidable, since it uses waitress to serve.  So, I have two questions:

1) is the issue linked above indeed the reason for the segfault I am observing?

2) is there a workaround?

Concerning 2, in case it helps: all code in FindStat accessing any "mathematical" functionality is isolated (in fact, because it is user-contributed).  Unfortunately, I don't really understand how the threads in FindStat are started.

Also concerning 2, ChatGPT suggested to replace waitress using gunicorn.  However, this comes with its own problems, because there is a lot of dynamic data which must be shared, and access to which must be fast.

Many thanks and best wishes,

Martin

# Example for context

sage: import threading
sage: fun = lambda: print(BinaryTrees(3).cardinality())
sage: t = threading.Thread(target=fun)
sage: t.start()
5
sage: fun = lambda: print(RootedTrees(3).cardinality())
sage: t = threading.Thread(target=fun)
sage: t.start()
sage: Exception in thread Thread-3 (<lambda>):
Traceback (most recent call last):
  File "sage/misc/cachefunc.pyx", line 1019, in sage.misc.cachefunc.CachedFunction.__call__
KeyError: ((3,), ())

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "<ipython-input-16-3e6183557abd>", line 1, in <lambda>
  File "/home/martin/sage/src/sage/combinat/rooted_tree.py", line 737, in cardinality
    return number_of_rooted_trees(self._n)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "sage/misc/cachefunc.pyx", line 1024, in sage.misc.cachefunc.CachedFunction.__call__
  File "/home/martin/sage/src/sage/combinat/rooted_tree.py", line 53, in number_of_rooted_trees
    return sum(sum(d * number_of_rooted_trees(d) for d in k.divisors()) *
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/martin/sage/src/sage/combinat/rooted_tree.py", line 53, in <genexpr>
    return sum(sum(d * number_of_rooted_trees(d) for d in k.divisors()) *
                                                          ^^^^^^^^^^^^
  File "sage/rings/integer.pyx", line 3184, in sage.rings.integer.Integer.divisors
    return pari_divisors_small(self)
  File "sage/libs/pari/convert_sage.pyx", line 431, in sage.libs.pari.convert_sage.pari_divisors_small
  File "sage/libs/pari/convert_sage.pyx", line 468, in sage.libs.pari.convert_sage.pari_divisors_small
cysignals.signals.SignalError: Segmentation fault

[David Roe]

See https://github.com/sagemath/cypari2/issues/107 and the other issues linked there.  This has been an issue for the LMFDB for 4 years (it prevents auto-restarting the webserver when files are changed, which is a convenient feature when developing, and also prevents us from offering large file downloads because of single-thread timeouts); we tried fixing it but did not succeed.  There is an attempt started at https://github.com/sagemath/cypari2/pull/116, but it's not working yet.

Using gunicorn (multiple processes rather than multiple threads) should work.  The problem arises in the LMFDB's context when using the Werkzeug server that comes with Flask by default; things seem to work fine on our production servers that use gunicorn.  But, as you say, that means you don't get shared memory.

The actual solution probably requires changing how cypari2 interacts with Pari's memory management.  There have been a few attempts, but nothing that seems to have fixed this threading issue.  I don't have time to dive into this (again), but I'm happy to test fixes if someone else wants to tackle it.

David

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/sage-devel/817b1a12-4258-43f3-a1ce-a8ebd1937983n%40googlegroups.com.

[julian...@fsfe.org]

As David said, threading usually leads to trouble with cypari at the moment so you want to use forks instead (also SageMath doesn't claim to be too thread-safe so if your machinery spawns multiple threads you'll see crashes anyway.)

With forks that are created by external libraries it can also be tricky to get things right, see https://github.com/flatsurf/flatsurvey/blob/master/flatsurvey/dask/task.py#L54 for some ideas what was needed to get SageMath to work with dask spawning processes.

Feel free to reach out directly if you need some help here.

julian

[Martin R]

Thank you so much for the offer!

I must admit that this does not look very encouraging :-(