sage is slowding down PARI/GP by factor x2 (roughly)
116 views
Skip to first unread message
Vincent Delecroix
Dec 14, 2022, 3:23:51 PM12/14/22
to sage-devel
Dear all,
Some strange phenomenon makes execution of PARI/GP up to twice slower
when ran inside a Python process compared to its execution in GP.
Though this is not systematic, in a fresh python environment timing
are comparable with GP.
The slowdown has been first observed in the context of pari_jupyter
[1] and then with cypari2 when sage.all is loaded [2]. There is a
short code snippet in the description of [2] that you can try on your
own sage version (must be ran with "sage -python" and not "sage").
Any idea on what could be the source of the slowdown or some profiling
advices would be very welcome.
[1] https://github.com/sagemath/pari-jupyter/issues/27
[2] https://trac.sagemath.org/ticket/34850
Best
Vincent
Some strange phenomenon makes execution of PARI/GP up to twice slower
when ran inside a Python process compared to its execution in GP.
Though this is not systematic, in a fresh python environment timing
are comparable with GP.
The slowdown has been first observed in the context of pari_jupyter
[1] and then with cypari2 when sage.all is loaded [2]. There is a
short code snippet in the description of [2] that you can try on your
own sage version (must be ran with "sage -python" and not "sage").
Any idea on what could be the source of the slowdown or some profiling
advices would be very welcome.
[1] https://github.com/sagemath/pari-jupyter/issues/27
[2] https://trac.sagemath.org/ticket/34850
Best
Vincent
Dima Pasechnik
Dec 14, 2022, 3:30:49 PM12/14/22
to sage-devel
I vaguely recall something about the way gp is built, without linking libpari dynamically, and something about gp getting slower if built with threads.
So, probably, more info is needed on how exactly your comparisons are done, in regard of using, or not, threads, and dynamic linking.
--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAGEwAAm_%2BVfg%3D6po8-7B4eb7kW2UT%3DHjvXaR7yfUOxL930HhQQ%40mail.gmail.com.
David Roe
Dec 14, 2022, 3:38:24 PM12/14/22
to sage-...@googlegroups.com
It could be related to #31572, which notes a dramatic speed regression based on pari being compiled with pthread.
David
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq2yJEvXbRa8znksXtDm%2B-hkXN_EVpwN09X4tb8EY-0-kA%40mail.gmail.com.
Dima Pasechnik
Dec 14, 2022, 3:57:49 PM12/14/22
to sage-...@googlegroups.com
Does #31572 actually mean that pari threads are more harmful than useful?
If so, perhaps, we can create a special multithreaded, differerently
named, libpari for giac, and
build our "main" libpari single-threaded?
Dima
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAChs6_mqWOpnHAe4-cYTxg63y5ri4vFuJxZRasMUzsEMYzTNAQ%40mail.gmail.com.
If so, perhaps, we can create a special multithreaded, differerently
named, libpari for giac, and
build our "main" libpari single-threaded?
Dima
Vincent Delecroix
Dec 14, 2022, 4:06:18 PM12/14/22
to sage-...@googlegroups.com
Note that in #34850 I execute the very same command, namely
pari("quadclassunit(1 - 2^100)"), once in a fresh python session and
once preceeded by "import sage.all". So whatever linking and whatever
threading option has been used to compile pari these are the same for
both executions. In particular, I do not see any relations between
#31572 (which tests two pari versions) and #34850 (which tests a
single pari version). Could you either read carefully the description
of #34850 or explain the link between #31572 and #34850 if any?
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq0dH8%3DTp1z_owfAbCw-ED_W%3DHjuh0NQp%2B9MUatapfxKXw%40mail.gmail.com.
pari("quadclassunit(1 - 2^100)"), once in a fresh python session and
once preceeded by "import sage.all". So whatever linking and whatever
threading option has been used to compile pari these are the same for
both executions. In particular, I do not see any relations between
#31572 (which tests two pari versions) and #34850 (which tests a
single pari version). Could you either read carefully the description
of #34850 or explain the link between #31572 and #34850 if any?
Dima Pasechnik
Dec 14, 2022, 7:41:58 PM12/14/22
to sage-devel
On Wed, 14 Dec 2022, 21:06 Vincent Delecroix, <20100.d...@gmail.com> wrote:
Note that in #34850 I execute the very same command, namely
pari("quadclassunit(1 - 2^100)"), once in a fresh python session and
once preceeded by "import sage.all". So whatever linking and whatever
threading option has been used to compile pari these are the same for
both executions.
oops, sorry, I was too quick. I guess here it is a change in memory layout that causes the slowdown.
You also mention that it might be ipython - can this be tested by building Sage without it?
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAGEwAAmj4FkYzJC07v80S%3DqU%3DVOH8vtysU%3Dg5tOs10-qV8Sqxg%40mail.gmail.com.
Vincent Delecroix
Dec 15, 2022, 4:30:01 AM12/15/22
to sage-...@googlegroups.com
On Thu, 15 Dec 2022 at 01:42, Dima Pasechnik <dim...@gmail.com> wrote:
>
>
>
> On Wed, 14 Dec 2022, 21:06 Vincent Delecroix, <20100.d...@gmail.com> wrote:
>>
>> Note that in #34850 I execute the very same command, namely
>> pari("quadclassunit(1 - 2^100)"), once in a fresh python session and
>> once preceeded by "import sage.all". So whatever linking and whatever
>> threading option has been used to compile pari these are the same for
>> both executions.
>
>
>
> oops, sorry, I was too quick. I guess here it is a change in memory layout that causes the slowdown.
I don't quite believe it. The PARI/GP stack is allocated before the
>
>
>
> On Wed, 14 Dec 2022, 21:06 Vincent Delecroix, <20100.d...@gmail.com> wrote:
>>
>> Note that in #34850 I execute the very same command, namely
>> pari("quadclassunit(1 - 2^100)"), once in a fresh python session and
>> once preceeded by "import sage.all". So whatever linking and whatever
>> threading option has been used to compile pari these are the same for
>> both executions.
>
>
>
> oops, sorry, I was too quick. I guess here it is a change in memory layout that causes the slowdown.
`import sage.all` and is not affected by this call. What kind of
change in memory could cause the slowdown here?
> You also mention that it might be ipython - can this be tested by building Sage without it?
slowdown in pari_jupyter. That is just a supposition.
Dima Pasechnik
Dec 15, 2022, 7:00:31 AM12/15/22
to sage-...@googlegroups.com
On Thu, Dec 15, 2022 at 9:30 AM Vincent Delecroix
latter happened at least once earlier on in Sage, was caused by a
clang bug, in fact). This time, it is caused by sympy.
If you replace
import sage.all
with
import sympy
you'll get the same slowdown...
You guys should invite me to Luminy and buy me a beer there ;-)
>
> > You also mention that it might be ipython - can this be tested by building Sage without it?
>
> I meant the base kernel from ipython that would be responsible for the
> slowdown in pari_jupyter. That is just a supposition.
This one I can rule out - I ran your tests in iPython shell, with
basically the same results, so no, iPython has
an alibi. I also tried cProfile, to see if there are extra things
executed for some reason
if sage.all is imported - but no, it's a genuine Pari slowdown.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAGEwAA%3DmHuOF%3DbERuFY2s0gN4RkWdAxxBKObPZDfie-tfqWQyQ%40mail.gmail.com.
<20100.d...@gmail.com> wrote:
>
> On Thu, 15 Dec 2022 at 01:42, Dima Pasechnik <dim...@gmail.com> wrote:
> >
> >
> >
> > On Wed, 14 Dec 2022, 21:06 Vincent Delecroix, <20100.d...@gmail.com> wrote:
> >>
> >> Note that in #34850 I execute the very same command, namely
> >> pari("quadclassunit(1 - 2^100)"), once in a fresh python session and
> >> once preceeded by "import sage.all". So whatever linking and whatever
> >> threading option has been used to compile pari these are the same for
> >> both executions.
> >
> >
> >
> > oops, sorry, I was too quick. I guess here it is a change in memory layout that causes the slowdown.
>
> I don't quite believe it. The PARI/GP stack is allocated before the
> `import sage.all` and is not affected by this call. What kind of
> change in memory could cause the slowdown here?
It's caused by changes in FPU status, or something like this (the
>
> On Thu, 15 Dec 2022 at 01:42, Dima Pasechnik <dim...@gmail.com> wrote:
> >
> >
> >
> > On Wed, 14 Dec 2022, 21:06 Vincent Delecroix, <20100.d...@gmail.com> wrote:
> >>
> >> Note that in #34850 I execute the very same command, namely
> >> pari("quadclassunit(1 - 2^100)"), once in a fresh python session and
> >> once preceeded by "import sage.all". So whatever linking and whatever
> >> threading option has been used to compile pari these are the same for
> >> both executions.
> >
> >
> >
> > oops, sorry, I was too quick. I guess here it is a change in memory layout that causes the slowdown.
>
> I don't quite believe it. The PARI/GP stack is allocated before the
> `import sage.all` and is not affected by this call. What kind of
> change in memory could cause the slowdown here?
latter happened at least once earlier on in Sage, was caused by a
clang bug, in fact). This time, it is caused by sympy.
If you replace
import sage.all
with
import sympy
you'll get the same slowdown...
You guys should invite me to Luminy and buy me a beer there ;-)
>
> > You also mention that it might be ipython - can this be tested by building Sage without it?
>
> I meant the base kernel from ipython that would be responsible for the
> slowdown in pari_jupyter. That is just a supposition.
basically the same results, so no, iPython has
an alibi. I also tried cProfile, to see if there are extra things
executed for some reason
if sage.all is imported - but no, it's a genuine Pari slowdown.
John Cremona
Dec 15, 2022, 7:05:14 AM12/15/22
to sage-...@googlegroups.com
On Thu, 15 Dec 2022 at 12:00, Dima Pasechnik <dim...@gmail.com> wrote:
>
> On Thu, Dec 15, 2022 at 9:30 AM Vincent Delecroix
> <20100.d...@gmail.com> wrote:
> >
> > On Thu, 15 Dec 2022 at 01:42, Dima Pasechnik <dim...@gmail.com> wrote:
> > >
> > >
> > >
> > > On Wed, 14 Dec 2022, 21:06 Vincent Delecroix, <20100.d...@gmail.com> wrote:
> > >>
> > >> Note that in #34850 I execute the very same command, namely
> > >> pari("quadclassunit(1 - 2^100)"), once in a fresh python session and
> > >> once preceeded by "import sage.all". So whatever linking and whatever
> > >> threading option has been used to compile pari these are the same for
> > >> both executions.
> > >
> > >
> > >
> > > oops, sorry, I was too quick. I guess here it is a change in memory layout that causes the slowdown.
> >
> > I don't quite believe it. The PARI/GP stack is allocated before the
> > `import sage.all` and is not affected by this call. What kind of
> > change in memory could cause the slowdown here?
>
> It's caused by changes in FPU status, or something like this (the
> latter happened at least once earlier on in Sage, was caused by a
> clang bug, in fact). This time, it is caused by sympy.
>
> If you replace
>
> import sage.all
>
> with
>
> import sympy
>
> you'll get the same slowdown...
>
> You guys should invite me to Luminy and buy me a beer there ;-)
A propos, which Sage developers will be in Luminy for the COUNT
>
> On Thu, Dec 15, 2022 at 9:30 AM Vincent Delecroix
> <20100.d...@gmail.com> wrote:
> >
> > On Thu, 15 Dec 2022 at 01:42, Dima Pasechnik <dim...@gmail.com> wrote:
> > >
> > >
> > >
> > > On Wed, 14 Dec 2022, 21:06 Vincent Delecroix, <20100.d...@gmail.com> wrote:
> > >>
> > >> Note that in #34850 I execute the very same command, namely
> > >> pari("quadclassunit(1 - 2^100)"), once in a fresh python session and
> > >> once preceeded by "import sage.all". So whatever linking and whatever
> > >> threading option has been used to compile pari these are the same for
> > >> both executions.
> > >
> > >
> > >
> > > oops, sorry, I was too quick. I guess here it is a change in memory layout that causes the slowdown.
> >
> > I don't quite believe it. The PARI/GP stack is allocated before the
> > `import sage.all` and is not affected by this call. What kind of
> > change in memory could cause the slowdown here?
>
> It's caused by changes in FPU status, or something like this (the
> latter happened at least once earlier on in Sage, was caused by a
> clang bug, in fact). This time, it is caused by sympy.
>
> If you replace
>
> import sage.all
>
> with
>
> import sympy
>
> you'll get the same slowdown...
>
> You guys should invite me to Luminy and buy me a beer there ;-)
meeting? I will.
John
Dima Pasechnik
Dec 15, 2022, 7:19:44 AM12/15/22
to sage-...@googlegroups.com
people who haven't made it on the list, I have no idea.
I should form a 1-person union and declare myself on strike w.r.t.,
quoting their announcement, "informal collaboration and for coding
projects related to PARI/GP, SageMath,..." :-)
Dima
John Cremona
Dec 15, 2022, 7:25:17 AM12/15/22
to sage-...@googlegroups.com
applicants. It may help that one of the organisers was my postdoc for
3 years...
David Lowry-Duda
Dec 16, 2022, 11:58:05 AM12/16/22
to sage-...@googlegroups.com
On Thu, Dec 15, 2022 at 12:19:28PM +0000, Dima Pasechnik wrote:
>On Thu, Dec 15, 2022 at 12:05 PM John Cremona <john.c...@gmail.com> wrote:
>>
>> A propos, which Sage developers will be in Luminy for the COUNT
>> meeting? I will.
>>
>I'm still waiting to hear from them. Or perhaps they don't notify
>people who haven't made it on the list, I have no idea.
>I should form a 1-person union and declare myself on strike w.r.t.,
>quoting their announcement, "informal collaboration and for coding
>projects related to PARI/GP, SageMath,..." :-)
>
>Dima
I'm also waiting to hear from them. But in principle, I'd like to be
>On Thu, Dec 15, 2022 at 12:05 PM John Cremona <john.c...@gmail.com> wrote:
>>
>> A propos, which Sage developers will be in Luminy for the COUNT
>> meeting? I will.
>>
>I'm still waiting to hear from them. Or perhaps they don't notify
>people who haven't made it on the list, I have no idea.
>I should form a 1-person union and declare myself on strike w.r.t.,
>quoting their announcement, "informal collaboration and for coding
>projects related to PARI/GP, SageMath,..." :-)
>
>Dima
there too.
- DLD
Dima Pasechnik
Dec 16, 2022, 3:47:46 PM12/16/22
to sage-...@googlegroups.com
https://www.cirm-math.fr/Listes/liste_pre_verif.php?id_renc=2805&num_semaine=0
Cheers
Dima
> - DLD
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/Y5yjk1mCDBY9kkIL%40icerm-dld.
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
Gonzalo Tornaria
Dec 18, 2022, 6:56:19 PM12/18/22
to sage-devel
Reply all
Reply to author
Forward
0 new messages