Sage 8.1 eats memory while 7.5.1 did not

[Marco Caliari]

Hi, the following script

def test(m,c,precision):
    M = 3*m
    RRR = RealField(prec = precision)
    coef02 = [RRR(1/i) for i in [1..M+1]]
    g = coef02[M]
    for i in [M-1..2,step=-1]:
        g = x*g+coef02[i]
    ME = 32
    disk = [exp (2*pi.n(precision)*I*i/ME) for i in range(ME)]
    epsilon1 = max([abs(g(x=z)) for z in disk])
    return
m = 40
for c in [1/2..10,step=1/2]:
    for ell in [1..10]:
        test(m,c,165)

runs smoothly on my Sage 7.5.1 (ppa for Ubuntu 14.04/Mint 17.3) and takes 2.5% of my memory. But in Sage 8.1 (64bit bynary for Ubuntu 14.04) memory consumption slowly increases up to 7.3%. If I increase the length of the loops, memory consumption continues to increase. I already got a better implementation of the script on ask.sagemath, but I think this is a regression wrt 7.5.1.

[Nils Bruin]

On Friday, February 9, 2018 at 11:03:11 AM UTC, Marco Caliari wrote:

Hi, the following script

def test(m,c,precision):
    M = 3*m
    RRR = RealField(prec = precision)
    coef02 = [RRR(1/i) for i in [1..M+1]]
    g = coef02[M]
    for i in [M-1..2,step=-1]:
        g = x*g+coef02[i]
    ME = 32
    disk = [exp (2*pi.n(precision)*I*i/ME) for i in range(ME)]
    epsilon1 = max([abs(g(x=z)) for z in disk])
    return
m = 40
for c in [1/2..10,step=1/2]:
    for ell in [1..10]:
        test(m,c,165)

Indeed, comparing the objects on the heap that weren't there before the loop I find:

[(<type 'builtin_function_or_method'>, 1),
 (<type 'sage.rings.rational.Rational'>, 1),
 (<type 'instancemethod'>, 1),
 (<type 'set'>, 1),
 (<type 'dict'>, 1),
 (<type 'tuple'>, 3),
 (<type 'list'>, 3),
 (<type 'frame'>, 3),
 (<type 'weakref'>, 28),
 (<type 'sage.rings.real_mpfi.RealIntervalFieldElement'>, 6200),
 (<type 'sage.rings.real_mpfr.RealNumber'>, 29999),
 (<type 'sage.rings.complex_number.ComplexNumber'>, 1457000)]

The real numbers mostly seem to be approximations to pi. Using objgraph.show_backrefs I'm not getting anything useful. We're definitely leaking but I wasn't able to identify a cache that's keeping references. Is that any change that we're doing something wrong with an INCREF/DECREF ?

[Marco Caliari]

I tested sage 8.0 and it works fine.

[Marco Caliari]

Maybe it was clear, but the offending part is

 

g = coef02[M]
for i in [M-1..2,step=-1]:
    g = x*g+coef02[i]


Something like

P = PolynomialRing(RRR,"x")
g = P(coef02[2:])

works without any problem.

On Friday, 9 February 2018 14:44:23 UTC+1, Nils Bruin wrote:

[Vincent Delecroix]

I isolated a leak in

a = pi * I * RR.one()

I opened #24745 (https://trac.sagemath.org/ticket/24745)

Vincent

[Dima Pasechnik]

guppy.hpy() finds 149 objects 

dict of sympy.core.assumptions.ManagedProperties
on the heap

Not sure where sympy is called here. While doing max() ?

[Simon King]

On 2018-02-16, Vincent Delecroix <20100.d...@gmail.com> wrote:
> I isolated a leak in
>
> a = pi * I * RR.one()

For the record: a = I*RR.one() leaks, a=pi*I resp. a=pi*RR.on() doesn't
leak.

[Dima Pasechnik]

in the original example, removing '*I'  from the line "disk = ..." still produces a leak.

So there are >1 leaks...

 

[Ralf Stephan]

On Friday, February 16, 2018 at 10:05:02 AM UTC+1, Dima Pasechnik wrote:

guppy.hpy() finds 149 objects 

dict of sympy.core.assumptions.ManagedProperties
on the heap

The solution of the original issue did not involve sympy so I was trying to reproduce this, but when trying to SAGE_ROOT/local/bin/pip install guppy I got

pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.

Collecting guppy

  Could not fetch URL https://pypi.python.org/simple/guppy/: There was a problem confirming the ssl certificate: Can't connect to HTTPS URL because the SSL module is not available. - skipping

  Could not find a version that satisfies the requirement guppy (from versions: )

No matching distribution found for guppy

How did you install guppy?

[Dima Pasechnik]

I ran ./sage --pip install guppy

Anyhow, it seems that my guess about a leak in sympy is wrong.

These sympy.core.assumptions.ManagedProperties is something one gets by

importing sympy (or one of its components)

[Samuel Lelievre]

Sat 2018-02-17 07:38:20 UTC, Ralf Stephan:

Installing extra packages with Sage's pip as in

    sage --pip install guppy

will only succeed if Sage's Python has ssl support.

For this to be the case, you need one of the following:

- have system-wide ssl support before building Sage.
  In Ubuntu, this would be achieved by running

      sudo apt full-upgrade
      sudo apt install openssl libssl-dev

- install ssl support before building Sage:

      ./sage -i openssl
      make

- install ssl support once Sage is already built:

      sage -i openssl
      sage -f python2
      sage -i pyopenssl

[Travis Scrimshaw]

As a variant of option 3: install a system-wide ssl after building Sage: I believe this just requires rebuild python:

     sage -f python2

At least, this was all I had to do on some older version of Sage (I forget which since it has been over a year, maybe even 2, since I did this).

Best,
Travis