ZODB update required?

[Nils Bruin]

Thank you, Volker and JP, for excellent instructions on #13864

Unfortunately(?) module.py now completes successfully for me. I
haven't been able to nudge it into giving the "import segfault" we
were seeing before. No surprise if pydebug changes the memory layout
of objects. That issue is obviously a delicate memory corruption.

I am getting lots of "visit_decref: Assertion `gc->gc.gc_refs != 0'".
If I go up the stack a little and see on what "op" the refcount fails
(in gdb, do something like:)

(gdb) p PyString_AsString(PyObject_Repr(op))

I saw that BTree from ZODB was implied every time. Indeed, this bug
report:

https://bugs.launchpad.net/zodb/+bug/641481

seems to indicate was a problem with zodb3-3.7.0 we're using with
python 2.7. This supposedly is fixed in the current ZODB3-3.10.5.
Installing that seems to have worked for me, so probably just dropping
the new source into the spkg provides a valid new package. Can someone
confirm?

[Simon King]

Hi!

On 2012-12-28, Nils Bruin <nbr...@sfu.ca> wrote:
> Thank you, Volker and JP, for excellent instructions on #13864

+10

> I saw that BTree from ZODB was implied every time.

I saw different crashes. One is triggered by
sage: from sage.tests.benchmark import FiniteExtFieldMult
sage: B = FiniteExtFieldMult(GF(Integer(9), 'x'), Integer(2))

This crashes in the debug version of Sage from #13864, stating
python: Modules/gcmodule.c:326: visit_decref: Assertion `gc->gc.gc_refs != 0' failed.


There is some unpickling function involved, according to the gdb
backtrace. Perhaps the unpickler is not correctly raising the refcount?

Best regards,
Simon

[Simon King]

PS:

On 2012-12-28, Simon King <simon...@uni-jena.de> wrote:
> I saw different crashes. One is triggered by
> sage: from sage.tests.benchmark import FiniteExtFieldMult
> sage: B = FiniteExtFieldMult(GF(Integer(9), 'x'), Integer(2))

Actually,
sage: K = GF(9, 'x')
is enough to get a wrong refcount.

Could that be ZODB related?

Cheers,
Simon

[Nils Bruin]

On Dec 28, 2:17 pm, Simon King <simon.k...@uni-jena.de> wrote:
> Could that be ZODB related?

My guess is that ZODB data structures are lying around with fields
that Python 2.7 recognizes as pointers, but which haven't increased
the refcount. Thus, if GC examines them it may find more pointers than
explained by the refcount ... ERROR! These objects are just lying on
the python heap. It's a matter of chance whether a GC gets triggered
that stumbles into them. What you SHOULD do is investigate on what
object the assertion fails. That's how I found BTree to be implicated.
Its implication has nothing to do with what code actually gets
executed (other than triggering the right kind of garbage
collection).

Note that with "print ..." you can basically execute arbitrary C
instructions in gdb, which means you have the entire Python library at
your disposal. Of course, executing these routines will foul up the
heap even further, so don't expect this to go very far, but examining
the string representation of one or two objects on the stack should be
doable with a bit of luck.

I'm pretty sure we have to upgrade ZODB. The linked bugreport upstairs
declares "python 2.7 will be unsupported unless we have a
breakthrough", and luckily they did have a breakthrough, so 3.10 does
support 2.7.

ZODB is just a database that gets used (and several instances
initialized at startup!) by sage.

[Simon King]

On 2012-12-28, Nils Bruin <nbr...@sfu.ca> wrote:

> On Dec 28, 2:17 pm, Simon King <simon.k...@uni-jena.de> wrote:
>> Could that be ZODB related?
>
> My guess is that ZODB data structures are lying around with fields
> that Python 2.7 recognizes as pointers, but which haven't increased
> the refcount. Thus, if GC examines them it may find more pointers than
> explained by the refcount ... ERROR!

The following function call crashes
sage.databases.conway.ConwayPolynomials().has_polynomial(3,2)
and tracing through it, one sees that ZODB is involved:
> /home/simon/SAGE/debug/sage-5.6.beta1/local/lib/python2.7/site-packages/sage/databases/db.py(125)_load_()
124
--> 125 def _load_(self):
126 import BTrees.OOBTree

ipdb>
> /home/simon/SAGE/debug/sage-5.6.beta1/local/lib/python2.7/site-packages/sage/databases/db.py(126)_load_()
125 def _load_(self):
--> 126 import BTrees.OOBTree
127 from ZODB import FileStorage, DB

> What you SHOULD do is investigate on what
> object the assertion fails.

> I'm pretty sure we have to upgrade ZODB. The linked bugreport upstairs
> declares "python 2.7 will be unsupported unless we have a
> breakthrough", and luckily they did have a breakthrough, so 3.10 does
> support 2.7.
>
> ZODB is just a database that gets used (and several instances
> initialized at startup!) by sage.

See above: It *is* involved.

How can I upgrade it? Is there a trac ticket?

Cheers,
Simon

[Nils Bruin]

On Dec 28, 2:37 pm, Simon King <simon.k...@uni-jena.de> wrote:

> How can I upgrade it? Is there a trac ticket?

see
http://trac.sagemath.org/sage_trac/ticket/13864#comment:79

I haven't made a ticket yet because I didn't know if it was a real
problem or just a silly artifact. Making a new spkg should be easy:
just drop in the new source. Coming up ...

[Simon King]

On 2012-12-28, Nils Bruin <nbr...@sfu.ca> wrote:

Then you may look at #10352.

But alas, the zodb3-3.10.1.spkg from #10352 seems to not fix the problem.

Cheers,
Simon

[Simon King]

Hi Nils,

On 2012-12-28, Simon King <simon...@uni-jena.de> wrote:

Hooray! zodb3-3.10.1 does not solve the problem, but zodb3-3.10.5 does!

Best regards,
Simon