Test case that will segfault

[Ewald de Wit]

Hi guys,

I have attached some code below that will lead to a segmentation fault on Linux. The code iterates over a linked list of nodes and removes a node while doing so. Adding an extra ref to the removed node will prevent the crash, so the problem might be a refcount going to zero before its time. This is with Cython 0.17.1 - not sure if bug or known limitation.

Ewald

-------------------------------------------------------------------------------------------------------------------------

cdef class Node:

    cdef public int value

    cdef public Node nextNode

    def __init__(self, value):

        self.value = value

        self.nextNode = None

cdef class Chain:

    cdef public Node firstNode

    def __init__(self):

        self.firstNode = None

    def __iter__(self):

        n = self.firstNode

        while n is not None:

            yield n 

            n = n.nextNode

    

    def add(self, Node node):

        if self.firstNode is None:

             self.firstNode = node

        else:           

            for n in self: pass

            n.nextNode = node

    def remove(self, Node node):

        if node is self.firstNode:

            self.firstNode = node.nextNode 

        else:               

            for n in self:  

                if node is n.nextNode: 

                    n.nextNode = node.nextNode

                    break   

    def dump(self):         

        for n in self:          

            print n.value       

def test():                     

    c = Chain()

    c.add(Node(1))

    c.add(Node(2))

    c.add(Node(3))

    c.add(Node(4))

    n = c.firstNode

    while n is not None:

        if n.value == 3:

            c.remove(n)

        #nn = n         # Adding extra ref will prevent crash

        n = n.nextNode  # Crash will happen here at value 3

    c.dump()

test()

-------------------------------------------------------------------------------------------------------------------------

[Stefan Behnel]

Hi,

thanks for the test case.

Ewald de Wit, 16.11.2012 23:09:

> I have attached some code below that will lead to a segmentation fault on
> Linux. The code iterates over a linked list of nodes and removes a node
> while doing so. Adding an extra ref to the removed node will prevent the
> crash, so the problem might be a refcount going to zero before its time.
> This is with Cython 0.17.1 - not sure if bug or known limitation.

> -------------------------------------------------------------------------------------------------------------------------
> cdef class Node:
>
> cdef public int value
> cdef public Node nextNode
>
> def __init__(self, value):
> self.value = value
> self.nextNode = None
>
>
> cdef class Chain:
>
> cdef public Node firstNode
>
> def __init__(self):
> self.firstNode = None

Note that Python object attributes are automatically initialised to None,
so the above line is redundant.

The C code generated for the last line is as follows:

__Pyx_INCREF(((PyObject *)__pyx_v_n->nextNode));
__Pyx_DECREF(((PyObject *)__pyx_v_n));
__pyx_v_n = __pyx_v_n->nextNode;

That's definitely unhealthy code. For a test case, the following would be
enough:

"""
cdef class Node:
cdef public Node next

def test():
cdef Node n = Node()
n.next = Node()
n = n.next

test()
"""

However, this doesn't crash for me, likely because the DECREF() above
doesn't free the memory, just release it to CPython's memory allocator. So
we're just lucky here.

Stefan

[Stefan Behnel]

Stefan Behnel, 17.11.2012 12:08:

Here is a proposed fix:

https://github.com/cython/cython/commit/ea6a71acb5c79afb080855be1cb6ca30d283ec25

If Jenkins is happy with it, I'll add it to the release branch for 0.17.2.

Stefan