maybe a bug at utils/lib.py, in function decref
66 views
Skip to first unread message
Miguel Alarcos
Nov 6, 2009, 12:52:16 PM11/6/09
to rpyc
Hi tomer, hi everyone. I'm new here and I will try to explain in
English my best.
First, I want to tell about a bug I think there is at utils/lib.py,
in:
def decref(self, key):
self._lock.acquire()
try:
slot = self._dict[key]
#if slot[1] <= 1: # now
if slot[1] < 1: # <-- what I think is
best
del self._dict[key]
else:
slot[1] -= 1
self._dict[key] = slot
finally:
self._lock.release()
I had two problems with it, because an object is garbage collected
while it is really alive:
- KeyError trying to access an object that shoud be alive.
- I got an object that was not what I was looking for: an object is
garbage collected,
another object is created and Python gives it the same id.
If you need some code I can give it to you.
Second, just to say that this project is a really good work. Thanks
thanks very much.
=== layer5 === well, what can I say. I think this is the good point of
view. If I can help and it is
in my hands and knowledge, I will be pleased to do.
What I am trying to do now is to use elixir + py-notify + rpyc. I am
thinking in a GUI
where you simply declare what you want to see (even complex predicates
like "the surname of propietaries
whose cars are red and have gone to cinema last night", for example)
and the actions to the model.
With the persistence done by elixir and all of this distribuited in
imaginative ways thanks to rpyc.
Uf! I still have a lot to learn...
Best regards.
English my best.
First, I want to tell about a bug I think there is at utils/lib.py,
in:
def decref(self, key):
self._lock.acquire()
try:
slot = self._dict[key]
#if slot[1] <= 1: # now
if slot[1] < 1: # <-- what I think is
best
del self._dict[key]
else:
slot[1] -= 1
self._dict[key] = slot
finally:
self._lock.release()
I had two problems with it, because an object is garbage collected
while it is really alive:
- KeyError trying to access an object that shoud be alive.
- I got an object that was not what I was looking for: an object is
garbage collected,
another object is created and Python gives it the same id.
If you need some code I can give it to you.
Second, just to say that this project is a really good work. Thanks
thanks very much.
=== layer5 === well, what can I say. I think this is the good point of
view. If I can help and it is
in my hands and knowledge, I will be pleased to do.
What I am trying to do now is to use elixir + py-notify + rpyc. I am
thinking in a GUI
where you simply declare what you want to see (even complex predicates
like "the surname of propietaries
whose cars are red and have gone to cinema last night", for example)
and the actions to the model.
With the persistence done by elixir and all of this distribuited in
imaginative ways thanks to rpyc.
Uf! I still have a lot to learn...
Best regards.
tomer filiba
Nov 11, 2009, 8:48:49 AM11/11/09
to rp...@googlegroups.com
hey!
it's nice to get feedback from you! also, i'm sorry i've delayed so much in my replied, but the semester has started and i'm overbooked :)
anyway,
it's nice to get feedback from you! also, i'm sorry i've delayed so much in my replied, but the semester has started and i'm overbooked :)
anyway,
def decref(self, key):
self._lock.acquire()
try:
slot = self._dict[key]
#if slot[1] <= 1: # now
if slot[1] < 1: # <-- what I think is best
del self._dict[key]
else:
slot[1] -= 1
self._dict[key] = slot
finally:
self._lock.release()
if you have code snippets that recreate the bug it would be great... but i have a feeling that your change means objects will never be collected: the dictionary will hold objects whose refcount is zero. i can investigate it, but i will need to be able to recreate the bug.
thanks,
-tomer
An NCO and a Gentleman
thanks,
-tomer
An NCO and a Gentleman
Miguel Alarcos
Nov 13, 2009, 9:57:12 AM11/13/09
to rpyc
ups, maybe there was an error when I sent the post. I do it again.
-----------------------------------------------------------------
Hi tomer,
First of all, what I think. If we look at method add in class
RefCountingColl, we see that initially the value is 0. So, I
understand that it should be collected when that value is 0 when we
call decref.
We can see the next secuences:
add 0, decref --> object removed , OK
add 0, add 1, decref --> object removed, because of <= 1 , not OK.
You first check about the value, and only if it doesn't match then you
subtract.
And the code I promised: (by the way, I'm using Python 2.6.3)
A) when I get an object that I'm not looking for. The one I wanted had
been collected.
client:
#-------------------------------------
import rpyc
conn = rpyc.connect ("localhost", 18861)
store = conn.root.store
Order = conn.root.Order
estrella_levante = conn.root.estrella_levante #beer supplier
client = conn.root.client
Product = conn.root.Product
def show_reserved_products (product): #callback
print 'You have reserved %s' % product
#Product.reserveds is the py-notify signal that is thrown when you
output a product from de store
conn.root.connect (Product.reserveds, show_reserved_products)
p = store.output ('quinto', estrella_levante) #store.output gives me a
Product object
order = Order (client)
order.add_product (p) # <---------- this is the interesting
moment
# we pass as argument a Product object
conn.close ()
#-------------------------------------
server: please, pay attention at line 'print type(product)' inside
Order in method add_product
we will see Order and not Product class
#-------------------------------------
from itertools import count
from datetime import datetime
from notify.all import *
import rpyc
class Supplier(object):
def __init__(self, name):
self.name = name
def __repr__(self):
return 'Supplier: %s' % self.name
class Product(object):
reserveds = Signal() #when a product is set reserved, we throw
this signal
exposed_reserveds = reserveds
def __init__(self, name, supplier, id):
self.name = name
self.supplier = supplier
self.id = id
self.reserved = 'N'
def set_reserved (self):
self.reserved = 'Y'
Product.reserveds(self) #notify the observers
def __repr__(self):
return '<Product: %s, %s, Id: %s>' % (self.name,
self.supplier, self.id)
class Store(object):
def __init__(self, name):
self.name = name
self._products = []
def input (self, product):
self._products.append(product)
def output (self, product_name, supplier):
for i, p in enumerate(self._products):
if p.name == product_name and p.supplier == supplier:
ret = self._products.pop(i)
ret.set_reserved()
return ret
raise Exception('There is no: %s, %s' % (product_name,
supplier))
exposed_output = output
def __repr__(self):
ret = 'Store: %s\n' % self.name
for p in self._products:
ret += '%s\n' % p
return ret
class Client(object):
def __init__(self, name):
self.name = name
def __repr__(self):
return 'Client: %s' % self.name
class Order(object):
new_id = count()
def __init__(self, client):
self.id = Order.new_id.next()
self.client = client
self.products = []
self.datetime = datetime.now()
def add_product(self, product):
print type(product) # <--------- look at the output:
it is not Product but Order class
self.products.append(product)
exposed_add_product = add_product
def __repr__(self):
ret = ['<Order: %d, %s, Date: %s>' % (self.id, self.client,
self.datetime.strftime('%d-%m-%Y %H:%M'))]
for p in self.products:
ret.append('\t%s' % p)
return '\n'.join(ret)
##
#creating some objects
estrella_levante = Supplier('Estrella Levante')
store = Store('Trastienda')
client = Client('Miguel Angel')
for i in range(5):
store.input(Product('quinto', estrella_levante, '#%d' % i))
store.input(Product('tercio', estrella_levante, '#%d' % i))
class MyService(rpyc.Service):
exposed_store = store
exposed_Order = Order
exposed_estrella_levante = estrella_levante
exposed_client = client
exposed_Product = Product
def __init__(self, *args):
rpyc.Service.__init__(self, *args)
self.signal_handler = [] #when registering signal of py-
notify with the handler
def on_disconnect(self):
for s, h in self.signal_handler:
s.disconnect(h)
#we register py-notify signal with the handler this way, so when
closing we disconnect every connection
def exposed_connect(self, signal, handler):
handler = rpyc.async(handler)
self.signal_handler.append((signal, handler))
signal.connect(handler)
def exposed_disconnect(self, signal, handler):
handler = rpyc.async(handler) #py-notify disconnects passing
as argument the handler object, as we connected
try:
self.signal_handler.remove((signal, handler))
signal.disconnect(handler)
except:
pass
if __name__ == "__main__":
from rpyc.utils.server import ThreadedServer
t = ThreadedServer(MyService, port = 18861)
t.start()
#-------------------------------------
B) I'm trying to reproduce the other bug, when I get a KeyError. I
will post it if needed.
Regards and thanks for your attention.
-----------------------------------------------------------------
On 11 nov, 14:48, tomer filiba <tomerfil...@gmail.com> wrote:
> hey!
>
> it's nice to get feedback from you! also, i'm sorry i've delayed so much in
> my replied, but the semester has started and i'm overbooked :)
> anyway,
>
> def decref(self, key):
> self._lock.acquire()
> try:
> slot = self._dict[key]
> #if slot[1] <= 1: # now
> if slot[1] < 1: # <-- what I think is best
> del self._dict[key]
> else:
> slot[1] -= 1
> self._dict[key] = slot
> finally:
> self._lock.release()
>
> if you have code snippets that recreate the bug it would be great... but i
> have a feeling that your change means objects will never be collected: the
> dictionary will hold objects whose refcount is zero. i can investigate it,
> but i will need to be able to recreate the bug.
>
> thanks,
> -tomer
>
> An NCO and a Gentleman
>
-----------------------------------------------------------------
Hi tomer,
First of all, what I think. If we look at method add in class
RefCountingColl, we see that initially the value is 0. So, I
understand that it should be collected when that value is 0 when we
call decref.
We can see the next secuences:
add 0, decref --> object removed , OK
add 0, add 1, decref --> object removed, because of <= 1 , not OK.
You first check about the value, and only if it doesn't match then you
subtract.
And the code I promised: (by the way, I'm using Python 2.6.3)
A) when I get an object that I'm not looking for. The one I wanted had
been collected.
client:
#-------------------------------------
import rpyc
conn = rpyc.connect ("localhost", 18861)
store = conn.root.store
Order = conn.root.Order
estrella_levante = conn.root.estrella_levante #beer supplier
client = conn.root.client
Product = conn.root.Product
def show_reserved_products (product): #callback
print 'You have reserved %s' % product
#Product.reserveds is the py-notify signal that is thrown when you
output a product from de store
conn.root.connect (Product.reserveds, show_reserved_products)
p = store.output ('quinto', estrella_levante) #store.output gives me a
Product object
order = Order (client)
order.add_product (p) # <---------- this is the interesting
moment
# we pass as argument a Product object
conn.close ()
#-------------------------------------
server: please, pay attention at line 'print type(product)' inside
Order in method add_product
we will see Order and not Product class
#-------------------------------------
from itertools import count
from datetime import datetime
from notify.all import *
import rpyc
class Supplier(object):
def __init__(self, name):
self.name = name
def __repr__(self):
return 'Supplier: %s' % self.name
class Product(object):
reserveds = Signal() #when a product is set reserved, we throw
this signal
exposed_reserveds = reserveds
def __init__(self, name, supplier, id):
self.name = name
self.supplier = supplier
self.id = id
self.reserved = 'N'
def set_reserved (self):
self.reserved = 'Y'
Product.reserveds(self) #notify the observers
def __repr__(self):
return '<Product: %s, %s, Id: %s>' % (self.name,
self.supplier, self.id)
class Store(object):
def __init__(self, name):
self.name = name
self._products = []
def input (self, product):
self._products.append(product)
def output (self, product_name, supplier):
for i, p in enumerate(self._products):
if p.name == product_name and p.supplier == supplier:
ret = self._products.pop(i)
ret.set_reserved()
return ret
raise Exception('There is no: %s, %s' % (product_name,
supplier))
exposed_output = output
def __repr__(self):
ret = 'Store: %s\n' % self.name
for p in self._products:
ret += '%s\n' % p
return ret
class Client(object):
def __init__(self, name):
self.name = name
def __repr__(self):
return 'Client: %s' % self.name
class Order(object):
new_id = count()
def __init__(self, client):
self.id = Order.new_id.next()
self.client = client
self.products = []
self.datetime = datetime.now()
def add_product(self, product):
print type(product) # <--------- look at the output:
it is not Product but Order class
self.products.append(product)
exposed_add_product = add_product
def __repr__(self):
ret = ['<Order: %d, %s, Date: %s>' % (self.id, self.client,
self.datetime.strftime('%d-%m-%Y %H:%M'))]
for p in self.products:
ret.append('\t%s' % p)
return '\n'.join(ret)
##
#creating some objects
estrella_levante = Supplier('Estrella Levante')
store = Store('Trastienda')
client = Client('Miguel Angel')
for i in range(5):
store.input(Product('quinto', estrella_levante, '#%d' % i))
store.input(Product('tercio', estrella_levante, '#%d' % i))
class MyService(rpyc.Service):
exposed_store = store
exposed_Order = Order
exposed_estrella_levante = estrella_levante
exposed_client = client
exposed_Product = Product
def __init__(self, *args):
rpyc.Service.__init__(self, *args)
self.signal_handler = [] #when registering signal of py-
notify with the handler
def on_disconnect(self):
for s, h in self.signal_handler:
s.disconnect(h)
#we register py-notify signal with the handler this way, so when
closing we disconnect every connection
def exposed_connect(self, signal, handler):
handler = rpyc.async(handler)
self.signal_handler.append((signal, handler))
signal.connect(handler)
def exposed_disconnect(self, signal, handler):
handler = rpyc.async(handler) #py-notify disconnects passing
as argument the handler object, as we connected
try:
self.signal_handler.remove((signal, handler))
signal.disconnect(handler)
except:
pass
if __name__ == "__main__":
from rpyc.utils.server import ThreadedServer
t = ThreadedServer(MyService, port = 18861)
t.start()
#-------------------------------------
B) I'm trying to reproduce the other bug, when I get a KeyError. I
will post it if needed.
Regards and thanks for your attention.
-----------------------------------------------------------------
On 11 nov, 14:48, tomer filiba <tomerfil...@gmail.com> wrote:
> hey!
>
> it's nice to get feedback from you! also, i'm sorry i've delayed so much in
> my replied, but the semester has started and i'm overbooked :)
> anyway,
>
> def decref(self, key):
> self._lock.acquire()
> try:
> slot = self._dict[key]
> #if slot[1] <= 1: # now
> if slot[1] < 1: # <-- what I think is best
> del self._dict[key]
> else:
> slot[1] -= 1
> self._dict[key] = slot
> finally:
> self._lock.release()
>
> if you have code snippets that recreate the bug it would be great... but i
> have a feeling that your change means objects will never be collected: the
> dictionary will hold objects whose refcount is zero. i can investigate it,
> but i will need to be able to recreate the bug.
>
> thanks,
> -tomer
>
> An NCO and a Gentleman
>
Reply all
Reply to author
Forward
0 new messages