confused by bindings
Sam Falkner
can't make this work.
What I want to do is to have my test module set "stuff" in one of the
other modules, do its tests, and then check the results. Code might
be more clear:
--- file data.py ---
class Cell:
count = 0
def __init__(self, stuff):
self.count += 1
self.stuff = stuff
print 'count is', self.count
--- file test.py ---
class CellTestCase(unittest.TestCase):
def testforleaks(self):
data.Cell.count = 0
# do a whole bunch of stuff
# x = a gnarly collection of Cell objects
# count number of Cell objects in x
self.failUnlessEqual(count-of-Cells-from-x, data.Cell.count)
In other words, I'm trying to test for a leak of Cell objects. But,
no matter what I've tried, this doesn't work. I've tried
global-to-the-data-module variables for the count, I've tried the Borg
pattern from the cookbook, but nothing works.
When I run it this way, I get
count is 1
count is 1
count is 1
count is 1
...
and the test fails with 5499 != 0.
Okay, I feel stupid here. What am I doing wrong?
Thanks!
- Sam
Mitchell Morris
news:ii7k7yv...@central.sun.com:
> I'm confused. I'm a relative newbie, but am still surprised that I
> can't make this work.
>
> What I want to do is to have my test module set "stuff" in one of the
> other modules, do its tests, and then check the results. Code might
> be more clear:
>
> --- file data.py ---
>
> class Cell:
> count = 0
> def __init__(self, stuff):
> self.count += 1
> self.stuff = stuff
> print 'count is', self.count
Well, I learn something new every day. I would have expected this to raise a
NameError on the "self.count += 1" line. It doesn't though ... I'll have to
revisit my understanding of scoping now.
The short answer is that "self.count" isn't bound to the same thing as
"Cell.count". "self.count" is an attribute of the instance: each new Cell
instance gets its own copy. If you want to manipulate the class attribute,
then you have to say so via "Cell.count += 1" instead.
[snip]
> Okay, I feel stupid here. What am I doing wrong?
My guess is you're still wearing your Java hat, but I'm not entirely sure.
> Thanks!
>
> - Sam
so-much-for-my-resolve-not-to-followup-on-Usenet-any-more-ly y'rs
+Mitchell
Markus Schaber
Sam Falkner <samf+...@frii.com> schrub:
> class Cell:
> count = 0
This is the "class variable" called count.
> def __init__(self, stuff):
> self.count += 1
Here, you set an "instance variable" count. That's why you always get 1.
Try Cell.count += 1 here.
markus
--
"The strength of the Constitution lies entirely in the determination of
each citizen to defend it. Only if every single citizen feels duty
bound to do his share in this defense are the constitutional rights
secure." -- Albert Einstein
Don O'Donnell
Hi Sam,
Not stupid, just confusing instance variables with class variables. The
argument 'self' in your __init__ method refers to the class *instance*
that is being initialized, so that the statement self.count += 1 in
Cell.__init__ is really incrementing the new instance variable named
count. Here's how that statement works:
1. Looks for a variable named count in the dictionary of the new
instance.
2. Doesn't find it, so searches the Cell class dictionary.
3. Finds it, with a value of 0.
4. Adds 1 to it.
5. Stores it in the local instance dict.
So every instance you create will have a count value of 1, as you
noticed.
What you need to do to maintain an instance counter by incrementing the
class variable directly. Your class definition should look like this:
>>> class Cell:
count = 0
def __init__(self, stuff):
Cell.count += 1
self.stuff = stuff
print 'count is', Cell.count
>>> x = []
>>> x.append(Cell('a'))
count is 1
>>> x.append(Cell('b'))
count is 2
>>> x.append(Cell('c'))
count is 3
>>> Cell.count
3
To be complete, you should also define a __del__ method in your Cell
class which will decrement Cell.count when an instance is deleted:
def __del__(self):
Cell.count -= 1
Hope this helps.
Cheers,
Don
Chris Barker
> What I want to do is to have my test module set "stuff" in one of the
> other modules, do its tests, and then check the results. Code might
> be more clear:
In general, it is, but it helps a lot if it is working code (or, in this
case, broken code that runs)
A) count-of-Cells-from-x is not a legal variable name so I get a syntax
error (no "-" are allowed in variable names
B) you should probably provide a shortened version of:
# do a whole bunch of stuff
# x = a gnarly collection of Cell objects
# count number of Cell objects in x
That does something
C) I'm betting that this could all be in one file and exhibit the same
symptoms, so you can make an easy test case.
D) All that being said, I can take a guess at what you are trying to do:
you want a class attribute, count, that is increased every time an
instance is created.
The way you have it written, Cell.count is a reference to the object, 0.
At each instance creation you now point the instance variable,
self.count to a new object, 1 (0+1). the original Cell.count is still
pointing to the original 0. (I know I havn't explained this very well.
search the archives, there was an involved discussion about this a month
or so ago.
The solutions:
A) You can use an immutable type for count, so that you can change it's
contents, rather than creating a new one:
class Cell:
count = [0]
def __init__(self):
self.count[0] += 1
print 'count is', self.count
a = Cell()
b = Cell()
c = Cell()
d = Cell()
e = Cell()
# this works, but note that when you delete a few of these instances,
the count isnot affected:
del a,b,c
f = Cell()
# so you have a count of how many are created, not how many currently
exist.
Another option is to reference the attribute of the Cell class directly:
class Cell2:
count = 0
def __init__(self):
Cell2.count += 1
print 'count is', Cell2.count
a = Cell2()
b = Cell2()
c = Cell2()
d = Cell2()
e = Cell2()
--
Christopher Barker,
Ph.D.
ChrisH...@home.net --- --- ---
http://members.home.net/barkerlohmann ---@@ -----@@ -----@@
------@@@ ------@@@ ------@@@
Oil Spill Modeling ------ @ ------ @ ------ @
Water Resources Engineering ------- --------- --------
Coastal and Fluvial Hydrodynamics --------------------------------------
------------------------------------------------------------------------
Sam Falkner
> Sam Falkner <samf+...@frii.com> wrote in
> news:ii7k7yv...@central.sun.com:
>
> > I'm confused. I'm a relative newbie, but am still surprised that I
> > can't make this work.
> >
> > What I want to do is to have my test module set "stuff" in one of the
> > other modules, do its tests, and then check the results. Code might
> > be more clear:
> >
> > --- file data.py ---
> >
> > class Cell:
> > count = 0
> > def __init__(self, stuff):
> > self.count += 1
> > self.stuff = stuff
> > print 'count is', self.count
> Well, I learn something new every day. I would have expected this to
> raise a NameError on the "self.count += 1" line. It doesn't though
> ... I'll have to revisit my understanding of scoping now.
I might have thought this too, but look at the wonderful Borg recipe:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66531
Aren't they doing exactly what I was doing?
self.__dict__ = self.__shared_state
As I mentioned before, I tried using the "borg pattern" for my test
suite, and it didn't work either. If anyone is curious, I can
re-create the failure.
> The short answer is that "self.count" isn't bound to the same thing as
> "Cell.count". "self.count" is an attribute of the instance: each new Cell
> instance gets its own copy. If you want to manipulate the class attribute,
> then you have to say so via "Cell.count += 1" instead.
I've tried it this way before. I just (re)tried this, and the test
passes, if I run only that test case. But, if I run all my test
cases, it fails again with 5677 != 0; in other words, data.Cell.count
is 0 when looked at from my test module.
re-inserting my test case:
--- file test.py ---
class CellTestCase(unittest.TestCase):
def testforleaks(self):
data.Cell.count = 0
# do a whole bunch of stuff
# x = a gnarly collection of Cell objects
# count number of Cell objects in x
self.failUnlessEqual(count-of-Cells-from-x, data.Cell.count)
You can see that I reset the counter, via data.Cell.count = 0, to
clean up from the other test cases. As I mentioned above, my test now
succeeds when I try it your way, but not if I run the whole test suite.
> > Okay, I feel stupid here. What am I doing wrong?
> My guess is you're still wearing your Java hat, but I'm not entirely sure.
I'm offended, but only slightly. ;-)
Help! I'm running Python 2.1.1; I don't yet suspect a Python bug, but
I'm getting more confused by the minute.
- sam
Chris Barker
> I might have thought this too, but look at the wonderful Borg recipe:
>
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66531
>
> Aren't they doing exactly what I was doing?
> self.__dict__ = self.__shared_state
Nope. what is happening here is that the instance variable self.__dict__
is being set to refer to the the class variable self.__shared_state.
This would be the same as if you did, in your example:
class Cell:
count = 0
def __init__(self, stuff):
self.local_count = self.count + 1
print 'count is', self.count
however, that wouldn't be useful, as count is immutable, so the Class
variable, count would never get changed. What I suggested in an earlier
post is similar to this borg pattern:
A) You can use an immutable type for count, so that you can change it's
contents, rather than creating a new one:
class Cell:
count = [0]
def __init__(self):
self.count[0] += 1
print 'count is', self.count
Now Cell.count and self.count both refer to the same mutable object, and
thus changes are seen everywhere.
Of course, the whole point of the borg pattern is to replicate the
ENTIRE class, so you make sure that all instances have the same
.__dict__. That's a special case, and may or may not be what you want.
> re-inserting my test case:
>
> --- file test.py ---
>
> class CellTestCase(unittest.TestCase):
> def testforleaks(self):
> data.Cell.count = 0
> # do a whole bunch of stuff
> # x = a gnarly collection of Cell objects
> # count number of Cell objects in x
> self.failUnlessEqual(count-of-Cells-from-x, data.Cell.count)
>
> You can see that I reset the counter, via data.Cell.count = 0, to
> clean up from the other test cases. As I mentioned above, my test now
> succeeds when I try it your way, but not if I run the whole test suite.
Again, post some runnable code that exhibits your problem, it will be a
whole lot easier for us to figure out what is going on, and you may even
find your problem when you try to narrow it down.
Also, unless you decrease Cell.count when you delete an instance, I
can't see how this would be useful.
The name of this function implies that you are testing for memory leaks.
Python's refence counting scheme makes it pretty hard to have such
things. The only time they show up is with circular references, and I
think newer versions of Python even clean those up, so what are you
really trying to do here?
-Chris
Alex Martelli
news:ii7itef...@central.sun.com...
...
> I might have thought this too, but look at the wonderful Borg recipe:
>
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66531
>
> Aren't they doing exactly what I was doing?
> self.__dict__ = self.__shared_state
>
> As I mentioned before, I tried using the "borg pattern" for my test
> suite, and it didn't work either. If anyone is curious, I can
> re-create the failure.
As Borg's author (thanks for the "wonderful" -- I hope you have
also given it a rating of 5/5...?-), I am indeed quite curious
about the failure.
Alex
Sam Falkner
complex app. It has extensions written in PRO*C (Oracle
C-preprocessor thing), yadda yadda yadda. That's why I've been
writing watered down stuff in these postings.
I'll re-write two modules that reproduce the problems. Or, when
writing the modules, I fail to reproduce the problems, I'll hopefully
figure out what silly thing is happening. Either way, expect a
follow-up from me within 24 hours.
Thanks for all the help!
- Sam
Sam Falkner
into two modules: work and test. The four methods:
(1) Borg class,
(2) class variable, set as self.count,
(3) class variable, set as Class.count, and
(4) global-to-module variable.
When I tried this, (1) and (2) failed, (3) and (4) worked. But, I
went back and re-applied (3) and (4) to my existing application, and
they still failed.
I would have expected all four of these to work, except perhaps (2),
which I would have expected either to work or to fail with a NameError
or somesuch.
Here are the two modules. My output is as follows:
$ ./test.py -v
testFour (test.WorkTestCase) ... ok
testOne (test.WorkTestCase) ... FAIL
testThree (test.WorkTestCase) ... ok
testTwo (test.WorkTestCase) ... FAIL
======================================================================
FAIL: testOne (test.WorkTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./test.py", line 13, in testOne
self.failUnlessEqual(len(junk), b.countone)
File "/opt/etext/lib/python2.1/unittest.py", line 273, in failUnlessEqual
raise self.failureException, (msg or '%s != %s' % (first, second))
AssertionError: 5 != 1
======================================================================
FAIL: testTwo (test.WorkTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./test.py", line 19, in testTwo
self.failUnlessEqual(len(junk), work.Two.count)
File "/opt/etext/lib/python2.1/unittest.py", line 273, in failUnlessEqual
raise self.failureException, (msg or '%s != %s' % (first, second))
AssertionError: 5 != 0
----------------------------------------------------------------------
Ran 4 tests in 0.003s
FAILED (failures=2)
Note that I still can't get *anything* to work in my real application.
Can anyone explain what's going on? Or, if this was discussed a month
or so ago, can someone tell me the subject line, so I can look it up
on google?
Thanks!
- Sam
#! /usr/bin/env python
# work.py module
class Borg:
__shared_state = {}
def __init__(self):
self.__dict__ = self.__shared_state
self.countone = 0
class One:
def __init__(self):
borg = Borg()
borg.countone += 1
class Two:
count = 0
def __init__(self):
self.count += 1
class Three:
count = 0
def __init__(self):
Three.count += 1
fourcount = 0
class Four:
def __init__(self):
global fourcount
fourcount += 1
def work1():
junk = []
for i in range(5):
junk.append(One())
return junk
def work2():
junk = []
for i in range(5):
junk.append(Two())
return junk
def work3():
junk = []
for i in range(5):
junk.append(Three())
return junk
def work4():
junk = []
for i in range(5):
junk.append(Four())
return junk
#! /usr/bin/env python
# test.py module
import unittest
import work
class WorkTestCase(unittest.TestCase):
def testOne(self):
b = work.Borg()
b.countone = 0
junk = work.work1()
self.failUnlessEqual(len(junk), b.countone)
def testTwo(self):
work.Two.count = 0
junk = work.work2()
self.failUnlessEqual(len(junk), work.Two.count)
def testThree(self):
work.Three.count = 0
junk = work.work3()
self.failUnlessEqual(len(junk), work.Three.count)
def testFour(self):
work.fourcount = 0
junk = work.work4()
self.failUnlessEqual(len(junk), work.fourcount)
def buildall():
return unittest.makeSuite(WorkTestCase, 'test')
if __name__ == '__main__':
unittest.main('test', 'buildall')
Joshua Macy
Doing self.count += 1 in the __init__ creates an instance variable. You
have to either refer to it by the explicit class variable name, e.g.
A.count, or you have to use a mutable type like a list and do e.g.
self.count[0]. Look at what's in the dictionaries:
>>> class A:
...
count = 0
...
def __init__(self):
...
self.count += 1
...
>>> a = A()
>>> b = A()
>>> dir(A)
['__doc__','__init__','__module__','count']
>>> A.count
0
>>> A.__dict__['count']
0
>>> dir(a)
['count']
>>> a.count
1
>>> a.__dict__['count']
1
>>> class B:
...
count = []
...
def __init__(self):
...
self.count[0] += 1
...
>>> c = B()
>>> d = B()
>>> dir(B)
['__doc__','__init__','__module__','count']
>>> B.count
[2]
>>> dir(c)
[]
Does that help any?
Joshua
Christian Tanzer
> class Borg:
> __shared_state = {}
> def __init__(self):
> self.__dict__ = self.__shared_state
> self.countone = 0
>
> class One:
> def __init__(self):
> borg = Borg()
> borg.countone += 1
Your code resets `countone` to zero every time a new Borg is created.
Try this instead:
class Borg:
__shared_state = {"countone" : 0}
def __init__(self):
self.__dict__ = self.__shared_state
--
Christian Tanzer tan...@swing.co.at
Glasauergasse 32 Tel: +43 1 876 62 36
A-1130 Vienna, Austria Fax: +43 1 877 66 92
Alex Martelli
> Here's what I've found. I tried four different methods, putting them
> into two modules: work and test. The four methods:
>
> (1) Borg class,
I'm going to examine this one only, as it's the one that
specifically deals with Borg.
> class Borg:
> __shared_state = {}
> def __init__(self):
> self.__dict__ = self.__shared_state
> self.countone = 0
So you're asking for Borg._Borg__shared_state['countone'] to
be reset to 0 each and every time an instance of Borg is
created. I'm not sure why, but, OK -- this IS what you're
doing here.
> class One:
> def __init__(self):
> borg = Borg()
> borg.countone += 1
And here, each time an instance of One is created, the
shared state 'countoune' of class Borg is first reset
to 0 (by instantiating Borg, see above), then set to 1
(through a specific instance of Borg, but of course by
definition of Borg that doesn't matter). Then the
specific Borg instance is thrown away, being a local
variable of One.__init__, but that doesn't matter
either.
> def work1():
> junk = []
> for i in range(5):
> junk.append(One())
> return junk
So you're repeating five times the "set Borg's shared
state "countone" to 1", as well as returning a list of
five (empty) instances of class One. OK.
> class WorkTestCase(unittest.TestCase):
> def testOne(self):
> b = work.Borg()
> b.countone = 0
This last statement is redundant, of course, since you
already set the 'countone' state of all Borgs to 0 (by
instantiating a Borg) in the previous statement. Of
course, it's innocuous to repeat this.
> junk = work.work1()
> self.failUnlessEqual(len(junk), b.countone)
Now this is the one I really don't understand! How
could len(junk), which we KNOW is five, POSSIBLY
equal the 'countone' state of all Borgs, which we
KNOW just as well is ONE?! Since "work1"'s job IS
to set that count to one -- hey, it sets it to one
FIVE TIMES just to make sure...!
I'm starting to suspect that the "self.countone = 0"
second statement of Borg.__init__, which you wrote,
is *NOT* what you actually MEANT to write. It *IS*
clear to you that, since all Borg instances share
all state (that's the POINT of class Borg!), this is
zeroing out the shared state of 'countone' each time
an instance is created, right?
If your class Borg was totally different, e.g.:
class Borg:
__shared_state = {'countone':0}
def __init__(self):
self.__dict__ = self.__shared_state
THEN there would be no zeroing of the 'countone'
shared-state entry at each Borg instantiation, and
things might work a bit more like you appear to
expect them to (judging by the failUnlessEqual
that we see in WorkTestCase.testOne).
To repeat, that's the POINT of Borg: when you
'bloobet' one of them, you 'bloobet' them all,
where 'bloobet' is any verb having to do with
alteration of and/or access to instance state.
Borg instance have separate _identity_ (id()
builtin function); subclasses of Borg may
give instances with separate _behavior_ (by
adding methods to the subclass-object) and/or
produce strange effects by failing to call
Borg.__init__ in a subclass-specific
initialization (as for other such cases in
Python) or playing dirty tricks with the
_Borg__shared_state class attribute; and
that's about it.
Alex
Sam Falkner
efforts have been failing in bizarre ways. I tried four different
ways to write the tests, detailed in the message that this is
following-up.
Alex's follow-up pointed out the flaw in my use of the Borg class.
But, I was much more confused about why the other methods were
failing.
The bottom line: I was doing a bad thing in one of my C extensions.
I started getting other very odd behavior in my app, such as
isinstance(c, data.Cell)
returning false, but c.__class__ is data.Cell, and c is obviously
behaving as a Cell. When this happened, I went over my C code line by
line. In one place, I was importing "mod" (name changed to protect
the guilty), and I meant to be importing "pkg.mod". Because I was
sitting in the same directory as "mod", it didn't fail. I fixed this,
and these problems have vanished.
* * *
Thanks for all the replies, this has been a good learning experience
for me. Whew!
- Sam