Need help with Python scoping rules
kj
I have many years of programming experience, and a few languages,
under my belt, but still Python scoping rules remain mysterious to
me. (In fact, Python's scoping behavior is the main reason I gave
up several earlier attempts to learn Python.)
Here's a toy example illustrating what I mean. It's a simplification
of a real-life coding situation, in which I need to initialize a
"private" class variable by using a recursive helper function.
class Demo(object):
def fact(n):
if n < 2:
return 1
else:
return n * fact(n - 1)
_classvar = fact(5)
This code fails to compile, with the error "global name 'fact' not
defined".
Scanning the Python Language Reference page I see nothing that
would suggest to me a discussion of Python's scoping rules, let
alone anything that would help me solve the specific problem above.
I'm sure it's in there, *somewhere*, but it's anyone's guess where.
Likewise, my book, Beazley's "Python: Essential Reference" is of
no help. I don't doubt that the answer to my question "is in there,
*somewhere*", but finding it (without reading the book sequentially
from page 1) is a tall order.
All that's left is trial-and-error, the worst kind of programming.
And still I can't find the right way to do this... I've tried
every variant of this code that I can imagine, including decorating
fact with @staticmethod or @classmethod, etc., etc. (In the latter
case, I get cryptic errors saying that the staticmethod or classmethod
object is not callable. Or something like that.)
Needless to say, I'm pretty beat by this point. Any help would be
appreciated.
Thanks,
kynn
Martin P. Hellwig
<cut>
>
> Here's a toy example illustrating what I mean. It's a simplification
> of a real-life coding situation, in which I need to initialize a
> "private" class variable by using a recursive helper function.
eh?
>
> class Demo(object):
> def fact(n):
> if n < 2:
> return 1
> else:
> return n * fact(n - 1)
>
> _classvar = fact(5)
>
<cut>
Sorry still doesn't make sense, I'll give it a try though:
class Demo(object):
"""Apparently a 'real-life coding situation'"""
def __init__(self):
# Look at http://docs.python.org/tutorial/classes.html
# for init explanation.
self.__class_var = self.fact(5)
def fact(self, number):
"""Look at http://docs.python.org/tutorial/classes.html why you
need
self."""
if number < 2:
return(1)
else:
return_value = number * self.fact(number -1)
return(return_value)
TEST = Demo()
# Print the 'private' class variable
print(TEST._Demo__class_var)
--
MPH
http://blog.dcuktec.com
'If consumed, best digested with added seasoning to own preference.'
Diez B. Roggisch
Classes are not scopes.
So the above doesn't work because name resolution inside functions/methods
looks for local variables first, then for the *global* scope. There is no
class-scope-lookup.
If you want anything else, you need to qualify it with a full name,
Demo.fact. Which isn't working for your example because "Demo" isn't
available yet.
But if you insist on the above methodology, you can do this:
class Demo(object):
def fact(n):
def inner(n):
if n < 2:
return 1
else:
return n * inner(n - 1)
return inner(n)
_classvar = fact(5)
This makes inner a *local* variable, which is found.
Diez
Jean-Michel Pichavant
>
> I have many years of programming experience, and a few languages,
> under my belt, but still Python scoping rules remain mysterious to
> me. (In fact, Python's scoping behavior is the main reason I gave
> up several earlier attempts to learn Python.)
>
> Here's a toy example illustrating what I mean. It's a simplification
> of a real-life coding situation, in which I need to initialize a
> "private" class variable by using a recursive helper function.
>
> class Demo(object):
> def fact(n):
> if n < 2:
> return 1
> else:
> return n * fact(n - 1)
>
> _classvar = fact(5)
>
> This code fails to compile, with the error "global name 'fact' not
> defined".
>
fact is defined within the Demo class, so to access it, you'll need to
prefix it with Demo:
_classvar = Demo.fact(5).
The problem is, Demo will raise a NameError exception.
The solution is pretty simple, cause your fact function seems to be
unrelated to the Demo class :
def _fact(n):
# some code
class Demo(object):
_classvar = _fact(5)
It may happen that you *want* your fact within the Demo, in you example
it's not that obvious, but I think Java would allow such following patern:
class Color:
def __init__(self, r, g,b):
pass
BLACK = Color(0,0,0)
It make sens from a design point of view to put BLACK in the Color
namespace. But I don't think it's possible with python.
You could trick the system using inheritance:
class Chrome:
def __init__(self, r,g,b)
pass
Putting all your code in the Chrome class then:
class Color(Chrome):
BLACK = Chrome(0,0,0)
I'm not sure this is satisfying.
JM
Jean-Michel Pichavant
> Classes are not scopes.
>
>
Too bad, could have been handy.
JM
Diez B. Roggisch
Nope. Because then a lot of people would write something like this:
class Foo(object):
def bar(self):
bar() # note the missing self.
And this would lead to errors because self was missing from the call
to "bar".
And you'd need a disambiguation for methodname/global-name-clashes. The
global-statement would work, but then code could break when all of a
sudden a subclass defines a method that hitherto was only known as global.
So via subclassing, you introduce *arbitray* and hard to debug errors.
No. I'm certain, not a good idea.
Diez
John Posner
>
> Classes are not scopes.
>
> So the above doesn't work because name resolution inside functions/methods
> looks for local variables first, then for the *global* scope. There is no
> class-scope-lookup.
But http://docs.python.org/tutorial/classes.html says, in Section 9.3 "A
First Look at Classes":
When a class definition is entered, a new namespace is created,
and used as the local scope � thus, all assignments to local variables
go into this new namespace. In particular, function definitions bind
the name of the new function here.
The following example confirms this:
class Spam(object):
clsvar_1 = 555
clsvar_2 = clsvar_1 + 222
def __init__(self):
print "Spam instance initialized"
sp = Spam()
print sp.clsvar_1, sp.clsvar_2
output:
Spam instance initialized
555 777
Does the OP (kj) have a legitimate gripe, though? I confess that I know
nothing about Python's implementation -- I'm strictly a user. So it's
just a suspicion of mine that
something special enables a recursive function definition to refer to
the function's own name before the definition has been completed. It
works at the module-namespace (i.e. global) level, and Diez's "toy
example" shows that it works at function-namespace level:
class Demo(object):
def fact(n):
def inner(n):
if n < 2:
return 1
else:
return n * inner(n - 1)
return inner(n)
_classvar = fact(5)
So why can't it work at the class-namespace level, too?
(BTW, Diez, your toy example is another demonstration that there *is* a
class-scope-lookup: the "def" statement binds the name "fact" in the
class scope, and the assignment statement looks up the name "fact" in
that scope.)
-John
7stud
> Diez said:
>
>
>
> > Classes are not scopes.
>
> > So the above doesn't work because name resolution inside functions/methods
> > looks for local variables first, then for the *global* scope. There is no
> > class-scope-lookup.
>
> First Look at Classes":
>
> When a class definition is entered, a new namespace is created,
> go into this new namespace. In particular, function definitions bind
> the name of the new function here.
>
> The following example confirms this:
>
> class Spam(object):
> clsvar_1 = 555
> clsvar_2 = clsvar_1 + 222
>
> def __init__(self):
> print "Spam instance initialized"
>
> sp = Spam()
> print sp.clsvar_1, sp.clsvar_2
>
> output:
> Spam instance initialized
> 555 777
>
> Does the OP (kj) have a legitimate gripe, though? I confess that I know
I guess a counter example would be something like this:
y = "hello"
class Demo(object):
y = "goodbye"
def __init__(self):
self.x = 10
print y
Demo()
--output:--
hello
>
> nothing about Python's implementation -- I'm strictly a user. So it's
> just a suspicion of mine that
> something special enables a recursive function definition to refer to
> the function's own name before the definition has been completed.
>
python ignores the names inside a function when it creates the
function. This "program" will not produce an error:
def f():
print x
python parses the file and creates the function object and assigns the
function object to the variable f. It's not until you execute the
function that python will raise an error. The same thing happens with
the recursive function.
John Posner
> python ignores the names inside a function when it creates the
> function. This "program" will not produce an error:
>
>
> def f():
> print x
>
> python parses the file and creates the function object and assigns the
> function object to the variable f. It's not until you execute the
> function that python will raise an error. The same thing happens with
> the recursive function.
>
Thanks for that explanation. So in the OP's example:
Class Demo(object):
def fact(n):
if n < 2:
return 1
else:
return n * fact(n - 1)
_classvar = fact(5)
... no failure occurs when "fact(5)" is invoked, because the lookup of
"fact" in the local scope is a class-scope-lookup, which succeeds. The
failure occurs on the first recursive invocation of fact() in the
statement "return n * fact(n - 1)": the function-scope-lookup of "fact"
fails, and then the interpreter falls back to a global-scope-lookup of
"fact", which also fails.
Diez B. Roggisch
See my other post on what name lookup with class-scope *inside*
functions would mean.
Diez
Ryniek90
> Ryniek90 wrote:
>> Referring to my earlier posts:
>> http://groups.google.pl/group/comp.lang.python/browse_thread/thread/4e34f995800f5352?hl=pl
>>
>>
>> and
>>
>> http://groups.google.pl/group/comp.lang.python/browse_thread/thread/abf5573b8fceb37e?hl=pl#
>>
>>
>> I've dealt with those errors. but now have another.
>> When my backup scripts starts to backup chosen file, python gave me
>> Traceback:
>>
>> "
>> C:\Users\Ryniek's WinSe7en\Documents\My
>> Dropbox\Aplikacje\Moje_aplikacje\Pythonowe_aplikacje\Skrypty>python ba
>> ckuper.py -f F:\APLIKACJE\nowegg.exe F:\ MyGGBackup
>> Checking permissions for reading and writing...
>> Have permissions on [F:\APLIKACJE\nowegg.exe] for reading.
>> Have permissions on [F:\] for writing.
>> Preparing for backup [nowegg.exe]...
>> Starting backup...
>> Now adding [nowegg.exe]...
>> Traceback (most recent call last):
>> File "backuper.py", line 197, in <module>
>> main_meth()
>> File "backuper.py", line 189, in main_meth
>> paq.backup_file(pars.options.filename[0],
>> pars.options.filename[1], pars.options.filename[2])
>> File "backuper.py", line 127, in backup_file
>> backup_obj.add(read_bin_obj)
>> File "E:\WinSe7en Apps\APLIKACJE\ActiveState Python
>> 2.6\lib\tarfile.py", line 1948, in add
>> if self.name is not None and os.path.abspath(name) == self.name:
>> File "E:\WinSe7en Apps\APLIKACJE\ActiveState Python
>> 2.6\lib\ntpath.py", line 458, in abspath
>> path = _getfullpathname(path)
>> TypeError: _getfullpathname() argument 1 must be (buffer overflow),
>> not str
>> "
>>
>> I've searched over google, but found only that is bug in Python (
>> http://bugs.python.org/issue4071 - in that issue bug is inside
>> Python 2.5.2 - i've got ActiveState Active Python with core Python
>> 2.6.2), and another info: *http://tinyurl.com/lvyn7o and
>> **http://tinyurl.com/kn49vk
>> *
>>
>> Here's my script code:
>> *http://paste.ubuntu.com/259310/
>>
>> *Shouldn't that bug be patched already :-?
> Are you giving it the contents of the file when it's actually expecting
> the filename?
>
>
Of course the content of the file:
"
C:\Users\Ryniek's WinSe7en\Documents\My
Dropbox\Aplikacje\Moje_aplikacje\Pythonowe_aplikacje\Skrypty>python ba
ckuper.py -f F:\APLIKACJE\nowegg.exe F:\ MyGGBackup
Checking permissions for reading and writing...
Have permissions on [F:\APLIKACJE\nowegg.exe] for reading.
Have permissions on [F:\] for writing.
Preparing for backup [nowegg.exe]...
Starting backup...
Now adding [nowegg.exe]...
Traceback (most recent call last):
File "backuper.py", line 196, in <module>
main_meth()
File "backuper.py", line 188, in main_meth
paq.backup_file(pars.options.filename[0], pars.options.filename[1],
pars.options.filename[2])
File "backuper.py", line 126, in backup_file
backup_obj.add(read_obj.read())
File "E:\WinSe7en Apps\APLIKACJE\ActiveState Python
2.6\lib\tarfile.py", line 1948, in add
if self.name is not None and os.path.abspath(name) == self.name:
File "E:\WinSe7en Apps\APLIKACJE\ActiveState Python
2.6\lib\ntpath.py", line 458, in abspath
path = _getfullpathname(path)
TypeError: _getfullpathname() argument 1 must be (buffer overflow), not str
"
See? *backup_obj.add(read_obj.read())*
In previous pasted Traceback you can see *backup_obj.add(read_bin_obj)*
- read_obj_bin was a reference to the *read_obj.read()* .
Do only I have this problem or Python programming under Windows is
nightmare?
John Posner
>
>
> But http://docs.python.org/tutorial/classes.html says, in Section
> 9.3 "A First Look at Classes":
>
> When a class definition is entered, a new namespace is created,
> go into this new namespace. In particular, function definitions bind
> the name of the new function here.
>
>
> (BTW, Diez, your toy example is another demonstration that there
> *is* a class-scope-lookup: the "def" statement binds the name
> "fact" in the class scope, and the assignment statement looks up
> the name "fact" in that scope.)
>
>
> equivalent to a scope, really, I think.
My apologies if I was being too loose with the terms "namespace" and
"scope". I was trying to use Diez's terminology (e.g.
"class-scope-lookup"). But I think you agree with me in *almost*
everything you say below.
>
> The def statement creates a function object, and assigns it to the
> given name, in its /local scope/ which is the class's namespace -- but
> that is not a new scope. Its just, during class creation, the local scope.
>
> When you enter a class definition, a new namespace is created, and
> that is used as the local scope during the class's definition. Under
> "class Foo", the local scope is this class's namespace-- and the
> global scope is the module's namespace. Any lookup operation checks
> first in the local scope, then in the global scope...
>
> When the "def fact(...)" is executed (note that class bodies are
> executed on load, function bodies aren't), a new namespace is created
> as well-- the function's namespace-- and its used as the local scope.
> During execution of the code, it looks up in the local scope first--
> and then it looks up in the global scope if it doesn't find anything.
> There /is/ no class-lookup-scope... but there IS a class namespace.
>
> There's just those two scopes: but the namespace bound to those scopes
> changes as you enter different parts of the code.
>
> For a long time that's all there was, just the local and global scope:
> to access any other namespace required you to explicitly address it.
> The class namespace remains accessible only via explicit addressing,
> but PEP 227 in Python 2.1/2.2 introduced static nested scopes. But
> that applies only to enclosing functions: embedding one function into
> another.
>
> You can only call recursive functions if the function is able to refer
> to itself according to the same lookup rules: is the function's name
> in the local scope? No it's not... is it in the global scope? No? Then
> it can't call itself recursively... well, unless its a method, and it
> addresses itself specifically-- with "self.".
All of the above is consistent with my previous post.
>
> The OP's probably is that during the execution of the class body, the
> class doesn't exist... so the 'def fact' -can't- use Classname.fact to
> address itself explicitly, and within fact the locals don't contain a
> reference to the function itself, and its globals don't either. You
> just can't do that.
Here's where we disagree, and I'm sticking to my guns. The fact that the
class definition has not been completely processed is irrelevant. The
OP's problem was attempting to implement a recursive function. A
non-recursive implementation of fact() works fine:
class ThisWorks(object):
def fact(n):
answer = 1
i = 1
while i <= n:
answer *= i
i += 1
return answer
clsvar = fact(4)
print ThisWorks.clsvar # output: 24
> The right way, IMHO, is to move 'fact' up and out of the class ...
We're back to agreement on this point!
-John
MRAB
>>
>> Ryniek90 wrote:
[snip]
>>> Here's my script code:
>>> *http://paste.ubuntu.com/259310/
>>>
>>> *Shouldn't that bug be patched already :-?
>> Are you giving it the contents of the file when it's actually expecting
>> the filename?
>>
>
> Of course the content of the file:
> See? *backup_obj.add(read_obj.read())*
> In previous pasted Traceback you can see *backup_obj.add(read_bin_obj)*
> - read_obj_bin was a reference to the *read_obj.read()* .
>
provide the values it's expecting. For example, if you choose "add" then
provide the filename, not the contents of the file.
> Do only I have this problem or Python programming under Windows is
> nightmare?
It's only a nightmare if you don't follow the documentation. :-)
Jan Kaliszewski
> The OP's probably is that during the execution of the class body, the
> class
> doesn't exist... so the 'def fact' -can't- use Classname.fact to address
> itself explicitly, and within fact the locals don't contain a reference
> to
> the function itself, and its globals don't either. You just can't do
> that.
> The right way, IMHO, is to move 'fact' up and out of the class into a
> _fact global variable. Alternatively, you can use a metaclass.
Note that you can also pass a function to the function itself, and then
it works:
>>> class Foo:
... def func(foo=None):
... if foo:
... return foo()
... else:
... return '2nd step'
... x = func(func)
...
>>> Foo.x
'2nd step'
Note that when called from class definition body, func is an ordinary
function, not a method. It become a method *when it's called as a method*
(what is possible *after* creating the class => outside the definition).
Cheers,
*j
--
Jan Kaliszewski (zuo) <z...@chopin.edu.pl>
Stephen Fairchild
def Demo():
def fact(n):
if n < 2:
return 1
else:
return n * fact(n - 1)
return type("Demo", (object,), {"fact": staticmethod(fact), "_classvar":
fact(5)})
Demo = Demo()
d = Demo()
print d._classvar # prints 120
print d.fact(7) # prints 5040
print Demo # prints <class '__main__.Demo'>
--
Stephen Fairchild
Dave Angel
fact() has no self argument. Seems to me that makes it a staticmethod,
so it should be declared that way. But you can't call a static method
from the class scope, since the class hasn't been completed yet. That's
the part I consider a problem, not all the rest. I've seen the same
problem in Forth, where 'smudge' is used to prevent a definition from
accidentally calling itself. But I don't recall whether that was in the
standard, or just vendor's helpful additions.
Anyway, my first choice is to move the static method out of the class.
Second choice workaround follows, moving the initialization of the class
variable to after the class, when it can properly use the class name.:
class Demo(object):
@staticmethod
def fact(n):
if n < 2:
return 1
else:
return n * Demo.fact(n - 1)
Demo._classvar = Demo.fact(5)
print Demo._classvar
xx = Demo()
print xx.fact(6)
DaveA
Ulrich Eckhardt
> class Color:
> def __init__(self, r, g,b):
> pass
> BLACK = Color(0,0,0)
>
> It make sens from a design point of view to put BLACK in the Color
> namespace. But I don't think it's possible with python.
class Color:
...
setattrib(Color, "BLACK", Color(0,0,0))
Uli
--
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
kj
>Stephen Hansen said:
>> This sounds like a fundamental confusion -- a namespace is not
>> equivalent to a scope, really, I think.
<snip>
Hmm. I can't find Stephen Hansen's original post anywhere. Where
did you come across it?
Is there an *official* write-up where these issues are discussed?
To put it differently, where exactly in the Python docs would
someone learning Python go to answer my original query?
TIA!
kynn
kj
>Classes are not scopes.
This looks to me like a major wart, on two counts.
First, one of the goals of OO is encapsulation, not only at the
level of instances, but also at the level of classes. Your comment
suggests that Python does not fully support class-level encapsulation.
Second, my example shows that Python puts some peculiar restrictions
on recursion. Recursion! One of the central concepts in the theory
of functions! This is shown most clearly by the following elaboration
of my original example:
class Demo(object):
def fact_rec(n):
if n < 2:
return 1
else:
return n * fact_rec(n - 1)
def fact_iter(n):
ret = 1
for i in range(1, n + 1):
ret *= i
return ret
classvar1 = fact_iter(5)
classvar2 = fact_rec(5)
This code won't compile as shown, but it does compile if the last
line (the call to the recursive fact_rec) is commented out. There
is no justification for discriminating against recursive functions
in this context. Recursive functions should be OK wherever functions
are OK. I can't think of any other language that allows recursion
but not anywhere.
Is there any good reason (from the point of view of Python's overall
design) for not fixing this?
kynn
Ulrich Eckhardt
> Jean-Michel Pichavant wrote:
>> class Color:
>> def __init__(self, r, g,b):
>> pass
>> BLACK = Color(0,0,0)
>>
>> It make sens from a design point of view to put BLACK in the Color
>> namespace. But I don't think it's possible with python.
>
> class Color:
> ...
>
> setattrib(Color, "BLACK", Color(0,0,0))
Apart from it being "setattr" and not "setattrib", a simple
Color.BLACK = Color(0,0,0)
should have done the job here. However, what I had in mind was this:
class Color:
_colors = [ ("BLACK", (0,0,0)),
("WHITE", (1,1,1))
]
def __str__(self):
# try to locate a name
for name, rgb in Color._colors:
if self.rgb==rgb:
return name
# no name found, just format as a triplet
return "(%s, %s, %s)" % self.rgb
# add symbolic names
for name, rgb in Color._colors:
setattr(Colors, name, Color(rgb))
...which I got as suggestion on my question how to model C enumeration
lookalikes.
Carl Banks
> In <7figv3F2m3p0...@mid.uni-berlin.de> "Diez B. Roggisch" <de...@nospam.web.de> writes:
>
> >Classes are not scopes.
>
> This looks to me like a major wart, on two counts.
Class statements *are* scopes, they are just not accessible from
scopes nested within them.
> First, one of the goals of OO is encapsulation, not only at the
> level of instances, but also at the level of classes. Your comment
> suggests that Python does not fully support class-level encapsulation.
I can't answer this, I don't even know what you are talking about.
The OO notion of encapsulation that I'm familar means that an object
can seal off access to private data. This has nothing to do with
class statement scoping.
> Second, my example shows that Python puts some peculiar restrictions
> on recursion. Recursion! One of the central concepts in the theory
> of functions! This is shown most clearly by the following elaboration
> of my original example:
>
> class Demo(object):
> def fact_rec(n):
> if n < 2:
> return 1
> else:
> return n * fact_rec(n - 1)
>
> def fact_iter(n):
> ret = 1
> for i in range(1, n + 1):
> ret *= i
> return ret
>
> classvar1 = fact_iter(5)
> classvar2 = fact_rec(5)
>
> This code won't compile as shown, but it does compile if the last
> line (the call to the recursive fact_rec) is commented out. There
> is no justification for discriminating against recursive functions
> in this context. Recursive functions should be OK wherever functions
> are OK. I can't think of any other language that allows recursion
> but not anywhere.
Inside class statements are not the place for this kind of thing.
Define functions like fact_rec outside the class statement.
> Is there any good reason (from the point of view of Python's overall
> design) for not fixing this?
I suspect that no reason, however profound, would satisfy you.
Therefore I will merely answer your question without fanfare, and you
can take it as you will.
1. One of the key aspects of Python's design is that attributes must
be accessed explicitly with dot notation. Accessing class scopes from
nested functions would (seemingly) allow access to class attributes
without the dotted notation. Therefore it is not allowed.
2. This is considered more important that your ability to define
recursive functions in the class statement.
Carl Banks
Dave Angel
> Jean-Michel Pichavant wrote:
>
>> class Color:
>> def __init__(self, r, g,b):
>> pass
>> BLACK = Color(0,0,0)
>>
>> It make sens from a design point of view to put BLACK in the Color
>> namespace. But I don't think it's possible with python.
>>
>
> class Color:
> ...
>
> setattrib(Color, "BLACK", Color(0,0,0))
>
> Uli
>
>
Color.BLACK = Color(0,0,0)
DaveA
Martin P. Hellwig
<cut>
> First, one of the goals of OO is encapsulation, not only at the
> level of instances, but also at the level of classes.
Anyway, you could be right (I am not capable to judge it) and Python
should change on this issue but from what I gathered, Pythons OO is
inspired by the following:
- Don't repeat yourself
- Containing code into logical units makes it easier to understand and
maintain.
Note the absence of 'OO is defined as X so we are going to do X without
thinking if X is actually the best way to do it'
> Your comment suggests that Python does not fully support
> class-level encapsulation.
Probably right too, but being intrigued, why is that necessary, and
given an example is that the best way to do it?
> Second, my example shows that Python puts some peculiar restrictions
> on recursion. Recursion! One of the central concepts in the theory
> of functions!
<cut>
It is also one of the best ways to shoot yourself in the foot, but I ask
you again, is the reason why you encountered this 'limitation' actually
the best way to solve your problem?
Please forgive my incompetence if this reply sounds harsh and patronizing.
7stud
> Stephen Fairchild wrote:
> > You are trying to run code in a class that does not exist yet.
>
> > def Demo():
> > def fact(n):
> > if n < 2:
> > return 1
> > else:
> > return n * fact(n - 1)
> > return type("Demo", (object,), {"fact": staticmethod(fact), "_classvar":
> > fact(5)})
> > Demo = Demo()
>
> > d = Demo()
> > print d._classvar # prints 120
> > print d.fact(7) # prints 5040
> > print Demo # prints <class '__main__.Demo'>
>
>
>
> In all these messages, something I haven't seen pointed out is that
> fact() has no self argument.
>
An "argument" is something that is specified in the the function
call. I assume you are trying to state something like, "fact() is not
defined with a parameter variable named self". However, that has
never been a requirement in python:
class A(object):
def fact(n):
print n
fact("hello")
a = A()
a.fact()
--output:--
hello
<__main__.A object at 0x7faf0>
kj
>kj wrote:
><cut>
>> First, one of the goals of OO is encapsulation, not only at the
>> level of instances, but also at the level of classes.
>Who says?
Python itself: it already offers a limited form of class encapsulation
(e.g. class variables). It would be nice if it went all the way
and gave classes their own bona fide scope. (But I hasten to add:
I *still* don't understand the Python scope model, and not for lack
of trying. I've only succeeded in finding fragments of this model
explained here and there, like pottery shards: a bit lost in a
tutorial, or some comment in a newsgroup thread, etc. Maybe the
full, authoritative documentation of Python's scope model got lost
somewhere, and will be found by archaeologists in the 25th century...)
>Anyway, you could be right (I am not capable to judge it) and Python
>should change on this issue but from what I gathered, Pythons OO is
>inspired by the following:
>- Don't repeat yourself
>- Containing code into logical units makes it easier to understand and
>maintain.
...except, apparently, when that code is a recursive function, in
which case one's out of luck.
>> Second, my example shows that Python puts some peculiar restrictions
>> on recursion. Recursion! One of the central concepts in the theory
>> of functions!
><cut>
>It is also one of the best ways to shoot yourself in the foot...
If recursion is so evil, and Python so intent in saving us from
shooting ourselves in the foot, why does it allow recursion at all?
kynn
kj
>> First, one of the goals of OO is encapsulation, not only at the
>> level of instances, but also at the level of classes. =A0Your comment
>> suggests that Python does not fully support class-level encapsulation.
>I can't answer this, I don't even know what you are talking about.
Yes, you do. As I said in another post, Python offers some degree
of class-level encapsulation (e.g. class variables). But it is
limited by the fact that these class-encapsulated elements can't
always be accessed from within the class itself, which is kind of
silly.
>1. One of the key aspects of Python's design is that attributes must
>be accessed explicitly with dot notation. Accessing class scopes from
>nested functions would (seemingly) allow access to class attributes
>without the dotted notation. Therefore it is not allowed.
It would be trivial to define a keyword (e.g. this, or if you
prefer, __this__), valid only within a class statement, and that
the interpreter would recognize as "the current class", even before
this class is full defined.
kynn
Dave Angel
> On Aug 25, 7:26 pm, Dave Angel <da...@ieee.org> wrote:
>
>> Stephen Fairchild wrote:
>>
>>> You are trying to run code in a class that does not exist yet.
>>>
>>> def Demo():
>>> def fact(n):
>>> if n < 2:
>>> return 1
>>> else:
>>> return n * fact(n - 1)
>>> return type("Demo", (object,), {"fact": staticmethod(fact), "_classvar":
>>> fact(5)})
>>>
>>> d =emo()
>>> print d._classvar # prints 120
>>> print d.fact(7) # prints 5040
>>> print Demo # prints <class '__main__.Demo'>
>>>
>>
>> In all these messages, something I haven't seen pointed out is that
>> fact() has no self argument.
>>
>>
>
> An "argument" is something that is specified in the the function
> call. I assume you are trying to state something like, "fact() is not
> defined with a parameter variable named self". However, that has
> never been a requirement in python:
>
>
> class A(object):
>
> def fact(n):
> print n
>
> fact("hello")
>
>
> a.fact()
>
> --output:--
> hello
> <__main__.A object at 0x7faf0>
>
>
>
>
and formal parameter. And self is a convention, not a requirement.
But the fact is that the method as written would never have worked, when
called from outside the class, since you'd have to call it with either
the class name or an instance, and in either case, the method was then
trying to do arithmetic on one of those.
Thanks for diluting my point. The OP is chasing the wrong problem. Who
cares whether a class initializer can call a method, if the method
doesn't meet its original requirements, to be callable outside the class?
And the arguments about how recursion is restricted are ridiculous.
Nothing wrong with a method calling itself, once it's got a proper
signature. You just have to make the call agree with the signature.
The problem is only that the method may not be actually called until the
class definition is complete.
DaveA
Mel
> Is there any good reason (from the point of view of Python's overall
> design) for not fixing this?
Python is not a compiled language, in the sense that a compiler can go back
and forth over the program, filling in the details that make the program
runnable. Python is an interpreted language in a sense that makes a `class`
statement an executable statement, with a start time and an end time.
Before the start time, the affected class doesn't exist. After the start
time, the class does exist. In between, while the `class` statement is
executing, it's best to make no promises so as not to constrain present and
future implementations.
Mel.
Carl Banks
> In <16b72319-8023-471c-ba40-8025aa6d4...@a26g2000yqn.googlegroups.com> Carl Banks <pavlovevide...@gmail.com> writes:
>
> >> First, one of the goals of OO is encapsulation, not only at the
> >> level of instances, but also at the level of classes. =A0Your comment
> >> suggests that Python does not fully support class-level encapsulation.
> >I can't answer this, I don't even know what you are talking about.
>
> Yes, you do. As I said in another post, Python offers some degree
> of class-level encapsulation (e.g. class variables). But it is
> limited by the fact that these class-encapsulated elements can't
> always be accessed from within the class itself, which is kind of
> silly.
Nope, you're wrong. Class variables are accessible wherever a class
exists. The apparent silliness of this is because you are confusing
classes with class statements.
You know that the class isn't created until the class statement exits,
don't you?
Yeah, it's a little surprising that you can't access class scope from
a function, but that has nothing to do with encapsulation.
> >1. One of the key aspects of Python's design is that attributes must
> >be accessed explicitly with dot notation. Accessing class scopes from
> >nested functions would (seemingly) allow access to class attributes
> >without the dotted notation. Therefore it is not allowed.
>
> It would be trivial to define a keyword (e.g. this, or if you
> prefer, __this__), valid only within a class statement, and that
> the interpreter would recognize as "the current class", even before
> this class is full defined.
Your solution to this "problem" is to add a keyword to Python.
Laughable. (BTW, it's not trivial.)
Carl Banks
kj
>Stephen Fairchild wrote:
>> You are trying to run code in a class that does not exist yet.
>>
>> def Demo():
>> def fact(n):
>> if n < 2:
>> return 1
>> else:
>> return n * fact(n - 1)
>> return type("Demo", (object,), {"fact": staticmethod(fact), "_classvar":
>> fact(5)})
>> Demo = Demo()
>>
>> d = Demo()
>> print d._classvar # prints 120
>> print d.fact(7) # prints 5040
>> print Demo # prints <class '__main__.Demo'>
>>
>>
>In all these messages, something I haven't seen pointed out is that
>fact() has no self argument. Seems to me that makes it a staticmethod,
>so it should be declared that way.
No, the fact() function here represents an internal "helper"
function. It is meant to be called only once to help initialize
a class variable that would be inconvenient to initialize otherwise;
this helper function is not meant to be called from outside the
class statement. Granted, in the example I gave, the "helper"
function (factorial) is a bit silly, but that was just intended as
a simple and familiar example of a recursive function. The actual
function that motivated this post would be considerably more
difficult to explain and would have obscured the point of the post.
kynn
Dave Angel
> In <7figv3F...@mid.uni-berlin.de> "Diez B. Roggisch" <de...@nospam.web.de> writes:
>
>
>> Classes are not scopes.
>>
>
> This looks to me like a major wart, on two counts.
>
> First, one of the goals of OO is encapsulation, not only at the
> level of instances, but also at the level of classes. Your comment
> suggests that Python does not fully support class-level encapsulation.
>
> Second, my example shows that Python puts some peculiar restrictions
> on recursion. Recursion! One of the central concepts in the theory
> of functions! This is shown most clearly by the following elaboration
> of my original example:
>
> class Demo(object):
> def fact_rec(n):
> if n < 2:
> return 1
> else:
> return n * fact_rec(n - 1)
>
>
the class name or an instance object, depending on whether it's a
instance method or not. By default, a method is unbound, and its first
argument is by convention called self. If you want to recurse, then add
self in the appropriate places.
As you have it, neither of the methods can be called outside the class:
class Demo(object):
def fact_rec(n):
if n < 2:
return 1
else:
return n * fact_rec(n - 1)
....
obj = Demo()
print obj.fact_rec(5)
gives error:
Traceback (most recent call last):
File "M:\Programming\Python\sources\dummy\stuff2.py", line 20, in <module>
print obj.fact_rec(5)
TypeError: fact_rec() takes exactly 1 argument (2 given)
To fix it, you need to either change the signature (add in 'self'
argument before the n argument) or do some form of decorator to the
function. If I assume you never intended this method to use 'self'
(ie. it's a static method), then declare it so. And call it accordingly.
class Demo(object):
@staticmethod
def fact(n):
if n < 2:
return 1
else:
return n * Demo.fact(n - 1)
print Demo.fact(6)
On the other hand, if I assume you're just giving a simple example of
what's really intended to be a normal method (called with an instance,
that it uses), then you'd change it this way.
class Demo2(object):
def fact(self, n):
if n<2:
return 1
else:
return n * self.fact(n-1)
obj = Demo2()
print obj.fact(5)
Now the only real restriction, as opposed to all these red-herrings, is
that the class may not be used before it's complete. That only comes to
play when class attributes (non-instance "variables") are defined in
terms of class methods. So if you have such attributes to define, move
them outside of the class, perhaps immediately after it.
class Demo(object):
@staticmethod
def fact(n):
if n < 2:
return 1
else:
return n * Demo.fact(n - 1)
Demo._classvar = Demo.fact(5)
> def fact_iter(n):
> ret = 1
> for i in range(1, n + 1):
> ret *= i
> return ret
>
> classvar1 = fact_iter(5)
> classvar2 = fact_rec(5)
>
> This code won't compile as shown,
Sure it will compile. It just won't run. You get the error after
compiling the function when *calling* it from the classvar* line. But
at that time the class is not defined, and not everything is ready for use.
You can probably work around this by replacing the staticmethod
decorator with an equivalent function call:.
class Demo9(object):
def fact(n):
if n < 2:
return 1
else:
return n * Demo.fact(n - 1)
_classvar = fact(5)
fact = staticmethod(fact)
print Demo9._classvar
xx = Demo9()
print xx.fact(6)
print Demo9.fact(8)
kj
>Thanks for diluting my point. The OP is chasing the wrong problem. Who
>cares whether a class initializer can call a method, if the method
>doesn't meet its original requirements, to be callable outside the class?
>And the arguments about how recursion is restricted are ridiculous.
>Nothing wrong with a method calling itself, once it's got a proper
>signature. You just have to make the call agree with the signature.
>The problem is only that the method may not be actually called until the
>class definition is complete.
As I described at length in another reply, the function in question
is not intended to be "callable outside the class". And yes,
recursive functions in Python *are* restricted in ways that
non-recursive functions aren't. The examples I've posted prove
this point unambiguously.
At this point the only defense for this restriction is to claim
that it is somehow negligible. But I disagree. It's easy to come
up with equally negligible, and equally indefensible, restrictions
to the language would be found quite unacceptable by most users.
E.g. suppose that Python's specification prohibited the use of
upper case X in an identifier. That's certainly a negligible
restriction: it is not harder at all to write great software without
the services of the letter X than with them. Still, I think most
users would find this restriction infuriatingly idiotic. Which
pretty much captures what I feel about Python's prohibition of
recursive functions within class statements.
kynn
kj
>On Aug 26, 7:09=A0am, kj <no.em...@please.post> wrote:
>> In <16b72319-8023-471c-ba40-8025aa6d4...@a26g2000yqn.googlegroups.com> Ca=
>rl Banks <pavlovevide...@gmail.com> writes:
>>
>> >> First, one of the goals of OO is encapsulation, not only at the
>> >> level of instances, but also at the level of classes. =3DA0Your commen=
>t
>> >> suggests that Python does not fully support class-level encapsulation.
>> >I can't answer this, I don't even know what you are talking about.
>>
>> Yes, you do. =A0As I said in another post, Python offers some degree
>> of class-level encapsulation (e.g. class variables). =A0But it is
>> limited by the fact that these class-encapsulated elements can't
>> always be accessed from within the class itself, which is kind of
>> silly.
>Nope, you're wrong. Class variables are accessible wherever a class
>exists. The apparent silliness of this is because you are confusing
>classes with class statements.
Repeating an earlier example (though I've switched the order of
the two internal functions):
class Demo(object):
def fact_iter(n):
ret = 1
for i in range(1, n + 1):
ret *= i
return ret
def fact_rec(n):
if n < 2:
return 1
else:
return n * fact_rec(n - 1)
classvar1 = fact_iter(5)
classvar2 = fact_rec(5)
In the initialization of classvar1, fact_iter is invoked without
any problem even though the class is not yet created: no need to
qualify it with the name of the class. This is completely inconsistent
with the requirement that fact_rec be so qualified when invoked
within fact_rec.
kynn
Carl Banks
> You can probably work around this by replacing the staticmethod
> decorator with an equivalent function call:.
>
> class Demo9(object):
> def fact(n):
> if n < 2:
> return 1
> else:
> return n * Demo.fact(n - 1)
>
> _classvar = fact(5)
> fact = staticmethod(fact)
>
> print Demo9._classvar
> xx = Demo9()
> print xx.fact(6)
> print Demo9.fact(8)
This won't work normally. It only worked for you because you made a
typo.
Carl Banks
Steven D'Aprano
> In <7figv3F...@mid.uni-berlin.de> "Diez B. Roggisch"
> <de...@nospam.web.de> writes:
>
>>Classes are not scopes.
>
> This looks to me like a major wart, on two counts.
>
> First, one of the goals of OO is encapsulation, not only at the level of
> instances, but also at the level of classes. Your comment suggests that
> Python does not fully support class-level encapsulation.
I don't even know what that is supposed to mean. If anything, your
problem could be because Python has TOO MUCH "class-level encapsulation"
compared to what you are expecting: functions inside a class don't see
the class attributes you expect them too.
Actually, I think your problem is that you are interpreting what you're
seeing in terms of C++ or Java, and assuming that whatever they do is the
One True Definition of OO. That's hardly valid -- if any language
deserves the label of the One True OO Design, it might be Smalltalk, on
the basis that it was the first OO language. But even that is silly --
just because something was the first, that doesn't make it the best
example of something.
> Second, my example shows that Python puts some peculiar restrictions on
> recursion.
Incorrect. Recursive calls are just like any other function call in
Python: when you call *any* function from the body of a function, it must
be in scope at runtime.
def fact(n):
print g() # g must be in scope when fact is called
if n < 2: return 1
return n*fact(n-1) # fact must be in scope when fact is called
Python doesn't treat recursive functions specially in any way. It is
*classes*, not functions, that are special. This is explained in the docs:
http://docs.python.org/reference/executionmodel.html
It is also discussed in the PEP introducing nested scopes to Python:
http://www.python.org/dev/peps/pep-0227/
It's even eluded to in the tutorial:
http://docs.python.org/tutorial/classes.html
> Recursion! One of the central concepts in the theory of
> functions! This is shown most clearly by the following elaboration of
> my original example:
>
> class Demo(object):
> def fact_rec(n):
> if n < 2:
> return 1
> else:
> return n * fact_rec(n - 1)
Why are you defining a method without a self parameter?
In any case, the problem has nothing to do with recursion:
>>> class Broken:
... def f():
... return g()
... def g():
... return "gee"
... x = f()
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 6, in Broken
File "<stdin>", line 3, in f
NameError: global name 'g' is not defined
> def fact_iter(n):
> ret = 1
> for i in range(1, n + 1):
> ret *= i
> return ret
>
> classvar1 = fact_iter(5)
> classvar2 = fact_rec(5)
>
> This code won't compile as shown,
That's nonsense. It compiles, and then it suffers a runtime error when
you *execute* it. (NameError occurs at runtime, not at compile time.)
You can prove this for yourself by putting that above class definition
inside a string, s:
>>> x = compile(s, '', 'exec')
>>> exec(x)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "", line 1, in <module>
File "", line 15, in Demo
File "", line 6, in fact_rec
NameError: global name 'fact_rec' is not defined
If you expect us to take your criticisms seriously, at least get your
basic facts right.
[...]
> Is there any good reason (from the point of view of Python's overall
> design) for not fixing this?
It doesn't need to be "fixed" because it's not broken. The question is,
should it be *changed* to match C++ programmers' expectations?
--
Steven
kj
>Yeah, it's a little surprising that you can't access class scope from
>a function, but that has nothing to do with encapsulation.
It does: it thwarts encapsulation. The helper function in my
example is one that clearly rightfully belongs nowhere else than
the class itself, i.e. encapsulated within the class. It is only
this silly prohibition against recursive functions in a class
statement that forces one to put it outside the class statement.
kynn
Steven D'Aprano
> In <jeqdncAMUYvTrwjX...@bt.com> "Martin P. Hellwig"
> <martin....@dcuktec.org> writes:
>
>>kj wrote:
>><cut>
>>> First, one of the goals of OO is encapsulation, not only at the level
>>> of instances, but also at the level of classes.
>>Who says?
>
> Python itself: it already offers a limited form of class encapsulation
> (e.g. class variables).
An int variable is an int. A string variable is a string. A bool variable
is a bool. A class variable is a class.
Perhaps you mean a class attribute?
> It would be nice if it went all the way and
> gave classes their own bona fide scope.
Classes have their own bona fide scope. It just doesn't work the way you
expect it to.
> (But I hasten to add: I *still*
> don't understand the Python scope model, and not for lack of trying.
> I've only succeeded in finding fragments of this model explained here
> and there, like pottery shards: a bit lost in a tutorial, or some
> comment in a newsgroup thread, etc. Maybe the full, authoritative
> documentation of Python's scope model got lost somewhere, and will be
> found by archaeologists in the 25th century...)
It's all in the Fine Manual:
http://docs.python.org/reference/executionmodel.html
>>Anyway, you could be right (I am not capable to judge it) and Python
>>should change on this issue but from what I gathered, Pythons OO is
>>inspired by the following:
>>- Don't repeat yourself
>>- Containing code into logical units makes it easier to understand and
>>maintain.
>
> ...except, apparently, when that code is a recursive function, in which
> case one's out of luck.
Incorrect. As I showed in my previous post, your problem has nothing to
do with recursion.
>>> Second, my example shows that Python puts some peculiar restrictions
>>> on recursion. Recursion! One of the central concepts in the theory
>>> of functions!
>><cut>
>>It is also one of the best ways to shoot yourself in the foot...
>
> If recursion is so evil, and Python so intent in saving us from shooting
> ourselves in the foot, why does it allow recursion at all?
Recursion isn't evil, and Python isn't intent on preventing foot-shooting.
--
Steven
kj
>On Wed, 26 Aug 2009 10:57:32 +0000, kj wrote:
>> Recursion! One of the central concepts in the theory of
>> functions! This is shown most clearly by the following elaboration of
>> my original example:
>>
>> class Demo(object):
>> def fact_rec(n):
>> if n < 2:
>> return 1
>> else:
>> return n * fact_rec(n - 1)
>Why are you defining a method without a self parameter?
Because, as I've explained elsewhere, it is not a method: it's a
"helper" function, meant to be called only once, within the class
statement itself.
Well, this is not strictly true, because the function is recursive,
so it will also call itself a few times. But still, all these
calls happen (loosely speaking) "within the class statement".
In fact the only reason to use a function for such initialization
work is when one needs recursion; otherwise there's no point in
defining a function that will only be invoked exactly once.
kynn
Nigel Rantor
>
> Needless to say, I'm pretty beat by this point. Any help would be
> appreciated.
>
> Thanks,
Based on your statement above, and the fact that multiple people have
now explained *exactly* why your attempt at recursion hasn't worked, it
might be a good idea to step back, accept the advice and walk away
instead of trying to convince people that the language forbids recursion
and doesn't provide decent OO ecapsulation.
Otherwise I'd wager you'll soon be appearing in multiple kill-files.
n
Steven D'Aprano
>>1. One of the key aspects of Python's design is that attributes must be
>>accessed explicitly with dot notation. Accessing class scopes from
>>nested functions would (seemingly) allow access to class attributes
>>without the dotted notation. Therefore it is not allowed.
>
> It would be trivial to define a keyword (e.g. this, or if you prefer,
> __this__), valid only within a class statement, and that the interpreter
> would recognize as "the current class", even before this class is full
> defined.
Python doesn't treat the addition of new keywords, and the corresponding
breakage of code which used that word as a normal name, as "trivial" --
the Python dev team takes their responsibilities to not casually break
people's code seriously. That pretty much rules out "this", although it
would allow "__this__".
However, what's your use-case?
class Demo(object):
def f(n):
return n+1
x = f(10)
is a poorly written class, because the user will expect to do this:
instance = Demo()
assert instance.c == 11 # suceeds
instance.f(10) == 11 # fails
Since the function f doesn't refer to a Demo instance, or the Demo class,
why do you put it inside the Demo class? It doesn't belong there, it
belongs in the global (module) scope. Move it out, and your problem goes
away.
The only use-case I can think of for __this__ is the following:
class Example(object):
@staticmethod
def rec(n):
if n < 2: return "x"
return "x" + __this__.rec(n/2)
Example.rec(15)
but again, why include rec() in the class if it doesn't actually have
anything to do with the class or the instance? Or why make it a
staticmethod? Just make it a class method:
class Example(object):
@classmethod
def rec(cls, n):
if n < 2: return "x"
return "x" + cls.rec(n/2)
--
Steven
Carl Banks
> In <1bf83a7e-f9eb-46ff-84fe-cf42d9608...@j21g2000yqe.googlegroups.com> Carl Banks <pavlovevide...@gmail.com> writes:
>
> >Yeah, it's a little surprising that you can't access class scope from
> >a function, but that has nothing to do with encapsulation.
>
> It does: it thwarts encapsulation. The helper function in my
> example is one that clearly rightfully belongs nowhere else than
> the class itself, i.e. encapsulated within the class. It is only
> this silly prohibition against recursive functions in a class
> statement that forces one to put it outside the class statement.
Oh well, I guess that sucks for you.
Carl Banks
Steven D'Aprano
> In <1bf83a7e-f9eb-46ff...@j21g2000yqe.googlegroups.com>
> Carl Banks <pavlove...@gmail.com> writes:
>
>>Yeah, it's a little surprising that you can't access class scope from a
>>function, but that has nothing to do with encapsulation.
>
> It does: it thwarts encapsulation. The helper function in my example is
> one that clearly rightfully belongs nowhere else than the class itself,
> i.e. encapsulated within the class.
There's nothing "clearly" about it.
This helper function doesn't reference the class, or any instance of the
class. Why should it be encapsulated in the class? It's not *part* of the
class, it shouldn't be *inside* the class.
Look at it this way: classes are made from building blocks. Just because
a building block ends up in a class, doesn't mean the function that makes
the building block has to be inside the class too.
It's an accident of Python's execution model that a single function call
*sometimes* works as you expect inside the class statement:
class Demo:
def f():
return 2
def g():
return f()+1
x = f() # works
y = g() # fails
As you can see, the problem is not because of recursion, but because the
class scope is not inserted into the function scope. As I said earlier,
your problem isn't too little class encapsulation, but too much: the
class scope doesn't leak into the function scope.
Python could, of course, behave the way you want, but it would lead to
some odd interactions:
class Weird(object):
x = 1
def __init__(self):
self.y = 2
def test(self):
print self.x # refers to attribute x with value 1
print x # refers to attribute x with value 1
print self.y # refers to attribute y with value 2
print y # refers to global y
In existing Python, both x and y will refer to globals, because it's
considered more important for all attribute access to be consistently
explicit than to insert the class scope into the function scope. This
isn't a design flaw, or a bug, it's a feature. Yes, it makes it hard for
you to solve your problem the way you want to solve it, but it doesn't
stop you from solving your problem. The module is encapsulation enough.
--
Steven
Ulrich Eckhardt
> class Demo(object):
> def fact_iter(n):
> ret = 1
> for i in range(1, n + 1):
> ret *= i
> return ret
>
> def fact_rec(n):
> if n < 2:
> return 1
> else:
> return n * fact_rec(n - 1)
>
> classvar1 = fact_iter(5)
> classvar2 = fact_rec(5)
>
>
> In the initialization of classvar1, fact_iter is invoked without
> any problem even though the class is not yet created: no need to
> qualify it with the name of the class. This is completely inconsistent
> with the requirement that fact_rec be so qualified when invoked
> within fact_rec.
Let me take a shot at explaining this, maybe it helps that I'm mostly a C++
guy....
Firstly, if Python sees a name, it looks up that name first in the local
scope and then in the global scope. This is very simple (especially
compared to C++ when adding ADL there...), but it is something most people
can live with. So, even inside a class function (static or member), you
have to qualify a function call like 'ClassName.function_name', because
neither is 'function_name' a local nor is it a global.
Secondly, and that is due to the very dynamic nature of Python's types, the
class doesn't exist until its definition is finished. Therefore, any
attempt (directly or indirectly) to access a class member will usually fail
before that time. The exception is that inside the class definition you can
access the members directly, because they are in the same scope (access to
locals). Note that the scope already seems to exist but that it is not yet
available under any name! Looking at your example, you can not
write 'classvar1 = Demo.fact_iter(42)', because the lookup of 'Demo' will
fail.
Now, what actually causes problems is that 'fact_rec' is not universally
accessible until class 'Demo' is fully defined, but you need this in order
to fully define it. Here comes a drawback of the dynamic nature of Python,
that the declaration phase (compiling) is not separate from the execution.
I fully agree that this case is rather vexing to my (and obviously your)
biased brain. I wouldn't call this a bug before fully understanding why
e.g. you can not access a class before its definition is finished. I think
someone mentioned one or two PEPs, which are the design documents that
explain Python including the rationale, I'd use that as a starting point.
Suggestion: someone mentioned that you should make the function a normal
function. You can mark it as private using an underscore as prefix. If you
want to make it foolproof, you could even delete ("del fact_rec") the
function after use. Other alternatives would be to put it into a separate
baseclass, use a local function as implementation or to add the class
attributes after defining the class. Neither of these provide the 'perfect'
encapsulation of things in a class that you might be used to from e.g.
Java, but they have proven to be very useful nonetheless.
Terry Reedy
> In <7figv3F...@mid.uni-berlin.de> "Diez B. Roggisch" <de...@nospam.web.de> writes:
>
>> Classes are not scopes.
Classes are objects. In particular, they are (by default) instances of
class 'type'. Unless 'scopes' were instances of some other metaclass,
the statement has to be true. I understand 'scope' as referring to a
section of code, as opposed to a runtime object.
Class statements introduce a new local namespace used to define the
class. Whether one considers that as introducing a new 'scope' depends,
I suppose, on one's exact definition of scope.
> This looks to me like a major wart, on two counts.
It is a 'wart' that a Python object is not a 'scope', whatever that is
to you? I have trouble comprehending that claim.
> First, one of the goals of OO is encapsulation, not only at the
> level of instances, but also at the level of classes. Your comment
> suggests that Python does not fully support class-level encapsulation.
I really do not see how your claim follows from the comment. The irony
of your 'two counts' is that the example of 'count 2' fails precisely
because of the encapsulation that you claim does not exist as 'count 1'.
> Second, my example shows that Python puts some peculiar restrictions
> on recursion.
I claim it does not. Name-based recursion inherently requires that a
function be able to access itself by name at the time it is called. This
can be a bit tricky, especially in a language with dynamic rather than
static name binding and resolution, as it requires that code within the
function be able to access a scope outside itself in which the function
name is defined. In other words, it requires that function code *not* be
completely encapsulated. It also require that the appropriate name be
used when there is one. Neither of these is a 'peculiar restriction'
imposed by Python.
> class Demo(object):
> def fact_rec(n):
> if n < 2:
> return 1
> else:
> return n * fact_rec(n - 1)
This function is just a function. It is not an instance method. It is
not even a class method. It does not belong here and should not be here.
If you insist on putting it where it does not belong, then you have to
call it by a name that works. If you only call it after the class
statement has finished, then 'Demo.fact_rec' works, as I believe someone
else pointed out.
If you want to call the function during class creation, before (in this
case) Demo exists, then binding it to a local name works. With 3.1
class Demo:
def f1(n):
if n < 2: return 1
else: return n * Demo.f1(n - 1)
def f2(n,f):
if n < 2: return 1
else: return n * f(n - 1, f)
cvar = f2(5, f2)
print(Demo.f1(5), Demo.cvar, Demo.f2(5,Demo.f2))
# prints
>>>
120 120 120
> Recursive functions should be OK wherever functions are OK.
Iteration can and has been viewed as within-frame recursion. When
iterative rather than recursive syntax is used, the naming issue is
avoided.
> Is there any good reason (from the point of view of Python's overall
> design) for not fixing this?
After reading the above, what, if anything, do you think still needs to
be fixed?
Before 2.2, Python functions were more encapsulated than they are today
in that they could only access local and global namespaces but not outer
function namespaces. It would be possible to further de-encapsulate
them by giving them direct access to lexically surrounding class
namespaces, but this would increase the problem of name clashes without
solving any real problems in proper Python code. It could also break the
intentional design principle that function code should mean the same
thing whether placed within or without a class statement. This principle
allows functions to be added to existing classes as attributes and work
as methods that same as if they had been defined with the class.
Terry Jan Reedy
Ethan Furman
>
>
> I have many years of programming experience, and a few languages,
> under my belt, but still Python scoping rules remain mysterious to
> me. (In fact, Python's scoping behavior is the main reason I gave
> up several earlier attempts to learn Python.)
>
> Here's a toy example illustrating what I mean. It's a simplification
> of a real-life coding situation, in which I need to initialize a
> "private" class variable by using a recursive helper function.
>
> class Demo(object):
> def fact(n):
> if n < 2:
> return 1
> else:
>
> _classvar = fact(5)
As has been pretty thoroughly discussed, the issue here is not
recursion, but name lookup.
Going back through the archives I found Arnaud's post with this decorator:
def bindfunc(f):
def boundf(*args, **kwargs):
return f(boundf, *args, **kwargs)
return boundf
If you use it on your fact function like so...
class Demo(object):
@bindfunc
def fact(recurse, n) # recurse can be any name you like
if n < 2:
return 1
else:
return n * recurse(n-1)
_classvar = fact(5)
del fact # no longer needed, and won't work
# once class is created
This should do as you want.
As a side note, if you're going to bother asking questions on this list,
you really should try to understand the answers. I won't gripe at you
too much, though, 'cause I learned a lot from the many responses given
due to your refusal to do so. ;-)
~Ethan~
kj
>http://docs.python.org/reference/executionmodel.html
>It is also discussed in the PEP introducing nested scopes to Python:
>http://www.python.org/dev/peps/pep-0227/
>It's even eluded to in the tutorial:
>http://docs.python.org/tutorial/classes.html
Thanks!
kynn
Dave Angel
> On Aug 26, 8:13 am, Dave Angel <da...@ieee.org> wrote:
>
>> You can probably work around this by replacing the staticmethod
>> decorator with an equivalent function call:.
>>
>> class Demo9(object):
>> def fact(n):
>> if n < 2:
>> return 1
>> else:
>> return n * Demo.fact(n - 1)
>>
>> fact =taticmethod(fact)
>>
>> print Demo9._classvar
>> xx =emo9()
>> print xx.fact(6)
>> print Demo9.fact(8)
>>
>
> This won't work normally. It only worked for you because you made a
> typo.
>
>
> Carl Banks
>
>
the class in the same module, and forgot to include to change Demo to
Demo9 in the recursive call.
I didn't like that approach anyway, as it smacked of taking advantage of
some implementation accident. The other approaches are more
straightforward.
DaveA
kj
>Going back through the archives I found Arnaud's post with this decorator:
>def bindfunc(f):
> def boundf(*args, **kwargs):
> return f(boundf, *args, **kwargs)
> return boundf
>If you use it on your fact function like so...
>class Demo(object):
> @bindfunc
> def fact(recurse, n) # recurse can be any name you like
> if n < 2:
> return 1
> else:
> return n * recurse(n-1)
> _classvar = fact(5)
> del fact # no longer needed, and won't work
> # once class is created
>This should do as you want.
Thanks, this is instructive.
>As a side note, if you're going to bother asking questions on this list,
>you really should try to understand the answers.
I think I understand the answers well enough. What I *really*
don't understand is why this particular "feature" of Python (i.e.
that functions defined within a class statement are forbidden from
"seeing" other identifiers defined within the class statement) is
generally considered to be perfectly OK. IMO it's a bizarre,
inexplicable blindspot (which, among other things, gives rise to
a certain worry about what other similar craziness lurks under
Python's image of rationality). I have never seen even a half-hearted
justification, from a language design point of view, for why this
particular "feature" is worth having. Maybe some day the BDFL will
deign to give one.
kynn
kj
>But if you insist on the above methodology, you can do this:
>class Demo(object):
> def fact(n):
> def inner(n):
> if n < 2:
> return 1
> else:
> return n * inner(n - 1)
> return inner(n)
> _classvar = fact(5)
>This makes inner a *local* variable, which is found.
Thanks for this. I think this is the most straightforward workaround.
kynn
kj
>kj wrote:
>> class Demo(object):
>> def fact_iter(n):
>> ret = 1
>> for i in range(1, n + 1):
>> ret *= i
>> return ret
>>
>> def fact_rec(n):
>> if n < 2:
>> return 1
>> else:
>> return n * fact_rec(n - 1)
>>
>> classvar1 = fact_iter(5)
>> classvar2 = fact_rec(5)
>>
>>
>> In the initialization of classvar1, fact_iter is invoked without
>> any problem even though the class is not yet created: no need to
>> qualify it with the name of the class. This is completely inconsistent
>> with the requirement that fact_rec be so qualified when invoked
>> within fact_rec.
>Let me take a shot at explaining this, maybe it helps that I'm mostly a C++
>guy....
Thanks for your reply.
>I fully agree that this case is rather vexing to my (and obviously your)
>biased brain. I wouldn't call this a bug before fully understanding why
>e.g. you can not access a class before its definition is finished.
I understand this, what I don't understand is why do it this way.
I've been trying to understand this particular design point for
*literally* years.
>I think
>someone mentioned one or two PEPs, which are the design documents that
>explain Python including the rationale, I'd use that as a starting point.
I'm reading PEP 227 now, suggested by Steven D'Aprano. In it I
find statements like the following alert:
(Note: If a region is contained within a class definition, the
name bindings that occur in the class block are not visible to
enclosed functions.)
I've seen this before, but never an explanation for why having this
restriction. It's just one of life's mysteries. (Incidentally,
the fact that the author of PEP 227 felt it necessary to add that
parenthetical remark suggests that the expectation it warns against
is not so crazy after all.)
But I'm still not done with PEP 227. Maybe I'll see the light by
the time I'm done.
kynn
Jan Kaliszewski
> Jean-Michel Pichavant wrote:
>> class Color:
>> def __init__(self, r, g,b):
>> pass
>> BLACK = Color(0,0,0)
>>
>> It make sens from a design point of view to put BLACK in the Color
>> namespace. But I don't think it's possible with python.
>
> class Color:
> ...
>
> setattrib(Color, "BLACK", Color(0,0,0))
Or simpler:
class Color:
...
Color.BLACK = Color(...)
Then
(Color.BLACK is Color.BLACK.BLACK.BLACK.BLACK) == True
:-)
*j
PS. Obviously, that's nothing special (there is no problem with
creating such "recursive" references in Python).
--
Jan Kaliszewski (zuo) <z...@chopin.edu.pl>
Ethan Furman
> I think I understand the answers well enough. What I *really*
> don't understand is why this particular "feature" of Python (i.e.
> that functions defined within a class statement are forbidden from
> "seeing" other identifiers defined within the class statement) is
> generally considered to be perfectly OK. IMO it's a bizarre,
> inexplicable blindspot (which, among other things, gives rise to
> a certain worry about what other similar craziness lurks under
> Python's image of rationality). I have never seen even a half-hearted
> justification, from a language design point of view, for why this
> particular "feature" is worth having. Maybe some day the BDFL will
> deign to give one.
>
> kynn
You keep using that word. I do not think it means what you think it
means. :)
Keep studying. Listen. Learn how it *is*. Understanding may come later.
~Ethan~
Jan Kaliszewski
1. I don't understand then... Why do you desire to both define and
run it *within* that class statement as if it was this class's
method?
2. Could you please show me how it could be done in C++ or Java?
(Or you want to say that that languages also are not fully valuable
OO languages?)
3. Python makes function bodies "agnostic" about the context of their
definition -- generally any non-global information must be passed
explicitly to their interior. *It has nothing to do with recursion.*
If you really must both define and use such a function within the
class definition, pass function object to itself explicitly, and
everybody will be happy:
class Demo(object):
def fact(fact, n):
if n < 2:
return 1
else:
return n * fact(fact, n - 1)
fact(fact, 3)
*j
Hendrik van Rooyen
> As I described at length in another reply, the function in question
> is not intended to be "callable outside the class". And yes,
I think this might go to nub of your problem - It might help you to think as
follows:
A Python class, even after it has been executed, does not really exist except
as a kind of template or pattern - it is mostly useless until an instance of
the class is made by calling it with whatever it needs to set up the
instance. And once you have an instance, you can do stuff with that
particular instance. Before that time, the class is mostly just a promise of
things to come.
<aside>
The other thing to bear in mind is that you cannot really hide stuff in python
classes - there are no really private things.
</aside>
> recursive functions in Python *are* restricted in ways that
> non-recursive functions aren't. The examples I've posted prove
> this point unambiguously.
Yes and no - mostly no - your examples just illustrate the point I tried to
make above.
Pythons object model, and its classes, are different from what you are used
to. A bare class is mostly useless without an instance, which is ultimately
why accessing a function in a class from itself like you are doing, without
reference to an instance, does not work - the function does not exist yet to
a degree that it can be referenced. It is kind of subtle, and different from
other languages.
8<------------- rant against the way things are -----------------------
Welcome to python, and the newsgroup, by the way.
- Hendrik
Hendrik van Rooyen
> In <02a54597$0$20629$c3e...@news.astraweb.com> Steven D'Aprano
<st...@REMOVE-THIS-cybersource.com.au> writes:
> >Why are you defining a method without a self parameter?
>
> Because, as I've explained elsewhere, it is not a method: it's a
> "helper" function, meant to be called only once, within the class
> statement itself.
If the sole purpose of the function is to be used to define what will become a
constant, why do you not just calculate the constant on your calculator, or
at the interactive prompt, and assign it to the attribute, and be done with
it?
Why waste run time recalculating every time the programme is called?
>
> Well, this is not strictly true, because the function is recursive,
> so it will also call itself a few times. But still, all these
> calls happen (loosely speaking) "within the class statement".
Why *must* it be there?
>
> In fact the only reason to use a function for such initialization
> work is when one needs recursion; otherwise there's no point in
> defining a function that will only be invoked exactly once.
Seems no point to me even when there is recursion - a number is a number is a
number. If you know the arguments when you write it, and if the function is
known at time of writing, storing a literal is the best solution.
Conversely, if the arguments are not known at time of writing, then they
should be passed at run time, when an instance is created, and assigned as
part of the __init__ method.
And the same is true if the function is not known at time of writing - then it
must be passed to the constructor of the instance.
Storm in a teacup.
- Hendrik
Terry Reedy
> I think I understand the answers well enough. What I *really*
> don't understand is why this particular "feature" of Python (i.e.
> that functions defined within a class statement are forbidden from
> "seeing" other identifiers defined within the class statement) is
> generally considered to be perfectly OK. IMO it's a bizarre,
> inexplicable blindspot (which, among other things, gives rise to
> a certain worry about what other similar craziness lurks under
> Python's image of rationality). I have never seen even a half-hearted
> justification, from a language design point of view, for why this
> particular "feature" is worth having. Maybe some day the BDFL will
> deign to give one.
I attempted to give an explanation in my previous post.
Bruno Desthuilliers
(snip)
> Now, what actually causes problems is that 'fact_rec' is not universally
> accessible until class 'Demo' is fully defined, but you need this in order
> to fully define it. Here comes a drawback of the dynamic nature of Python,
> that the declaration phase (compiling) is not separate from the execution.
It is. You can manually compile a module, remove the source .py and
still import (or execute) the compiled .pyc.
The "problem" here is not with compilation, but with the fact that
'class' (and 'def') statements are *executable* statements. IOW, the
class statement is executed when encountered, that is, usually (most
class statements being top-level statements), when the module is first
loaded.
> I fully agree that this case is rather vexing to my (and obviously your)
> biased brain. I wouldn't call this a bug before fully understanding why
> e.g. you can not access a class before its definition is finished.
Nor why it's a GoodThing(tm) for quite a lot of use case - while the
OP's problem is in the splitting-hairs category - only a problem if yoçu
insist in imposing your views on the language instead of learning how to
use it.
> Suggestion: someone mentioned that you should make the function a normal
> function.
(snip)
Someone (Diez Roggisch IIRC) also mentionned a simple solution that
mostly fulfills the OP's desires: use a nested function.
Stephen Fairchild
> Because, as I've explained elsewhere, it is not a method: it's a
> "helper" function, meant to be called only once, within the class
> statement itself.
So why didn't you delete it after you were done with it?
Class Demo(object):
def fact(x):
...
_classvar = fact(5)
del fact()
--
Stephen Fairchild
Bruno Desthuilliers
(snip)
>
>> I fully agree that this case is rather vexing to my (and obviously your)
>> biased brain. I wouldn't call this a bug before fully understanding why
>> e.g. you can not access a class before its definition is finished.
>
> I understand this, what I don't understand is why do it this way.
> I've been trying to understand this particular design point for
> *literally* years.
It's quitye simple: 'class' is an executable statement that creates a
class object (yes, all this happens at runtime) and bind it in the
current namespace. So until the class statement is executed, the class
object doesn't exist and it's name is not bound.
Now if you don't understand why it's a GoodThing(tm) that all this
happens at runtime, have a look at metaclasses, how they are used in
most major Python frameworks, and how this allows to vastly reduce the
amount of needed boring and erreor-prone boilerplate code one cand find
elsewhere.
>> I think
>> someone mentioned one or two PEPs, which are the design documents that
>> explain Python including the rationale, I'd use that as a starting point.
>
> I'm reading PEP 227 now, suggested by Steven D'Aprano. In it I
> find statements like the following alert:
>
> (Note: If a region is contained within a class definition, the
> name bindings that occur in the class block are not visible to
> enclosed functions.)
>
> I've seen this before, but never an explanation for why having this
> restriction.
You should re-read Carl Bank's answers more carefully then.
> It's just one of life's mysteries. (Incidentally,
> the fact that the author of PEP 227 felt it necessary to add that
> parenthetical remark suggests that the expectation it warns against
> is not so crazy after all.)
It's not so crazy for someone coming from a more static language.
Python's object model is indeed a bit peculiar, and it can take some
times getting the big picture, but be sure it has been carefully
designed, and is incredibly powerfull once you start understanding the
whole thing.
> But I'm still not done with PEP 227. Maybe I'll see the light by
> the time I'm done.
My 2 cents : learn about metaclasses, attributes lookup rules, and the
descriptor protocol (and some practical applications of these features).
Then you might decide that the few resulting restrictions are really
worth the price.
Bruno Desthuilliers
> In <1bf83a7e-f9eb-46ff...@j21g2000yqe.googlegroups.com> Carl Banks <pavlove...@gmail.com> writes:
>
>> Yeah, it's a little surprising that you can't access class scope from
>> a function, but that has nothing to do with encapsulation.
>
> It does: it thwarts encapsulation. The helper function in my
> example is one that clearly rightfully belongs nowhere else than
> the class itself, i.e. encapsulated within the class.
I don't see what's wrong with having this function at the top-level.
Sorry to have to say it this way, but IMHO you're border on cargo-cult
thinking here. Python is an all-object language (ie : everything is an
object, including functions, classes and modules), but it's by no mean
'pure-object', and module level functions are perfectly ok (and quite
often the right thing).
Now if you really want to "encapsulate" the function and the class
together, you do have simple and and working solutions : using a nested
recursive function (as explained by Diez), or nesting both the class and
function in a factory function, ie:
def Demo():
def fact(n):
if n < 2:
return 1
else:
return n * fact(n - 1)
class Demo(object):
_classvar = fact(5)
return Demo
Demo = Demo()
But anyway: this is really a WTF. Python's notion of encapsulation is
very weak (by design), and you're really wasting time trying to fight
the language instead of learning it. Just put that f... helper function
at the top level (prefixing it's name with a '_' to mark it as
implementation detail), and move on to something else !-)
> It is only
> this silly prohibition against recursive functions in a class
> statement that forces one to put it outside the class statement.
The "prohibition" only happens for recursive functions defined *and*
called in the class statement. And once you get the big picture, it's
certainly not "silly".
Bruno Desthuilliers
> In <jeqdncAMUYvTrwjX...@bt.com> "Martin P. Hellwig" <martin....@dcuktec.org> writes:
>
>> kj wrote:
>> <cut>
>> Who says?
>
> Python itself: it already offers a limited form of class encapsulation
> (e.g. class variables).
"class variables" - which FWIW include the "methods" - are just
attributes of the class *object* (reminder : Python's classes and
functions *are* objects too). Any name bound at the top level of the
class statement scope becomes an attribute of the class object. IOW,
this "limited form of class encapsulation" is just the ordinary "form of
encapsulation" you'll get with Python objects.
> It would be nice if it went all the way
> and gave classes their own bona fide scope. (But I hasten to add:
> I *still* don't understand the Python scope model,
I think that what you should first understand are Python's execution and
object models.
>> Anyway, you could be right (I am not capable to judge it) and Python
>> should change on this issue but from what I gathered, Pythons OO is
>> inspired by the following:
>> - Don't repeat yourself
>> - Containing code into logical units makes it easier to understand and
>> maintain.
>
> ...except, apparently, when that code is a recursive function, in
> which case one's out of luck.
I wrote quite a few recursive functions in Python and never had any
problem.
>>> Second, my example shows that Python puts some peculiar restrictions
>>> on recursion.
It's not a restriction on recursion, it's a scoping rule that apply to
any other name. You can't access the class statement's scope from within
a function, period.
>>> Recursion! One of the central concepts in the theory
>>> of functions!
>> <cut>
>> It is also one of the best ways to shoot yourself in the foot...
>
> If recursion is so evil, and Python so intent in saving us from
> shooting ourselves in the foot, why does it allow recursion at all?
Recursion is not evil, and Python doesn't try to prevent anyone from
doing stupid things anyway. FWIW, do you know that you can add / replace
/ remove attributes (including methods) at runtime, both on a
per-instance or per-class basis ? And even dynamically change the class
of an object ?-)
Steven D'Aprano
> On Wednesday 26 August 2009 17:14:27 kj wrote:
>
>> As I described at length in another reply, the function in question is
>> not intended to be "callable outside the class". And yes,
>
> I think this might go to nub of your problem - It might help you to
> think as follows:
>
> A Python class, even after it has been executed, does not really exist
> except as a kind of template or pattern - it is mostly useless until an
> instance of the class is made by calling it with whatever it needs to
> set up the instance. And once you have an instance, you can do stuff
> with that particular instance. Before that time, the class is mostly
> just a promise of things to come.
Oh my! I couldn't disagree more strongly! I think the above is totally
incorrect.
Classes and types are first class objects, you can treat them like
anything else in Python. In fact, classes themselves are instances of
type, so you can say:
>>> class C(object): # must inherit from object
... pass
...
>>> issubclass(C, object)
True
>>> isinstance(C, type)
True
Classes are themselves instances of their metaclass. By default, classes
have a metaclass of type, but you can easily change that. Metaclass
programming is advanced but very powerful.
Because classes are themselves objects, you can (with a bit of metaclass
jiggery-pokery) make them do all sorts of interesting things. For
instance, huge amounts of effort are often put into creating a Singleton
class, a class that has a single instance. Well, okay... but why not just
use the class object itself, instead of an instance? There's already one
of those, and you can't (easily) make a copy of it, and even if you did,
it would be an independent class. Instead of this:
singleton = SingletonClass(args)
do_something_with(singleton)
just do this:
do_something_with(SingletonClass)
Of course SingletonClass needs to be designed to work that way, which
will probably need some metaclass magic. It would be interesting to see
which requires less effort.
When it comes to built-in classes (types), I often use the class object
itself as an object. E.g. I might do something like this:
def convert(seq):
t = type(seq) # remember the original type
result = process(seq) # always produces a list
if t is list:
return result # don't bother making a copy of the result
elif t is str or t is unicode:
empty = t()
return empty.join(result)
else:
return t(result) # return the original type
>> recursive functions in Python *are* restricted in ways that
>> non-recursive functions aren't. The examples I've posted prove this
>> point unambiguously.
>
> Yes and no - mostly no - your examples just illustrate the point I
> tried to make above.
Completely no. You may have missed the examples I've given, but the
problems the Original Poster were having had nothing to do with recursion.
> Pythons object model, and its classes, are different from what you are
> used to. A bare class is mostly useless without an instance, which is
> ultimately why accessing a function in a class from itself like you are
> doing, without reference to an instance, does not work - the function
> does not exist yet to a degree that it can be referenced.
That is incorrect. What's going on is more subtle.
>>> class Demo:
... def function(x):
... print "Calling function with argument %s" % x
... function(None)
... function(1)
... function(function)
...
Calling function with argument None
Calling function with argument 1
Calling function with argument <function function at 0xb7d495dc>
>>> Demo
<class __main__.Demo at 0xb7d4e0ec>
--
Steven
Steven D'Aprano
> On Wednesday 26 August 2009 17:45:54 kj wrote:
>> In <02a54597$0$20629$c3e...@news.astraweb.com> Steven D'Aprano
> <st...@REMOVE-THIS-cybersource.com.au> writes:
>
>> >Why are you defining a method without a self parameter?
>>
>> Because, as I've explained elsewhere, it is not a method: it's a
>> "helper" function, meant to be called only once, within the class
>> statement itself.
>
> If the sole purpose of the function is to be used to define what will
> become a constant, why do you not just calculate the constant on your
> calculator, or at the interactive prompt, and assign it to the
> attribute, and be done with it?
>
> Why waste run time recalculating every time the programme is called?
What you are calculating might actually be quite complicated to enter as
a literal. You might have something like:
class C(object):
TABLE = []
for i in xrange(100):
row = []
for j in xrange(100):
row.append((i**3 + 2*i**2 - i*j + 7*j**2 + 9*i*j**2 - j)/3)
TABLE.append(row)
It's a little hard to type out C.TABLE as a literal.
And even if you could, which would you rather see if you were debugging
this class? The above, or:
class C(object):
TABLE = [
[0.0, 2.0, 8.6666666666666661, 20.0, 36.0, ... ],
[1.0, 5.666666666666667, 21.0, 47.0, 83.666666666666671, ...],
[5.333333333333333, 12.666666666666666, 36.666666666666664, ...],
...
[... , 3143161.0, 3201497.6666666665, 3260433.0]
]
For obvious reasons I haven't typed the whole thing out! And imagine
trying to check it for typos!
Clearly this is a made-up example, but the principle is sound. If Python
can calculate values for you, why not let it do so? It is easier for you,
easier to check that you've calculated them correctly, easier to debug
and read, it makes the algorithm clearer (fewer "magic constants"). Yes,
there is a run-time cost, but you only pay it once, when you import the
module, and it's likely to be not that much more expensive than parsing
the literals anyway.
Of course, for something as big and complicated as the above table, I'd
almost certainly put the code to calculate it in a function outside of
the class, but that's a matter of style, and it will work to put it
inside the class.
--
Steven
greg
> No, the fact() function here represents an internal "helper"
> function. It is meant to be called only once to help initialize
> a class variable that would be inconvenient to initialize otherwise;
> this helper function is not meant to be called from outside the
> class statement.
That, to me, is an excellent indication that the function
does *not* belong inside the class!
The only things that belong inside the class are things
that users of the class will need to use, or that instances
of the class will need to do their job.
Your helper function belongs outside the class, in the
module where the class is defined.
--
Greg
Jean-Michel Pichavant
> Ulrich Eckhardt wrote:
>
>> Jean-Michel Pichavant wrote:
>>
>>> class Color:
>>> def __init__(self, r, g,b):
>>> pass
>>> BLACK = Color(0,0,0)
>>>
>>> It make sens from a design point of view to put BLACK in the Color
>>> namespace. But I don't think it's possible with python.
>>>
>> class Color:
>> ...
>>
>> setattrib(Color, "BLACK", Color(0,0,0))
>>
>
>
> Color.BLACK = Color(0,0,0)
>
>
Obviously ... I'll remember that.
JM
Jean-Michel Pichavant
> I think I understand the answers well enough. What I *really*
> don't understand is why this particular "feature" of Python (i.e.
> that functions defined within a class statement are forbidden from
> "seeing" other identifiers defined within the class statement) is
> generally considered to be perfectly OK. IMO it's a bizarre,
> inexplicable blindspot (which, among other things, gives rise to
> a certain worry about what other similar craziness lurks under
> Python's image of rationality). I have never seen even a half-hearted
> justification, from a language design point of view, for why this
> particular "feature" is worth having. Maybe some day the BDFL will
> deign to give one.
>
> kynn
>
I think I got your point.
I guess many people may not be receptive to your question, cause guess
what, we're all python fans :o)
in foo.py:
a = 5
b = a # works fine
class A:
c = 5
d = c # broken
d = A.c # broken either
def foo(self):
e = 5
f = e #works fine
We should all acknowledge that any newcomer to python will not expect
such behavior. There are plenty of good answers to that thread
explaining why the fact that classes are not scopes is much better.
Still this design fails at one point : insight.
It may be solved by creating the class upon the "class" statement. If
the class A object is created, then c is added as a property of that
object, there's no problem accession one object property with A.c.
JM
kj
>in foo.py:
Thanks! I was beginning to think I'd gone crazy.
kynn
Steven D'Aprano
> in foo.py:
>
> a = 5
> b = a # works fine
>
> class A:
> c = 5
> d = c # broken
Incorrect. That works fine.
>>> class A:
... c = 5
... d = c # not actually broken
...
>>> A.c
5
>>> A.d
5
The class is a scope, and inside the class scope, you can access local
names. What you can't do is access the class scope from inside nested
functions.
--
Steven
kj
>in foo.py:
>a = 5
>b = a # works fine
>
> def foo(self):
> e = 5
> f = e #works fine
>It may be solved by creating the class upon the "class" statement. If
>the class A object is created, then c is added as a property of that
>object, there's no problem accession one object property with A.c.
I thought one could implement something like
class A:
c = 5
d = __thisclass__.c
But apparently this is a *huge* deal. I'm not sure if the reason
it is a huge deal is that it is technically difficult to implement,
or that there is an implicit "lie" in having something (__thisclass__)
standing for something else that doesn't yet exist (and the potential
errors that may arise from this "lie" in sufficiently unfortunate
code). Or maybe something else beyond my noobish ken altogether.
kynn
kj
>On Thu, 27 Aug 2009 09:09:21 +0200, Hendrik van Rooyen wrote:
>> On Wednesday 26 August 2009 17:45:54 kj wrote:
>>> In <02a54597$0$20629$c3e...@news.astraweb.com> Steven D'Aprano
>> <st...@REMOVE-THIS-cybersource.com.au> writes:
>>
>>> >Why are you defining a method without a self parameter?
>>>
>>> Because, as I've explained elsewhere, it is not a method: it's a
>>> "helper" function, meant to be called only once, within the class
>>> statement itself.
>>
>> If the sole purpose of the function is to be used to define what will
>> become a constant, why do you not just calculate the constant on your
>> calculator, or at the interactive prompt, and assign it to the
>> attribute, and be done with it?
>>
>> Why waste run time recalculating every time the programme is called?
>What you are calculating might actually be quite complicated to enter as
>a literal.
Thank you!
kynn
Bruno Desthuilliers
> kj wrote:
>> I think I understand the answers well enough. What I *really*
>> don't understand is why this particular "feature" of Python (i.e.
>> that functions defined within a class statement are forbidden from
>> "seeing" other identifiers defined within the class statement) is
>> generally considered to be perfectly OK. IMO it's a bizarre,
>> inexplicable blindspot (which, among other things, gives rise to
>> a certain worry about what other similar craziness lurks under
>> Python's image of rationality). I have never seen even a half-hearted
>> justification, from a language design point of view, for why this
>> particular "feature" is worth having. Maybe some day the BDFL will
>> deign to give one.
>>
>> kynn
>>
>
> I think I got your point.
> I guess many people may not be receptive to your question, cause guess
> what, we're all python fans :o)
>
> in foo.py:
>
> a = 5
> b = a # works fine
>
> class A:
> c = 5
> d = c # broken
Err... Did you actually tried this ?
>>> class A:
... c = 5
... d = c
...
>>> A.c
5
>>> A.d
5
>>>
> d = A.c # broken either
Not "broken" : the class doesn't yet exists, nor is it bound to global
name 'A'. FWIW, *this* works (for some definitions of 'works') juts fine:
>>> class Foo(object):
... c = 42
...
>>> A = Foo()
>>> class A(object):
... d = A.c
...
>>> A.d
42
>
> We should all acknowledge that any newcomer to python will not expect
> such behavior.
Any newcomer to any language should aknowledge that her expectations
based on previous experience with any other language should be kept
aside or just plain abandoned, and start by learning the new language.
The only thing one is entitled to expect when learning a new language is
that the language's implementation follows the language specs.
> There are plenty of good answers to that thread
> explaining why the fact that classes are not scopes is much better.
> Still this design fails at one point : insight.
Oh, really ?
> It may be solved by creating the class upon the "class" statement. If
> the class A object is created, then c is added as a property of that
> object, there's no problem accession one object property with A.c.
Please, write a pep...
D'Arcy J.M. Cain
Stephen Fairchild <some...@somewhere.com> wrote:
> So why didn't you delete it after you were done with it?
>
> Class Demo(object):
That should be "class".
> _classvar = fact(5)
> del fact()
I suppose you mean "del fact".
--
D'Arcy J.M. Cain <da...@druid.net> | Democracy is three wolves
http://www.druid.net/darcy/ | and a sheep voting on
+1 416 425 1212 (DoD#0082) (eNTP) | what's for dinner.
Hendrik van Rooyen
> On Thu, 27 Aug 2009 08:38:29 +0200, Hendrik van Rooyen wrote:
> > On Wednesday 26 August 2009 17:14:27 kj wrote:
> >> As I described at length in another reply, the function in question is
> >> not intended to be "callable outside the class". And yes,
> >
> > I think this might go to nub of your problem - It might help you to
> > think as follows:
> >
> > A Python class, even after it has been executed, does not really exist
> > except as a kind of template or pattern - it is mostly useless until an
> > instance of the class is made by calling it with whatever it needs to
> > set up the instance. And once you have an instance, you can do stuff
> > with that particular instance. Before that time, the class is mostly
> > just a promise of things to come.
>
> Oh my! I couldn't disagree more strongly! I think the above is totally
> incorrect.
It was intended to help the OP out of the mental hole he finds himself in.
Most of the classes I use fall directly into such a classification - they are
useless until an instance is created. And I would be so bold as to say that
_all_ gui classes are like that.
This is the pattern I am talking about:
class thing(object):
def __init__(self,foo,bar):
stuff to do things with foo and bar,
creating or modifying attributes of
the instance.
def somemethod(self,baz,bling):
instance method to do further operations on
the attributes of the instance
Now kindly explain to me how a class like that is usefull before an instance
of it is created, and I might agree with you that what I said is "totally
incorrect"
8< --trivial examples showing that there is something there ------------
> Classes are themselves instances of their metaclass. By default, classes
> have a metaclass of type, but you can easily change that. Metaclass
> programming is advanced but very powerful.
>
> Because classes are themselves objects, you can (with a bit of metaclass
> jiggery-pokery) make them do all sorts of interesting things. For
> instance, huge amounts of effort are often put into creating a Singleton
> class, a class that has a single instance. Well, okay... but why not just
> use the class object itself, instead of an instance? There's already one
> of those, and you can't (easily) make a copy of it, and even if you did,
> it would be an independent class. Instead of this:
>
> singleton = SingletonClass(args)
> do_something_with(singleton)
>
> just do this:
>
> do_something_with(SingletonClass)
>
> Of course SingletonClass needs to be designed to work that way, which
> will probably need some metaclass magic. It would be interesting to see
> which requires less effort.
*nods* yes this would make sense - but it is not quite what either the OP or I
was on about.
>
> When it comes to built-in classes (types), I often use the class object
> itself as an object. E.g. I might do something like this:
>
> def convert(seq):
> t = type(seq) # remember the original type
> result = process(seq) # always produces a list
> if t is list:
> return result # don't bother making a copy of the result
> elif t is str or t is unicode:
> empty = t()
> return empty.join(result)
> else:
> return t(result) # return the original type
nice. now try doing it with object:
thing = object()
After that, thing will exist, but it is a bit like write only memory -
completely useless, until you have done some more work with it.
>
> >> recursive functions in Python *are* restricted in ways that
> >> non-recursive functions aren't. The examples I've posted prove this
> >> point unambiguously.
> >
> > Yes and no - mostly no - your examples just illustrate the point I
> > tried to make above.
>
> Completely no. You may have missed the examples I've given, but the
> problems the Original Poster were having had nothing to do with recursion.
The yes was in there to make the man feel a bit better, and because from where
he stood, it _was_ the recursion that did him in. : - )
>
> > Pythons object model, and its classes, are different from what you are
> > used to. A bare class is mostly useless without an instance, which is
> > ultimately why accessing a function in a class from itself like you are
> > doing, without reference to an instance, does not work - the function
> > does not exist yet to a degree that it can be referenced.
>
> That is incorrect. What's going on is more subtle.
>
Right - I can see that you are reading that to mean that there must be an
instance. That is not what I intended to bring across. I was talking about
the lack of a reference that is his original problem, which he only
encountered with recursion.
> >>> class Demo:
>
> ... def function(x):
> ... print "Calling function with argument %s" % x
> ... function(None)
> ... function(1)
> ... function(function)
> ...
> Calling function with argument None
> Calling function with argument 1
> Calling function with argument <function function at 0xb7d495dc>
>
> >>> Demo
>
> <class __main__.Demo at 0xb7d4e0ec>
two points:
Of what earthly use is a bare class like that - except possibly as a container
for all the good stuff those functions produced out of nothing? - You might
as well use a list, or better a dict, to keep the results of such functions,
unless the functions are procedures, doing things by side effect - and if
that is the case, why bother putting them in a class?
Secondly:
Now what you have done is carefully avoided the OP's original problem, which
arises when the function in the class is called, and tries to call itself at
the time of execution of the class suite, and fails, for the reasons so
nicely explained by a lot of people. So the OP had a point when he
complained that python does not like recursion, because, until he understands
what is happening, the simplest theory that fits his observed facts is that
the failure is caused by the recursion. - a completely rational conclusion
based on Occam's razor. It just happens to be wrong in this case. This was
the source for my "Yes" above.
This should also work for the OP - I cannot recall if it has been shown:
class Demo(object):
def fact(x,f):
if x < 2:
return 1
else:
return x*f(x-1,f)
thing = fact(42,fact)
I still think it is stupid, though - thing should simply be set to
1405006117752879898543142606244511569936384000000000L
and that would be the end of it for this use case, without the bother of a
function and the concomitant waste of time.
- Hendrik
Hendrik van Rooyen
>
> What you are calculating might actually be quite complicated to enter as
> a literal. You might have something like:
8< ---------- Complicated Table Example -------------------------------
Me? - never! I am just an assembler programmer. I would not touch a thing
like that with a barge pole. Unless of course, I have to.
> Clearly this is a made-up example, but the principle is sound. If Python
> can calculate values for you, why not let it do so? It is easier for you,
> easier to check that you've calculated them correctly, easier to debug
> and read, it makes the algorithm clearer (fewer "magic constants"). Yes,
> there is a run-time cost, but you only pay it once, when you import the
> module, and it's likely to be not that much more expensive than parsing
> the literals anyway.
>
> Of course, for something as big and complicated as the above table, I'd
> almost certainly put the code to calculate it in a function outside of
> the class, but that's a matter of style, and it will work to put it
> inside the class.
It is a hell of a thing if it needs recursion to calculate - If it was really
that complex, I would calculate it, check it (if I can), document it and put
it in a module of its own, with "This side up", "Fragile", and other warning
stickers all over it.
- Hendrik
kj
>The only thing one is entitled to expect when learning a new language is
>that the language's implementation follows the language specs.
In fact, the official docs, when they discuss scopes, are off to
a bad start:
Names refer to objects. Names are introduced by name binding
operations. Each occurrence of a name in the program text refers
to the binding of that name established in the innermost function
block containing the use.
The first paragraph implies that binding can only occur within
functions, which is either incorrect, or presupposes a definition
of "function" that is not what most programmers would recognize.
In general, I found the docs very unclear on the subject of scoping.
PEP 227 is much better, but I wouldn't have thought of it as "the
specs".
kynn
Jan Kaliszewski
> The class is a scope, and inside the class scope, you can access local
> names. What you can't do is access the class scope from inside nested
> functions.
s/from inside nested functions/from inside nested scopes
Besides that detail, I fully agree.
Miles Kaufmann
> I think I understand the answers well enough. What I *really*
> don't understand is why this particular "feature" of Python (i.e.
> that functions defined within a class statement are forbidden from
> "seeing" other identifiers defined within the class statement) is
> generally considered to be perfectly OK. IMO it's a bizarre,
> inexplicable blindspot (which, among other things, gives rise to
> a certain worry about what other similar craziness lurks under
> Python's image of rationality). I have never seen even a half-hearted
> justification, from a language design point of view, for why this
> particular "feature" is worth having.
Guido's design justifications:
http://mail.python.org/pipermail/python-dev/2000-November/010598.html
--
My personal justification:
Python has used the same basic method of class creation since the very
beginning: create a new local namespace, execute the class suite in
that namespace, and then create a class, using the contents of the
namespace as the class attributes. The important thing to note here
is that there are really *two* namespaces--the local namespace that
exists while the class suite is being executed (what I call the "suite
namespace"), and the namespace of the class itself--and the first
ceases to exist when the second is created. The two namespaces
generally contain the same names at the point that the transfer
occurs, but they don't have to; the metaclass (which constructs the
class) is free to mess with the dictionary of attributes before
creating the class.
Suppose for a moment that the suite namespace *were* visible to nested
scopes. The simplest and most consistent implementation would be to
have a closure generated by a class statement be similar to that
generated by a function--i.e., the closure would be over the suite
namespace. This hardly seems desirable, though, because the suite
namespace and the class namespace would get out of sync when different
objects were assigned to the class namespace:
class C:
x = 1
def foo(self):
print x
print self.x
>>> o = C()
>>> o.foo()
1
1
>>> o.x = 2
>>> o.foo()
1
2
Surely such an implementation would be considered an even larger
Python wart then not having the suite namespace visible to nested
scopes at all. But it's better than the alternative of trying to
unify the class suite namespace and the class namespace, which would
be a nightmare of special cases (adding/deleting class attributes?
descriptors? __getattr__?) and require an implementation completely
separate from that of normal nested scopes.
-Miles
P.S. Just for fun:
import types
def make_class(*bases):
"""Decorator to allow you to (ab)use a function as a class definition.
The function must take no arguments and end with 'return locals()';
bases are (optionally) specified as arguments to make_class;
metaclasses other than 'type' are not supported.
>>> @make_class
... def C():
... greeting = 'Hello'
... target = 'world'
... def greet(self):
... print '%s, %s' % (self.greeting, target)
... return locals()
...
>>> C().greet()
Hello, world
"""
def decorator(func):
return type(func.func_name, bases, func())
if len(bases) == 1 and isinstance(bases[0], types.FunctionType):
func = bases[0]
bases = (object,)
return decorator(func)
if not bases:
bases = (object,)
return decorator
Piet van Oostrum
>k> No, the fact() function here represents an internal "helper"
>k> function. It is meant to be called only once to help initialize
>k> a class variable that would be inconvenient to initialize otherwise;
>k> this helper function is not meant to be called from outside the
>k> class statement. Granted, in the example I gave, the "helper"
>k> function (factorial) is a bit silly, but that was just intended as
>k> a simple and familiar example of a recursive function. The actual
>k> function that motivated this post would be considerably more
>k> difficult to explain and would have obscured the point of the post.
Classes don't have helper functions; they have methods. Instance
methods, static methods or class methods. Your's isn't either of these.
Methods are to be called like `something.method(...)'.
--
Piet van Oostrum <pi...@cs.uu.nl>
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: pi...@vanoostrum.org
kj
Miles Kaufmann <mil...@umich.edu> writes:
>On Aug 26, 2009, at 1:11 PM, kj wrote:
>> I think I understand the answers well enough. What I *really*
>> don't understand is why this particular "feature" of Python (i.e.
>> that functions defined within a class statement are forbidden from
>> "seeing" other identifiers defined within the class statement) is
>> generally considered to be perfectly OK. IMO it's a bizarre,
>> inexplicable blindspot (which, among other things, gives rise to
>> a certain worry about what other similar craziness lurks under
>> Python's image of rationality). I have never seen even a half-hearted
>> justification, from a language design point of view, for why this
>> particular "feature" is worth having.
>Guido's design justifications:
>http://mail.python.org/pipermail/python-dev/2000-November/010598.html
Ah! Clarity! Thanks! How did you find this? Did you know of
this post already? Or is there some special way to search Guido's
"design justifications"?
>...because the suite
>namespace and the class namespace would get out of sync when different
>objects were assigned to the class namespace:
>class C:
> x = 1
> def foo(self):
> print x
> print self.x
> >>> o = C()
> >>> o.foo()
>1
>1
> >>> o.x = 2
> >>> o.foo()
>1
>2
But this unfortunate situation is already possible, because one
can already define
class C:
x = 1
def foo(self):
print C.x
print self.x
which would lead to exactly the same thing.
I need to learn more about metaclasses, though, to fully understand
your post.
Many thanks!
kynn
Stephen Fairchild
> But this unfortunate situation is already possible, because one
> can already define
>
> class C:
> x = 1
> def foo(self):
> print C.x
> print self.x
How is this a problem? There is no ambiguity between the global scope and
the local from within foo.
From within foo C is accessed from the global scope having not found it as a
local. Once you have C C.x is just one step away.
Variable self is local to function foo since it was passed in as a parameter
from the method which wraps it.
Variable self refers to a class instance which contains a dictionary.
Variable x is absent from the instance dictionary so the class
self.__class__ is referenced, again from local scope to find x. If it
wasn't found there the superclasses would have been searched before
throwing an attribute error.
It seems to me, you are confusing instance creation with class creation.
--
Stephen Fairchild
Miles Kaufmann
> Miles Kaufmann <mil...@umich.edu> writes:
>> Guido's design justifications:
>> http://mail.python.org/pipermail/python-dev/2000-November/010598.html
>
> Ah! Clarity! Thanks! How did you find this? Did you know of
> this post already? Or is there some special way to search Guido's
> "design justifications"?
I just checked the python-dev archives around the time that PEP 227
was written.
You're right, of course. If I had been thinking properly, I would
have posted this:
... the suite namespace and the class namespace would get out of sync
when different objects were assigned to the class namespace:
# In a hypothetical Python with nested class suite scoping:
class C:
x = 1
@classmethod
def foo(cls):
print x
print cls.x
>>> C.foo()
1
1
>>> C.x = 2
>>> C.foo()
1
2
With your example, the result is at least easily explainable: self.x
is originally 1 because the object namespace inherits from the class
namespace, but running 'o.x = 2' rebinds 'x' in the object namespace
(without affecting the class namespace). It's a distinction that
sometimes trips up newbies (and me, apparently ;) ), but it's
straightforward to comprehend once explained. But the distinction
between the class suite namespace and the class namespace is far more
subtle; extending the lifetime of the first so that it still exists
after the second is created is, IMO, asking for trouble (and trying to
unify the two double so).
-Miles
Bruno Desthuilliers
> In <4a967b2f$0$19301$426a...@news.free.fr> Bruno Desthuilliers <bruno.42.de...@websiteburo.invalid> writes:
>
>> The only thing one is entitled to expect when learning a new language is
>> that the language's implementation follows the language specs.
>
> In fact, the official docs, when they discuss scopes, are off to
> a bad start:
>
> Names refer to objects. Names are introduced by name binding
> operations. Each occurrence of a name in the program text refers
> to the binding of that name established in the innermost function
> block containing the use.
>
> The first paragraph implies that binding can only occur within
> functions, which is either incorrect, or presupposes a definition
> of "function" that is not what most programmers would recognize.
Indeed, and you're right to point it. I strongly suggest you submit a
ticket to the doc maintainers.
> In general, I found the docs very unclear on the subject of scoping.
> PEP 227 is much better, but I wouldn't have thought of it as "the
> specs".
Well, here again, it sure would help to make clear that peps (and
release specific "what's new") *are* part of the doc. But for sure, what
used to be a simple and clear reference is now a bit of a mess, despite
the hard work of the doc team. Side-effect of the rapid growth of the
language I guess...
Ethan Furman
This is not the same thing, and definitely not exactly the same thing.
In your example you are explicitly stating whether you want the original
class variable, or the current, and possibly different, instance
variable. Further, the instance variable will not be different from the
class variable, even after C.x = whatever, unless the instance has had
that variable set specifically for it.
In [1]: class C(object):
...: x = 9
...: def doit(self):
...: print C.x
...: print self.x
...:
In [2]: test = C()
In [3]: test.doit()
9
9
In [4]: C.x = 10
In [5]: test.doit()
10
10
In [6]: test.x = 7
In [7]: test.doit()
10
7
~Ethan~
kj
Thanks for the clarification!
kynn