Why nested scope rules do not apply to inner Class?
Cousson, Benoit
I'd like to be able to use a nested class (C1) from another sibling nested class (C3). This looks very similar to the nested scopes of functions except that it does not work.
class A(object):
pass
class B(object):
class C1(object):
pass
class C2(C1):
foo = A
class C3(object):
foo = C1
The funny thing is that C2 can inherit from C1 but C3 cannot reference C1. B.C1 does not work either, but in that case it makes sense since B is still being defined.
Is this a language limitation or something that does not make sense at all?
I'm wondering as well if the new nonlocal statement will fix that in py3k?
Thanks in advance,
Benoit
Maric Michaud
> Hi,
>
> I'd like to be able to use a nested class (C1) from another sibling nested
> class (C3). This looks very similar to the nested scopes of functions
> except that it does not work.
>
> class A(object):
> pass
>
> class B(object):
>
> class C1(object):
> pass
>
> class C2(C1):
> foo = A
>
> class C3(object):
> foo = C1
>
> The funny thing is that C2 can inherit from C1 but C3 cannot reference C1.
> B.C1 does not work either, but in that case it makes sense since B is still
> being defined. Is this a language limitation or something that does not
> make sense at all?
>
This is a language limitation.
This is because nested scope is implemented for python function only since 2.3
allow late binding of free variables. the scope in class statment is not a
closure, so there is only two possible scope in it : local and global.
When "class C2(C1):" statment is interpreted, it is in the scope of class B
for which a name C1 exists, but it doesn't within the new scope of class C2
(the future C2.__dict__).
If you really need such a nested thing, this could help :
>>>[39]: class A(object) :
class B(object) : pass
C = type('C', (B,), {'ref' : B})
....:
>>>[29]:
>>>[30]: A
...[30]: <class '__main__.A'>
>>>[31]: A.B
...[31]: <class '__main__.B'>
>>>[32]: A.C
...[32]: <class '__main__.C'>
>>>[33]: A.C.__bases__
...[33]: (<class '__main__.B'>,)
>>>[34]: A.C.ref
...[34]: <class '__main__.B'>
> I'm wondering as well if the new nonlocal statement will fix that in py3k?
>
nonlocal doesn't adress this issue an I doubt python 3.0 fix it at all.
You have the same problem with generator expressions, which bind lately outer
loop variables :
>>>[70]: def f() :
l = list( a for a in range(5) )
return list( e + f for f in range(5) for e in l )
....:
>>>[73]: list(f())
...[73]: [0, 1, 2, 3, 4, 1, 2, 3, 4, 5, 2, 3, 4, 5, 6, 3, 4, 5, 6, 7, 4, 5, 6,
7, 8]
>>>[74]: class A(object) :
l = list( a for a in range(5) )
m = [ e + f for f in range(5) for e in l ]
....:
....:
>>>[77]: A.m
...[77]: [0, 1, 2, 3, 4, 1, 2, 3, 4, 5, 2, 3, 4, 5, 6, 3, 4, 5, 6, 7, 4, 5, 6,
7, 8]
>>>[78]: class A(object) :
l = list( a for a in range(5) )
m = list( e + f for f in range(5) for e in l )
....:
....:
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
/home/maric/<ipython console> in <module>()
/home/maric/<ipython console> in A()
/home/maric/<ipython console> in <genexpr>((f,))
NameError: global name 'l' is not defined
--
_____________
Maric Michaud
Salim Fadhley
The "class C3" statement is executing before the "class B" statement
has concluded, so at that time B does not exist in any scope at all,
not even globals(). You could reference B.C1 inside a method because a
method is executed AFTER the class is defined.
class A(object):
pass
class B(object):
class C1(object):
pass
class C2(C1):
foo = A
class C3(object):
@staticmethod
def test( ):
print repr( B.C1 )
print repr( B.C2 )
B.C3.test()
:-)
Calvin Spealman
What you are seeing is really an artifact of how classes are built.
Basically, everything inside the class body has to exist before it can
run, so the inner classes code objects are actually created first.
However, the class object itself isnt created out of that until the
body of the outer class is evaluated.
On Tue, Aug 12, 2008 at 5:29 AM, Cousson, Benoit <b-co...@ti.com> wrote:
> Hi,
>
> I'd like to be able to use a nested class (C1) from another sibling nested class (C3). This looks very similar to the nested scopes of functions except that it does not work.
>
> class A(object):
> pass
>
> class B(object):
>
> class C1(object):
> pass
>
> class C2(C1):
> foo = A
>
> class C3(object):
> foo = C1
>
> The funny thing is that C2 can inherit from C1 but C3 cannot reference C1. B.C1 does not work either, but in that case it makes sense since B is still being defined.
> Is this a language limitation or something that does not make sense at all?
>
> I'm wondering as well if the new nonlocal statement will fix that in py3k?
>
> Thanks in advance,
> Benoit
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/
Cousson, Benoit
> This is because nested scope is implemented for python function only since
> 2.3
> allow late binding of free variables. the scope in class statment is not a
> closure, so there is only two possible scope in it : local and global.
That was my understanding as well, but I think it is a pity to have that limitation. Don't you think that the same improvement that was done for method nested scope could be done as well for nested class?
I can easily fix my current issue by doing the binding after the class declaration.
My concern is more about the lack of symmetry of that approach; meaning that if both classes are in the global scope, one can access the others, whereas if they are in the body of another class they cannot.
This is OK:
class A(object):
pass
class B(object):
foo=A
I have to add the binding after the declaration in the case of nested:
class C(object):
class A(object):
pass
class B(object):
foo=None
B.foo = A
That extra step is a little bit painful and should not be necessary for my point of view.
> > I'm wondering as well if the new nonlocal statement will fix that in
> py3k?
> >
>
> nonlocal doesn't adress this issue an I doubt python 3.0 fix it at all.
>
> You have the same problem with generator expressions, which bind lately
> outer
> loop variables :
Good to know, I was not aware of that.
Thanks,
Benoit
Calvin Spealman
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy
Cousson, Benoit
> of code, but IMHO makes the result just a bit more cluttered, while
> reducing the elegance of reusing the metaclass.
The whole point of nested class is to avoid polluting the namespace with classes that are only used locally. So the argument about the elegance of reusing is not very valid in that case.
I agree that they are other ways using module to avoid namespace pollution, but in some case it is easier to use nested class instead and keep everything in the same file.
In my case, I'm trying to use a similar approach as XIST's one, meaning using Python class to model hierarchical data. So clearly nested class is a very nice and easy understandable way to do that.
> Here are only a few examples of threads giving good reasons against
> class nesting. I've never seen any good arguments for it. There are
> dozens of good reasons we don't encourage it and won't actively
> support it.
>
> http://mail.python.org/pipermail/python-dev/2005-March/052454.html
> http://mail.python.org/pipermail/python-dev/2002-November/029872.html
>From what I quickly read in these email threads, except for the pickling issue and the fact that Guido does not like nested classes, I do not see strong argument against it either.
Regards,
Benoit
Calvin Spealman
>> Defining it as a nested class saves you one line
>> of code, but IMHO makes the result just a bit more cluttered, while
>> reducing the elegance of reusing the metaclass.
>
> The whole point of nested class is to avoid polluting the namespace with classes that are only used locally. So the argument about the elegance of reusing is not very valid in that case.
There is no point of nested classes because nested classes _are not_
supported by python. They are simply an artifact of not actively
denying the syntax non-globally. I would fully support a change to the
language to actively forbid a class definition that is not
module-level.
> I agree that they are other ways using module to avoid namespace pollution, but in some case it is easier to use nested class instead and keep everything in the same file.
I don't consider it pollution. If you want it considered "private" to
the module, name it with an underscore.
> In my case, I'm trying to use a similar approach as XIST's one, meaning using Python class to model hierarchical data. So clearly nested class is a very nice and easy understandable way to do that.
I don't find that this is clear in anyway. I can't imagine why you'd
think a nested class is even useful here, rather than an instance with
some understandable attributes. I've seen a lot of places nested
classes are used and not one of them that should be been doing it.
But, on that note, there is a point where a discussion is obviously
not going to resolve with either side changing their minds. This is
obviously such a case.
Cousson, Benoit
> There is no point of nested classes because nested classes _are not_
> supported by python. They are simply an artifact of not actively
> denying the syntax non-globally. I would fully support a change to the
> language to actively forbid a class definition that is not
> module-level.
>
> using Python class to model hierarchical data. So clearly nested class is
> a very nice and easy understandable way to do that.
>
> I don't find that this is clear in anyway. I can't imagine why you'd
> think a nested class is even useful here, rather than an instance with
> some understandable attributes. I've seen a lot of places nested
> classes are used and not one of them that should be been doing it.
>
> But, on that note, there is a point where a discussion is obviously
> not going to resolve with either side changing their minds. This is
> obviously such a case.
I don't think so; my original email was mainly a question. I do agree that they are other ways to do what I'm trying to achieve; there are always several ways to solve an issue.
Few days ago, I decided to use nested class because I realized that it was the most convenient way to implement my need.
Since this feature is supported in many languages, I was just surprised that Python did support it only partially, hence my original email.
Now if you say; it is not supported, don't do that, we will deprecate that feature, fine, I will use an alternative solution.
I was just not aware of that "nested class is evil" group in the Python community. I still not understand why, but if it is the BDFL decision...
Regards,
Benoit
Rhamphoryncus
It has nothing to do with nested classes, but rather methods trying to
access other methods:
class X(object):
foo = 42
def bar(self):
print foo
print self.foo
X.foo = 7
When the class is created it *copies* the scope it used, meaning it's
left intact. "print foo" would print 42, rather than the correct and
desired 7 you get from "print self.foo". It is this subtle bug, as
well as "There should be one—and preferably only one—obvious way to do
it", that leads to prohibiting nested access to a class's scope.
Maric Michaud
Le Tuesday 12 August 2008 23:15:23 Calvin Spealman, vous avez écrit :
> The issue has been brought up several times before. There have been
> proposals to make nested classes better supported, but they are always
> shut down. The vote is always against it. Your metaclass example is
> one thing, but its not really applicable here. I've never seen or
> would encourage nested classes even in that case. The metaclass is
> typically defined on its own, and then referenced by one or more
> classes that use it. Defining it as a nested class saves you one line
> of code, but IMHO makes the result just a bit more cluttered, while
> reducing the elegance of reusing the metaclass.
>
> class nesting. I've never seen any good arguments for it. There are
> dozens of good reasons we don't encourage it and won't actively
> support it.
>
> http://mail.python.org/pipermail/python-dev/2005-March/052454.html
> http://mail.python.org/pipermail/python-dev/2002-November/029872.html
>
I was not aware of any "nested classes are unsupported" before and didn't
consider nested classes as bad practice till now, even with the pickle
limitation (not every class are intended to be pickled), more you didn't give
any evidence or any pertinent quote of this and at least one of Guido in the
above threads seems contradict you :
http://mail.python.org/pipermail/python-dev/2002-November/029906.html
BTW my concern was not really about nested classes but the asymetry between
function scope and classes scope, there are at least two examples that come
to my mind and for which I don't see good argument to keep the situation as
it is. First is the very odd behavior of genexp, second is the dynamic type
factory, I know it's not a big deal to in each case and can live with this,
but these pieces of code fail unexpectedly while even a experimented python
programmer would have thought they work at first glance, and the explanation
of why they fail to a newcomer is not easy (specially for the first)
(1) this fail only on the last line while it would have work well with
listcomp and works well in a function definition
class A(object) :
l = range(5)
m = list(a+b for a in l for b in range(5))
n = list(a+b for a in range(5) for b in l)
(2) This won't work as it would with nested functions, you need to build the
new calss directly with type('dynamic', (object,), {"type_": type_})
def create_type(type_) :
class dynamic(object) :
type_ = type_
return dynamic
--
_____________
Maric Michaud
Calvin Spealman
> I was not aware of any "nested classes are unsupported" before and didn't
> consider nested classes as bad practice till now, even with the pickle
> limitation (not every class are intended to be pickled), more you didn't give
> any evidence or any pertinent quote of this and at least one of Guido in the
> above threads seems contradict you :
> http://mail.python.org/pipermail/python-dev/2002-November/029906.html
Of course, I don't have to agree with Guido, but I don't really think
his comment there counts as some encouragement of the practice. It
also isn't an example that actually demonstrates the things people
misuse about nesting classes, although its definitely an example that
I think could be done much better without them, at all.
The use of nested classes to define some hierarchical structure is
subject, at best. And, the examples I see aren't even hierarchical.
The nesting doesn't seem to gain them anything. As a matter of fact,
wouldn't that example be a lot more clear with one class and some
"Attribute" properties describing the structure?
What rubs me the wrong way about nested classes is that they aren't
even "real" classes. That is, they aren't being used to instantiate
instances, but just as dummy scopes to hold some attributes. They are
abusing classes as property bags.
> BTW my concern was not really about nested classes but the asymetry between
> function scope and classes scope, there are at least two examples that come
> to my mind and for which I don't see good argument to keep the situation as
> it is. First is the very odd behavior of genexp, second is the dynamic type
> factory, I know it's not a big deal to in each case and can live with this,
> but these pieces of code fail unexpectedly while even a experimented python
> programmer would have thought they work at first glance, and the explanation
> of why they fail to a newcomer is not easy (specially for the first)
>
> (1) this fail only on the last line while it would have work well with
> listcomp and works well in a function definition
> class A(object) :
> l = range(5)
> m = list(a+b for a in l for b in range(5))
> n = list(a+b for a in range(5) for b in l)
I can admit that this behavior can be surprising, but it isn't
something I really see cropping up as a problem. How often do you see
that you need to have any kind of procedural behavior in a class body?
In other words, in all but rare cases, the order of the class body
lines shouldn't change the resulting class, but this example breaks
that expectation.
> (2) This won't work as it would with nested functions, you need to build the
> new calss directly with type('dynamic', (object,), {"type_": type_})
>
> def create_type(type_) :
> class dynamic(object) :
> type_ = type_
> return dynamic
This is simply completely invalid. That code at the beginning of any
body will _always_ raise either an UnboundLocalError or a NameError,
because you are assigning to the same name you are dereferencing. That
line will always be either an error or a no-op, anywhere it appears.
It has nothing to do with nesting of any sort.
>
> --
> _____________
>
> Maric Michaud
> --
> http://mail.python.org/mailman/listinfo/python-list
Maric Michaud
> class X(object):
> foo = 42
>
> def bar(self):
> print foo
> print self.foo
> X.foo = 7
Yes this is a problem, function could not bind their free vars to the class
namespace without introducing a subtle incompatibility with actual code.
This is exactly this case which would be a problem (your example raise an
error and is not exactly a "running code"):
G = 1
class A(object) :
G = 0
def f(self) : return G
--
_____________
Maric Michaud
Maric Michaud
> On Wed, Aug 13, 2008 at 7:41 PM, Maric Michaud <ma...@aristote.info> wrote:
>> I was not aware of any "nested classes are unsupported" before and didn't
>> consider nested classes as bad practice till now, even with the pickle
>> limitation (not every class are intended to be pickled), more you didn't give
>> any evidence or any pertinent quote of this and at least one of Guido in the
>> above threads seems contradict you :
>> http://mail.python.org/pipermail/python-dev/2002-November/029906.html
>
> Of course, I don't have to agree with Guido,
neither do I, for sure.
> but I don't really think
> his comment there counts as some encouragement of the practice. It
> also isn't an example that actually demonstrates the things people
> misuse about nesting classes, although its definitely an example that
> I think could be done much better without them, at all.
>
Yes this example is some sort of metalanguage using class statment in a
derived way, It comes in mind sometimes by the fact there is no other
way to extend the language to make DSL in python (perhaps easy extend
will be popular one day, but I doubt). I did something like that before
in an application and won't encourage anyone to do so.
> The use of nested classes to define some hierarchical structure is
> subject, at best. And, the examples I see aren't even hierarchical.
> The nesting doesn't seem to gain them anything. As a matter of fact,
> wouldn't that example be a lot more clear with one class and some
> "Attribute" properties describing the structure?
>
> What rubs me the wrong way about nested classes is that they aren't
> even "real" classes. That is, they aren't being used to instantiate
> instances, but just as dummy scopes to hold some attributes. They are
> abusing classes as property bags.
>
Here, I disagree, the helper classes example is good IMO, defining an
Iterator as an inner class and referencing it by MyIterable.Iterator is
good style.
That said, I understood from your first post that python community in
general will discourage nested classes as bad practice and consider them
not supported by the language, statment for which I can't find or
remember no evidence.
>> BTW my concern was not really about nested classes but the asymetry between
>> function scope and classes scope, there are at least two examples that come
>> to my mind and for which I don't see good argument to keep the situation as
>> it is. First is the very odd behavior of genexp, second is the dynamic type
>> factory, I know it's not a big deal to in each case and can live with this,
>> but these pieces of code fail unexpectedly while even a experimented python
>> programmer would have thought they work at first glance, and the explanation
>> of why they fail to a newcomer is not easy (specially for the first)
>>
>> (1) this fail only on the last line while it would have work well with
>> listcomp and works well in a function definition
>> class A(object) :
>> l = range(5)
>> m = list(a+b for a in l for b in range(5))
>> n = list(a+b for a in range(5) for b in l)
>
> I can admit that this behavior can be surprising, but it isn't
> something I really see cropping up as a problem. How often do you see
> that you need to have any kind of procedural behavior in a class body?
> In other words, in all but rare cases, the order of the class body
> lines shouldn't change the resulting class, but this example breaks
> that expectation.
>
I wrote something like that a few months ago, and found it both simple
concise and elegant :
class test_suite1(test_suite) :
roles = (role1, ...)
actors = (actor1, ...)
use_cases = [ (actor, role) for actor in actors for role in roles ]
which I wrote at first tuple((actor, role) for ...) and came to the problem.
Of course use_cases can be made a true generator, but in this particular
case building the tuple/list together with the class is good enough.
>> (2) This won't work as it would with nested functions, you need to build the
>> new calss directly with type('dynamic', (object,), {"type_": type_})
>>
>> def create_type(type_) :
>> class dynamic(object) :
>> type_ = type_
>> return dynamic
>
> This is simply completely invalid. That code at the beginning of any
> body will _always_ raise either an UnboundLocalError or a NameError,
> because you are assigning to the same name you are dereferencing. That
> line will always be either an error or a no-op, anywhere it appears.
> It has nothing to do with nesting of any sort.
>
Yes, of course, sorry, it should be :
def create_type(t) :
class dynamic(object) :
type_ = t
return dynamic
but this work to my surprise ! I just thought the "t" in the class body
would be looked up in global namespace. The design is that "free vars
doesn't bind to class namespaces" not that "free vars of class
namespaces bind to global namespace", my mistake.
Carl Banks
> On Wed, Aug 13, 2008 at 11:32 AM, Cousson, Benoit <b-cous...@ti.com> wrote:
> >> Defining it as a nested class saves you one line
> >> of code, but IMHO makes the result just a bit more cluttered, while
> >> reducing the elegance of reusing the metaclass.
>
> > The whole point of nested class is to avoid polluting the namespace with classes that are only used locally. So the argument about the elegance of reusing is not very valid in that case.
>
> There is no point of nested classes because nested classes _are not_
> supported by python. They are simply an artifact of not actively
> denying the syntax non-globally. I would fully support a change to the
> language to actively forbid a class definition that is not
> module-level.
I think that's taking it a little too far. It's not unreasonable to
throw small, private use classes into the class definition, like so:
class Someting(object):
class PrivateException(Exception):
pass
And inside function a class definition can make a lot of sense.
Oftentimes I write a quick adaptor class because I want to pass
something to code that expects something else, for instance:
def some_function(lines):
class FileMimicker(object):
def __init__(self):
self.index = 0
def readline(self):
line = lines[self.index]
self.index += 1
return line
function_that_calls_readline(FileMimicker())
(Why would I want to clutter up my module's namespace for that silly
thing?)
So I see no good reason for the compiler to disallow nested class
statements; it's occasionally useful and not a common pitfall.
Carl Banks
Calvin Spealman
I know every rule has its exceptions. I put "don't nest classes" in
with other similar rules I will claim, where I think its safest to say
"Never do this!", because only then will you know that, should you
actually do it at some point, you've got a reason good enough to break
a "rule".
As for the cluttering of the namespace, I don't buy it. Sure there are
cases to be made to reduce the number of names in any scope, but I
don't see a big advantage here. Its still accessible from the module,
just though the class. I also don't see any examples of nested classes
that wouldn't be useful elsewhere, your example included. For that
matter, your example should just being using StringIO.
Steven D'Aprano
> I know every rule has its exceptions. I put "don't nest classes" in with
> other similar rules I will claim, where I think its safest to say "Never
> do this!", because only then will you know that, should you actually do
> it at some point, you've got a reason good enough to break a "rule".
I would put it like this:
"Never nest classes, unless you need to, or to win a bet."
I've never yet needed to nest a class inside a class, but I've used a
"class factory function", a function that returned classes.
--
Steven
Bruno Desthuilliers
> On Wed, Aug 13, 2008 at 11:32 AM, Cousson, Benoit <b-co...@ti.com> wrote:
(snip)
> There is no point of nested classes because nested classes _are not_
> supported by python.
Depending on the definition of "supported".
(snip)
>> In my case, I'm trying to use a similar approach as XIST's one,
>> meaning using Python class to model hierarchical data. So clearly nested
>> class is a very nice and easy understandable way to do that.
>
> I don't find that this is clear in anyway. I can't imagine why you'd
> think a nested class is even useful here, rather than an instance with
> some understandable attributes.
I onced played with the same idea, ie using nested classes as a
declarative approach to XML/HTML generation. It didn't quite work - at
least not well enough to be worth spending more time on the topic - but
at least I can imagine why the OP thinks "true" nested classes would
have help here.
Bruno Desthuilliers
(snip)
> I know every rule has its exceptions. I put "don't nest classes" in
> with other similar rules I will claim, where I think its safest to say
> "Never do this!",
I know (from experience) and understand why goto's and globals are
(usually) evil, I know why public attributes can be harmful - in
languages that don't support computed attributes, that is -, well, I can
understand and explain most of the usual and some less known "golden
rules" (law of demeter etc), and even a couple very local or
yet-unwritten ones (monkeypatch anyone ?), but I definitively fail to
see what's your problem with nested classes.
Bruno Desthuilliers
(snip)
> Now if you say; it is not supported, don't do that, we will deprecate
> that feature, fine, I will use an alternative solution.
>
> I was just not aware of that "nested class is evil" group in the
> Python community.
Probably because this group is mostly composed of Mr Calvin Spealman ?
I've been lurking here for quite a few years now, and it's the very
first time I read anything about nested classes being evil....