Class body integrity needs to be preserved to support class decorators and metaclasses

[Alejandro Martinez]

I've started to look at class decorators (details in issue #64) and metaclasses (details in issue #65), in both cases but specially in the second, attributes assigned in the body of the class can receive different treatment than attributes assigned after class creation, the problem is in our lexical phase(s) such distinction is lost before class statement reaches desugar, for example:

     class C(Base1, Base2):

         a = expr

         b = expr

is desugared to something like:

    class C(Base1, Base2): pass

    C.a = expr

    C.b = expr

so the body expression in PyClass is not used and the integrity of the class body is lost.

I think this problem could be solved if the sequence of assignments to which the class body reduces to are placed inside the body field of PyClass instead of as separate assignments without a clear delimitation.

Two questions:

1) This seems reasonable or there are objection and/or better ideas?

2) How difficult is to do this change to the lexical phase(s)?

Alejandro.

[Matthew Milano]

1 - it's reasonable; I had been flattening classes because I was not
aware of the class decorators issue. I presume you mean that this is
fixed by just changing

class C(base1, base2): pass

C.a = expr
C.b = expr

to

class C(base1, base2):

C.a = expr
C.b = expr

2 - it'll be kinda hard to change phase2 for this. But I was supposed
to be re-writing that anyway, so that work should be done regardless
of the decorators bug.

~matthew

On 5/3/13, Alejandro Martinez <amtri...@gmail.com> wrote:
> I've started to look at class decorators (details in issue

> #64<https://github.com/brownplt/lambda-py/issues/64>) and
> metaclasses (details in issue
> #65<https://github.com/brownplt/lambda-py/issues/65>),

> --
> You received this message because you are subscribed to the Google Groups
> "lambda-py" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to lambda-py+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

[Alejandro Martinez]

Yes, that's what I mean, it would allow to implement class decorators and metaclasses, class creation would be something like this:

    a) create an empty class

    b) execute the body in the current scope

    c) if the metaclass is other than type, call it passing the name, bases, the class dict and the rest of the class statement arguments.

    d) if there is decorators, apply them

I think I could implement this just now since the body has no effect, but it will be really functional once the lexical phase populates the class body.

Do you agree?

Alejandro.

2013/5/3 Matthew Milano <mat...@cs.brown.edu>

--
Alejandro.

[Joe Gibbs Politz]

Related to the topic, but not the implementation:

Thinking about scope, it's actually an interesting point of design
that Python doesn't allow referencing class variables from method
bodies. Things like metaclasses and dynamic subclassing in Python
would make it next to impossible to tell which variables are free,
since later mutations of the class object might cause those variables
to simply not exist anymore. Consider:

def f(name, bases, dict):
dict.__delitem__("x")

x = "global x"
class C():
x = "class var"
__metaclass__ = f
def m(self):
return x


If the reference to x above was allowed, this would be a weird
situation for `x' indeed. Is it simply undefined? Does it drop
through to "global x"? Does it still manage to capture the "class
var" value? Python's class scope, while weird (as we've noticed),
does neatly sidestep these particular issues.

[Alejandro Martinez]

Interesting, with Python semantics x inside m refers to the global one without problems, two observations on this example:

1) the __metaclass__ attribute doesn't seem to be supported anymore in Python 3.2, it was replaced by the keyword argument metaclass, i.e. class C(metaclass=f): ...

2) the metaclass callable should return the newly created class, in this case probable return type(name, bases, dict)

Back to the original topic, I have good news: I think I found a very simple change to phase 2 to solve the issue: in https://github.com/brownplt/lambda-py/blob/master/base/python-phase2.rkt#L308 to keep the (deal-with-class new-body class-expr) inside the LexClass-body instead to put it in a sequence after the LexClass and to modify LexClass desugar accordingly to execute the body after the class object is created and assigned to the name in the corresponding scope. I need to do more testing, and some cleanup since currently there is a double assignment, but it seems to be working without problems and this test is passing:

def d2(cls):

    try:

        cls.dec_attr = cls.dec_attr + "d2"

    except:

        cls.dec_attr = "d2"

    return cls

# Attribute already exists in the class body

@d2

class C2body:

    dec_attr = "C"

___assertEqual(C2body.dec_attr, "Cd2")

2013/5/7 Joe Gibbs Politz <joe.p...@gmail.com>

--
Alejandro.

[Matthew Milano]

While that change is easy and solves your bug, it's unfortunately slightly orthogonal to the other bug fix/cleanup changes that need to be done. 

That being said, I don't know when I will have time to work on it, so it is definitely the right call right now. 

None of the things I want to change are causing active bugs anyway. 

~matthew

--

[Alejandro Martinez]

I did the minimum changes to phase2 for class decorators (and metaclasses going forward) to work, regression tests are all passing and if this changes are undone only class decorators (and metaclasses) should fail. They are in this commit.

When you have time to explain the changes you would like to make, the problems they solve and how to maintain the class body integrity with them, I would like to understand them.

2013/5/7 Matthew Milano <mat...@cs.brown.edu>

--
Alejandro.

[Alejandro Martinez]

FWIW, the partial implementation of metaclasses is passing this test, based on Joe's example:

def f(name, bases, dict):

  dict.__delitem__("x")

  return type(name, bases, dict)

x = "global x"

class C(metaclass=f):

  x = "class var"

  def m(self):

    return x

___assertFalse(hasattr(C, x))

___assertEqual(C().m(), "global x")