Proposal to deprecate PyClassFunc and to add decorators to PyFunc

[Alejandro Martinez]

The syntax:

@classmethod

def cm(cls, ...):

    ...

is just syntactic sugar for

def cm(cls, ...):

    ...

cm = classmethod(cm)

and, more generally, this applies to every function decorator.

I've just implemented classmethod as a class and modified PyClassFunc desugar to this scheme in multiple_inheritance branch, this and method class were necessary for super() and multiple inheritance support, but I think there is no need to have PyClassFunc in the AST and it should be replaced by a list of decorators in PyFunc, which are just interpreted as HOF applied to the function before assign it to its name.

If there is no objection I will implement this change in desugar, it will enable @staticmethod.decorator, which is also implemented.

[Alejandro Martinez]

When looking at this I've found two other problems related with functions: limited support for default arguments(issue #18) and lack of support for function attributes (issue #38), perhaps we should try to solve all of them at once, the idea would be:

1) To have only one form of PyFun with vararg, defaults and decorators.

2) Decorators would be desugared as proposed above

3) Functions could be represented as VObjects with a new MetaClosure replacing the original VClosure and class "function", so they can have attributes in general and, in particular, __defaults__ should be a tuple of default arguments, __name__ the function name, etc.

Any thoughts or better ideas ?

[Alejandro Martinez]

This is feature is ready: we have support for function decorators in general, PyClassFunc is not needed anymore since classmethod class can be used as decorator, also staticmethod is available.

There is also a test case using a decorator to convert the naive recursive fib definition to O(n) using @memo decorator.

This is the commit: https://github.com/brownplt/lambda-py/commit/a4a0db48229eecc96b7d2b08dd17a2b4ec9bdb16

El miércoles, 23 de enero de 2013 21:24:16 UTC-3, Alejandro Martinez escribió:

[Alejandro Martinez]

Now we have decorators and functions-as-objects I think it would be interesting to solve defaults handling limitations, the changes I propose are as follows:

1) Consolidate [Py|Lex]Func and [Py|Lex]FuncVarArg into [Py|Lex]Func which will have both [defaults : (listof [Py|Lex]Expr)] and [vararg : (optionof [Py|Lex]Expr)]

2) Desugar LexFunc in the same way LexFuncVarArg is being desugared until now but, add two attributes:

  - __defaults__: a tuple built with the results of the evaluation of defaults expressions

  - __name__: a string with the function name (for completeness)

3) Extend [Py|Lex]Lam to support both vararg and defaults in the same way, except __name__ = '(Lambda)'

4) Change py-app to use __defaults__:

  - if there is no stararg and there are sufficient positional arguments, no changes WRT existent desugar (I think it

    convenient to simplify the "normal" case for performance and bootstrapping reasons) 

  - else, let starvals be the result of tuple(stararags) or tuple() if no starargs present, appended with elements from

    the tail of function.__defaults__ up to match the positional parameters of the function and desugar as usual.

IMHO this is worth doing since it extends the supported functionality while it simplifies the current desugar and, in principle, it doesn't require modifications to the core/interpreter.

To add support for keyword arguments, **kwarg and **expr is another kind of beast, I'm afraid.

[Joe Gibbs Politz]

On Saturday, April 6, 2013 6:16:33 PM UTC-4, Alejandro Martinez wrote:

Now we have decorators and functions-as-objects I think it would be interesting to solve defaults handling limitations, the changes I propose are as follows:

1) Consolidate [Py|Lex]Func and [Py|Lex]FuncVarArg into [Py|Lex]Func which will have both [defaults : (listof [Py|Lex]Expr)] and [vararg : (optionof [Py|Lex]Expr)]

2) Desugar LexFunc in the same way LexFuncVarArg is being desugared until now but, add two attributes:

  - __defaults__: a tuple built with the results of the evaluation of defaults expressions

  - __name__: a string with the function name (for completeness)

3) Extend [Py|Lex]Lam to support both vararg and defaults in the same way, except __name__ = '(Lambda)'

4) Change py-app to use __defaults__:

  - if there is no stararg and there are sufficient positional arguments, no changes WRT existent desugar (I think it

    convenient to simplify the "normal" case for performance and bootstrapping reasons) 

  - else, let starvals be the result of tuple(stararags) or tuple() if no starargs present, appended with elements from

    the tail of function.__defaults__ up to match the positional parameters of the function and desugar as usual.

IMHO this is worth doing since it extends the supported functionality while it simplifies the current desugar and, in principle, it doesn't require modifications to the core/interpreter.

This sounds great, but see my question below.  If we do run with it, we should see if we can simplify the Py/Lex data structures first in case Matthew is going to go off and re-write code that uses those structures.

 

To add support for keyword arguments, **kwarg and **expr is another kind of beast, I'm afraid.

Agreed that this is trickier but also worthwhile.  At a high level, do we think there's any chance of handling these purely in desugaring, or are they deeply embedded in how functions work?  It might be worth spending a little time thinking about them if they'll significantly change the way application works, so we can settle on a model of functions once and for all.  I haven't played with these as much as I'd like, does anyone have a good sense of what's involved here?

I could see us, for example, desugaring all user-defined Python functions to have the first two arguments are always be the positional tuple and keyword dictionary, and desugaring to wrap the body in the appropriate let-bindings for any named or default arguments in the original function signature.  That would be a big change to how we do desugaring, but (I think) not require changing the core.

[Alejandro Martinez]

2013/4/7 Joe Gibbs Politz <joe.p...@gmail.com>

If we do run with it, we should see if we can simplify the Py/Lex data structures first in case Matthew is going to go off and re-write code that uses those structures.

Agree, perhaps we could go further than I proposed: to unify [Py|Lex]App and [Py|Lex]AppStarArg and to add the remaining fields necessary to support keywords/kwarg/etc. even when initially they be ignored at desugar, IOW try to match the AST data structures to avoid further modifications.

 

To add support for keyword arguments, **kwarg and **expr is another kind of beast, I'm afraid.

Agreed that this is trickier but also worthwhile.  At a high level, do we think there's any chance of handling these purely in desugaring, or are they deeply embedded in how functions work?  It might be worth spending a little time thinking about them if they'll significantly change the way application works, so we can settle on a model of functions once and for all.  I haven't played with these as much as I'd like, does anyone have a good sense of what's involved here?

I could see us, for example, desugaring all user-defined Python functions to have the first two arguments are always be the positional tuple and keyword dictionary, and desugaring to wrap the body in the appropriate let-bindings for any named or default arguments in the original function signature.  That would be a big change to how we do desugaring, but (I think) not require changing the core.

I think it would be not too difficult to extend the proposed scheme to handle keyword arguments by desugaring, it would be an additional "massaging" of the starargs tuple, even *kwarg seems possible but, the major problem seems to be "mapping expansion" feature in calls (**expr), since it allows the dynamic building of keyword/value pairs and, it probably will require some kind of additional core/interpreter support.

--
Alejandro.

[Alejandro Martinez]

I was thinking a little more about **expression and now I'm pretty sure it can be handled in desugaring using the same proposed scheme, which is to convert all arguments to positional arguments: we can build a tuple of strings with the name of the parameters to match the dynamic keyword arguments in core code.

BTW, this process is similar to how call semantics is explained in the reference manual: http://docs.python.org/3.2/reference/expressions.html#calls

I've started the cleanup of defaults with this commit: Removed [Py|Lex]FuncVarArg and extended [Py|Lex]Func with [vararg : (optionof symbol)] parameter, it is passing all the regression tests but, the lack of typechecking is really a bummer for this kind of changes.

Next steps would be to complete defaults handling, to extend lambda and to start with keywords.

2013/4/7 Alejandro Martinez <amtri...@gmail.com>

I think it would be not too difficult to extend the proposed scheme to handle keyword arguments by desugaring, it would be an additional "massaging" of the starargs tuple, even *kwarg seems possible but, the major problem seems to be "mapping expansion" feature in calls (**expr), since it allows the dynamic building of keyword/value pairs and, it probably will require some kind of additional core/interpreter support.

--
Alejandro.

[Alejandro Martinez]

FWIW here is the implementation of defaults handling using the proposed scheme: Generalized default argument handling

Extension to lambda is straightforward and I think keywords handling is somewhat more clear now.

2013/4/9 Alejandro Martinez <amtri...@gmail.com>

--
Alejandro.

[Alejandro Martinez]

Calls with keywords and **expression are finished, there remains to handle **kwarg and keyword-only parameters in FunctionDef and Lambda.

2013/4/10 Alejandro Martinez <amtri...@gmail.com>

--
Alejandro.

[Joe Gibbs Politz]

Awesome!

> --
> You received this message because you are subscribed to the Google Groups
> "lambda-py" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to lambda-py+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Alejandro Martinez]

Well, keyword-only parameters and **kwarg are supported now, so function call is complete now.

Actually there is a limitation to use keyword arguments in class initializers, in theory the solution would be simple: add **kwarg parameter to type.__call__ and use them when to call __new__ and __init__, the problem is this same mechanism is used to create builtin classes, such as tuple and list, which are used to desugar **kwarg calling sequence... 

Probably we should special case the creation of base builtin classes so type.__call__ could be generalized to support keywords as above.

For example the property class use keywords in __init__ as can be seen in this test: https://github.com/brownplt/lambda-py/blob/master/tests/python-reference/property/test_property_getter_doc_override.py

This example also uses docstring but, this seems to be simpler to implement at desugar.

2013/4/14 Joe Gibbs Politz <joe.p...@gmail.com>

--
Alejandro.