[RFC] Lambdas
ZyX
============================================================================================
lambda *expr-lambda*
------
\arg1, ...: expr1 lambda function
Lambda functions are the lightweight variant of |user-functions|. Differences
between them are:
1. Body of the lambda function is an |expr1| and not a sequence of |Ex|
commands.
2. Lambda function does not have its own scope dictionary. It does have scope
dictionary for |a:var| though. |l:| scope dictionary is inherited from the
place where lambda was defined (thus it is either |g:| scope dictionary or
|l:| dictionary of the function that defined the lambda).
You see, it is poor man closure.
3. If |a:var| is not found in lambda own scope dictionary it is searched in
|a:| scope dictionary of the function where lambda was defined (if any).
This does not apply to |a:000|, |a:0|, |a:1|, ... variables.
4. There are no dictionary lambdas.
5. Lambda is not globally recorded anywhere like |anonymous-function|.
NOTE: lambda function records the whole scope dictionary in its body and also
|a:| dictionary. That means that extensive generation of lambdas and
exporting them outside of generator function may lead to a huge memory
overhead.
============================================================================================
This is one of the easiest implementation. Commit with this implementation was pushed to https://bitbucket.org/ZyX_I/vim, branch extended-funcref-lambdas, but not to the mailing list (implementation is mostly untested and definitely experimental). I have a few other suggestion for lambdas:
0. Current state.
Pros:
1. Simplicity of implementation.
2. No allocations for l: variables for lambdas.
3. Has access to a: of the caller.
Cons:
1. Lambda disallows freeing funccal structure until lambda is deleted.
2. No access to a:N variables. Requires modification of find_var_in_ht to work. This likely may be worked around.
3. No local variables. Though if one is using expressions it is hard to manipulate with them.
1. Make lambdas have exactly one local dictionary allocated once at creation. Modify eval7 to copy all used variables to this dictionary on creation. Arguments still go to a: dictionary allocated on each lambda call.
Pros:
1. Does not hold funccal structure.
2. Only those variables that are actually used will be recorded.
3. It is a python-like closure.
4. One less hash allocation compared to calling user function.
Cons:
1. Requires likely not so trivial modifications of eval7 and also modification of other eval* to pass down the reference to l: dictionary.
2. One l: dictionary for all calls. Though if one is using … (see cons 3. from p. 0.).
3. No access to a: of the caller (or we will get cons 1. from p. 0.).
2. Like above, but arguments go to local dictionary. Dictionary itself is copied on lambda invocation.
Pros:
Same as above.
4. Copying dictionary should probably be faster then creating and populating a new one. Needs some profiling.
Cons:
Same as above.
3. Like the current state (0.), but also has its own l: dictionary.
Pros:
Same as 0., except for 2.
4. Own scope where you can place something.
5. Adds a facility to create user functions that are real closures (likely, but not necessary, limited to 2 nesting level; no limit on nesting is a bit harder to implement, but it seems far from hard).
Cons:
Same as 0., except for 3.
// Note about the doc excerpt: anonymous functions are not recorded anywhere *in extended-funcref branch*.
Bram Moolenaar
ZyX wrote:
> Since request for comments in thread with extended-funcref branch
> patches has gone unnoticed I will repost it here. I currently have
> somewhat working implementation of lambda functions in this branch and
> want to discuss where to move on since any implementation I can
> imagine has its downsides. �Somewhat working� implementation I
> have was done in one of the easiest ways possible. Here is the doc I
> have written for new lambdas:
Especially before/after comparisons.
--
ARTHUR: You fight with the strength of many men, Sir knight.
I am Arthur, King of the Britons. [pause]
I seek the finest and the bravest knights in the land to join me
in my Court of Camelot. [pause]
You have proved yourself worthy; will you join me? [pause]
You make me sad. So be it. Come, Patsy.
BLACK KNIGHT: None shall pass.
The Quest for the Holy Grail (Monty Python)
/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
LCD 47
>
> ZyX wrote:
>
> > Since request for comments in thread with extended-funcref branch
> > patches has gone unnoticed I will repost it here. I currently have
> > somewhat working implementation of lambda functions in this branch
> > and want to discuss where to move on since any implementation I can
> > written for new lambdas:
>
> How about a few examples that show how awesome this feature is?
> Especially before/after comparisons.
Reducing call overhead? Making complicated key maps less awkward?
Something else? I understand what Lambda functions are in Python (which
seems to be your inspiration for this design), but I can't really
translate the common use cases of Python lambda to Vim. So, what
problem does this solve? What would be the main use cases in Vim?
/lcd
Nikolay Pavlov
1. Partial function application.
2. Sorting with sort().
In frawor I constantly use generated functions that look like
function d.F(...)
return call(s:features['plugid'].name.func, [s:plugdicts['userplugid'], s:shadowdicts['userplugid']]+a:000, {})
endfunction
'plugid', .name, .func, 'userplugid' are all generated. This can be reduced to just "\...: call(feature[name].func, [plugdict, shadowdict]+a:000, {})". Still needs eval, unless p 1. is used, but already more lightweight.
Third use is completely hiding the variables: shadowdict in the example can be generated in the function that exports features and not ever recorded in any global.
Based on this uses I should stick with 1. If partial application is dropped or be considered to be allowed to be suboptimal sticking with 0. is an option.
By the way, the first person I saw requesting lambdas is Bram in discussion to sort() proposed patch that will make sort() accept expressions. Thus the main reasoning was "sort() and partial application are good, but not enough. As it is relatively simple to add lambdas to my patch let us add them and see whether they will make Bram accept patch faster." Patch is incomplete without making map() and filter() accept funcrefs, but this is really trivial. RFC is here because maybe Bram sees more uses for lambdas then me. Quote:
> Using strings here was just a quick way of making it work. What we
> would really need is some kind of lambda function. So the argument
> would be one of three types:
> expression - mainly for backwards compatibility, but also allows
> building the functionality from pieces.
> function name or reference - for bigger functions
> lamba function - for small functions
>
> That could be used in several places.
>
> Implementing a lambda function is not trivial though.
> /lcd
Tony Mechelynck
an expression in string form, so that it can later be :execute'd, as is
already done for several options: ":help expr'<Ctrl-D>" shows me ten of
those (not necessarily all different).
Best regards,
Tony.
--
"It is always tempting to divide men into two lots: Greeks and barbarians,
Muslims and infidels, those who believe in God and those who don't.
But who
does not fear to understand things that threaten his beliefs? Of course,
one is not consciously afraid; but everybody who is honest with himself
finds that often he does not try very hard to understand what clashes
with his deep convictions."
[Walter Kaufmann, "The Faith of a Heretic"]
Nikolay Pavlov
Expressions are invalid for :execute. I do not understand how &*expr options are related.
Note that no string expressions take variable values with them. I am not much fond of the workaround where expression accesses them from some global variable. It is bad for GC: generally you cannot know when expression and attached variables are no longer needed. Globals are always bad for threading, if this will ever go into vim. It adds a bunch of unneeded code: generate global name, record value there, put name into the expression.
>
> Best regards,
> Tony.
> --
> "It is always tempting to divide men into two lots: Greeks and barbarians,
> Muslims and infidels, those who believe in God and those who don't. But who
> does not fear to understand things that threaten his beliefs? Of course,
> one is not consciously afraid; but everybody who is honest with himself
> finds that often he does not try very hard to understand what clashes
> with his deep convictions."
> [Walter Kaufmann, "The Faith of a Heretic"]
>
>
> --
> --
> You received this message from the "vim_dev" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> --- You received this message because you are subscribed to the Google Groups "vim_dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
Tony Mechelynck
>
> On Oct 19, 2013 10:05 AM, "Tony Mechelynck"
>
> Note that no string expressions take variable values with them. I am not
> much fond of the workaround where expression accesses them from some
> global variable.
using an argument with a predetermined name; then
:let lambda_arg = 123 * 456
:exe "let result =" lambda
:unlet lambda_arg " unless it has a persistent meaning
" one reason to omit the :unlet would be if you want to use
" the expression several times (possibly from different places)
" with the same argument (or one that changes less often than
" the expression is used); or else if the expression
" modifies its own argument (but the latter would be
" more appropriate for a full-fledged user function).
or for a number of arguments, the "argument" variable could be a List or
a Dictionary instead.
> It is bad for GC: generally you cannot know when
> expression and attached variables are no longer needed. Globals are
> always bad for threading, if this will ever go into vim. It adds a bunch
> of unneeded code: generate global name, record value there, put name
> into the expression.
be regarded as a kind of "expression" with a peculiar syntax; but of
course it needs neither unletting nor garbage collection.
Best regards,
Tony.
--
<joost> Do you mean to say that I can read mail with vi too? ;-)
<Joey> Didn't you know that?
<Joey> :r /var/spool/mail/jk
-- Seen on #debian-mentors
ZyX
> > to an expression in string form, so that it can later be :execute'd, as
> > is already done for several options: ":help expr'<Ctrl-D>" shows me ten
> > of those (not necessarily all different).
> >
> > Expressions are invalid for :execute. I do not understand how &*expr
> > options are related.
>
> :execute "let x =" lambda
:execute expr
for an expression like `tr(s, "abc", "def")` is not valid. You execute here not an expression, but an `:let` assignment containing an expression. Options do not use `:execute` or anything `:execute` uses directly, they use the same function `eval()` uses.
> Why not? I don't see this as a workaround: Write your expression as
> using an argument with a predetermined name; then
>
> :let lambda_arg = 123 * 456
> :exe "let result =" lambda
> :unlet lambda_arg " unless it has a persistent meaning
> " one reason to omit the :unlet would be if you want to use
> " the expression several times (possibly from different places)
> " with the same argument (or one that changes less often than
> " the expression is used); or else if the expression
> " modifies its own argument (but the latter would be
> " more appropriate for a full-fledged user function).
>
> or for a number of arguments, the "argument" variable could be a List or
> a Dictionary instead.
Also do not use `execute 'let a='.lambda`. There is `eval()` which has far less overhead and is more convenient to use in many cases.
> > It is bad for GC: generally you cannot know when
> > expression and attached variables are no longer needed. Globals are
> > always bad for threading, if this will ever go into vim. It adds a bunch
> > of unneeded code: generate global name, record value there, put name
> > into the expression.
>
> What is currently done for 'indentexpr' et al.? Even 'statusline' could
> be regarded as a kind of "expression" with a peculiar syntax; but of
> course it needs neither unletting nor garbage collection.
Ingo Karkat
> On Oct 15, 2013 2:26 AM, "LCD 47" <lcd...@gmail.com
they're written from someone with a Python or Ruby background and make
intensive use of Dictionaries and anonymous functions. But this is
rather atypical. The usual plugin is structured like a shell script,
with a few (named) functions, nothing more.
Even though I've written plenty of plugins, I only remember a few
occasions where a lambda would have been really handy, and the
workaround was an ugly script-scoped variable or explicit function. The
string evaluation in map() et al. certainly isn't beautiful, but it's
consistent with options like 'statusline'.
Now, especially with your recent work on extending the Python interface
towards a fully-featured Vim integration, I wonder whether it isn't
sufficient for those people who want to do fancy object-oriented and
functional stuff to implement that Python altogether?! Adding stuff like
Lists and Dictionaries in Vim 7 was a right step; many people were
already crudely emulating them. With lambdas, I'm not so sure,
especially because there isn't an abundance of Vim developers, and many
patches and features are already waiting for Bram's review and approval.
-- regards, ingo
Nikolay Pavlov
As I said earlier I do not need lambdas much. They are good for frawor, but I sometimes think about writing python version of frawor rather then improving its VimL variant.
I do find references to python callables handy though. Lambdas are an example of what one can relatively easy to do with a new branch which provides funcrefs to python functions. They also fulfill a request from Bram I quoted earlier. If I really was writing this for frawor then you would see some better designed code as a part of extended-funcref branch and no RFC.
> -- regards, ingo