[vim/vim] Loss of scope data when using nested partials/funcref (#812)

[Andrey Gavrikov]

Now when we have job_start there are many more cases when nested callbacks may be required
Presently it seems that when there is a chain of partials (one called from another) the scope of all partials (except the last one) gets lost.

Steps to reproduce:

• Given the file demo.vim
  

function! Callback_Test()
    let obj1 = {"test": 'test1', "other_test": "other_test"}
    function obj1.func1() 
        " next line will use value from wrong scope
        echomsg "inside obj1.func1: test=".self.test 
        " next line will result in: 'Key not present in Dictionary: other_test'
        echomsg "inside obj1.func1: other_test=".self.other_test 
    endfunction

    let obj2 = {"test": 'test2', 'callback': obj1.func1}
    function obj2.func2(...)
        echomsg "inside obj2.func2: ".self.test 
        call self.callback()
    endfunction

    call obj2.func2()
endfunction

vim -u NONE -U NONE -S demo.vim

:call Callback_Test()

Expected:

inside obj2.func2: test2
inside obj1.func1: test=test1
inside obj1.func1: other_test=other_test

Actual:

 inside obj2.func2: test2
 inside obj1.func1: test=test2 " NOTE the wrong value here
 Error detected while processing function 2[2]..1:
 line    4:
 E716: Key not present in Dictionary: other_test
 E15: Invalid expression: "inside obj1.func1: other_test=".self.other_test

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub

[mister fish]

Hi, I don't have special knowledge of vimscript, but it appears to me that this is the expected behavior. I don't believe that the function stores the information about the dictionary it lives in; also, if it lives in many dictionaries, I would assume it's still just one function reference and not copies. In Javascript for example it works exactly this way.

These also do what I expect:

function! Callback_Test2()
    function! GetColor() dict
        return self.color
    endfunction
    let reddog  = { 'color': 'red', 'get_color': function('GetColor') }
    let bluedog = { 'color': 'blue', 'get_color': function('GetColor') }
    echomsg "First dog is " . reddog.get_color()
    " --- prints blue, not red.
    echomsg "Second dog is " . bluedog.get_color()
endfunction

function! Callback_Test3()
    let reddog = { 'color': 'red' }
    function! reddog.get_color()
        return self.color
    endfunction
    let bluedog = { 'color': 'blue', 'get_color': reddog.get_color }
    echomsg "First dog is " . reddog.get_color()
    " --- and do does this:
    echomsg "Second dog is " . bluedog.get_color()
endfunction 

[Nikolay Aleksandrovich Pavlov]

2016-05-18 0:54 GMT+03:00 mister fish <vim-dev...@256bit.org>:

Hi, I don't have special knowledge of vimscript, but it appears to me that this is the expected behavior. I don't believe that the function stores the information about the dictionary it lives in; also, if it lives in many dictionaries, I would assume it's still just one function reference and not copies. In Javascript for example it works exactly this way.

​Not function. `dict.func` (as well as `dict['func']`) is now a equivalent of `function(get(dict, 'func'), dict)`: indexing a dictionary creates a partial. So Andrey expected to store this partial in a dictionary and retrieve as-is.

To actually do this one needs to use `get(dict, 'func')` now: this retrieves exactly what is stored in the dictionary. In Python partials are also not created automatically: `pyeval('vim.bindeval("dict")["func"]')` will return a copy of original partial, not create the new one with different binding.

 

These also do what I expect:

function! Callback_Test2()
    function! GetColor() dict
        return self.color
    endfunction
    let reddog  = { 'color': 'red', 'get_color': function('GetColor') }
    let bluedog = { 'color': 'blue', 'get_color': function('GetColor') }
    echomsg "First dog is " . reddog.get_color()
    " --- prints blue, not red.
    echomsg "Second dog is " . bluedog.get_color()
endfunction

function! Callback_Test3()
    let reddog = { 'color': 'red' }
    function! reddog.get_color()
        return self.color
    endfunction
    let bluedog = { 'color': 'blue', 'get_color': reddog.get_color }
    echomsg "First dog is " . reddog.get_color()
    " --- and do does this:
    echomsg "Second dog is " . bluedog.get_color()
endfunction 

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub

--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrey Gavrikov]

 To actually do this one needs to use `get(dict, 'func')` 

Does not this imply that 'dict' is available at the point when 'func' is about to be invoked?

if I was to extend my original example and instead of calling call obj2.func2() directly we had something like this:

" obj2.func2 will be called when channel is closed
call job_start("ping -c 3 google.com", {"close_cb": obj2.func2}) 

here is a full version

function! Callback_Test()
    let obj1 = {"test": 'test1', "other_test": "other_test"}
    function obj1.func1() 
        " next line will use value from wrong scope
        echomsg "inside obj1.func1: test=".self.test 
        " next line will result in: 'Key not present in Dictionary: other_test'
        echomsg "inside obj1.func1: other_test=".self.other_test 
    endfunction

    let obj2 = {"test": 'test2', 'callback': obj1.func1}
    function obj2.func2(...)
        echomsg "inside obj2.func2: ".self.test 
        call self.callback()
    endfunction

    "call obj2.func2()

    call job_start("ping -c 3 google.com", {"close_cb": obj2.func2})
endfunction

In this instance - is there a way to ensure that when obj1.func1 is finally called we have access to its obj1 'dict' in order to be able to do get(obj1, 'func1')

—
You are receiving this because you commented.

[Nikolay Aleksandrovich Pavlov]

Since you store `obj1.func1` in obj2 you need to do `get(obj2, 'callback​')` to get partial that is attached to `obj1` because indexing `obj2` (i.e. `obj2.callback` or `self.callback` in your code) creates new partial with `obj2` attached in place of `obj1`. I do not see why you think you need access to `obj1`, expression `obj1.func1` creates a partial which references this dictionary and you store this partial in `obj2` (and you can use `pyeval('vim.bindeval("obj2")["callback"].self')` to get `obj1` back if needed).

But if you do `get(obj1, 'func1')` you will *not* get a partial, you will get a funcref without `self` dictionaries attached: when defining a function no partial is created. Calling such funcref needs explicit `self` (i.e. `call(get(obj1, 'func1'), [], obj1)`, not `get(obj1, 'func1')()`).

 

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

--

[Christian Brabandt]

Am I understanding correctly, that we can close this issue?

—
You are receiving this because you commented.

[Nikolai Aleksandrovich Pavlov]

Not sure. Depends on whether dict.my_partial was intentionally meant to create a new partial with dict or it is a bug (I personally think this was intentional; was this already discussed in the mailing list?).

—
You are receiving this because you commented.

[Andrey Gavrikov]

Am I understanding correctly, that we can close this issue?

Can not say that I am satisfied with the outcome because nested callbacks (preserving scope of partial) appear to be pretty convoluted. I would expect that the more plugin developers take advantage of job_start the more questions like this will arise.

However, based on the explanations above it would appear that this is expected behaviour. So I am going to close this issue.

Thank you everyone for your input.

—
You are receiving this because you commented.

[Andrey Gavrikov]

Closed #812.

—
You are receiving this because you commented.

[Bram Moolenaar]

> Now when we have job_start there are many more cases when nested callbacks may be required
> Presently it seems that when there is a chain of partials (one called from another) the scope of all partials (except the last one) gets lost.
>
> Steps to reproduce:

> - Given the file `demo.vim`
>
> ```vim

So what would you expect? That assigning a function to a dict creates a
partial, but assigning a partial to a dict stores the partial unchanged?
Or only when it already has an associated dictionary?

This gets confusing very quickly. We could drop automatically creating
a partial on assignment, but that increases the difference between a
partial and a funcref.

Perhaps the best solution is just to document the current behavior.

--
Mrs Abbott: I'm a paediatrician.
Basil: Feet?
Mrs Abbott: Children.
Sybil: Oh, Basil!
Basil: Well, children have feet, don't they? That's how they move
around, my dear. You must take a look next time, it's most
interesting. (Fawlty Towers)

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

—
You are receiving this because you commented.

[Nikolai Aleksandrovich Pavlov]

So what would you expect? That assigning a function to a dict creates a
partial, but assigning a partial to a dict stores the partial unchanged?
Or only when it already has an associated dictionary?

This gets confusing very quickly. We could drop automatically creating
a partial on assignment, but that increases the difference between a
partial and a funcref.

Perhaps the best solution is just to document the current behavior.

I do not understand what you are saying. Assigning function somewhere does not create a partial, currently coded behaviour is that indexing a dictionary creates partial (in handle_subscript) if value at index is a funcref or a partial. If assignment created partial I would not be able to resolve the issue by suggesting to use get(dict, 'callback').

—
You are receiving this because you commented.

[Andrey Gavrikov]

So what would you expect?

Thank you for asking.
Let me preface this by confirming that based on the explanations above I now realise that my expectations are not compatible with current vim script behaviour.

I had hopped that self inside dict function works same way as this inside object method in many OO languages.

i.e. every time I pass and then call obj.funct1 that funct1 would have access to the scope of the original obj (dict) via self.

What happens now is (as my example demonstrates) self actually gets bound to the scope of the last dict in which funct1 was invoked, hence (in my original example)

echomsg "inside obj1.func1: test=".self.test

returns

inside obj1.func1: test=test2

instead of (what I expected)

inside obj1.func1: test=test1

This behaviour (although correct from vim design point of view, as I now understand) makes it quite difficult to program a chain of nested callbacks.

Before I began exploring vim's new (and awesome) job_start & ch_open/ch_sendraw functionality this was a non issue, because all calls in my vim scripts were blocking and there was no real need in callbacks.

Now however I am having to resort to a very dirty hack - every time (before I pass obj1.funcXas a callback to another obj2.funcY) I copy data which I want to preserve from obj1 dict to obj2 dict.
This way when I get to the bottom of my callback chain final objX dict contains data belonging to all dict-s in the callback stack.
When callback stack is rolled back up each objX.funcX can find its data via self. This is prone to key collisions, generally inconvenient and potentially insecure, but that's the best way I could come up with for now.

I hope this makes sense.

—
You are receiving this because you commented.

[Bram Moolenaar]

> > So what would you expect?
>
> Thank you for asking.
> Let me preface this by confirming that based on the explanations above I now realise that my expectations are not compatible with current vim script behaviour.
>
> I had hopped that `self` inside `dict` function works same way as `this` inside object method in many OO languages.
>

> i.e. every time I pass and then call `obj.funct1` that `funct1` would have access to the scope of the **original** `obj` (dict) via `self`.

>
> What happens now is (as my example demonstrates) `self` actually gets bound to the scope of the last `dict` in which `funct1` was invoked, hence (in my original example)

> ```vim

> echomsg "inside obj1.func1: test=".self.test

> ```

> returns
> > inside obj1.func1: test=test2
>
> instead of (what I expected)
> > inside obj1.func1: test=test1
>
> This behaviour (although correct from vim design point of view, as I now understand) makes it quite difficult to program a chain of nested callbacks.
>
> Before I began exploring vim's new (and awesome) `job_start` & `ch_open/ch_sendraw` functionality this was a non issue, because all calls in my vim scripts were blocking and there was no real need in callbacks.
>

> Now however I am having to resort to a very dirty hack - every time (before I pass `obj1.funcX`as a callback to another `obj2.funcY`) I copy data which I want to preserve from `obj1` dict to `obj2` dict.

> This way when I get to the bottom of my callback chain final `objX` dict contains data belonging to all dict-s in the callback stack.

> When callback stack is rolled back up each `objX.funcX` can find its data via `self`. This is prone to key collisions, generally inconvenient and potentially insecure, but that's the best way I could come up with for now.

>
> I hope this makes sense.

So, one option is that when one uses:

dict.member = SomeFunctionVar

we do not automatically bind the function that SomeFunctionVar refers to
to "dict", but keep whatever it was bound to (including nothing).

A slightly different mechanism would be to bind to "dict" only when
there is no binding yet. This would mostly do what you expect, but it
hides the details, which is tricky and may lead to strange mistakes.
The main problem is that exactly the same text does something different,
depending on what the variable contains (has a dict bound or not).

So one would always need to use:

dict.member = funcion(SomeFunctionVar, dict)

Opinions?

--
BEDEVERE: Wait. Wait ... tell me, what also floats on water?
ALL: Bread? No, no, no. Apples .... gravy ... very small rocks ...
ARTHUR: A duck.
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

—
You are receiving this because you commented.

[Nikolai Aleksandrovich Pavlov]

So, one option is that when one uses:

dict.member = SomeFunctionVar

we do not automatically bind the function that SomeFunctionVar refers to
to "dict", but keep whatever it was bound to (including nothing).

? This is the current behaviour, SomeFunctionVar is kept whatever it was bound to (including nothing) when it is being assigned. It is handle_subscript that creates binding.

Current variant is backwards compatible: in Vim without partials what @neowit is seeing is exactly what is expected because there no partials are created. If you care only exactly about callbacks, then, assuming that handle_subscript will handle dict.func() the old way (partial is created only once all subscripts were handled, so it is different from (dict.func)()), the solution may be “create new partial if funcref/partial at given index has dict attached, otherwise leave this as-is”, but this will make Vim show strange differences between dict.func() (call(dict.func, dict.func.args + [], dict)*) and (dict.func) (call(dict.func, dict.func.args + [], dict.func.self)*) if dict.func is a partial with dictionary attached which is not dict. In any case any solution that will not break plugins will exhibit the same behaviour as expressed in the first message.

* pseudocode: assumes that partials/funcrefs support indexing and two indexes may be used args to get a list of arguments or an empty list and self to get a self dictionary or zero, and also assumes that call(name, args, 0) is like call(name, args). BTW, I think these are good ideas to actually implement.

—
You are receiving this because you commented.

[Bram Moolenaar]

Nikolai Pavlov wrote:

> > So, one option is that when one uses:
> >
> > dict.member = SomeFunctionVar
> >
> > we do not automatically bind the function that SomeFunctionVar refers to
> > to "dict", but keep whatever it was bound to (including nothing).
>
> ? This is the current behaviour, SomeFunctionVar is kept whatever it
> was bound to (including nothing) when it is being assigned. It is
> `handle_subscript` that creates binding.

OK, we could use the difference between doing it at assignment and usage
perhaps. So keep it that the assignment takes whatever is on the RHS,
and when using it we could do something different.

> Current variant is backwards compatible: in Vim without partials what
> @neowit is seeing is exactly what is expected because there no
> partials are created. If you care only exactly about callbacks, then,
> assuming that `handle_subscript` will handle `dict.func()` the old way
> (partial is created only once all subscripts were handled, so it is

> different from `(dict.func)()`), the solution may be “create new

> partial if funcref/partial at given index has dict attached, otherwise
> leave this as-is”, but this will make Vim show strange differences

> between `dict.func()` (`call(dict.func, dict.func.args + [], dict)`\*)
> and `(dict.func)` (`call(dict.func, dict.func.args + [],
> dict.func.self)`\*) if `dict.func` is a partial with dictionary
> attached which is not `dict`. In any case any solution that will not

> break plugins will exhibit the same behaviour as expressed in the
> first message.
>
> \* pseudocode: assumes that partials/funcrefs support indexing and two
> indexes may be used `args` to get a list of arguments or an empty list
> and `self` to get a self dictionary or zero, and also assumes that

> `call(name, args, 0)` is like `call(name, args)`. BTW, I think these

> are good ideas to actually implement.

The new thing is the partial itself. So a member of a dict can already
have a dict bound to it. Previously this didn't exist, so only using
dict.member would bind "dict".

Clearly we need to keep the old behavior:

let dict1.f1 = function('Name')
call dict1.f1() " binds dict1

let dict2.f2 = dict1.f1
call dict2.f2() " binds dict2

But with a partial this could behave differently:

let dict1.f1 = function('Name', dict2)
call dict1.f1() " binds dict2

Thus if a partial is used that already binds a dict, using dict.member
does not bind another dict. If you see this as storing a partial
somewhhere and using it, it would be equivalent to:

let f1 = function('Name', dict2)
call f1() " binds dict2

So that's consistent. But it's inconsistent with the old behavior
(still backwards compatible, since partials didn't exist before).

From a functionality perspective, there should be a way to take the
function from dict1.f1 (which is bound to dict2) and apply it to dict1.
That's easy:

let dict1.f1 = function('Name', dict2)
call function(dict1.f1, dict1) " binds dict1

If we do it the other way around, that dict1.func() always binds dict1,
even when using a partial that binds another dict (this is the current
behavior), how would we keep that other dict that was already bound?
Getting the dict out of the partial and using it?

let dict1.f1 = function('Name', dict2)
call function(dict1.f1, getdict(dict1.f1)) " binds dict2

This doesn't really work, because evaluating dict1.f1 would normally
add the binding with dict1, unless we treat the argument to getdict() in
a very special way.

So it looks like when a dict member is a partial with a bound
dictionary, we should not have handle_subscript() change to use the
dictionary it's a member of.

We need to be able to explain this. Perhaps we can state that when a
binding already exists it's kept, if there is no binding and the form
dict.member is used then the binding is added.

[Disclaimer: any typos are caused by the bus I'm in being very shaky
:-)]

--
Never under any circumstances take a sleeping pill
and a laxative on the same night.

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

—
You are receiving this because you commented.

[Andrey Gavrikov]

From a functionality perspective, there should be a way to take the
function from dict1.f1 (which is bound to dict2) and apply it to dict1

Perhaps an attempt to re-bind a (already bound) function should be explicit.
e.g.

let dict1 = {"test": "dict1"}
function dict1.f1() {
    return self.test " returns 'dict1'
}

" 1) - explicit binding (scope change) using function(func, dict)
let dict2 = {"test": "dict2"}
let dict2.f2 = function(dict1.f1, dict2) " explicit binding to dict2
call dict2.f2() " returns 'dict2' because dict2.f2 is explicitly bound to 'dict2'

" 2) - desire to change scope is not expressed explicitly
let dict2.f2 = function(dict1.f1)
call dict2.f2() " returns 'dict1', i.e. scope is preserved

if above does not work from backwards compatibility point of view then perhaps function() needs another (optional) parameter used to explicitly specify if scope must be preserved or can be changed. If such parameter is not specified then keep current behaviour.

—
You are receiving this because you commented.

[Bram Moolenaar]

> From a functionality perspective, there should be a way to take the
> function from dict1.f1 (which is bound to dict2) and apply it to dict1

You can always do that with function().

> Perhaps an attempt to re-bind a (already bound) function should be explicit.
> e.g.

> ```vim

> let dict1 = {"test": "dict1"}
> function dict1.f1() {
> return self.test " returns 'dict1'
> }

When specifying a function like this, it's always bound to the
dictionary it's specified for.

> " 1) - explicit binding (scope change) using function(func, dict)
> let dict2 = {"test": "dict2"}
> let dict2.f2 = function(dict1.f1, dict2) " explicit binding to dict2
> call dict2.f2() " returns 'dict2' because dict2.f2 is explicitly bound to 'dict2'
>
> " 2) - desire to change scope is not expressed explicitly
> let dict2.f2 = function(dict1.f1)
> call dict2.f2() " returns 'dict1', i.e. scope is preserved
> ```
>
> if above does not work from backwards compatibility point of view then
> perhaps `function()` needs another (optional) parameter used to
> explicitly specify if scope must be preserved or can be changed. If
> such parameter is not specified then keep current behaviour.

That is an extra flag to change behavior. Generally this makes things
more complicated, since there are more values possible. I would only do
this when it can't be avoided.


--
Team-building exercises come in many forms but they all trace their roots back
to the prison system. In your typical team-building exercise the employees
are subjected to a variety of unpleasant situations until they become either a
cohesive team or a ring of car jackers.
(Scott Adams - The Dilbert principle)

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

—
You are receiving this because you commented.

[Nikolay Aleksandrovich Pavlov]

2016-05-21 21:23 GMT+03:00 Bram Moolenaar <vim-dev...@256bit.org>:
>
>
> > From a functionality perspective, there should be a way to take the
> > function from dict1.f1 (which is bound to dict2) and apply it to dict1
>
> You can always do that with function().
>
> > Perhaps an attempt to re-bind a (already bound) function should be explicit.
> > e.g.
> > ```vim
> > let dict1 = {"test": "dict1"}
> > function dict1.f1() {
> > return self.test " returns 'dict1'
> > }
>
> When specifying a function like this, it's always bound to the
> dictionary it's specified for.

It is not. Automatic binding occurs *only* in handle_subscript, this
variant just defines a new anonymous function and saves a funcref (not
partial) to it in a dictionary without any bindings to any
dictionaries. To change this it is needed to modify ex_function().

[Andrey Gavrikov]

When specifying a function like this, it's always bound to the
dictionary it's specified for.

Initially it is, but as soon as you assign it as a value in another dictionary the original binding is lost.

function! Callback_Test()
    let obj1 = {"test": 'test1'

}
    

function obj1.func1() 
        " next line will use value from wrong scope
        echomsg "inside obj1.func1: test=".self.

test 
    endfunction

    let obj2 = {"test": 'test2', 'callback': function(obj1.func1, obj1)}
    

function obj2.func2(...)
        echomsg "inside obj2.func2: ".self.test 
        call self.callback()
    endfunction

    call job_start("ping -c 3 google.com", {"close_cb": obj2.func2})
endfunction

results in the same outcome as in my very first message in this thread

inside obj2.func2: test2
inside obj1.func1: test=test2

Last line should have printed ...test=test1 but it prints ...test=test2

—
You are receiving this because you commented.

[Nikolai Aleksandrovich Pavlov]

21.05.2016, 22:35, "Andrey Gavrikov" <notifi...@github.com>:When specifying a function like this, it's always bound to thedictionary it's specified for.Initially it is, but as soon as you assign it as a value in another dictionary the original binding is lost. It is not. Assignment does *not* alter the value. Original funcref also *is not* bound to anything. There are only two ways to get a partial with bound dictionary: subscripting a dictionary or explicitly using function(). When you assign to the second dictionary you use `dict.func` and *that* creates a partial, *not* `:function`. When you try to call assigned function `handle_subscript()` uses whatever dictionary was indexed to get the funcref/partial (note: `(dict.func)()` and `dict.func()` go via rather different code paths, the latter does not create any partials: better think of it as a `.()` operator, though this is again not completely true: just this function handels all subscripts in an expression in cycle, saving some state (like dictionary funcref was taken from) from previous iterations), this has nothing to do with `:let` assigment performed earlier which correctly saved partial bound to the first dictionary. function! Callback_Test()

let obj1 = {"test": 'test1'}

function obj1.func1()
" next line will use value from wrong scope
echomsg "inside obj1.func1: test=".self.test

endfunction

let obj2 = {"test": 'test2', 'callback': function(obj1.func1, obj1)}

function obj2.func2(...)
echomsg "inside obj2.func2: ".self.test
call self.callback()
endfunction

call job_start("ping -c 3 google.com", {"close_cb": obj2.func2})

endfunctionresults in the same outcome as in my very first message in this threadinside obj2.func2: test2inside obj1.func1: test=test2Last line should have printed ...test=test1 but it prints ...test=test2—You are receiving this because you commented.Reply to this email directly or view it on GitHub 

—
You are receiving this because you commented.

[Andrey Gavrikov]

There are only two ways to get a partial with bound dictionary: subscripting a dictionary or explicitly using function().

Excuse my ignorance, but does not this:

let obj2 = {"test": 'test2', 'callback': function(obj1.func1, obj1)}

explicitly use function() to bind obj1.func1 to obj1 dict ?

If it does then why (when I later call self.callback() which (in theory) should invoke obj1.func1 bound to obj1) I get test=test2 instead of test=test1 ?

Is there another way to call self.callback() (from my last example) such as to ensure that whatever I previously saved to obj2.callback has not been altered, and call self.callback() would in fact resolve to func1() with self linked to obj1 instead of obj2 as it happens now?

—
You are receiving this because you commented.

[Nikolai Aleksandrovich Pavlov]

Is there another way to call self.callback() (from my last example) such as to ensure that whatever I previously saved to obj2.callback has not been altered, and call self.callback() would in fact resolve to func1() with self linked to obj1 instead of obj2 as it happens now?

That was in the very first message: use get().

If it does then why (when I later call self.callback() which (in theory) should invoke obj1.func1 bound to obj1) I get test=test2 instead of test=test1 ?

That was explained in my messages. A few times, if I am not mistaking.

—
You are receiving this because you commented.

[Nikolay Aleksandrovich Pavlov]

2016-05-21 23:50 GMT+03:00 Nikolai Aleksandrovich Pavlov
<vim-dev...@256bit.org>:

>
> > Is there another way to call self.callback() (from my last example) such as to ensure that whatever I previously saved to obj2.callback has not been altered, and call self.callback() would in fact resolve to func1() with self linked to obj1 instead of obj2 as it happens now?
>
> That was in the very first message: use get().

Very first my message.

>
> > If it does then why (when I later call self.callback() which (in theory) should invoke obj1.func1 bound to obj1) I get test=test2 instead of test=test1 ?
>
> That was explained in my messages. A few times, if I am not mistaking.
>
> —
> You are receiving this because you commented.
> Reply to this email directly or view it on GitHub
>

[Andrey Gavrikov]

That was in the very first my message: use get().

You mean get(dict, 'func'), e.g. get(obj1, 'func1') ?

Not sure how this helps in a situation with real callbacks - where method which invokes a callback has no access/knowledge of the relevant dict and only has a funcref of the method to call.

I do realise that in my simplistic example instead of doing call self.callback() I could just call obj1.func1(), but in the real world scenario obj2.func2() may not have access/visibility of obj1 dict, so get(obj1, 'func1') does not look like a viable option.

I am sure I misunderstood you because get(dict, 'func') looks like an obvious no go.
Please can you point out where my train of thought went wrong?

—
You are receiving this because you commented.

[Nikolai Aleksandrovich Pavlov]

@neowit In your simplistic example you can use call call(get(self, 'callback'), []) (using call() because you can’t write call get(self, 'callback')(), in e.g. let that would be fine). get() does not create new partial, it gets exactly what is there in a dictionary and thus it does help in your situation.

—
You are receiving this because you commented.

[Nikolai Aleksandrovich Pavlov]

But to use this you must know that callback in dictionary is a partial. In less simple example that may need to be

python import vim
if pyeval('vim.bindeval("self")["callback"].self is None')
    return self.callback()
else
    return get(self, 'callback')()
endif

(there currently is no way to get self dictionary presense out of the partial in VimL, except for using string() which may throw an error; and exactly no way to get self dictionary itself, one of the latest updates to Python bindings may do this).

—
You are receiving this because you commented.

[Bram Moolenaar]

> > > From a functionality perspective, there should be a way to take the
> > > function from dict1.f1 (which is bound to dict2) and apply it to dict1
> >
> > You can always do that with function().
> >
> > > Perhaps an attempt to re-bind a (already bound) function should be explicit.
> > > e.g.
> > > ```vim
> > > let dict1 = {"test": "dict1"}
> > > function dict1.f1() {
> > > return self.test " returns 'dict1'
> > > }
> >

> > When specifying a function like this, it's always bound to the
> > dictionary it's specified for.
>

> It is not. Automatic binding occurs *only* in handle_subscript, this
> variant just defines a new anonymous function and saves a funcref (not
> partial) to it in a dictionary without any bindings to any
> dictionaries. To change this it is needed to modify ex_function().

Since the normal way to call the function is by using dict1.f1() or
dict1['f1']() the function is bound to dict1. Where that is implemented
doesn't really matter for the user.

It only matters when using get(dict1, 'f1'), but that's unusual. I
don't think we want to force using get() to avoid a partial bound to one
dict to be rebound to another dict.

--
Our job was to build a computer information system for the branch banks. We
were the perfect people for the job: Dean had seen a computer once, and I had
heard Dean talk about it.

(Scott Adams - The Dilbert principle)

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

—

[Bram Moolenaar]

> > When specifying a function like this, it's always bound to the
> > dictionary it's specified for.
>

> Initially it is, but as soon as you assign it as a value in *another* dictionary the original binding is lost.
>
> ```vim

> function! Callback_Test()
> let obj1 = {"test": 'test1'}
> function obj1.func1()
> " next line will use value from wrong scope
> echomsg "inside obj1.func1: test=".self.test
> endfunction
>

> let obj2 = {"test": 'test2', 'callback': function(obj1.func1, obj1)}

> function obj2.func2(...)
> echomsg "inside obj2.func2: ".self.test
> call self.callback()
> endfunction
>
> call job_start("ping -c 3 google.com", {"close_cb": obj2.func2})
> endfunction

> ```

>
> results in the same outcome as in my very first message in this thread
> > inside obj2.func2: test2
> > inside obj1.func1: test=test2
>

> Last line should have printed `...test=test1` but it prints `...test=test2`

Yeah, that's what we have been discussing: the statement
call self.callback()
rebinds the callback to "self", which happens to be obj2

I tend to think that when we have a partial that is bound to a dict,
invoking it from another dict should not rebind it. We'll have to see
if that causes more confusion than it solves though.

--
If your company is not involved in something called "ISO 9000" you probably
have no idea what it is. If your company _is_ involved in ISO 9000 then you
definitely have no idea what it is.

[Nikolai Aleksandrovich Pavlov]

Since the normal way to call the function is by using dict1.f1() or

dict1'f1' the function is bound to dict1. Where that is implemented

doesn't really matter for the user.

It only matters when using get(dict1, 'f1'), but that's unusual. I
don't think we want to force using get() to avoid a partial bound to one
dict to be rebound to another dict.

—

[Nikolai Aleksandrovich Pavlov]

Since the normal way to call the function is by using dict1.f1() or
dict1'f1' the function is bound to dict1. Where that is implemented
doesn't really matter for the user.

It does. When you e.g. copy()/deepcopy()/extend() dictionary funcref is copied as funcref and partial is copied as partial, so new dictionary will not receive a bound partial. Also

It only matters when using get(dict1, 'f1'), but that's unusual. I
don't think we want to force using get() to avoid a partial bound to one
dict to be rebound to another dict.

is not the only way to get data from dictionary as-is. In python when using bindeval() no automatic binding ever happens.

I tend to think that when we have a partial that is bound to a dict,
invoking it from another dict should not rebind it. We'll have to see
if that causes more confusion than it solves though.

I can immediately say that this will break at least https://github.com/Bashka/vim_lib/blob/ca50aed17f51d76ad84c2a8fc2d2112f3477b40e/autoload/vim_lib/base/Object.vim#L27 (it is not the only place which will break in this plugin). I guess this may be the case for other plugins that use objects, just this is the one code of which I most recently viewed. If you want to implement something like this then you need to remove new code from handle_subscript which creates partials automatically.

Basically your suggestion breaks every time somebody writes something like

let s:Parent = {}
function s:Parent.overridden_function() dict
    let self.num_calls = get(self, 'num_calls', 0) + 1
    return self.num_calls
endfunction
let s:Class = copy(s:Parent)
let s:Class._overridden_function = s:Class.overridden_function
function! s:Class.overridden_function() dict
    call self._overridden_function()
    return self.num_calls * 2
endfunction

let s:instance = copy(s:Class)
call self.overridden_function()

: s:instance._overridden_function will be bound to s:Class, but it is expected and should be bound to s:instance.

[Bram Moolenaar]

Nikolai Pavlov wrote:

> > Since the normal way to call the function is by using dict1.f1() or

> > dict1['f1']() the function is bound to dict1. Where that is implemented

> > doesn't really matter for the user.
>

> It does. When you e.g. `copy()`/`deepcopy()`/`extend()` dictionary

> funcref is copied as funcref and partial is copied as partial, so new
> dictionary will not receive a bound partial. Also

I don't see how this matters, it's just a copy. Only if we would make
two changes (bind when assigning and not rebinding) then it would
change.

> > It only matters when using get(dict1, 'f1'), but that's unusual. I
> > don't think we want to force using get() to avoid a partial bound to one
> > dict to be rebound to another dict.
>
> is not the only way to get data from dictionary as-is. In python when
> using `bindeval()` no automatic binding ever happens.

I don't see that documented. The help says:
Evaluates the expression str using the vim internal expression
evaluator (see |expression|).
Thus it should do the same thing.

> > I tend to think that when we have a partial that is bound to a dict,
> > invoking it from another dict should not rebind it. We'll have to see
> > if that causes more confusion than it solves though.
>
> I can immediately say that this will break at least
> https://github.com/Bashka/vim_lib/blob/ca50aed17f51d76ad84c2a8fc2d2112f3477b40e/autoload/vim_lib/base/Object.vim#L27
> (it is not the only place which will break in this plugin). I guess
> this may be the case for other plugins that use objects, just this is
> the one code of which I most recently viewed. If you want to implement
> something like this then you need to remove new code from
> `handle_subscript` which creates partials automatically.

Looks like that was written before partials existed, thus it won't
break, since we would only change how partials behave. If you think it
does break, please give an example (I can't read Russian comments :-).

> Basically your suggestion breaks every time somebody writes something like
>

> ```VimL

> let s:Parent = {}
> function s:Parent.overridden_function() dict
> let self.num_calls = get(self, 'num_calls', 0) + 1
> return self.num_calls
> endfunction
> let s:Class = copy(s:Parent)
> let s:Class._overridden_function = s:Class.overridden_function
> function! s:Class.overridden_function() dict
> call self._overridden_function()
> return self.num_calls * 2
> endfunction
>
> let s:instance = copy(s:Class)
> call self.overridden_function()

This doesn't work, there is no "self" here. Did you mean:
call s:instance.overridden_function()

> ```
>
> : `s:instance._overridden_function` will be bound to `s:Class`, but it
> is expected and should be bound to `s:instance`.

This should not break, the behavior must be backwards compatible. We
need to make a difference between using function() to create a partial
and passing on a function reference. This gets a bit tricky internally,
but if it's explainable to the users it can work.

If that gets too complicated, we could have a simple way to get a
partial from a dict without having it rebound to that dict. The get()
way still looks a bit ugly.

--
A salesperson says: Translation:
"backward compatible" Old technology
"Premium" Overpriced
"Can't keep it on the shelf" Unavailable
"Stands alone" Piece of shit
"Proprietary" Incompatible

(Scott Adams - The Dilbert principle)

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

—

[Nikolai Aleksandrovich Pavlov]

Thus it should do the same thing.

I meant vim.bindeval('self')['callback'], not vim.bindeval('self.callback'). This has no reason to do the same thing.

[Nikolai Aleksandrovich Pavlov]

Looks like that was written before partials existed, thus it won't
break, since we would only change how partials behave. If you think it
does break, please give an example (I can't read Russian comments :-).

No. After partials were added dict.func behaviour changed. If you add suggested change (not rebounding partials with still existing bind-on-index) this will break because self.expand (and others) in that function should be bound to l:child. Comments are not needed to understand that (in fact, I did not read them).

[Nikolai Aleksandrovich Pavlov]

This doesn't work, there is no "self" here. Did you mean:

call s:instance.overridden_function()

Yes, this is what I meant.

[Bram Moolenaar]

> > Thus it should do the same thing.
>
> I meant `vim.bindeval('self')['callback']`, not

> `vim.bindeval('self.callback')`. This has no reason to do the same
> thing.

Well, that's after converting to a Python type, so it will work
differently. I don't think anybody expect Python types to work the same
way as Vim types.

--
I recommend ordering large cargo containers of paper towels to make up
whatever budget underruns you have. Paper products are always useful and they
have the advantage of being completely flushable if you need to make room in
the storage area later.

(Scott Adams - The Dilbert principle)

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

—

[Nikolai Aleksandrovich Pavlov]

This should not break, the behavior must be backwards compatible. We
need to make a difference between using function() to create a partial
and passing on a function reference. This gets a bit tricky internally,
but if it's explainable to the users it can work.

It is explainable to user, but I see it more tricky to explain then to understand. Basically I see it as a same flag “this partial should (not) be rebound” which was already suggested (and rejected) earlier in this issue. Just implicit flag which I think is not the good idea.

If that gets too complicated, we could have a simple way to get a
partial from a dict without having it rebound to that dict. The get()
way still looks a bit ugly.

With current syntax I can suggest one of the following ways:

1. dict.func creates a partial, dict['func'] does not.
2. New function. I do not see how this would be different from get().
3. An option that controls automatic creation of partials. Looks worse then previous options.
4. Precommand modifier like norebind call self.callback().

[Nikolai Aleksandrovich Pavlov]

Well, that's after converting to a Python type, so it will work
differently. I don't think anybody expect Python types to work the same
way as Vim types.

That Python type is a mere proxy to a VimL type. So somebody may expect this.

[Nikolay Aleksandrovich Pavlov]

2016-05-22 23:27 GMT+03:00 Nikolai Aleksandrovich Pavlov
<vim-dev...@256bit.org>:
>

> Well, that's after converting to a Python type, so it will work
> differently. I don't think anybody expect Python types to work the same
> way as Vim types.
>
> That Python type is a mere proxy to a VimL type. So somebody may expect this.

Also because it is not much documented where exactly partial is
created. So one may think that partials with dictionary are created at
assignment like it was expressed a number of times in this thread.


>
> —

> You are receiving this because you commented.
> Reply to this email directly or view it on GitHub
>

[Andrey Gavrikov]

In your simplistic example you can use call call(get(self, 'callback'), [])

I am happy to report that call call(get(self, 'callback'), []) suggested by @ZyX-I seems to be doing exactly what I tried to achieve in the first place.

Given this code:

function! Callback_Test()
    let obj1 = {"test": 'test1'}
    function obj1.func1

() 
        echomsg "obj1.test=".self.test 
    endfunction

    call s:step2(obj1.func1)

endfunction    

function! s:step2(callback)
    let obj2 = {"test": 'test2', 'callback': a:callback}
    function obj2.internalCallback() 
        echomsg "obj2.test=".self.test
        call call(get(self, 'callback'), [])
    endfunction

    call s:step3(obj2.internalCallback)

endfunction    

function! s:step3(callback)
    let obj3 = {"test": 'test3', 'callback': a:callback}
    function obj3.internalCallback(...) 
        echomsg "obj3.test=".self.test
        call call(get(self, 'callback'), [])
    endfunction

    call job_start("ping -c 1 google.com", {"close_cb": obj3.internalCallback})
endfunction    

with the above code if I run call Callback_Test() the end result looks like so

obj3.test=test3
obj2.test=test2
obj1.test=test1

i.e. scope/dict of each function is preserved.

IMHO the Ideal solution would be one where - when we have a partial that is bound to a dict, invoking it from another dict would not rebind it.
But in the absence of that call call(get(self, 'callback'), []) does the job. I wonder how many people would be able to guess this though. Perhaps we need to add some examples in the documentation? I would be happy to send a pull request.

—

You are receiving this because you commented.

Reply to this email directly or view it on GitHub

[Bram Moolenaar]

Andrey Gavrikov wrote:

> > In your simplistic example you can use `call call(get(self, 'callback'), [])`
>
> I am happy to report that `call call(get(self, 'callback'), [])` suggested by @ZyX-I seems to be doing exactly what I tried to achieve in the first place.
>
> Given this code:

> ```vim

That's what I am also thinking of. We need to keep the automatic
binding for backwards compatibility. But Partials didn't exist until
recently, thus when someone explicitly binds a dict to a function with
function() we can have a flag that indicates it must not be changed when
passing the Partial around.

> But in the absence of that `call call(get(self, 'callback'), [])` does
> the job. I wonder how many people would be able to guess this though.
> Perhaps we need to add some examples in the documentation? I would be
> happy to send a pull request.

Right, this looks strange and no doubt will lead to obscure mistakes.

I'll try to write the help fist to check if I can explain how it works
and recommend how to use the old way (automatic binding) and the new way
(Partials).

--
I think that you'll agree that engineers are very effective in their social
interactions. It's the "normal" people who are nuts.

(Scott Adams - The Dilbert principle)

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

—

[Bram Moolenaar]

Nikolai wrote:

> > This should not break, the behavior must be backwards compatible. We
> > need to make a difference between using function() to create a partial
> > and passing on a function reference. This gets a bit tricky internally,
> > but if it's explainable to the users it can work.
>
> It is explainable to user, but I see it more tricky to explain then to
> understand. Basically I see it as a same flag “this partial should
> (not) be rebound” which was already suggested (and rejected) earlier

> in this issue. Just *implicit* flag which I think is not the good

> idea.
>
> > If that gets too complicated, we could have a simple way to get a
> > partial from a dict without having it rebound to that dict. The get()
> > way still looks a bit ugly.
>
> With current syntax I can suggest one of the following ways:
>

> 1. `dict.func` creates a partial, `dict['func']` does not.

I don't like that, since most of us will assume it's exactly the same
thing, with the only difference that the dict.func form has restrictions
on the characters in the string.

> 2. New function. I do not see how this would be different from `get()`.

To call a method it would be:

get(self, 'callback')(args)

Can't really make it shorter with a function. We could use another
character instead of ".":

self..callback(args)

That's too cryptic.


> 3. An option that controls automatic creation of partials. Looks worse
> then previous options.

Indeed.

> 4. Precommand modifier like `norebind call self.callback()`.

Can't be used when passing the function around in other places.

Thus the main alternatives are to use get() or to have some clever way
to avoid automatic binding when a partial was intentionally created.

--
I used to be indecisive, now I'm not sure.

[vim-dev ML]

Hey,

I'm so excited to tell you about yesterday that was full of pleasant surprises, just take a look at that <http://link.noah82.com/e4azodez>

Faithfully, paul

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub

[vim-dev ML]

Hi friend!

I've found a very easy tutorial on the stuff we were talking about, look here <http://quality.ultimatefoodusa.com/e4gvjm>

My Best, paul

[vim-dev ML]

Hello friend,


I wanted to tell you something really nice, please read it here <http://safe.datasciencetoolkit.org/e4ysyq>

Sincerely yours, paul