Patch 7.4.1577

[Bram Moolenaar]

Patch 7.4.1577
Problem: Cannot pass "dict.Myfunc" around as a partial.
Solution: Create a partial when expected.
Files: src/eval.c, src/testdir/test_partial.vim


*** ../vim-7.4.1576/src/eval.c 2016-03-15 13:33:50.709244886 +0100
--- src/eval.c 2016-03-15 19:19:48.733981126 +0100
***************
*** 110,115 ****
--- 110,116 ----
#ifdef FEAT_FLOAT
static char *e_float_as_string = N_("E806: using Float as a String");
#endif
+ static char *e_dict_both = N_("E924: can't have both a \"self\" dict and a partial: %s");

#define NAMESPACE_CHAR (char_u *)"abglstvw"

***************
*** 8912,8919 ****
name);
break;
case ERROR_BOTH:
! emsg_funcname(N_("E924: can't have both a \"self\" dict and a partial: %s"),
! name);
break;
}
}
--- 8913,8919 ----
name);
break;
case ERROR_BOTH:
! emsg_funcname(e_dict_both, name);
break;
}
}
***************
*** 11782,11793 ****
{
char_u *s;
char_u *name;

! s = get_tv_string(&argvars[0]);
! if (s == NULL || *s == NUL || VIM_ISDIGIT(*s))
EMSG2(_(e_invarg2), s);
/* Don't check an autoload name for existence here. */
! else if (vim_strchr(s, AUTOLOAD_CHAR) == NULL && !function_exists(s))
EMSG2(_("E700: Unknown function: %s"), s);
else
{
--- 11782,11810 ----
{
char_u *s;
char_u *name;
+ int use_string = FALSE;

! if (argvars[0].v_type == VAR_FUNC)
! {
! /* function(MyFunc, [arg], dict) */
! s = argvars[0].vval.v_string;
! }
! else if (argvars[0].v_type == VAR_PARTIAL
! && argvars[0].vval.v_partial != NULL)
! /* function(dict.MyFunc, [arg]) */
! s = argvars[0].vval.v_partial->pt_name;
! else
! {
! /* function('MyFunc', [arg], dict) */
! s = get_tv_string(&argvars[0]);
! use_string = TRUE;
! }
!
! if (s == NULL || *s == NUL || (use_string && VIM_ISDIGIT(*s)))
EMSG2(_(e_invarg2), s);
/* Don't check an autoload name for existence here. */
! else if (use_string && vim_strchr(s, AUTOLOAD_CHAR) == NULL
! && !function_exists(s))
EMSG2(_("E700: Unknown function: %s"), s);
else
{
***************
*** 11837,11842 ****
--- 11854,11865 ----
vim_free(name);
return;
}
+ if (argvars[0].v_type == VAR_PARTIAL)
+ {
+ EMSG2(_(e_dict_both), name);
+ vim_free(name);
+ return;
+ }
if (argvars[dict_idx].vval.v_dict == NULL)
dict_idx = 0;
}
***************
*** 11880,11886 ****
}
}

! if (dict_idx > 0)
{
pt->pt_dict = argvars[dict_idx].vval.v_dict;
++pt->pt_dict->dv_refcount;
--- 11903,11914 ----
}
}

! if (argvars[0].v_type == VAR_PARTIAL)
! {
! pt->pt_dict = argvars[0].vval.v_partial->pt_dict;
! ++pt->pt_dict->dv_refcount;
! }
! else if (dict_idx > 0)
{
pt->pt_dict = argvars[dict_idx].vval.v_dict;
++pt->pt_dict->dv_refcount;
***************
*** 21533,21539 ****
rettv->v_type = VAR_UNKNOWN;

/* Invoke the function. Recursive! */
! if (rettv->v_type == VAR_PARTIAL)
{
pt = functv.vval.v_partial;
s = pt->pt_name;
--- 21561,21567 ----
rettv->v_type = VAR_UNKNOWN;

/* Invoke the function. Recursive! */
! if (functv.v_type == VAR_PARTIAL)
{
pt = functv.vval.v_partial;
s = pt->pt_name;
***************
*** 21582,21587 ****
--- 21610,21632 ----
}
}
}
+
+ if (rettv->v_type == VAR_FUNC && selfdict != NULL)
+ {
+ partial_T *pt = (partial_T *)alloc_clear(sizeof(partial_T));
+
+ /* Turn "dict.Func" into a partial for "Func" with "dict". */
+ if (pt != NULL)
+ {
+ pt->pt_dict = selfdict;
+ selfdict = NULL;
+ pt->pt_name = rettv->vval.v_string;
+ func_ref(pt->pt_name);
+ rettv->v_type = VAR_PARTIAL;
+ rettv->vval.v_partial = pt;
+ }
+ }
+
dict_unref(selfdict);
return ret;
}
*** ../vim-7.4.1576/src/testdir/test_partial.vim 2016-03-15 12:36:04.417428858 +0100
--- src/testdir/test_partial.vim 2016-03-15 19:29:43.767744915 +0100
***************
*** 50,52 ****
--- 50,70 ----
call assert_equal("hello/xxx/yyy", Cb("xxx", "yyy"))
call assert_fails('Cb("fff")', 'E492:')
endfunc
+
+ func Test_partial_implicit()
+ let dict = {'name': 'foo'}
+ func dict.MyFunc(arg) dict
+ return self.name . '/' . a:arg
+ endfunc
+
+ call assert_equal('foo/bar', dict.MyFunc('bar'))
+
+ call assert_fails('let func = dict.MyFunc', 'E704:')
+ let Func = dict.MyFunc
+ call assert_equal('foo/aaa', Func('aaa'))
+
+ let Func = function(dict.MyFunc, ['bbb'])
+ call assert_equal('foo/bbb', Func())
+
+ call assert_fails('call function(dict.MyFunc, ["bbb"], dict)', 'E924:')
+ endfunc
*** ../vim-7.4.1576/src/version.c 2016-03-15 18:23:50.464921252 +0100
--- src/version.c 2016-03-15 19:20:57.225262978 +0100
***************
*** 745,746 ****
--- 745,748 ----
{ /* Add new patch number below this line */
+ /**/
+ 1577,
/**/

--
Never go to the toilet in a paperless office.

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[Kent Sibilev]

On Tuesday, March 15, 2016 at 2:33:55 PM UTC-4, Bram Moolenaar wrote:
> Patch 7.4.1577
> Problem: Cannot pass "dict.Myfunc" around as a partial.
> Solution: Create a partial when expected.
> Files: src/eval.c, src/testdir/test_partial.vim
>

This change doesn't allow functions like this:

function! s:cache_clear(...) dict

function! rails#cache_clear(...)

to be defined at the same time. Any particular reason for this, cause this change breaks vim-rails plugin.

Thanks,
Kent Sibilev

[Raymond Ko]

I think this patch is the cause for breaking plugins like incsearch.vim and vim-easymotion. If I revert back to 7.4.1576, then it works again.

[mattn]

If possible, please try this patch.

https://gist.github.com/mattn/394e8a95be26d9c7bce9

[Bram Moolenaar]

The patch should not change anything about what functions you can
define.

What is the error? Can you make a small example that fails?

--
"I can't complain, but sometimes I still do." (Joe Walsh)

[Bram Moolenaar]

Broken in what way? What's the error message?
Ideally we have a short example that we can add to the tests.

--
hundred-and-one symptoms of being an internet addict:
55. You ask your doctor to implant a gig in your brain.

[Raymond Ko]

I included the errors I got, the these plugins are massive I think, and would probably be really hard to track down unless you are the author.

It mainly seems to be variations of this:

E924: can't have both a "self" dict and a partial

https://gist.github.com/raymond-w-ko/3af7260575b9bea172ad
https://gist.github.com/raymond-w-ko/de9712e077221837ebf5

[Raymond Ko]

I get less errors, but plugins are still broken for me.

After getting these errors, eventually I get a fatal error:
Vim: Caught deadly signal SEGV
which probably means there is something wrong with the error handling cases?

E924: can't have both a "self" dict and a partial: 268
E924: can't have both a "self" dict and a partial: 268
E924: can't have both a "self" dict and a partial: 268
E924: can't have both a "self" dict and a partial: 268
E924: can't have both a "self" dict and a partial: 268
E924: can't have both a "self" dict and a partial: 268
E924: can't have both a "self" dict and a partial: 268
E924: can't have both a "self" dict and a partial: 268
E924: can't have both a "self" dict and a partial: 268
E924: can't have both a "self" dict and a partial: 268
Error detected while processing function incsearch#_go[5]..<SNR>143_get_input[12]..288:
line 4:
E924: can't have both a "self" dict and a partial: 286
Error detected while processing function incsearch#_go[5]..<SNR>143_get_input:
line 12:
E171: Missing :endif

[Ken Takata]

Hi,

2016/3/16 Wed 3:33:55 UTC+9 Bram Moolenaar wrote:
> Patch 7.4.1577
> Problem: Cannot pass "dict.Myfunc" around as a partial.
> Solution: Create a partial when expected.
> Files: src/eval.c, src/testdir/test_partial.vim

> + if (rettv->v_type == VAR_FUNC && selfdict != NULL)
> + {
> + partial_T *pt = (partial_T *)alloc_clear(sizeof(partial_T));
> +
> + /* Turn "dict.Func" into a partial for "Func" with "dict". */
> + if (pt != NULL)
> + {
> + pt->pt_dict = selfdict;
> + selfdict = NULL;
> + pt->pt_name = rettv->vval.v_string;
> + func_ref(pt->pt_name);
> + rettv->v_type = VAR_PARTIAL;
> + rettv->vval.v_partial = pt;
> + }
> + }

I'm not sure, but it seems that this if block has some problems.
When I comment out this block, the issue #690 disappears (even without mattn's
patch).

Regards,
Ken Takata

[Ken Takata]

Hi,

Ah! How about this.

--- a/src/eval.c
+++ b/src/eval.c
@@ -21733,6 +21733,7 @@ handle_subscript(
pt->pt_dict = selfdict;
selfdict = NULL;

pt->pt_name = rettv->vval.v_string;

+ pt->pt_refcount = 1;
func_ref(pt->pt_name);
rettv->v_type = VAR_PARTIAL;
rettv->vval.v_partial = pt;

Regards,
Ken Takata

[Kent Sibilev]

On Wednesday, March 16, 2016 at 10:43:13 AM UTC-4, Bram Moolenaar wrote:

> Kent Sibilev wrote:
>
> > On Tuesday, March 15, 2016 at 2:33:55 PM UTC-4, Bram Moolenaar wrote:
> > > Patch 7.4.1577
> > > Problem: Cannot pass "dict.Myfunc" around as a partial.
> > > Solution: Create a partial when expected.
> > > Files: src/eval.c, src/testdir/test_partial.vim
> > >
> >
> > This change doesn't allow functions like this:
> >
> > function! s:cache_clear(...) dict
> >
> > function! rails#cache_clear(...)
> >
> > to be defined at the same time. Any particular reason for this, cause
> > this change breaks vim-rails plugin.
>
> The patch should not change anything about what functions you can
> define.
>
> What is the error? Can you make a small example that fails?

Here is the small script to reproduce:

function s:cache_clear(...) dict
return self.name
endfunction

function! s:function(name)
return function(substitute(a:name,'^s:',matchstr(expand('<sfile>'), '<SNR>\d\+_'),''))
endfunction

let s:obj = {'name': 'cache'}

let s:obj['clear'] = s:function('s:cache_clear')

" this works
echo s:obj.clear()

" this fails with E924
echo call(s:obj.clear, [], s:obj)

[Bram Moolenaar]

Thanks. Apparently we didn't have a test for using function() this way.

That s:function() call is not needed to reproduce the problem.

Fix should be simple.

--
"I simultaneously try to keep my head in the clouds and my feet on the
ground. Sometimes it's a stretch, though." -- Larry Wall

[Bram Moolenaar]

Thanks. Let me include this fix first.

--
ERROR 047: Keyboard not found. Press RETURN to continue.

[Bram Moolenaar]

Yasuhiro Matsumoto wrote:

> ------=_Part_9179_1228888767.1458138804381
> Content-Type: text/plain; charset=UTF-8

Thanks!

I think there is still a remaining problem when the dictionary function
is a partial with bound arguments and ":call mydict.func()" is used.
Should use "mydict" and the arguments.

--
hundred-and-one symptoms of being an internet addict:

56. You leave the modem speaker on after connecting because you think it
sounds like the ocean wind...the perfect soundtrack for "surfing the net".

[lilydjwg]

On Wed, Mar 16, 2016 at 03:43:06PM +0100, Bram Moolenaar wrote:
>
> Kent Sibilev wrote:
>
> > On Tuesday, March 15, 2016 at 2:33:55 PM UTC-4, Bram Moolenaar wrote:
> > > Patch 7.4.1577
> > > Problem: Cannot pass "dict.Myfunc" around as a partial.
> > > Solution: Create a partial when expected.
> > > Files: src/eval.c, src/testdir/test_partial.vim
> > >
> >
> > This change doesn't allow functions like this:
> >
> > function! s:cache_clear(...) dict
> >
> > function! rails#cache_clear(...)
> >
> > to be defined at the same time. Any particular reason for this, cause
> > this change breaks vim-rails plugin.
>
> The patch should not change anything about what functions you can
> define.
>
> What is the error? Can you make a small example that fails?

I get these:

处理 function <SNR>30_import[11]..<SNR>30__import[20]..<SNR>30__build_module 时发生错误:
第 14 行:
E924: can't have both a "self" dict and a partial: <SNR>30_load
Mark: Invalid value type for g:mwPalettes[maximum]

At least two plugins are broken.

One of them is the mark.vim plugin, which does this:

elseif type(g:mwPalettes[g:mwDefaultHighlightingPalette]) == type(function('tr'))

The left side evaluates to a function ref. The result is 10, not 2.

The other is from an old version of neocomplete, autoload/vital/_b6a796b.vim. You can view it here:
https://github.com/lilydjwg/dotvim/blob/master/autoload/vital/_b6a796b.vim#L144

There are a lot of similar error messages from neocomplete too (both the
version I'm using and the latest version on GitHub).

--
Best regards,
lilydjwg

Linux Vim Python 我的博客:
http://lilydjwg.is-programmer.com/
--
A: Because it obfuscates the reading.
Q: Why is top posting so bad?

[Bram Moolenaar]

Please try the latest patch, hopefully this is fixed now.

--
Q: Should I clean my house or work on Vim?
A: Whatever contains more bugs.

[Raymond Ko]

On Thursday, March 17, 2016 at 10:04:40 AM UTC-4, Raymond Ko wrote:

> I upgraded to 1583 and everything works now.
>
> Thank you!

Oops spoke too soon :-(
Apparently I was using -O3 -flto and that miscompiled Vim and made me think everything was fixed. Switching splits would segfault Vim.

I downgraded the CFLAGS to just -O and now there are some errors instead of segfaulting. Unfortunately the error is plugin specific, which is not helpful.

E116: Invalid arguments for function <SNR>76_defplug
line 375:

[thinca]

This is not fixed in Vim 7.4.1603.

I think funcref and partial are same thing in Vim script, so
type({partial}) should return 2.

diff --git a/src/eval.c b/src/eval.c
index a2288f9..69969fe 100644
--- a/src/eval.c
+++ b/src/eval.c
@@ -20398,6 +20398,7 @@ f_type(typval_T *argvars, typval_T *rettv)
{
case VAR_NUMBER: n = 0; break;
case VAR_STRING: n = 1; break;
+ case VAR_PARTIAL:
case VAR_FUNC: n = 2; break;
case VAR_LIST: n = 3; break;
case VAR_DICT: n = 4; break;
@@ -20411,7 +20412,6 @@ f_type(typval_T *argvars, typval_T *rettv)
break;
case VAR_JOB: n = 8; break;
case VAR_CHANNEL: n = 9; break;
- case VAR_PARTIAL: n = 10; break;
case VAR_UNKNOWN:
EMSG2(_(e_intern2), "f_type(UNKNOWN)");
n = -1;


Or, a funcref in a dictionary must not be converted to a partial for
compatibility.

[thinca]

I thought once again to this problem.
I often use the idiom as below:


let s:base = {}
function! s:base.method()
" ...
endfunction

let instance = copy(s:base)

function IsInstance(instance)
" Check the instance has a same method as the base
return s:base.method == a:instance.method
endfunction


This doesn't work when a method is converted to a partial automatically.
This patch(7.4.1577) breaks backward compatibility.

[Bram Moolenaar]

That is actually very useful. E.g. to pass a callback to a function and
have it automatically bind the dictionary.

So let's make type() ignore the difference between a plain Funcref and a
partial. Perhaps it would be useful to have some way to get information
about the partial, that can be added later.

--
From "know your smileys":
:----} You lie like Pinocchio

[Raymond Ko]

I made a new build today with the current changes, and everything seems to work again.

Thanks!

[Bram Moolenaar]

Thinca wrote:

> I thought once again to this problem.
> I often use the idiom as below:
>
>
> let s:base = {}
> function! s:base.method()
> " ...
> endfunction
>
> let instance = copy(s:base)
>
> function IsInstance(instance)
> " Check the instance has a same method as the base
> return s:base.method == a:instance.method
> endfunction
>
>
> This doesn't work when a method is converted to a partial automatically.
> This patch(7.4.1577) breaks backward compatibility.

Hmm, it's a bit of a strange way to check this, but it's valid code.

So, when comparing functions, we need to ignore any arguments or bound
dictionary that a partial adds.

It's not really nice that two partials for the same function but with
different arguments are considered equal. But it's unavoidable if
comparing the plain function with the partial returns true.
I guess we can compare the string values.

--
From "know your smileys":

:-) Funny
|-) Funny Oriental
(-: Funny Australian

[Raymond Ko]

Okay, after using one plugin incsearch.vim for a period of time, I get this error. I think some behavior is still different from before this patch. Tested on 1615.

function incsearch#_go[5]..<SNR>117_get_input[12]..148[4]..146[1]..162[9]..161[10]..160[23]..159[22]..158[8]..131[2]..186[1]..<SNR>128_call[3]..329[1]..<SNR>162_on_searching[2]..<SNR>162_on_char[26]..<SNR>162_move_cursor[19]..<SNR>120_dictfunction, line 1 Vim(let):E724: variable nested too deep for displaying