Patch 7.3.859

[Bram Moolenaar]

Patch 7.3.859
Problem: 'ambiwidth' must be set by the user.
Solution: Detects East Asian ambiguous width (UAX #11) state of the terminal
at the start-up time and 'ambiwidth' accordingly. (Hayaki Saito)
Files: src/main.c, src/option.c, src/term.c, src/term.h,
src/proto/term.pro


*** ../vim-7.3.858/src/main.c 2013-01-17 13:59:56.000000000 +0100
--- src/main.c 2013-03-13 19:09:03.000000000 +0100
***************
*** 804,809 ****
--- 804,812 ----

starttermcap(); /* start termcap if not done by wait_return() */
TIME_MSG("start termcap");
+ #if defined(FEAT_TERMRESPONSE) && defined(FEAT_MBYTE)
+ may_req_ambiguous_character_width();
+ #endif

#ifdef FEAT_MOUSE
setmouse(); /* may start using the mouse */
*** ../vim-7.3.858/src/option.c 2013-02-13 15:44:22.000000000 +0100
--- src/option.c 2013-03-13 19:09:03.000000000 +0100
***************
*** 2900,2905 ****
--- 2900,2906 ----
p_term("t_op", T_OP)
p_term("t_RI", T_CRI)
p_term("t_RV", T_CRV)
+ p_term("t_u7", T_U7)
p_term("t_Sb", T_CSB)
p_term("t_Sf", T_CSF)
p_term("t_se", T_SE)
*** ../vim-7.3.858/src/term.c 2013-02-26 14:56:24.000000000 +0100
--- src/term.c 2013-03-13 19:18:22.000000000 +0100
***************
*** 111,116 ****
--- 111,121 ----
# define CRV_SENT 2 /* did send T_CRV, waiting for answer */
# define CRV_GOT 3 /* received T_CRV response */
static int crv_status = CRV_GET;
+ /* Request Cursor position report: */
+ # define U7_GET 1 /* send T_U7 when switched to RAW mode */
+ # define U7_SENT 2 /* did send T_U7, waiting for answer */
+ # define U7_GOT 3 /* received T_U7 response */
+ static int u7_status = U7_GET;
# endif

/*
***************
*** 933,938 ****
--- 938,944 ----
{(int)KS_CWP, IF_EB("\033[3;%d;%dt", ESC_STR "[3;%d;%dt")},
# endif
{(int)KS_CRV, IF_EB("\033[>c", ESC_STR "[>c")},
+ {(int)KS_U7, IF_EB("\033[6n", ESC_STR "[6n")},

{K_UP, IF_EB("\033O*A", ESC_STR "O*A")},
{K_DOWN, IF_EB("\033O*B", ESC_STR "O*B")},
***************
*** 1221,1226 ****
--- 1227,1233 ----
{(int)KS_CWP, "[%dCWP%d]"},
# endif
{(int)KS_CRV, "[CRV]"},
+ {(int)KS_U7, "[U7]"},
{K_UP, "[KU]"},
{K_DOWN, "[KD]"},
{K_LEFT, "[KL]"},
***************
*** 1596,1601 ****
--- 1603,1609 ----
{KS_TS, "ts"}, {KS_FS, "fs"},
{KS_CWP, "WP"}, {KS_CWS, "WS"},
{KS_CSI, "SI"}, {KS_CEI, "EI"},
+ {KS_U7, "u7"},
{(enum SpecialKey)0, NULL}
};

***************
*** 3183,3189 ****
/* May need to check for T_CRV response and termcodes, it
* doesn't work in Cooked mode, an external program may get
* them. */
! if (tmode != TMODE_RAW && crv_status == CRV_SENT)
(void)vpeekc_nomap();
check_for_codes_from_term();
}
--- 3191,3198 ----
/* May need to check for T_CRV response and termcodes, it
* doesn't work in Cooked mode, an external program may get
* them. */
! if (tmode != TMODE_RAW && (crv_status == CRV_SENT
! || u7_status == U7_SENT))
(void)vpeekc_nomap();
check_for_codes_from_term();
}
***************
*** 3245,3251 ****
# endif
{
/* May need to check for T_CRV response. */
! if (crv_status == CRV_SENT)
(void)vpeekc_nomap();
/* Check for termcodes first, otherwise an external program may
* get them. */
--- 3254,3260 ----
# endif
{
/* May need to check for T_CRV response. */
! if (crv_status == CRV_SENT || u7_status == U7_SENT)
(void)vpeekc_nomap();
/* Check for termcodes first, otherwise an external program may
* get them. */
***************
*** 3299,3304 ****
--- 3308,3355 ----
(void)vpeekc_nomap();
}
}
+
+ # if defined(FEAT_MBYTE) || defined(PROTO)
+ /*
+ * Check how the terminal treats ambiguous character width (UAX #11).
+ * First, we move the cursor to (0, 0) and print a test ambiguous character
+ * \u25bd (WHITE DOWN-POINTING TRIANGLE) and query current cursor position.
+ * If the terminal treats \u25bd as single width, the position is (0, 1),
+ * or if it is treated as double width, that will be (0, 2).
+ * This function has the side effect that changes cursor position, so
+ * it must be called immediately after entering termcap mode.
+ */
+ void
+ may_req_ambiguous_character_width()
+ {
+ if (u7_status == U7_GET
+ && cur_tmode == TMODE_RAW
+ && termcap_active
+ && p_ek
+ # ifdef UNIX
+ && isatty(1)
+ && isatty(read_cmd_fd)
+ # endif
+ && *T_U7 != NUL
+ && !option_was_set((char_u *)"ambiwidth"))
+ {
+ char_u buf[16];
+
+ term_windgoto(0, 0);
+ buf[mb_char2bytes(0x25bd, buf)] = 0;
+ out_str(buf);
+ out_str(T_U7);
+ u7_status = U7_SENT;
+ term_windgoto(0, 0);
+ out_str((char_u *)" ");
+ term_windgoto(0, 0);
+ /* check for the characters now, otherwise they might be eaten by
+ * get_keystroke() */
+ out_flush();
+ (void)vpeekc_nomap();
+ }
+ }
+ # endif
#endif

/*
***************
*** 4049,4061 ****
/* URXVT mouse uses <ESC>[#;#;#M, but we are matching <ESC>[ */
|| key_name[0] == KS_URXVT_MOUSE)
{
! /* Check for xterm version string: "<Esc>[>{x};{vers};{y}c". Also
! * eat other possible responses to t_RV, rxvt returns
! * "<Esc>[?1;2c". Also accept CSI instead of <Esc>[.
! * mrxvt has been reported to have "+" in the version. Assume
! * the escape sequence ends with a letter or one of "{|}~". */
! if (*T_CRV != NUL && ((tp[0] == ESC && tp[1] == '[' && len >= 3)
! || (tp[0] == CSI && len >= 2)))
{
j = 0;
extra = 0;
--- 4100,4121 ----
/* URXVT mouse uses <ESC>[#;#;#M, but we are matching <ESC>[ */
|| key_name[0] == KS_URXVT_MOUSE)
{
! /* Check for some responses from terminal start with "<Esc>[" or
! * CSI.
! *
! * - xterm version string: <Esc>[>{x};{vers};{y}c
! * Also eat other possible responses to t_RV, rxvt returns
! * "<Esc>[?1;2c". Also accept CSI instead of <Esc>[.
! * mrxvt has been reported to have "+" in the version. Assume
! * the escape sequence ends with a letter or one of "{|}~".
! *
! * - cursor position report: <Esc>[{row};{col}R
! * The final byte is 'R'. now it is only used for checking for
! * ambiguous-width character state.
! */
! if ((*T_CRV != NUL || *T_U7 != NUL)
! && ((tp[0] == ESC && tp[1] == '[' && len >= 3)
! || (tp[0] == CSI && len >= 2)))
{
j = 0;
extra = 0;
***************
*** 4067,4074 ****
if (i == len)
return -1; /* not enough characters */

/* eat it when at least one digit and ending in 'c' */
! if (i > 2 + (tp[0] != CSI) && tp[i] == 'c')
{
crv_status = CRV_GOT;

--- 4127,4153 ----
if (i == len)
return -1; /* not enough characters */

+ #ifdef FEAT_MBYTE
+ /* eat it when it has 2 arguments and ends in 'R' */
+ if (u7_status == U7_SENT && j == 1 && tp[i] == 'R')
+ {
+ char *p = NULL;
+
+ u7_status = U7_GOT;
+ if (extra == 2)
+ p = "single";
+ else if (extra == 3)
+ p = "double";
+ if (p != NULL)
+ set_option_value((char_u *)"ambw", 0L, (char_u *)p, 0);
+ key_name[0] = (int)KS_EXTRA;
+ key_name[1] = (int)KE_IGNORE;
+ slen = i + 1;
+ }
+ else
+ #endif
/* eat it when at least one digit and ending in 'c' */
! if (*T_CRV != NUL && i > 2 + (tp[0] != CSI) && tp[i] == 'c')
{
crv_status = CRV_GOT;

*** ../vim-7.3.858/src/term.h 2010-08-15 21:57:25.000000000 +0200
--- src/term.h 2013-03-13 19:09:03.000000000 +0100
***************
*** 83,92 ****
#ifdef FEAT_VERTSPLIT
KS_CSV, /* scroll region vertical */
#endif
! KS_OP /* original color pair */
};

! #define KS_LAST KS_OP

/*
* the terminal capabilities are stored in this array
--- 83,93 ----
#ifdef FEAT_VERTSPLIT
KS_CSV, /* scroll region vertical */
#endif
! KS_OP, /* original color pair */
! KS_U7 /* request cursor position */
};

! #define KS_LAST KS_U7

/*
* the terminal capabilities are stored in this array
***************
*** 158,163 ****
--- 159,165 ----
#define T_CEI (term_str(KS_CEI)) /* end insert mode */
#define T_CRV (term_str(KS_CRV)) /* request version string */
#define T_OP (term_str(KS_OP)) /* original color pair */
+ #define T_U7 (term_str(KS_U7)) /* request cursor position */

#define TMODE_COOK 0 /* terminal mode for external cmds and Ex mode */
#define TMODE_SLEEP 1 /* terminal mode for sleeping (cooked but no echo) */
*** ../vim-7.3.858/src/proto/term.pro 2012-02-05 22:05:44.000000000 +0100
--- src/proto/term.pro 2013-03-13 19:09:54.000000000 +0100
***************
*** 34,39 ****
--- 34,40 ----
void starttermcap __ARGS((void));
void stoptermcap __ARGS((void));
void may_req_termresponse __ARGS((void));
+ void may_req_ambiguous_character_width __ARGS((void));
int swapping_screen __ARGS((void));
void setmouse __ARGS((void));
int mouse_has __ARGS((int c));
*** ../vim-7.3.858/src/version.c 2013-03-13 19:02:37.000000000 +0100
--- src/version.c 2013-03-13 19:27:31.000000000 +0100
***************
*** 730,731 ****
--- 730,733 ----
{ /* Add new patch number below this line */
+ /**/
+ 859,
/**/

--
Microsoft is to software what McDonalds is to gourmet cooking

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[Hayaki Saito]

Hi Bram,

Some people report that this patch does not work well and causes wrong behavior in TERM=ansi environment.

The problem reproduces with the following shell command:

$ TERM=ansi vim -u NONE -N

Launching Vim with this option, The mode goes to REPLACE(not Normal) at the start-up time.
I think the patch#859 should be reverted until it come to include the workaround for this problem.

I confirm this problem on the following environments:
OSX Terminal.app
OSX iTerm2
OSX xterm
Ubuntu 12.10 gnome-terminal
Ubuntu 12.10 konsole
Ubuntu 12.10 xterm

Thanks,
Hayaki Saito

> --
> --
> You received this message from the "vim_dev" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups "vim_dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

[Hayaki Saito]

Hi, Bram
I wrote a quick fix patch for this.
I guess this patch does not fix the fundamental problem.
In $TERM=ansi environment, the incomplete sequence "ESC [" seems to be registered at termcode map.
Regards,
Hayaki Saito

[Bram Moolenaar]

Hayaki Saito wrote:

> Hi, Bram
> I wrote a quick fix patch for this.
> I guess this patch does not fix the fundamental problem.

> In $TERM=3Dansi environment, the incomplete sequence "ESC [" seems to be =
> registered at termcode map.

Thanks for the fix, I'll include it.

Hopefully you can think of a way to make this more robust.

--
"I can't complain, but sometimes I still do." (Joe Walsh)

[Gary Johnson]

On 2013-03-16, Bram Moolenaar wrote:
> Hayaki Saito wrote:
>
> > Hi, Bram
> > I wrote a quick fix patch for this.
> > I guess this patch does not fix the fundamental problem.
> > In $TERM=3Dansi environment, the incomplete sequence "ESC [" seems to be =
> > registered at termcode map.
>
> Thanks for the fix, I'll include it.
>
> Hopefully you can think of a way to make this more robust.

I discovered another problem with patch 7.3.859: when the
CursorHold event is triggered shortly after startup, the intro
message disappears.

To demonstrate this, start vim like this:

vim -N -u NONE --cmd 'autocmd CursorHold * if 0 | endif'

In four seconds ('updatetime') the intro message will disappear.

The "if 0 | endif" is just a command I chose that would have no
effect on the intro message by itself.

I noticed this behavior when I updated one of my vim installations
from 7.3.646 to the latest, 7.3.874, to test Christian Brabandt's
cedit patch. Shortly after starting the new version, the intro
message disappeared. I tracked the problem down to the CursorHold
autocommand in one of my plugins, but I've been using that plugin
for years without this behavior. I then used the hg bisect command
to determine the patch that introduced the behavior.

$ hg bisect -g
The first bad revision is:
changeset: 4215:ecf21be84def
tag: v7-3-859
user: Bram Moolenaar <br...@vim.org>
date: Wed Mar 13 19:29:28 2013 +0100
summary: updated for version 7.3.859

I looked briefly at the changeset (hg diff -c 4215) but nothing
jumped out at me as being the cause.

Regards,
Gary

[Hayaki Saito]

> To demonstrate this, start vim like this:
>
> vim -N -u NONE --cmd 'autocmd CursorHold * if 0 | endif'
>
> In four seconds ('updatetime') the intro message will disappear.

Thanks for your report.
I also reproduced this problem. This patch may fix it.

Regards,
Hayaki Saito

[Hayaki Saito]

Sorry, the patch I sent last time is wrong formatted.
I fix and resend it.

[Bram Moolenaar]

Thanks.

Do we also need this when setting crv_status to CRV_GOT?


--
From "know your smileys":
:^[/ mean-smiley-with-cigarette

[Bram Moolenaar]

Gary Johnson wrote:

> On 2013-03-16, Bram Moolenaar wrote:
> > Hayaki Saito wrote:
> >
> > > Hi, Bram
> > > I wrote a quick fix patch for this.
> > > I guess this patch does not fix the fundamental problem.

> > > In $TERM=ansi environment, the incomplete sequence "ESC [" seems to be > > registered at termcode map.

The intro message disappearing because Vim is doing something is not all
that bad. The intro message should not get in the say of anything.
For me it often isn't there because of the termresponse changing the
number of colors, which triggers a redraw.

Anyway, if you do find a way to keep the intro message, that's nice.

--
hundred-and-one symptoms of being an internet addict:
87. Everyone you know asks why your phone line is always busy ...and
you tell them to send an e-mail.

[Christian Brabandt]

Hi Hayaki!

On Mi, 20 M�r 2013, Hayaki Saito wrote:

> > To demonstrate this, start vim like this:
> >
> > vim -N -u NONE --cmd 'autocmd CursorHold * if 0 | endif'
> >
> > In four seconds ('updatetime') the intro message will disappear.
>
> Thanks for your report.
> I also reproduced this problem.

I also saw this problem. But it isn't really reproducable (at least not
when running under gdb or valgrind or under tmux.) I just noticed, that
resetting must_redraw didn't work for any reason, it seems to stay at a
value of CLEAR so after the cursorhold autocommand fires, this triggers
a redraw.

> This patch may fix it.

Unless I am missing something, this prevents the cursorhold command to
be triggered after startup. I am not sure, this is right.

Mit freundlichen Gr��en
Christian
--
Alle Menschen haben die Anlage, sch�pferisch t�tig zu sein. Nur merken
es die meisten nie.
-- Truman Capote

[Christian Brabandt]

Hi Bram!

On Mi, 13 M�r 2013, Bram Moolenaar wrote:

>
> Patch 7.3.859
> Problem: 'ambiwidth' must be set by the user.
> Solution: Detects East Asian ambiguous width (UAX #11) state of the terminal
> at the start-up time and 'ambiwidth' accordingly. (Hayaki Saito)
> Files: src/main.c, src/option.c, src/term.c, src/term.h,
> src/proto/term.pro

There is another problem with this patch:

term.c: In function 'check_termcode':
term.c:4137:13: warning: declaration of 'p' shadows a previous local [-Wshadow]
term.c:3875:13: warning: shadowed declaration is here [-Wshadow]


diff --git a/src/term.c b/src/term.c
--- a/src/term.c
+++ b/src/term.c
@@ -4134,15 +4134,15 @@

/* eat it when it has 2 arguments and ends in 'R' */

if (j == 1 && tp[i] == 'R')
{
- char *p = NULL;
+ char *q = NULL;

u7_status = U7_GOT;
if (extra == 2)
- p = "single";
+ q = "single";

else if (extra == 3)

- p = "double";
- if (p != NULL)
- set_option_value((char_u *)"ambw", 0L, (char_u *)p, 0);
+ q = "double";
+ if (q != NULL)
+ set_option_value((char_u *)"ambw", 0L, (char_u *)q, 0);
key_name[0] = (int)KS_EXTRA;
key_name[1] = (int)KE_IGNORE;

slen = i + 1;

Mit freundlichen Gr��en
Christian
--

I�t der Bauer Stoppelr�ben, kommt die Bl�hung dann in Sch�ben.

[Bram Moolenaar]

Christian Brabandt wrote:

> Hi Bram!
>
> On Mi, 13 M�r 2013, Bram Moolenaar wrote:
>
> >
> > Patch 7.3.859
> > Problem: 'ambiwidth' must be set by the user.
> > Solution: Detects East Asian ambiguous width (UAX #11) state of the terminal
> > at the start-up time and 'ambiwidth' accordingly. (Hayaki Saito)
> > Files: src/main.c, src/option.c, src/term.c, src/term.h,
> > src/proto/term.pro
>
> There is another problem with this patch:
>
> term.c: In function 'check_termcode':
> term.c:4137:13: warning: declaration of 'p' shadows a previous local [-Wshadow]
> term.c:3875:13: warning: shadowed declaration is here [-Wshadow]

Thanks! Somehow my compiler didn't complain.

--
From "know your smileys":

:~) A man with a tape recorder up his nose

[Tony Mechelynck]

On 20/03/13 21:23, Bram Moolenaar wrote:
[...]

> Anyway, if you do find a way to keep the intro message, that's nice.
>

I'm attaching a small global plugin which I wrote in 2009 for just that
purpose. Its name intentionally starts with zz in the hope that it will
run last; for that same reason it should be dropped in your user's
after-plugin directory: ~/.vim/after/plugin/ on *x, or
~/vimfiles/after/plugin/ on M$W.

I've tried to make that plugin as general as I could, and compatible
even with obsolete or feature-poor versions of Vim (e.g., if used in a
version so old that the :intro command wasn't even defined, it should
just do nothing.) I don't guarantee that it is perfect. If you can make
it better, go ahead.


Best regards,
Tony.
--
A remarkable race are the Persians;
They have such peculiar diversions.
They make love the whole day
In the usual way
And save up the nights for perversions.

[Hayaki Saito]

Hi, Bram

> Do we also need this when setting crv_status to CRV_GOT?

I guess it is necessary to do so.
In my environment, did_cursorhold was FALSE when setting crv_status to CRV_GOT.

Regards,
Hayaki Saito