Fwd: Mouse reporting and new standards
George Nachman
- implement xterm's horribly broken 1005 mouse reporting mode,
and an alternative 1015 mode that works in non-utf-8 locales
and has fewer limitations.
Thanks for your help,
Bram Moolenaar
George Nachman wrote:
There are no termcap entries about the mouse handling. This is hard
coded in the Vim sources. Have a look at src/term.c, search for
mouse_col.
I thought there was a todo entry to support more columns for xterm, but
I can't find it now.
If you can make a patch, that would be welcome. Do try to make it work
for everybody. If it's a mode switch then that is very tricky, because
it has to be restored when Vim exits (and when it crashes...?).
Everything depends on the name of the terminal, but that often makes
users set $TERM to xterm even though it's not actually an xterm and then
later something causes a problem. Perhaps we need a separate option for
this. But hopefully automatic detection works.
--
hundred-and-one symptoms of being an internet addict:
226. You sit down at the computer right after dinner and your spouse
says "See you in the morning."
/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
egmont
A lot happened since you had this discussion about a year ago. Vim has
added support for the urxvt mouse extension, and so did a couple of terminal
emulators (xterm, vte/gnome-terminal, George's iterm2, urxvt of course;
konsole is hopefully the next to join the line).
Unfortunately, this feature (or I'd rather call it bugfix) won't be
available for many users. First, vim has to be configured
--with-features=big which is not the default, so many distributions or users
might miss it. Second, vimrc needs to be updated to contain :set
ttymouse=urxvt, yet again something that's easy to miss. Third, this
ttymouse=urxvt setting is brittle; it breaks mouse support if your terminal
doesn't support the extension, so your vimrc becomes unportable across
various terminal emulators.
Bram, you said you were in favor of automatic detection. Luckily, the urxvt
extension allows this. The escape that switches to the new mode is
recognized by terminals that support it, and is ignored by others. I'm not
aware of any terminal that would misbehave seeing such an unrecognized
escape. The old and new style mouse coordinates, as reported by the
terminal, are distinguishable from each other, so an application doesn't
need to know whether the terminal supports the extension: it just asks the
terminal to switch it on, and then receives mouse events in either the old
or the new format.
In order for clicks beyond 223 to become a mainstream fix that works for all
users, rather than an extra bonus feature for a subset of power users, I
recommend doing the following:
- Make the feature compile by default whenever mouse support is compiled;
- Get rid of ttymouse=urxvt, put the feature of urxvt mouse coordinate
parsing into ttymouse=xterm;
- In case of ttymouse=xterm, make vim recognize both the old standard and
the new urxvt coordinates at the same time.
This way all vim users who have mouse support would automatically get
support for extended coordinates, they wouldn't have to modify their vimrc,
and the feature would be autodetected: it would work on terminals that
support it, while mouse would still work up to col/row 223 on terminals that
don't support the extension.
The same idea is already implemented in Midnight Commander 4.8.1, for
further technical discussions see http://midnight-commander.org/ticket/2662
.
What do you guys think? Should we make these changes so that this fix works
out of the box?
thanks,
egmont
--
View this message in context: http://vim.1045645.n5.nabble.com/Fwd-Mouse-reporting-and-new-standards-tp3378370p5161694.html
Sent from the Vim - General mailing list archive at Nabble.com.
Bram Moolenaar
Egmont wrote:
> Hi Bram, George, others...
>
> A lot happened since you had this discussion about a year ago. Vim has
> added support for the urxvt mouse extension, and so did a couple of terminal
> emulators (xterm, vte/gnome-terminal, George's iterm2, urxvt of course;
> konsole is hopefully the next to join the line).
>
> Unfortunately, this feature (or I'd rather call it bugfix) won't be
> available for many users. First, vim has to be configured
> --with-features=big which is not the default, so many distributions or users
> might miss it. Second, vimrc needs to be updated to contain :set
> ttymouse=urxvt, yet again something that's easy to miss. Third, this
> ttymouse=urxvt setting is brittle; it breaks mouse support if your terminal
> doesn't support the extension, so your vimrc becomes unportable across
> various terminal emulators.
>
> Bram, you said you were in favor of automatic detection. Luckily, the urxvt
> extension allows this. The escape that switches to the new mode is
> recognized by terminals that support it, and is ignored by others. I'm not
> aware of any terminal that would misbehave seeing such an unrecognized
> escape. The old and new style mouse coordinates, as reported by the
> terminal, are distinguishable from each other, so an application doesn't
> need to know whether the terminal supports the extension: it just asks the
> terminal to switch it on, and then receives mouse events in either the old
> or the new format.
Yes, I like this to work automatically. Please fill in the details:
- What is the code to switch it on?
- What is the format of mouse clicks when it is on and off (like urxvt
and xterm I suppose)
- How do we recognize both at the same time?
> In order for clicks beyond 223 to become a mainstream fix that works for all
> users, rather than an extra bonus feature for a subset of power users, I
> recommend doing the following:
>
> - Make the feature compile by default whenever mouse support is compiled;
> - Get rid of ttymouse=urxvt, put the feature of urxvt mouse coordinate
> parsing into ttymouse=xterm;
> - In case of ttymouse=xterm, make vim recognize both the old standard and
> the new urxvt coordinates at the same time.
I would like to see how this works.
> This way all vim users who have mouse support would automatically get
> support for extended coordinates, they wouldn't have to modify their vimrc,
> and the feature would be autodetected: it would work on terminals that
> support it, while mouse would still work up to col/row 223 on terminals that
> don't support the extension.
>
> The same idea is already implemented in Midnight Commander 4.8.1, for
> further technical discussions see http://midnight-commander.org/ticket/2662
> .
>
> What do you guys think? Should we make these changes so that this fix works
> out of the box?
Sounds good, but we need the details to decide if it's going to work
reliably.
BTW: Looking at the urxvt mouse detection code, it looks like there is
danger of getting stuck on a broken input sequence, when a semicolon is
missing it does "return -1", which means it gets more characters. If
the semicolon is any other non-digit character we loop forever.
--
Mrs Abbott: I'm a paediatrician.
Basil: Feet?
Mrs Abbott: Children.
Sybil: Oh, Basil!
Basil: Well, children have feet, don't they? That's how they move
around, my dear. You must take a look next time, it's most
interesting. (Fawlty Towers)
Egmont Koblinger
I think all your technical questions are answered in the first comment of http://www.midnight-commander.org/ticket/2662 -- please see that, and of course feel free to ask if you have any more questions.
thanks,
egmont
Bram Moolenaar
Egmont Koblinger wrote:
> I think all your technical questions are answered in the first comment of
> http://www.midnight-commander.org/ticket/2662 -- please see that, and of
> course feel free to ask if you have any more questions.
Thanks, that helps. However, it appears that the "second new extension"
is not actually supported by xterm yet. Did Thomas Dickey say something
about this?
> > BTW: Looking at the urxvt mouse detection code, it looks like there is
> > danger of getting stuck on a broken input sequence, when a semicolon is
> > missing it does "return -1", which means it gets more characters. If
> > the semicolon is any other non-digit character we loop forever.
This still holds. The mouse code should be recognized by the trailing
'M', not by the sequence starting with <Esc>[, most sequences start with
that.
--
Eye have a spelling checker, it came with my PC;
It plainly marks four my revue mistakes I cannot sea.
I've run this poem threw it, I'm sure your please to no,
It's letter perfect in it's weigh, my checker tolled me sew!
Egmont Koblinger
BTW: Looking at the urxvt mouse detection code, it looks like there is
danger of getting stuck on a broken input sequence, when a semicolon is
missing it does "return -1", which means it gets more characters. If
the semicolon is any other non-digit character we loop forever.
Yes that looks fishy. It should proceed in case of semicolon (or M), return -1 on NUL, and return 0 otherwise.
I'm also not sure what the rest of the code (LL 4274-4295) does, why it needs to find that 'M' once again and parse the mouse code again. But maybe it's just me.
e.
Egmont Koblinger
Complicated, not sure how much it was "designed to work" or just "happens to work" ;)
e.
dickey
> Egmont Koblinger wrote:
> > I think all your technical questions are answered in the first comment of
>
> Thanks, that helps. However, it appears that the "second new extension"
> is not actually supported by xterm yet. Did Thomas Dickey say something
> about this?
some
technical deficiencies with the 1015 code, and also noted a problem
with urvt's
implementation of 1005 (if the locale encoding isn't UTF-8, it won't
report positions
past 50x95).
I followed up by implementing a 1006 which lacks the defects that I
noted in urxvt's design.
Those points are summarized in the change-log for #277, as well as in
ctlseqs.
http://invisible-island.net/xterm/xterm.log.html#xterm_277
http://invisible-island.net/xterm/ctlseqs/ctlseqs.html
(Actually I made these changes at the beginning of December, but other
work
got in the way of a more rapid update- hence the late release for
vttest to
demonstrate the feature).
dickey
> I followed up by implementing a 1006 which lacks the defects that I
> noted in urxvt's design.
mainly to humor Egmont. I don't recommend using 1015,
because it opens up the possibility of confusing its responses
with other control sequences for deleting lines and scrolling.
egmont
I saw you applied the 1015 patch but I didn't see you coming up with 1006. I'm looking at it right now.
Could you please tell whether "^[[<" is a unique prefix that is used only for mouse reporting, and hence it can even be put in terminfo descriptions next to "^[[M"? If it's the case then I'm super glad. If not, I'll propose coming up with a 4th (arghhh) extension...
The story is: I tried to patch support to some applications. MC and Joe already done; I was studying someone else's patch to Vim; and I also looked at Emacs. My experience is:
Most of these applications only handle incoming escapes that don't have any parameters, mouse being the only exception. Their parsing code looks up a table or tree to recognize the escape and then they immediately know what escape they're facing. This table or tree contains "^[[M" for mouse. If this one is found, the escape lookup part has already done its job and the code responsible for handling mouse events simply reads three more bytes.
Current urxvt extension needs a completely different parsing logic, which means that for both MC and Joe my patch is extremely ugly: it handles 1015 mode as a special case.
For emacs I couldn't solve the problem (I could solve it with utterly ugly hacks), since the first part (recognizing the common mouse prefix) happens in C, while the second part (reading the coordinates and taking action) are in Lisp. For implementing the 1015 mode I'd have to cross this boundary in terrible ways.
So the conclusion is: A unique prefix that's used solely for reporting extended mouse coordinates makes it very easy to add support to existing applications. The lack thereof might make it extremely hard.
I'd like to see an extension that fulfills this criterium, and I would also prefer rxvt-unicode's developer to say OK to this (I don't want him to say it's broken and come up with yet another one...) If these two are done, I'd be happy to duplicate all my work so far, and push terminal emulators and applications towards implementing 1006.
thanks a lot,
egmont
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
http://vim.1045645.n5.nabble.com/Fwd-Mouse-reporting-and-new-standards-tp3378370p5163573.htmlIf you reply to this email, your message will be added to the discussion below:
View this message in context: Re: Fwd: Mouse reporting and new standards
Bram Moolenaar
Thomas Dickey wrote:
I was searching for "urxvt" in the version logs, but it turns out it is
called SGR 1006. That indeed looks like a more compatible mechanism.
So, if we first send DECSET 1005 and then send DECSET 1006 then,
depending on the version of the terminal emulator, we keep getting the
normal mouse codes or the new SGR 1006 mouse codes. We should be able
to decode both.
Note that in ctlseqs.html in the DECSET list 1006 and 1015 are not
documented.
--
CRONE: Who sent you?
ARTHUR: The Knights Who Say GNU!
CRONE: Aaaagh! (she looks around in rear) No! We have no licenses here.
"Monty Python and the Holy editor wars" PYTHON (MONTY) SOFTWARE LTD
Egmont Koblinger
So, if we first send DECSET 1005 and then send DECSET 1006 then,
depending on the version of the terminal emulator, we keep getting the
normal mouse codes or the new SGR 1006 mouse codes. We should be able
to decode both.
Bram, I guess you meant 1000 or 1002 instead of 1005 above. In that case you're right.
I'll push other terminal emulators towards supporting 1006. First, however, I'd like to hear Thomas explicitly confirming that ^[[< is (and will always remain) a unique prefix for extended mouse events only, and then hear an "okay" from urxvt's developer (I'll talk to him).
Once this is done, it'll be okay to add 1006 support to Vim. (In the mean time I guess 1015 support could be dropped; this would make extended mouse coords stop working in some versions of some terminals, but would probably also simplify Vim's code. But it's okay to keep 1015 too.)
thanks,
egmont
Note that in ctlseqs.html in the DECSET list 1006 and 1015 are not
documented.
--
CRONE: Who sent you?
ARTHUR: The Knights Who Say GNU!
CRONE: Aaaagh! (she looks around in rear) No! We have no licenses here.
"Monty Python and the Holy editor wars" PYTHON (MONTY) SOFTWARE LTD
/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
Thomas Dickey
>
> Thomas Dickey wrote:
>
> > > Egmont Koblinger wrote:
> > > > I think all your technical questions are answered in the first comment of
> > > >http://www.midnight-commander.org/ticket/2662-- please see that, and of
> > > > course feel free to ask if you have any more questions.
> > >
> > > is not actually supported by xterm yet. ?Did Thomas Dickey say something
> > > about this?
> >
> > yes - in discussing this with Egmont a few months ago, I pointed out
> > some technical deficiencies with the 1015 code, and also noted a
> > problem with urvt's implementation of 1005 (if the locale encoding
> > isn't UTF-8, it won't report positions past 50x95).
> >
> > I followed up by implementing a 1006 which lacks the defects that I
> > noted in urxvt's design. Those points are summarized in the
> > change-log for #277, as well as in ctlseqs.
> > http://invisible-island.net/xterm/xterm.log.html#xterm_277
> > http://invisible-island.net/xterm/ctlseqs/ctlseqs.html
> >
> > (Actually I made these changes at the beginning of December, but
> > other work got in the way of a more rapid update- hence the late
> > release for vttest to demonstrate the feature).
>
> I was searching for "urxvt" in the version logs, but it turns out it is
> called SGR 1006. That indeed looks like a more compatible mechanism.
This is the relevant section (more recent changes generally at the top):
* add SGR 1006, as a better technical solution than SGR 1015:
+ the responses will not be confused with line-deletion and
scrolling controls.
+ the button encoding is a little simpler, since it does not
add an unnecessary 32 because the integer parameter does not
have to be represented as a printable character.
+ the control responses for pressing and releasing a mouse
button differ, allowing an application to tell which button
was released.
Besides these improvements, in discussion, it was noted that
urxvt's implementation of 1005 is incorrect, relying upon a locale
that provides UTF-8 encoding. In contrast, vttest demonstrates a
correct decoding, independent of locale.
* add support for urxvt SGR 1015 to address shortcoming of SGR 1005
with luit (patch by Egmont Koblinger).
> So, if we first send DECSET 1005 and then send DECSET 1006 then,
> depending on the version of the terminal emulator, we keep getting the
> normal mouse codes or the new SGR 1006 mouse codes. We should be able
> to decode both.
That sounds right (though 1005 is not completely functional in urxvt,
as I noted).
> Note that in ctlseqs.html in the DECSET list 1006 and 1015 are not
> documented.
yes... I made a note about that earlier, but didn't remember to fix it
when closing out #277:
120108
forgot to document SGR 1005 and 1015 in the list of control sequences.
They _are_ documented in the mouse-specific stuff.
The #278 changes were closed out quickly because of the FreeBSD bug report.
At the moment, I've one "new" report which turns out to be at least a
couple of years old.
--
Thomas E. Dickey <dic...@invisible-island.net>
http://invisible-island.net
ftp://invisible-island.net
Thomas Dickey
> Hi,
>
> On Sun, Jan 22, 2012 at 14:31, Bram Moolenaar <Br...@moolenaar.net> wrote:
>
> >
> >
> > So, if we first send DECSET 1005 and then send DECSET 1006 then,
> > depending on the version of the terminal emulator, we keep getting the
> > normal mouse codes or the new SGR 1006 mouse codes. We should be able
> > to decode both.
> >
>
> Bram, I guess you meant 1000 or 1002 instead of 1005 above. In that case
> you're right.
>
> I'll push other terminal emulators towards supporting 1006. First,
> however, I'd like to hear Thomas explicitly confirming that ^[[< is (and
> will always remain) a unique prefix for extended mouse events only, and
> then hear an "okay" from urxvt's developer (I'll talk to him).
Actually it's the combination of \e[< and the final character that makes
the string unique. There are four characters which can be used to denote
a private mode: "?", "=", ">" and "<" (see ECMA-48 section 5.4).
The first three are used in various ways in different terminals. The last
seems to be rarely used, so I chose to use that.
Conceivably some other escape sequence could be implemented (I don't have
any plans for that). I'm assuming that vim will check the final character.
> Once this is done, it'll be okay to add 1006 support to Vim. (In the mean
> time I guess 1015 support could be dropped; this would make extended mouse
> coords stop working in some versions of some terminals, but would probably
> also simplify Vim's code. But it's okay to keep 1015 too.)
>
> thanks,
> egmont
>
>
>
> >
> > Note that in ctlseqs.html in the DECSET list 1006 and 1015 are not
> > documented.
> >
> > --
> > CRONE: Who sent you?
> > ARTHUR: The Knights Who Say GNU!
> > CRONE: Aaaagh! (she looks around in rear) No! We have no licenses here.
> > "Monty Python and the Holy editor wars" PYTHON (MONTY) SOFTWARE
> > LTD
> >
> > /// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net
> > \\\
> > /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/\\\
> > \\\ an exciting new programming language -- http://www.Zimbu.org
> > ///
> > \\\ help me help AIDS victims -- http://ICCF-Holland.org
> > ///
> >
> > --
> > You received this message from the "vim_use" maillist.
> > Do not top-post! Type your reply below the text you are replying to.
> > For more information, visit http://www.vim.org/maillist.php
> >
--
dickey
> Hi Thomas,
>
> I saw you applied the 1015 patch but I didn't see you coming up with 1006.
> I'm looking at it right now.
extend,
and then implemented 1006. I also modified vttest so I could see it
run.
(xterm uses 1006 for the X10 and highlighting modes as well). When
I'd
done that, I satisfied my curiousity regarding urxvt 1015 in a non-
UTF-8
locale, and saw that it indeed did not work properly. (I agree that
making
it work in that case would have been more than a line of code).
Prior to your comment about luit, no one had pointed out a technical
issue with 1005 (and modifying luit to work around 1005 would be
time not well spent).
> Could you please tell whether "^[[<" is a unique prefix that is used only
> for mouse reporting, and hence it can even be put in terminfo descriptions
> next to "^[[M"? If it's the case then I'm super glad. If not, I'll
> propose coming up with a 4th (arghhh) extension...
ncurses parse a
string, and use that for handling the different flavors). I was sort
of considering
that after 1005, but it wasn't urgent (and there's always something to
fix in
ncurses).
> The story is: I tried to patch support to some applications. MC and Joe
> already done; I was studying someone else's patch to Vim; and I also looked
> at Emacs. My experience is:
>
> Most of these applications only handle incoming escapes that don't have any
> parameters, mouse being the only exception. Their parsing code looks up a
> table or tree to recognize the escape and then they immediately know what
> escape they're facing. This table or tree contains "^[[M" for mouse. If
> this one is found, the escape lookup part has already done its job and the
> code responsible for handling mouse events simply reads three more bytes.
with this
general issue. I recall advising someone that it would be useful to
see CSI come
in (escape+"[" for instance) and collect the characters that could
comprise a
control sequence formatted according to the ANSI/ISO/ECMA rules.
Timeouts
or mismatches would handle things like a single escape character.
But I agree that modifying existing code can be hard.
> Current urxvt extension needs a completely different parsing logic, which
> means that for both MC and Joe my patch is extremely ugly: it handles 1015
> mode as a special case.
>
> For emacs I couldn't solve the problem (I could solve it with utterly ugly
> hacks), since the first part (recognizing the common mouse prefix) happens
> in C, while the second part (reading the coordinates and taking action) are
> in Lisp. For implementing the 1015 mode I'd have to cross this boundary in
> terrible ways.
>
> So the conclusion is: A unique prefix that's used solely for reporting
> extended mouse coordinates makes it very easy to add support to existing
> applications. The lack thereof might make it extremely hard.
this.
There could be responses to control strings from an application (at
some point),
but vim's unlikely to send those control strings without some
discussion and
modification.
> I'd like to see an extension that fulfills this criterium, and I would also
> prefer rxvt-unicode's developer to say OK to this (I don't want him to say
> it's broken and come up with yet another one...) If these two are done,
> I'd be happy to duplicate all my work so far, and push terminal emulators
> and applications towards implementing 1006.
> thanks a lot,
> egmont
>
> On Sun, Jan 22, 2012 at 02:57, Thomas Dickey [via VIM] <
>
>
>
>
>
>
>
>
>
> > On Jan 21, 11:19 am, Bram Moolenaar <[hidden email]<http://user/SendEmail.jtp?type=node&node=5163573&i=0>>
> > > Egmont Koblinger wrote:
> > > > I think all your technical questions are answered in the first comment
> > of
> > below:
>
> > To unsubscribe from Fwd: Mouse reporting and new standards, click here<http://vim.1045645.n5.nabble.com/template/NamlServlet.jtp?macro=unsub...>
> > .
> > NAML<http://vim.1045645.n5.nabble.com/template/NamlServlet.jtp?macro=macro...>
>
> --
> View this message in context:http://vim.1045645.n5.nabble.com/Fwd-Mouse-reporting-and-new-standard...
Egmont Koblinger
Actually it's the combination of \e[< and the final character that makes
the string unique. There are four characters which can be used to denote
a private mode: "?", "=", ">" and "<" (see ECMA-48 section 5.4).
I'm sad to hear this. Could we please then change the 1006 mode now, before it's widely deployed?
The problem, as I said, is that if there might be other escapes starting with \e[< then patching support into existing applications (most notably Emacs) becomes a nightmare (but see my patch for mc and joe, they're ugly too). The way currently most applicatoins parse is that there's a point during sequentially reading the bytes when they say "haha, I recognize this unique prefix, it means mouse", and then the mouse handling code processes the subsequent bytes as arguments. If the new escapes don't fit this scheme then application codes need to be modified heavily, in non-trivial ways, e.g. it's almost impossible for Emacs.
I think anything as simple as a fixed letter following the '<' sign, if it's not used anywhere else right now, would do it. Or alternatively \e[M followed by a single fixed letter that is currently out of use would also do it.
Also, I'm not sure about the purpose of "private" mode, but shouldn't we aim to provide something that will quickly become a new standard, not something "private"?
The first three are used in various ways in different terminals. The last
seems to be rarely used, so I chose to use that.
Conceivably some other escape sequence could be implemented (I don't have
any plans for that). I'm assuming that vim will check the final character.
Thanks a lot,
egmont
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iEYEARECAAYFAk8cG2IACgkQcCNT4PfkjtucogCfdZesPECwenLZNHe6xx/oc5s7
tiMAn2n1zVIgYEMMoZyl/RUwWkVJ1gtZ
=DJIQ
-----END PGP SIGNATURE-----
Egmont Koblinger
Prior to your comment about luit, no one had pointed out a technical
issue with 1005 (and modifying luit to work around 1005 would be
time not well spent).
Thomas, 1005 has a much more significant issue than the luit one, as described in the first comment of http://www.midnight-commander.org/ticket/2662 . I guess I've already pointed you to this but you seem to have overlooked. Copy-pasting from there:
- There is no way for an application to tell whether the underlying
terminal supports this new mode (whether DECSET 1005 did actually change
the behavior or not), but depending on this a completely different user
action might generate the same input. Example:
- If the terminal doesn't support this extension, then clicking at (162, 129) generates \e[M<32><194><161>
- If the terminal supports this extension, then clicking at (129, 1) [bit of math: 129+32 = 161, U+0161 in UTF-8 is 194 161] generates \e[M<32><194><161><33>
- so there's no way to tell whether the terminal ignored the 1005 escape sequence, the user clicked on (162, 129) and then typed an exclamation mark; or whether the terminal recognized the escape, and the user clicked on (129, 1).
- Due to this horrible brokenness, there's no way to implement support it without explicitly asking the user (via a setting) if the terminal can speak this extension.
I'm also keen on forgetting 1015, as 1006 is obviously superior to both. However, there's still one drawback of 1006, which is the lack of unique prefix, which makes it hard to patch it to existing application. Although as long as the application is not interested in any other escape beginning with \e[< it's not that bad, definitely better than 1015 ;)
egmont
Thomas Dickey
> On Sun, Jan 22, 2012 at 15:21, Thomas Dickey <dic...@his.com> wrote:
>
> >
> > Actually it's the combination of \e[< and the final character that makes
> > the string unique. There are four characters which can be used to denote
> > a private mode: "?", "=", ">" and "<" (see ECMA-48 section 5.4).
> >
>
> I'm sad to hear this. Could we please then change the 1006 mode now,
> before it's widely deployed?
I don't see a problem with it. I pointed out that it's unlikely that
there will be any confusion, but also pointed out that sometime in the
future the repertoire could be extended.
> The problem, as I said, is that if there might be other escapes starting
> with \e[< then patching support into existing applications (most notably
> Emacs) becomes a nightmare (but see my patch for mc and joe, they're ugly
> too). The way currently most applicatoins parse is that there's a point
> during sequentially reading the bytes when they say "haha, I recognize this
> unique prefix, it means mouse", and then the mouse handling code processes
> the subsequent bytes as arguments. If the new escapes don't fit this
> scheme then application codes need to be modified heavily, in non-trivial
> ways, e.g. it's almost impossible for Emacs.
Well that (a special magic prefix) is the X10 scheme, which
in turn is part of the problem we're discussing:
a) it doesn't follow the same rules as other escape sequences
b) there's no clear way to extend it.
The reader is still going to do a check at the end, and discard unexpected
input as an error. If the inputs follow the same syntax rules, then it's
less problem to implement and maintain.
I'm not expecting someone to just match "\e[<" and blindly sscanf looking
for a given number of integer parameters and then skip the next character
without examining it. 1006 uses the final character to distinguish
button press/release.
(Correctly parsing 1015 also would involve checking the final character,
of course).
> I think anything as simple as a fixed letter following the '<' sign, if
> it's not used anywhere else right now, would do it. Or alternatively \e[M
> followed by a single fixed letter that is currently out of use would also
> do it.
>
> Also, I'm not sure about the purpose of "private" mode, but shouldn't we
> aim to provide something that will quickly become a new standard, not
> something "private"?
"private" in this regard means that it's not part of ECMA-48.
Take a look at the various "CSI ?" and "CSI >" listed in ctlseqs to see
how xterm (and DEC) use private modes. The existing mouse controls are
for instance private modes.
CSI ? 1 0 0 1 h
Ps = 1 0 0 1 -> Use Hilite Mouse Tracking.
Egmont Koblinger
The reader is still going to do a check at the end, and discard unexpected
input as an error. If the inputs follow the same syntax rules, then it's
less problem to implement and maintain.
I'm not expecting someone to just match "\e[<" and blindly sscanf looking
for a given number of integer parameters and then skip the next character
without examining it. 1006 uses the final character to distinguish
button press/release.
Of course one has to check the final character :) And if I was to implement an application from scratch, I'd say that the standard is great, I'd implement a framework for collecting numeric parameters up to the final letter and then parse the whole stuff, it's easy.
Now, however, support should be patched into many already existing applications whose code I don't understand, don't maintain, and are not built along these lines. I'd like to patch quite a few of them, without spending too much time. Substantially rewriting them by me or anyone contributing a patch is not really feasible.
Please take a very quick look at emacs-23-3b. lisp/xt-mouse.el L246 registers the xterm-mouse-translate lisp-method to be called on \e[M. This, in turn, calls xterm-mouse-event-read three times to read the arguments, and then processes them. The input_decode_map is apparently looked up in src/term.c, although I don't understand that code yet. If there's a new unique prefix then that can be registered in the lisp file too, and the whole business of mouse coordinate parsing solved locally in lisp, which I guess even I would be able to do despite knowing nothing about lisp. If there's no unique prefix, i.e. if \e[< can mean mouse coordinate but can mean something else as well, then adding support would require much more complex architecture changes from someone who much more deeply understands emacs' structure.
I'm not asking you to come up with a patch to emacs or anything like that, but please take a very quick look at the source and estimate how much work it'd be to add support in one case or in the other. I hope you'll agree that it's much more work to add proper support if there is no unique prefix.
I personally think it's important that extended mouse support should be easy to add into applications, and should not require architecture redesign. I think it's a key to getting the feature deployed quickly and widely across apps.
thanks,
egmont
iEYEARECAAYFAk8cJDoACgkQcCNT4PfkjttF+ACeP3kEskL7sUExPndJIKSfgIBs
n0gAoJ7VGKBTLJJjIdn40t3hTNw6qeSC
=JCNQ
-----END PGP SIGNATURE-----
Bram Moolenaar
Egmont Koblinger wrote:
> On Sun, Jan 22, 2012 at 15:21, Thomas Dickey <dic...@his.com> wrote:
>
> >
> > Actually it's the combination of \e[< and the final character that makes
> > the string unique. There are four characters which can be used to denote
> > a private mode: "?", "=", ">" and "<" (see ECMA-48 section 5.4).
> >
>
> I'm sad to hear this. Could we please then change the 1006 mode now,
> before it's widely deployed?
>
> The problem, as I said, is that if there might be other escapes starting
> with \e[< then patching support into existing applications (most notably
> Emacs) becomes a nightmare (but see my patch for mc and joe, they're ugly
> too). The way currently most applicatoins parse is that there's a point
> during sequentially reading the bytes when they say "haha, I recognize this
> unique prefix, it means mouse", and then the mouse handling code processes
> the subsequent bytes as arguments. If the new escapes don't fit this
> scheme then application codes need to be modified heavily, in non-trivial
> ways, e.g. it's almost impossible for Emacs.
>
> I think anything as simple as a fixed letter following the '<' sign, if
> it's not used anywhere else right now, would do it. Or alternatively \e[M
> followed by a single fixed letter that is currently out of use would also
> do it.
>
> Also, I'm not sure about the purpose of "private" mode, but shouldn't we
> aim to provide something that will quickly become a new standard, not
> something "private"?
Thomas may say this better, but I know the idea is that the escape
sequence consists of:
A leader, usually CSI and another character. There is list of
known leaders.
A middle part, not containing a terminating character
A terminating character, identifying the meaning.
This allows for skipping codes that are not recognized. Thus the mouse
code *must* be recognized by the leader *and* the terminating character.
The old CSI M code for mouse was a weird exception and we should not go
that way.
Vim doesn't really do this parsing fully, also because there are
exceptions and the user can manually set escape sequences that must be
matched.
--
If you had to identify, in one word, the reason why the
human race has not achieved, and never will achieve, its
full potential, that word would be "meetings."
Thomas Dickey
> Hi,
>
> On Sun, Jan 22, 2012 at 15:59, Thomas Dickey <dic...@his.com> wrote:
>
> >
> > The reader is still going to do a check at the end, and discard unexpected
> > input as an error. If the inputs follow the same syntax rules, then it's
> > less problem to implement and maintain.
> >
> > I'm not expecting someone to just match "\e[<" and blindly sscanf looking
> > for a given number of integer parameters and then skip the next character
> > without examining it. 1006 uses the final character to distinguish
> > button press/release.
> >
>
> Of course one has to check the final character :) And if I was to
> implement an application from scratch, I'd say that the standard is great,
> I'd implement a framework for collecting numeric parameters up to the final
> letter and then parse the whole stuff, it's easy.
>
> Now, however, support should be patched into many already existing
> applications whose code I don't understand, don't maintain, and are not
> built along these lines. I'd like to patch quite a few of them, without
> spending too much time. Substantially rewriting them by me or anyone
> contributing a patch is not really feasible.
>
> Please take a very quick look at emacs-23-3b. lisp/xt-mouse.el L246
> registers the xterm-mouse-translate lisp-method to be called on \e[M.
> This, in turn, calls xterm-mouse-event-read three times to read the
Actually it first calls read-char three times, to read \e[M (three bytes).
It does a little error checking there, e.g.,
(unless is-click
(unless (and (eq (read-char) ?\e)
(eq (read-char) ?\[)
(eq (read-char) ?M))
(error "Unexpected escape sequence from XTerm")))
It then calls xterm-mouse-event-read to read the coordinates.
> arguments, and then processes them. The input_decode_map is apparently
> looked up in src/term.c, although I don't understand that code yet. If
> there's a new unique prefix then that can be registered in the lisp file
I'd assume that one would start by doing a
(define-key input-decode-map "\e[<" 'xterm-extended-mouse-translate))
...along with a suitable new function. It's been a while since I
programmed in lisp, but it's usually readable. Aside from cut/paste/massage
of text already in the file, there's not much to lookup.
> too, and the whole business of mouse coordinate parsing solved locally in
> lisp, which I guess even I would be able to do despite knowing nothing
That's the preferred method for emacs (changes to the C code are done
only to provide ways for the lisp code to do more).
> about lisp. If there's no unique prefix, i.e. if \e[< can mean mouse
> coordinate but can mean something else as well, then adding support would
> require much more complex architecture changes from someone who much more
> deeply understands emacs' structure.
>
> I'm not asking you to come up with a patch to emacs or anything like that,
> but please take a very quick look at the source and estimate how much work
> it'd be to add support in one case or in the other. I hope you'll agree
> that it's much more work to add proper support if there is no unique prefix.
It doesn't have to be unique - the reader simply has to check that the
input is in the expected format.
For someone who knows their way around emacs, this would be a couple of
hours work. If I had to do it, hmm - perhaps a day or two.
> I personally think it's important that extended mouse support should be
> easy to add into applications, and should not require architecture
> redesign. I think it's a key to getting the feature deployed quickly and
> widely across apps.
there are a lot of applications :-(
Egmont Koblinger
I'd assume that one would start by doing a
(define-key input-decode-map "\e[<" 'xterm-extended-mouse-translate))
...along with a suitable new function. It's been a while since I
programmed in lisp, but it's usually readable. Aside from cut/paste/massage
of text already in the file, there's not much to lookup.
It's clear as long as mouse is the only event beginning with \e[< that's processed by Emacs. As soon as there's a second, completely independent escape also beginning with \e[< which another module would like to process, problems begin. Same happens with apparently many other applications.
That being said, we can say that we stick to 1006, apps that are only interested in mouse-related \e[< codes can do the easy way of parsing, and apps that need to check for other \e[< codes need to do refactor more to implement a more correct parser.
egmont
iEYEARECAAYFAk8cqUYACgkQcCNT4Pfkjtt1iQCdE1o40QOZS69dIIJNkVHJLuZZ
B7EAn0Qr1RJaZgvy9uyy/7xJ3j5A76mu
=eQCD
-----END PGP SIGNATURE-----
Thomas Dickey
> On Mon, Jan 23, 2012 at 01:26, Thomas Dickey <dic...@his.com> wrote:
>
> >
> > I'd assume that one would start by doing a
> >
> > (define-key input-decode-map "\e[<"
> > 'xterm-extended-mouse-translate))
> >
> > ...along with a suitable new function. It's been a while since I
> > programmed in lisp, but it's usually readable. Aside from
> > cut/paste/massage
> > of text already in the file, there's not much to lookup.
> >
>
> It's clear as long as mouse is the only event beginning with \e[< that's
> processed by Emacs. As soon as there's a second, completely independent
> escape also beginning with \e[< which another module would like to process,
> problems begin. Same happens with apparently many other applications.
I did mention that there's the potential for confusion already with the
X10 scheme.
> That being said, we can say that we stick to 1006, apps that are only
> interested in mouse-related \e[< codes can do the easy way of parsing, and
> apps that need to check for other \e[< codes need to do refactor more to
> implement a more correct parser.
sounds good.
Egmont Koblinger
Returning to this old thread regarding mouse clicks beyond column 223...
Seems that xterm's 1006 extension is going to be the "winning" one, rather than urxvt's 1015 mode. Almost all terminals that support any mouse coordinate extension do support this extension in their newest released or soon-to-be-released development versions. Probably urxvt is the only exception, but it's always been a bit odd amongs emulators, and I hope the developer will include support one day. This extension is also getting stronger on application side, e.g. Emacs-25 will support it too (only xterm's 1006 mode, not urxvt's 1015).
I see that someone was kind enough to already provide support for 1006 in Vim.
My only pet peeve is that the 1006 mode is not compiled in by default. Trying to think from a user's point of view, this extension is not a feature, it's a bugfix (of a broken protocol). With the default compile options, one can use mouse in the first 223 columns only - I doubt anyone would want this. Having no mouse support at all might make sense, full mouse support sure does make sense, but something in between doesn't for me, still that's what one gets after a simple ./configure && make.
I think it would be great if people who have Vim with mouse support would be able to use it without limits, and without requiring to digging into configure options.
We're very close to that: all it would take is a one-line patch in feature.h, making FEAT_MOUSE_SGR guarded by FEAT_NORMAL.
If you agree that this is a bugfix and not an add-one feature, with minimal fingerprint in code size, and no risk at all (those terminals that don't support this extension will report old-fashioned codes and Vim will understand those too), then please consider making it a "normal" feature. This way most people would have this working automagically.
Thanks a lot,
egmont
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iEYEARECAAYFAk8dL/QACgkQcCNT4PfkjttD2ACg1nFYSgNgEKr6YoIjq/hIofTy
mt0AnRgFD5UN4JHkXCjnOqshDmCF1+W9
=9ETo
-----END PGP SIGNATURE-----
Bram Moolenaar
And the new feature only works with the latest terminal emulators. Thus
it doesn't appear all that important.
It's more a question of whether we want most things to be included in
the normal version. I must say that the huge version isn't much bigger
or slower. So, perhaps we should change it that the huge version is the
default, and those who want a small Vim can build with fewer features.
--
FATHER: Who are you?
PRINCE: I'm ... your son ...
FATHER: Not you.
LAUNCELOT: I'm ... er ... Sir Launcelot, sir.
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD
Egmont Koblinger
I do not think more than a few people run into the 223 column limit.
True, but even for them, I can't see why this shouldn't be fixed by default :)
And the new feature only works with the latest terminal emulators.
I think it's reasonable to assume that on most systems the age of the terminal and the age of vim doesn't differ significantly. Those having the latest-and-greatest vim are probably not stuck with 10 year old emulators. In about a year or two, most terminals on most reasonably up-to-date systems will have this feature.
Thus it doesn't appear all that important.
Well, this is true :) You know my opinion, but after all it's your call.
thanks a lot,
egmont