tigervnc 1.90: numlock state issue with RealVNC viewer?

[robert.r...@gmail.com]

After switching from tigervnc 1.80 server (as distributed with CentOS 7) to tigervnc 1.90 (installed using rpm's from tigervnc.org bintray distribution for RHEL7), we observed some issues when connecting with RealVNC viewer on MacOS. After some amount of time, some window manager shortcuts stop working. Looks like the numlock state in the server gets confused.

This effect was not observed so far in other combinations, i.e.:
- tigervnc 1.90 server --> RealVNC viewer (mac): effect/issue present
- tigervnc 1.80 server --> RealVNC viewer (mac): don't see this issue
- tigervnc 1.90 server --> tigervnc 1.90 viewer (mac): don't see this issue

Reason for using RealVNC viewer in this combo is its better support for MacOS native full-screen (focus switching), still slightly better performance (though tigervnc viewer is starting to get close).

This is all for "virtual desktop" type VNC sessions (not x0vncserver).

Any ideas for workarounds, etc. would be welcome.

Thanks, Robert

Some xev dumps and observations for these cases below:

*************************************************************************************
**** RealVNC Viewer 6.18.625 on MacOS 10.13 ==> TigerVNC 1.90 server (CentOS 7) *****
*************************************************************************************

Note: NumLock key switching behavior e.g. in nedit does not work (doesn't switch behavior).

======== pressing / releasing NumLock key: =========

KeyPress event, serial 25, synthetic NO, window 0x1000001,
root 0x268, subw 0x1000002, time 682557649, (54,43), root:(1000,68),
state 0x0, keycode 9 (keysym 0xff1b, Escape), same_screen YES,
XLookupString gives 1 bytes: (1b) "
mbLookupString gives 1 bytes: (1b) "
FilterEvent returns: False

KeyRelease event, serial 28, synthetic NO, window 0x1000001,
root 0x268, subw 0x1000002, time 682558784, (54,43), root:(1000,68),
state 0x0, keycode 9 (keysym 0xff1b, Escape), same_screen YES,
XLookupString gives 1 bytes: (1b) "
FilterEvent returns: False

--------------------------------------------------------
Initial state, fresh vncserver / client session, all OK:
--------------------------------------------------------

xmodmap: up to 4 keys per modifier, (keycodes in parentheses):

shift Shift_L (0x32), Shift_R (0x3e)
lock Caps_Lock (0x42)
control Control_L (0x25), Control_R (0x69)
mod1 Alt_L (0x40), Alt_R (0x6c), Meta_L (0xcd)
mod2 Num_Lock (0x4d)
mod3
mod4 Super_L (0x85), Super_R (0x86), Super_L (0xce), Hyper_L (0xcf)
mod5 ISO_Level3_Shift (0x5c), Mode_switch (0xcb)

======== pressing / releasing F5 key: =========

KeyPress event, serial 40, synthetic NO, window 0x1200001,
root 0x268, subw 0x1200002, time 373730236, (47,34), root:(808,313),
state 0x0, keycode 71 (keysym 0xffc2, F5), same_screen YES,
XLookupString gives 0 bytes:
XmbLookupString gives 0 bytes:
XFilterEvent returns: False

KeyRelease event, serial 40, synthetic NO, window 0x1200001,
root 0x268, subw 0x1200002, time 373730310, (47,34), root:(808,313),
state 0x0, keycode 71 (keysym 0xffc2, F5), same_screen YES,
XLookupString gives 0 bytes:
XFilterEvent returns: False

--------------------------------------------------------------------
... after some time: some shortcuts stop working. xev at that point:
--------------------------------------------------------------------

======== pressing / releasing F5 key: =========

KeyPress event, serial 40, synthetic NO, window 0x1400001,
root 0x268, subw 0x1400002, time 383479237, (37,39), root:(42,872),
state 0x10, keycode 71 (keysym 0xffc2, F5), same_screen YES,
XLookupString gives 0 bytes:
XmbLookupString gives 0 bytes:
XFilterEvent returns: False

KeyRelease event, serial 40, synthetic NO, window 0x1400001,
root 0x268, subw 0x1400002, time 383479284, (37,39), root:(42,872),
state 0x10, keycode 71 (keysym 0xffc2, F5), same_screen YES,
XLookupString gives 0 bytes:
XFilterEvent returns: False

==> NumLock state stuck ???
( haven't found a way to reset state yet )

**********************************************************************************
**** TigerVNC Viewer 1.90 on MacOS 10.13 ==> TigerVNC 1.90 server (CentOS 7) *****
**********************************************************************************

Note: NumLock key switching behavior e.g. in nedit works as expected.
Note: didn't observe "stuck" NumLock state behavior so far.

======== pressing / releasing NumLock key: =========

KeyPress event, serial 25, synthetic NO, window 0x1000001,
root 0x268, subw 0x1000002, time 682779725, (37,37), root:(983,62),
state 0x10, keycode 77 (keysym 0xff7f, Num_Lock), same_screen YES,
XLookupString gives 0 bytes:
XmbLookupString gives 0 bytes:
XFilterEvent returns: False

KeyRelease event, serial 28, synthetic NO, window 0x1000001,
root 0x268, subw 0x1000002, time 682780213, (37,37), root:(983,62),
state 0x10, keycode 77 (keysym 0xff7f, Num_Lock), same_screen YES,
XLookupString gives 0 bytes:
XFilterEvent returns: False

[Pierre Ossman]

On 20/08/18 13:30, robert.r...@gmail.com wrote:
> After switching from tigervnc 1.80 server (as distributed with CentOS 7) to tigervnc 1.90 (installed using rpm's from tigervnc.org bintray distribution for RHEL7), we observed some issues when connecting with RealVNC viewer on MacOS. After some amount of time, some window manager shortcuts stop working. Looks like the numlock state in the server gets confused.
>
> This effect was not observed so far in other combinations, i.e.:
> - tigervnc 1.90 server --> RealVNC viewer (mac): effect/issue present
> - tigervnc 1.80 server --> RealVNC viewer (mac): don't see this issue
> - tigervnc 1.90 server --> tigervnc 1.90 viewer (mac): don't see this issue
>

Could you test with TigerVNC viewer 1.8.0 as well? I suspect you should
see the same issue with that.

> Reason for using RealVNC viewer in this combo is its better support for MacOS native full-screen (focus switching), still slightly better performance (though tigervnc viewer is starting to get close).
>

Odd that you are seeing better performance with RealVNC. It doesn't
support Tight AFAIK, so it forces the server to use more CPU intensive
encodings.

> Some xev dumps and observations for these cases below:

Could you include a run with TigerVNC 1.8.0 server as well for reference?

Regards
--
Pierre Ossman Software Development
Cendio AB https://cendio.com
Teknikringen 8 https://twitter.com/ThinLinc
583 30 Linköping https://facebook.com/ThinLinc
Phone: +46-13-214600 https://plus.google.com/+CendioThinLinc

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[robert.r...@gmail.com]

On Monday, August 20, 2018 at 2:15:49 PM UTC+2, Pierre Ossman wrote:
> On 20/08/18 13:30, robert.r...@gmail.com wrote:
> > After switching from tigervnc 1.80 server (as distributed with CentOS 7) to tigervnc 1.90 (installed using rpm's from tigervnc.org bintray distribution for RHEL7), we observed some issues when connecting with RealVNC viewer on MacOS. After some amount of time, some window manager shortcuts stop working. Looks like the numlock state in the server gets confused.
> >
> > This effect was not observed so far in other combinations, i.e.:
> > - tigervnc 1.90 server --> RealVNC viewer (mac): effect/issue present
> > - tigervnc 1.80 server --> RealVNC viewer (mac): don't see this issue
> > - tigervnc 1.90 server --> tigervnc 1.90 viewer (mac): don't see this issue
> >
>
> Could you test with TigerVNC viewer 1.8.0 as well? I suspect you should
> see the same issue with that.

OK, I'll try to do some more tests tomorrow.

> > Reason for using RealVNC viewer in this combo is its better support for MacOS native full-screen (focus switching), still slightly better performance (though tigervnc viewer is starting to get close).
> >
>
> Odd that you are seeing better performance with RealVNC. It doesn't
> support Tight AFAIK, so it forces the server to use more CPU intensive
> encodings.

We keep going back to setting ZRLE encoding, seems to give the best overall performance for what we run (development stuff, text + CAE tools). I'd guess CPU is not hitting limits in our cases.

> > Some xev dumps and observations for these cases below:
>
> Could you include a run with TigerVNC 1.8.0 server as well for reference?

OK, I'll do that, will get back once I have all collected.

Thanks & best regards,
Robert Reutemann

[robert.r...@gmail.com]

Some more data on NumLock behavior across versions/viewers:

* TigerVNC Server 1.90 (running on CentOS 7):

- TigerVNC Viewer 1.90 on MacOS:
- NumLock showing up as NumLock in xev
- NumLock switching behavior functional (e.g. in nedit)

- TigerVNC Viewer 1.80 on MacOS:
- NumLock showing up as NumLock in xev
- BUT: NumLock switching behavior NOT functional (e.g. in nedit)

- RealVNC Viewer 6.18.625 on MacOS:
- NumLock showing up as Escape in xev
- AND: NumLock switching behavior NOT functional (e.g. in nedit)
==> for this combo, observed situations where NumLock got "stuck" in wrong state,
messing up some WM shortcuts, no obvious way to reset state

* TigerVNC Server 1.80 (running on CentOS 7):

- TigerVNC Viewer 1.90 on MacOS:
- no event generated in xev when pressing NumLock
- AND: NumLock switching behavior NOT functional (e.g. in nedit)

- TigerVNC Viewer 1.80 on MacOS:
- no event generated in xev when pressing NumLock
- AND: NumLock switching behavior NOT functional (e.g. in nedit)

- RealVNC Viewer 6.18.625 on MacOS:
- NumLock showing up as Escape in xev
- AND: NumLock switching behavior NOT functional (e.g. in nedit)
==> didn't seem to have the issue with "stuck" NumLock state in this combo (so far)

Also, on encodings (TigerVNC 1.90 Viewer --> TigerVNC 1.90 Server): ZRLE (w. full colors) really seems to be the most responsive combo, e.g. when moving a window with content. Tight is noticeably slower in this experiment. This is for a pretty fast server machine (dual Xeon, ...) with a "mid level" viewer machine (Core2 Duo), near-local network (via some firewall hops, but fast connections).

Best regards, Robert

PS: just for the record ... really love TigerVNC server, viewer getting to be really good as well. We have more issues with the "true" RealVNC-RealVNC combo (commercial) ...

[robert.r...@gmail.com]

Comment on NumLock below: the NumLock switching not being functional w. tigervnc 1.80 server isn't any issues for us, we have full-size keyboards. As long as it stays in the "numerical" state and doesn't interfere with other shortcuts, that's fine. Problem with the RealVNC viewer / TigerVNC 1.90 server combo is that the state seems to get stuck in a "funny" way after some time, messing up shortcuts.

[Pierre Ossman]

On 21/08/18 14:06, robert.r...@gmail.com wrote:
> Some more data on NumLock behavior across versions/viewers:
>
>
> * TigerVNC Server 1.90 (running on CentOS 7):
>
> - TigerVNC Viewer 1.90 on MacOS:
> - NumLock showing up as NumLock in xev
> - NumLock switching behavior functional (e.g. in nedit)
>
> - TigerVNC Viewer 1.80 on MacOS:
> - NumLock showing up as NumLock in xev
> - BUT: NumLock switching behavior NOT functional (e.g. in nedit)
>

Hmmm... That should work better than you describe. Exactly what are you
meaning with the nedit behaviour change?

Could you check what if "state" changes value in xev when you are using
the NumLock key here?

> - RealVNC Viewer 6.18.625 on MacOS:
> - NumLock showing up as Escape in xev
> - AND: NumLock switching behavior NOT functional (e.g. in nedit)

Well, if RealVNC doesn't actually send NumLock then we can't know that
we should toggle it. So this sounds like a bug in their viewer
unfortunately.

Just to verify though, could you start a Xvnc with -Log *:stderr:100 and
see what it prints when you press that key in the viewer?

> ==> for this combo, observed situations where NumLock got "stuck" in wrong state,
> messing up some WM shortcuts, no obvious way to reset state

The server will try to flip NumLock itself if it sees numpad keys
without NumLock active. I suspect that is what happens for you. The same
should happen with the 1.8.0 TigerVNC client.

I don't quite understand the problems this is causing though. Do you
have a WM that behaves differently depending on if NumLock is active or not?

>
> * TigerVNC Server 1.80 (running on CentOS 7):
>
> - TigerVNC Viewer 1.90 on MacOS:
> - no event generated in xev when pressing NumLock
> - AND: NumLock switching behavior NOT functional (e.g. in nedit)
>
> - TigerVNC Viewer 1.80 on MacOS:
> - no event generated in xev when pressing NumLock
> - AND: NumLock switching behavior NOT functional (e.g. in nedit)
>
> - RealVNC Viewer 6.18.625 on MacOS:
> - NumLock showing up as Escape in xev
> - AND: NumLock switching behavior NOT functional (e.g. in nedit)
> ==> didn't seem to have the issue with "stuck" NumLock state in this combo (so far)
>

Good. We actively filtered out lock keys in that version of the server,
so this verifies that things were sane then.

>
> Also, on encodings (TigerVNC 1.90 Viewer --> TigerVNC 1.90 Server): ZRLE (w. full colors) really seems to be the most responsive combo, e.g. when moving a window with content. Tight is noticeably slower in this experiment. This is for a pretty fast server machine (dual Xeon, ...) with a "mid level" viewer machine (Core2 Duo), near-local network (via some firewall hops, but fast connections).
>

Where did you get these builds? It sounds a bit like they're not using
an accelerated version of libjpeg. In that case ZRLE tends to be a lot
better.

> Comment on NumLock below: the NumLock switching not being functional w. tigervnc 1.80 server isn't any issues for us, we have full-size keyboards. As long as it stays in the "numerical" state and doesn't interfere with other shortcuts, that's fine. Problem with the RealVNC viewer / TigerVNC 1.90 server combo is that the state seems to get stuck in a "funny" way after some time, messing up shortcuts.

TigerVNC 1.8.0 will keep NumLock permanently off, not on. It will
generally inject fake Shift events, or use other keys, when it get
numpad keys that doesn't match the desired state. As such we don't
really want to revert to that model, so hopefully we can find some fix
for 1.9.0.

>
> PS: just for the record ... really love TigerVNC server, viewer getting to be really good as well. We have more issues with the "true" RealVNC-RealVNC combo (commercial) ...
>

Thank you. :)

[robert.r...@gmail.com]

Splitting reply. Viewer speed below, will follow up on NumLock/.... separately.

> >
> > Also, on encodings (TigerVNC 1.90 Viewer --> TigerVNC 1.90 Server): ZRLE (w. full colors) really seems to be the most responsive combo, e.g. when moving a window with content. Tight is noticeably slower in this experiment. This is for a pretty fast server machine (dual Xeon, ...) with a "mid level" viewer machine (Core2 Duo), near-local network (via some firewall hops, but fast connections).
> >
>
> Where did you get these builds? It sounds a bit like they're not using
> an accelerated version of libjpeg. In that case ZRLE tends to be a lot
> better.
>

This is using the TigerVNC 1.90 server rpm's for RHEL7 and MacOS viewer .dmg, both from the tigervnc bintray distribution.

The experiment here is moving around a relatively complex window (CAE waveform viewer, with content shown). Observation:
- for all cases: TigerVNC 1.90 server running on CentOS 7; all cases: full color, 1920x1200, fullscreen mode
- RealVNC viewer (MacOS), ZRLE encoding: "fluid" window movement, no noticeable lag
- TigerVNC viewer (MacOS), ZRLE encoding: near-fluid movement, slight lag, but workable
- TigerVNC viewer (MacOS), Tight encoding: not quite fluid, noticeable lag, getting tedious to place window as intended
Other operations (changing virtual desks, ... ) show similar relative behavior. Might be only for the types of "content" and setup we use, but forcing ZRLE has worked best for us across many versions of VNC server and viewer and different network connections for quite some years.

Thanks, Robert

[robert.r...@gmail.com]

On the NumLock stuff:

For one, dug around a bit for fvwm and nedit.
- both of these have their own quirks with NumLock, so maybe not the best tests for overall behavior
- have workarounds for NumLock for both these cases (for what we need/use)
- use "IgnoreModifiers L2" for fvwm (i.e. also ignore mod2=NumLock for fvwm key bindings)
--> solves the issue with key bindings in fvwm
- use xmodmap to hard-map numeric keys to their "number" side functions (KP_4, ...)
--> makes numeric digit entry work always (have full keyboards, separate arrow keys,
i.e. don't need secondary function of numeric keys).

More comments below.

My summary from those further experiments: looks like tignervnc 1.90 server does what is supposed to do according to your description. I guess one problem is NumLock mapping in RealVNC viewer. The other is applications/WM sensitive to NumLock state.

Thanks for all your work and input!
Robert

On Tuesday, August 21, 2018 at 2:53:02 PM UTC+2, Pierre Ossman wrote:
> On 21/08/18 14:06, robert.r...@gmail.com wrote:
> > Some more data on NumLock behavior across versions/viewers:
> >
> >
> > * TigerVNC Server 1.90 (running on CentOS 7):
> >
> > - TigerVNC Viewer 1.90 on MacOS:
> > - NumLock showing up as NumLock in xev
> > - NumLock switching behavior functional (e.g. in nedit)
> >
> > - TigerVNC Viewer 1.80 on MacOS:
> > - NumLock showing up as NumLock in xev
> > - BUT: NumLock switching behavior NOT functional (e.g. in nedit)
> >
>
> Hmmm... That should work better than you describe. Exactly what are you
> meaning with the nedit behaviour change?

Meaning it doesn't switch between number entry and arrow operation for e.g. KP_4. But looks like nedit is a bit special there anyway.

> Could you check what if "state" changes value in xev when you are using
> the NumLock key here?
>
> > - RealVNC Viewer 6.18.625 on MacOS:
> > - NumLock showing up as Escape in xev
> > - AND: NumLock switching behavior NOT functional (e.g. in nedit)
>
> Well, if RealVNC doesn't actually send NumLock then we can't know that
> we should toggle it. So this sounds like a bug in their viewer
> unfortunately.

May well be, see event log below.

> Just to verify though, could you start a Xvnc with -Log *:stderr:100 and
> see what it prints when you press that key in the viewer?

RealVNC 6.18 viewer MacOS --> TigerVNC 1.90 server:

* start fresh vncserver (with log enabled)
* connect with RealVNC viewer
* launch xev in session (to see state there)

* press/release "A"
--> Xvnc log, xev: press/release keycode 38 "a"
--> xev state: 0x0
(all as expected)

* press/relase "NumLock"
--> Xvnc log, xev: press/relase keycode 9 (= "Escape" according to xmodmap)
--> xev state: 0x0, unchanged
VNCSConnST: Key pressed: 0xff1b / 0x0
Input: keycode 9 down
VNCSConnST: Key released: 0xff1b / 0x0
Input: keycode 9 up

* press/release "A"
--> same as before, state still 0x0

* press/relase "KP_4"
--> Xvnc log:
VNCSConnST: Key pressed: 0xffb4 / 0x0
VNCSConnST: Inserting fake NumLock to get in sync with client
Input: keycode 77 down
Input: keycode 77 up
Input: keycode 83 down
VNCSConnST: Key released: 0xffb4 / 0x0
Input: keycode 83 up
--> xev:
KeyPress event, serial 28, synthetic NO, window 0x1000001,
state 0x0, keycode 77 (keysym 0xff7f, Num_Lock), same_screen YES,
XLookupString gives 0 bytes:

KeyRelease event, serial 28, synthetic NO, window 0x1000001,

state 0x10, keycode 77 (keysym 0xff7f, Num_Lock), same_screen YES,
XLookupString gives 0 bytes:

KeyPress event, serial 28, synthetic NO, window 0x1000001,
state 0x10, keycode 83 (keysym 0xffb4, KP_4), same_screen YES,
XLookupString gives 1 bytes: (34) "4"

KeyRelease event, serial 28, synthetic NO, window 0x1000001,

state 0x10, keycode 83 (keysym 0xffb4, KP_4), same_screen YES,
XLookupString gives 1 bytes: (34) "4"

==> with the fake NumLock event, the NumLock state in the session is now
stuck "on", i.e. xev states will always have the mod2 bit set (0x10).

> > ==> for this combo, observed situations where NumLock got "stuck" in wrong state,
> > messing up some WM shortcuts, no obvious way to reset state
>
> The server will try to flip NumLock itself if it sees numpad keys
> without NumLock active. I suspect that is what happens for you.

Yes, looks like this is the case.

> The same should happen with the 1.8.0 TigerVNC client.

Yes, exactly. With tigervnc 1.80 viewer:
- NumLock comes through as "Num_Lock"
- NumLock toggles the xev state betwen 0x00 and 0x10 as expected
- when pressing e.g. KP_4 while NumLock state is OFF, the server injects a fake NumLock event before that key event
- when pressing e.g. KP_4 with NumLock state already ON, no fake event is inserted

So, same behavior, except that realvnc viewer doesn't has this mapping issue for the NumLock key.

> I don't quite understand the problems this is causing though. Do you
> have a WM that behaves differently depending on if NumLock is active or not?

FVWM by default is sensitive to all modifiers, including mod2 (usually NumLock). Workaround see above.