Delayed Rendering

[Lennart Grosser]

Hi,

I'm using tigervnc 1.12.0 (built 2022-03-25 17:06). Seemingly, my shell is not rendered properly - only every now and then. 

Example:

Obviously "ll" should list files/directories. This also happens occasionally when switching terminal tabs (the tab that I want to switch to is not displayed, the old tab keeps being displayed).

However: on the next input (keyboard or mouse) the screen is rendered properly (including the next input action).

I also noticed two things:

1) this seems to happen only with ZSH (I tried bash and it SEEMS not happen there (could also be luck)) - I have never witnessed it with another application besides the terminal

2) it does not happen when using RDP so I assume its a VNC thing

Has anyone experienced the same? I'm wondering whether I should open a bug report.

Thanks in advance

[Petr Kulhanek]

Dear Lennart,

we observe similar symptoms since Ubuntu 22.04 but for the Konsole application in the GNOME environment. The refresh errors are so large that it discourages the regular use of this application. Since the default terminal is gnome-terminal, it is not (yet) a big problem in our environment (a PC classroom at a university).  I have attached the screenshot showing the problem. In the second tab, I have run the top command. Then I switched to the first tab.* The content of the terminal was not refreshed. Once I hover over some parts, which are interpreted as links, those parts have refreshed because they were re-rendered using the underlined font (see URL and email in the left middle part of the window). To fix the rendering, one must either move with a window or resize its contents.  

* actually, it can take several minutes to achieve this state, but once this happens, it is very frequent.

We did several tests to find the cause:

- it is independent of the TigerVNC client. We tested a native vncviewer (different versions) as well as noVNC

- it is independent of the TigerVNC server. Again we tested several, even very old, versions

- it is independent of OpenGL. Recently, we started to use VirtualGL over the EGL back end. We even put the entire desktop under VirtualGL. But the same problem we observed with OS Mesa3D employing the software rendering pipeline or even the most recent version taken from PPA.

I am not the expert, but it seems to me that the problem is related to incorrect XDAMAGE events. Either they are not generated at all, or they are shot sooner than the content is actually drawn. These events are generated by the X server (Xvnc) and the window and composite manager (Mutter in the case of GNOME).  And it must be some sort of race condition bug because it affects only terminals (in our case, Konsole; in your case, gnome-terminal).  Konsole is a QT application, but we do not observe any problems with other QT applications.

We also tested a different window manager (JWM), but here we did not observe any such problems. So the most probable cause is a bug in Mutter.

Best regards

Petr

[Pierre Ossman]

On 28/06/2023 09:34, 'Lennart Grosser' via TigerVNC User

Discussion/Support wrote:
>
> Has anyone experienced the same? I'm wondering whether I should open a bug
> report.
>

The only known issue we currently have is that there is some bug in GLAMOR:

https://github.com/TigerVNC/tigervnc/issues/1218

That only happens when using libvnc.so (and possibly x0vncserver), not
with Xvnc and vncserver.

How was this session started?

Regards
--
Pierre Ossman Software Development
Cendio AB https://cendio.com
Teknikringen 8 https://twitter.com/ThinLinc
583 30 Linköping https://facebook.com/ThinLinc
Phone: +46-13-214600

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[Pierre Ossman]

On 04/08/2023 09:48, Petr Kulhanek wrote:
> Dear Lennart,
>
> we observe similar symptoms since Ubuntu 22.04 but for the Konsole
> application in the GNOME environment. The refresh errors are so large that
> it discourages the regular use of this application. Since the default
> terminal is gnome-terminal, it is not (yet) a big problem in our
> environment (a PC classroom at a university). I have attached the
> screenshot showing the problem. In the second tab, I have run the top
> command. Then I switched to the first tab.* The content of the terminal was
> not refreshed. Once I hover over some parts, which are interpreted as
> links, those parts have refreshed because they were re-rendered using the
> underlined font (see URL and email in the left middle part of the window).
> To fix the rendering, one must either move with a window or resize its
> contents.

I can reproduce this issue here. This is not a previously known problem.

> I am not the expert, but it seems to me that the problem is related to
> incorrect XDAMAGE events.

That is the common problem in these cases, but it's unfortunately not
that simple this time.

You can easily rule these issues out by disconnecting and reconnecting.
If the problem remains, then the issue is some form of incorrect
rendering, not an overlooked update.

And when I test here, the konsole window is still not properly updated
after disconnecting and reconnecting.

So the bug is in one of three places:

* Konsole, that isn't pushing updates to the X server properly

* gnome-shell, that isn't doing the compositing correctly

* The Xorg parts in Xvnc that isn't respecting the draw commands from
gnome-shell properly

I can't seem to reproduce the issue on Fedora 38, so it might be an
issue that is resolved in later versions of the above components. I
would suggest filing a bug with Ubuntu.

[Petr Kulhanek]

Dear all,

I can reproduce this issue here. This is not a previously known problem.

So the bug is in one of three places:

* Konsole, that isn't pushing updates to the X server properly
* gnome-shell, that isn't doing the compositing correctly
* The Xorg parts in Xvnc that isn't respecting the draw commands from
gnome-shell properly

I can't seem to reproduce the issue on Fedora 38, so it might be an
issue that is resolved in later versions of the above components. I
would suggest filing a bug with Ubuntu.

for those who are facing the same problem, the efficient workaround I have found at the cost of slightly slower rendering is to set:

export MUTTER_SYNC=1

before the GNOME desktop startup. This is based on the information  here (very old bug with similar symptoms):

https://bugzilla.gnome.org/show_bug.cgi?id=657071

Note that the workaround with CLUTTER_PAINT does not work.

GNOME mutter was significantly rewritten between 42.9 (Ubuntu 22.04) and 44 (Fedora 38), and it would be pointless to dig deeper into this problem as it affects only a few people.

Best regards

Petr  Kulhanek

 

[Lennart Grosser]

Thank you Petr for looking into this!! I will try your suggested fix soon

[David Miller]

Chiming in because this sounds like what I'm seeing on a Rocky 9 Linux box with TigerVNC.  I posted the following problem description on the Rocky Linux forum: https://forums.rockylinux.org/t/weird-tigervnc-behavior/11591.  One additional data point since that post: I did a clean RL 9.2 install to a new VM and it seems to *NOT* be exhibiting the behavior.  That makes three VM running Rocky 9.2 that don't show a rendering delay and one Rocky 9 install on fairly beefy real hardware that has the rendering delay.  Also, all of the systems mentioned are running the Mate DE; not Gnome.

I see this behavior in both MateTerminal, xterm and the included mate text editor.  I don't see it in the Google search under Firefox nor in LibreOffice Writer.  Graphical rendering (e.g., paging down a long story on BBC) is also slower/choppier as compared to FF looking at the same BBC article on a VM.  Not as bad as text rendering but still noticeably slower and a little less smooth.  This seems to indicate that there is a hardware component to the problem but no idea why it would affect VNC.

The VMs are to help me move from my current CentOS 7 platforms to Rocky 9.  If there's anything I can do to capture data, let me know.  Same with the R9 install on the real hardware.  These systems are strictly to let me get familiar with Rocky 9 before I update my "real" systems.

Cheers,

Dave

[David Miller]

Follow-up to my previous post.  I ruled out that the network connection was the problem by installing iperf on both my desktop and the system exhibiting the problem.  Got the following:

[root@abuse ~]# iperf -c 192.168.0.2 -p 5001 Client connecting to 192.168.0.2, TCP port 5001

TCP window size: 85.0 KByte (default)

[ 1] local 192.168.0.4 port 52966 connected with 192.168.0.2 port 5001
[ ID] Interval Transfer Bandwidth
[ 1] 0.00-10.02 sec 1.10 GBytes 941 Mbits/sec
and going the other direction:

[root@bend ~]# iperf -c 192.168.0.4 -p 5001 Client connecting to 192.168.0.4, TCP port 5001
TCP window size: 510 KByte (default)

[ 3] local 192.168.0.2 port 48582 connected with 192.168.0.4 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 1.10 GBytes 943 Mbits/sec

Looks like network throughput isn’t the problem.

[David Miller]

Yet another update 16 October 2023: One other way to see if its VNC or something else is to allow ssh forwarding on my favorite desktop and then set the DISPLAY environment variable of an ssh session on the hardware having trouble with display rendering of a VNC sessions. Fired up an xterm and a mateTerminal which both popped up on my desktop as expected… and entering text was absolutely normal. No rendering lag.

So, why is tigerVNC having a display rendering lag?

[Andrew C Aitchison]

On Mon, 16 Oct 2023, David Miller wrote:

> Yet another update 16 October 2023: One other way to see if its VNC or
> something else is to allow ssh forwarding on my favorite desktop and then
> set the DISPLAY environment variable of an ssh session on the hardware
> having trouble with display rendering of a VNC sessions. Fired up an xterm
> and a mateTerminal which both popped up on my desktop as expected… and
> entering text was absolutely normal. No rendering lag.
>
> So, why is tigerVNC having a display rendering lag?

Sorry if these are obvious questions, I missed the start of the thread.

Am I correct in thinking the issue is the tigervnc server
- you have tried another VNC client to rule out the tigervnc client ?

Are you using what Debian/Ubuntu call the "tigervnc-standalone-server"
or the "tigervnc-scraping-server" ?
(Debian/Ubuntu also have a "tigervnc-xorg-extension" package,
but that is recommended when client and server are on the same machine.)

--
Andrew C. Aitchison Kendal, UK
and...@aitchison.me.uk

[David Miller]

Hi Andrew!

I'm on Rocky Linux 9 (RHEL 9 equivalent) for the servers and CentOS 7 for the client but everything is tigerVNC.  I took my experiment of running various character aps to its conclusion and started the tigerVNC client from the same forwarded ssh session and then connected to 127.0.0.1.  I got the expected VNC desktop but displayed back on my C7 desktop through the ssh forward.  It works as expected with no rendering delay.  I also tried a VNC client session from one of the R9 VMs to the real hardware box and it works as expected with no rendering delay.

So, with the Rocky 9 tigerVNC client connecting to the Rocky 9 server on real hardware, rendering works the way its supposed to with no delay.  For some reason CentOS 7 VNC client connecting to a Rocky 9 server on real hardware exhibits the delayed rendering.  CentOS 7 tigerVNC client connecting to a Rocky 9 tigerVNC server running on a VMware VM does not exhibit the delayed rendering problem.  Weird.

I'm getting ready to move everything here to Rocky 9.  I was concerned with the rendering delay I was seeing when connecting to the real hardware box from my C7 desktop.  Since the problem appears to go away when I've got a Rocky 9 client and server, I'm ready to start moving my C7 systems to Rocky 9 and the rendering lag problem will be "overcome by events."

I did notice one thing when I started the VNC client from the command line.  Some of the start up status that disappears when the client starts from the menuing system was visible.  I found the last line particularly interesting:

[dave@abuse ~]$ vncviewer &
[2] 220480
[dave@abuse ~]$
TigerVNC Viewer 64-bit v1.12.0
Built on: 2023-03-27 00:00
Copyright (C) 1999-2021 TigerVNC Team and many others (see README.rst)
See https://www.tigervnc.org for information on TigerVNC.

Mon Oct 16 22:45:19 2023
 DecodeManager: Detected 40 CPU core(s)
 DecodeManager: Creating 4 decoder thread(s)
 CConn:       Connected to host 127.0.0.1 port 5901
 CConnection: Server supports RFB protocol version 3.8
 CConnection: Using RFB protocol version 3.8
 CConnection: Choosing security type VeNCrypt(19)
 CVeNCrypt:   Choosing security type TLSVnc (258)

Mon Oct 16 22:45:24 2023
 CConn:       Using pixel format depth 24 (32bpp) little-endian rgb888
 CConnection: Enabling continuous updates

I didn't see anyway to set this but it had me wondering if it has something to do with the problem.

Cheers,

Dave

[DRC]

The TurboVNC Server would be a useful point of comparison, since we also
support the RFB flow control extensions but use slightly different logic
for pushing out updates.  When I attempted to merge the latest flow
control logic from TigerVNC into TurboVNC, I encountered a similar
delayed update issue:

https://github.com/TigerVNC/tigervnc/commit/a99d14d1939cb2338b6268d9aebe3850df66daed#r57748408

It would be useful to test whether the following patch works around the
issue, as this is essentially what I had to do to make the TurboVNC
Server work properly:


--- a/common/rfb/VNCSConnectionST.cxx
+++ b/common/rfb/VNCSConnectionST.cxx
@@ -859,8 +859,7 @@ bool VNCSConnectionST::isCongested()
     return false;

   eta = congestion.getUncongestedETA();
-  if (eta >= 0)
-    congestionTimer.start(eta);
+  congestionTimer.start(eta <= 0 ? 1 : eta);

   return true;
 }


If that patch works around the issue, then it may be the same issue I
encountered.

[DRC]

"Continuous updates" is part of the RFB flow control extensions that I mentioned in the previous message.  Only TigerVNC and TurboVNC support those extensions, to the best of my knowledge, so the extensions won't be enabled if you connect with a different VNC client.  Also, the issue I mentioned in the previous message was specific to the congestion control algorithms that the TigerVNC 1.9+ Server uses, so if this is the same issue I observed, then it should only be observed if the client reports that continuous updates are enabled when connecting to the TigerVNC 1.9+ Server.

--
You received this message because you are subscribed to the Google Groups "TigerVNC User Discussion/Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tigervnc-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tigervnc-users/0fc7c512-8f25-4e81-9858-fe824e4cf972n%40googlegroups.com.

[David Miller]

Not in a position to try the patch in your previous post.  Guessing from your post that the TigerVNC client the ships with CentOS 7 (tigervnc-1.8.0-25.el7_9.x86_64) is not correctly handling the congestion issue when connecting to a Rocky 9 server (tigervnc-server-1.12.0-13.el9_2.x86_64) on real hardware.  Seems to work fine with the same server on Rocky 9 but on a VM.  The virtual environment may be different enough that the issue doesn't show up.  I'm running a fairly old version of VMware ESXi which further complicates things.