In slashdot-24.rfb you are creating around 19000 frames, one frame for every update, but your code doesnt seem to set a i_keyint_max, so it defaults to 250 if i remember correct. So basically you force h264 to make a full screen update on every 250 frame, even if h264 detects no changes in the image. (Setting the i_keyint_max to 20000 reduces the keyframes from 80 to 4 and the size from 21.6mb at CQP 23 to 11 mb (hextile is at 9mb?, strange anyway), kde-hearts-24 is at 2.2 mb, photos at 900k at the same settings)
the output of the -o flag looks odd, but i cant verify cause i havent found a way to replay the fbs-dumped files from rfbsessions.tar.bz2
"x264 provides several methods for controlling the size of the encoded stream, including specifying a constant frame rate and a constant bit rate."
Are you sure X264_RC_CQP is constant quality? What is constant framerate?
This is just the upper bound after that an update is forced and shouldnt effect 2d or 3d workloads at all. It just makes your compression rate bad. It is good for "forward error correction" and one stream for all situations.
The value should be infinity, because you already dealing with packetloss and you only have one stream per client.
The reason i picked 20000 is cause it is bigger than all the testcases framecount and i havent found out how to set it to infinity, which is possible cause at http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-ugly-plugins/html/gst-plugins-ugly-plugins-x264enc.html#GstX264Enc--key-int-max
it its per default set to infinity (by setting it to zero).
(I tried setting it to zero too and got a full disk ( 19000 keyframes :( ) ;) )
Interesting about the slashdot testcase is that the framecount isnt that high compared too the length (i got around 50 fps). But the compressions is bad compared to all other codexes, even the losseless. But even the compression rate is bad, the bandwith demands are low.
>(I have seen one oil & gas app, for instance, that draws 100,000
>individual points on a grid without double buffering. Ugh.)
In my opinion frames are good concept. And i think it is questionable if you need "50 fps" if your network connection is the limiting factor, and you are trying to reduce network load at every other place (reducing resolution, reducing bitrate..).
(same topic: https://www.youtube.com/watch?v=RIctzAQOe44 Minute 29min)
That's not our experience. Are you using WinVNC?
The problem with GPU-assisted encoding is: how to get the images to and
from the GPU, given that VNC's framebuffer is in main memory. TurboVNC
is installed in sites that are serving up 3840x1200 desktops and putting
50 users on a single server machine, so if all of those users are
constantly sending 4-megapixel images back and forth to the GPU, bus
bandwidth is going to run out pretty quickly, not to mention that the
transfer overhead may kill any performance advantage that would
otherwise be realized from using the GPU.
Yeah, the ideal would be for VirtualGL to take advantage of that. The
issue, however, is how to get the encoded video stream to the client.
The most effective way I can think of seems to involve some sort of a
"compressed PutImage" extension, whereby VirtualGL tunnels the video
stream through the RFB protocol somehow, bypassing the VNC encoder.
Then the issue becomes how to handle it on the viewer end-- mainly how
to deal with compositing and window management, since the pixels are no
longer going through the X server. I know that IBM developed such
extensions (an X extension and an RFB extension) for RealVNC
Enterprise-- primarily because RealVNC wasn't fast enough to stream 3D
content on its own. I don't know if their RFB extension is officially
registered or not.
Another possibility is modifying VNC such that it stores its framebuffer
on the GPU, but I suspect that this would exhaust GPU memory pretty quickly.
> 1 significant downside of using hardware acceleration is limited number
> of streams. Intel Quick Sync (on their CPUs with integrated graphics)
> only supports 5 1080p streams for example. So this wouldn't work for
> people running machines with say 20 simultaneous users on a 16-core
> machine with very few GPUs.
I don't think any serious multi-user 3D servers are going to be running
Intel GPUs. Intel stuff is more consumer-oriented, so it would
primarily be a client-side thing.
Yeah, the other issue -- as stated in the report -- is that, in order to
minimize CPU usage with x264 as much as possible, I had to maintain a
YUV-encoded copy of the remote framebuffer (so as to avoid converting
the entire framebuffer from RGB to YUV each time a new H.264 frame was
encoded.) That wouldn't be an issue with GPU-based encoding. I have no
doubt that H.264 is going to be the best solution for games, and there
is already proof of this with companies like OnLive and Gaikai using
predictive codecs for their remote game streaming solutions. It's just
a matter of figuring out how best to implement it in VNC. Maybe the
deferred update timer could be leveraged as a frame rate governor. Much
testing would need to be done, which is going to require much money.
That's all really good info. With regards to bus bandwidth, though, what you didn't take into account is the frame rate. Per my previous message, there's no way to encode a partial frame with H.264, so if the remote desktop is 4MP, then all 4MP has to be downloaded to the GPU with every frame. 50 users * 4MP * 25 fps = 22.5 GB/s. OK, so we can make some simplifying assumptions-- in all likelihood, H.264 will be used only by employees connecting from remote locations and not a LAN, and with automatic desktop resizing, they will probably use a 3840x1200 desktop at work and then resize it to something smaller at home. And not all of them will be using the system at the same time or using H.264 at the same time. However, the other thing to bear in mind is that there is a lot of 3D data transiting the bus as well, particularly when dealing with volume visualization applications. So while I don't know for a fact that it will be an issue, I can certainly envision a scenario in which it would be. Storing a YUV copy of the framebuffer in GPU memory seems to make the most sense, because then you only have to send the changed regions down to the GPU.
--
You received this message because you are subscribed to the Google Groups "TigerVNC Developer Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tigervnc-deve...@googlegroups.com.
To post to this group, send email to tigervn...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tigervnc-devel/96086064-7f4c-4a13-a414-0bf3c17ad884%40googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tigervnc-devel/55046A51.4040101%40matthewlai.ca.
You received this message because you are subscribed to a topic in the Google Groups "TigerVNC Developer Discussion" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tigervnc-devel/kGS73nsjXBg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tigervnc-deve...@googlegroups.com.
To post to this group, send email to tigervn...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tigervnc-devel/ED45DCDE-B348-4DBD-A127-6B21BEF62D57%40virtualgl.org.
I am a windows system admin.
The most popular ones for home users are Splashtop and Steam Streamer. They both use H264 alike codec with variable bitrates(so in the quick motion I can see the quality drops to maintain remote control smoothness and responsive. When steady, the picture quick turn sharper). The result is so stunning, users actually can play 3D shooting games via remote desktop.
For the business, I see the nomachine using VP9 as NX protocol. And Sunlogin is so similar to VNC that uses "Mirror Driver" for capture and H264 for transfer. The result is so excellent that I can play youtube @30 fps on the remote server and watch it from home pc @20 fps without noticeable quality loss. (bad thing is most of those program are heavy commercial, closed source, all need to login to their server, keep popup "consider buying" dialog and so on.)
The real deal is, the smoothness of H264 for scrolling/drag drop/big dialog openning etc makes me enjoy my work much more. (the tiny laggy from vnc protocol create headache overtime)