WebRTC UWP(Metro) Application window capturing

[Andrei Gavrilescu]

Hello everyone,

Nice to meet you, I’m new to this community and to the webrtc code base/contribution process in general, so please forgive my ignorance in this matter.

I’m curious about why webRTC ignores UWP style applications when enumerating the list of windows available for capturing.

By looking around the code base it seems that the module responsible for window capturing and window listing on Windows systems is CroppingWindowCapturerWin which mostly wrapps the functionality from WindowCapturerWin.

The function that enumerates available windows has specific code that ignores UWP style windows as can be seen here https://chromium.googlesource.com/external/webrtc/+/master/modules/desktop_capture/window_capturer_win.cc#70 . Looking through the bugs related to this context the change was made as a result of https://bugs.chromium.org/p/chromium/issues/detail?id=526883 with cr https://codereview.webrtc.org/1371383003. There are a couple of more issues open that describe this problem like https://bugs.chromium.org/p/chromium/issues/detail?id=890928, but I still don’t understand the complete reasoning behind it.

From my somewhat limited understanding of how everything is glued together WindowCapturerWin::CaptureFrame uses GDI to capture the full screen and crop out the application window. That makes perfect sense to me as I’m aware that some applications don’t work with direct GDI window capture (applications that use Direct3D, if I’m not mistaken), and cropping is a generic method that works all around, with the caveat that you need to take care of overlapping/occluding windows by falling back to GDI capture for instance.

Now the part that I don’t understand is, why ignore UWP apps? if cropping is the de facto way of capturing anyway, shouldn’t it work as with any other application window?

Of course it’s going to have the limitations that Direct3D applications have, like a black thumbnail in the window list and black frames when occlusion occurs, but then again if this behaviour isn’t desired why aren’t such applications also filter out? one example would be any electron application like VisualCode.

If there aren’t any compelling reasons not to, I would very much like to add a fix for this behaviour, however like I mentioned initially, I'm farrelly new and it's going to take a while to get used to the whole contribution process ( any pointers and newbie advice is more than welcome :) ). Thank you in advance.

Best regards,
Andrei Gavrilescu

[Andrei Gavrilescu]

Update:

Managed to compile chromium with the https://chromium.googlesource.com/external/webrtc/+/master/modules/desktop_capture/window_capturer_win.cc#70 UWP filter removed, and it seems like metro apps behave just like applications that use Direct3D, i.e. the thumbnail in the window picker is black, and when occlusion occurs it switches to GDI window capture resulting in black frames, but in all other cases it works as intended. There are also some additional problems like Windows10 hidden uwp windows, but I'm guessing these can be filtered out. I used Skype Metro version for testing and can confirm that it doesn't show up in the window picker list without the changes mentioned above. Attached a couple of screenshots with the results.

[Andrei Gavrilescu]

Bump!

On Wednesday, February 6, 2019 at 7:15:23 PM UTC+2, Andrei Gavrilescu wrote:

[Andrei Gavrilescu]

Bump x2!