gUM Screen Capture source id

[Ajay Choudary]

When we are capturing the screen/window using getUserMedia, unable to trace the selected source.

Details from stream videoTrack: 

Chrome-44:

1. enabled: true
2. id: "e8f53418-78d1-4401-9414-cf15ad2ffdc0"
3. kind: "video"
4. label: "Screen"
5. muted: false
6. onended: null
7. onmute: null
8. onunmute: null
9. readyState: "live"

Firefox-40:

kind: "video", 

id: "{7a6a517a-7bbf-441b-9fe3-fae9c14497ca}", 

label: "", 

enabled: true

Is there any way to get source type (as described in CaptureSurfaceType ) and source identifier like screen-1/2/3 or Window-ID/Title/Process_Id?

When i try with navigator.getMediaDevices(), it is giving only microphone & webcam details but not getting about screens/ Z-order windows.

Please suggest.

[Christoffer Jansson]

Hi,

Answers inline:

On Thursday, August 20, 2015 at 6:08:33 PM UTC+2, Ajay Choudary wrote:

When we are capturing the screen/window using getUserMedia, unable to trace the selected source.

Details from stream videoTrack: 

Chrome-44:

1. enabled: true
2. id: "e8f53418-78d1-4401-9414-cf15ad2ffdc0"
3. kind: "video"
4. label: "Screen"
5. muted: false
6. onended: null
7. onmute: null
8. onunmute: null
9. readyState: "live"

Firefox-40:

kind: "video", 

id: "{7a6a517a-7bbf-441b-9fe3-fae9c14497ca}", 

label: "", 

enabled: true

Is there any way to get source type (as described in CaptureSurfaceType ) and source identifier like screen-1/2/3 or Window-ID/Title/Process_Id?

Check the desktop capture example https://github.com/webrtc/samples/blob/master/src/content/extensions/desktopcapture/.

Also check the https://developer.chrome.com/extensions/desktopCapture api info page.

When i try with navigator.getMediaDevices(), it is giving only microphone & webcam details but not getting about screens/ Z-order windows.

Screens/windows are not considered a media device, also sharing the screen is considered sensitive hence why it's handled via an extension, see the page above.

Please suggest.

[Ajay Choudary]

Hi Jansson,

I already have working ScreenShare setup (Try Demo) and uploaded my extension in chrome store.

Here user installed extension and shared his screen/window, everything is working fine.

But we need to identify which screen/window is shared, as we are trying to implement screen controlling with an app.

After selecting the window with chrome.desktopCapture.chooseDesktopMedia() it is not giving any details except random string (stream_id).

In VideoTrack object, label is always showing 'screen'(chrome) or ""(Firefox) and id is again a random string.

Security perspective: Here user installed extension, shared the screen/window from trusted URL and also installed our app.

Thanks,

Ajay

[Christoffer Jansson]

I see, I don't actually know but to me it looks like this is by design, maybe you could ask in the chromium-dev mailing list to be sure?

/Chris

[jib]

According to the screenshare spec "Each potential source of capture is treated by this API as a discrete media source" aka device, and since track.label points to the device label, it seems reasonable to expect more detail than just "Screen", like maybe "Screen (2880x1800)" and maybe even an id?

Comparatively, here's what Chrome 47 says for my camera: "FaceTime HD Camera (Built-in) (05ac:8510)"

-jib

[Christoffer Jansson]

jiayl@ what's your take on this?

[Jiayang Liu]

I don't understand the use case. What the information will be used for?

--

Jiayang Liu

[Ajay Choudary]

@Jiayang, we are implementing screen controlling functionality and  had an ambiguity while choosing the screen from my application if multiple screens are present.

We need a unique id/label from gUM stream/Track to detect the shared screen and should matches with NSScreen properties.

@Jib, in chrome the label is always "screen" for any screen/window.

[jib]

@Jiayang, regardless of use case, since the spec says each capture source is it's own device, and devices have labels, it follows that labels should be descriptive enough to tell devices apart, and using the same label "Screen" for all of them does not accomplish that.

On Thursday, September 3, 2015 at 2:06:37 PM UTC-4, Ajay Choudary wrote:

@Jib, in chrome the label is always "screen" for any screen/window.

Yes, I'm suggesting you file a bug on Chrome for that.