Try out CaptureStream from canvas and video!

[Niklas Enbom]

Hi all,

You can now start experimenting with creating MediaStreams from <canvas> and <video> elements in Chrome. You need Chrome 50 (current canary, soon dev channel) and enable the Experimental Web Platform features flag under chrome://flags.

Some samples:  https://cdn.rawgit.com/uysalere/js-demos/master/intro.html

This is still work in progress but feel free to file bugs. Known issues:

- Audio is still not supported from <video> elements. 

- Canvas in background tabs does not update properly.

[Niklas Enbom]

Sorry, wrong W3C spec in the link. Correct one: https://www.w3.org/TR/mediacapture-fromelement/.  

[Jeremy Noring]

This is awesome.  I'm really excited about this feature.  

I'm assuming ultimately I can call getUserMedia, render video to the canvas, overlay other stuff to the canvas, and all of it will get streamed?

[Niklas Enbom]

Yes that's an intended use case. Here's an example without a peerconnection: https://cdn.rawgit.com/uysalere/js-demos/master/canvascapture9.html, but it should work also over a PeerConnection. 

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/264ecbd6-71ae-4512-82ae-9495472330b0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Samson Timoner]

Hey Niklas,

 Thanks so much for the post.  Looking at: https://bugs.chromium.org/p/chromium/issues/detail?id=524218# , it looks like Chrome is going to support WebGL as well.

 By any chance, Do you know if it is going to support Chrome NaCl elements? I'm currently using GL within the chrome native client.  It shows up as an embedded element in the GL page. I'd love to capture the results as video...and possibly share it with a peer2peer stream.

 Thanks!

-- Samson

[Niklas Enbom]

Sorry, I don't now that much about rendering from NaCl. What type of object are you rendering into from NaCl / PPAPI ? 

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/8cba86e3-7e6e-46dd-9577-1cca1ad3d706%40googlegroups.com.

[Samson Timoner]

Thanks for getting back so quickly.  I'm taking a video and editing it in NaCl, and then plugging it into OpenGL. It shows up as an embedded element in the DOM.

It sounds like this might be a question for the NaCl group. Or, you CCed the owner of the ticket: Do you have insight?

 Thanks!

-- Samson

--

---
You received this message because you are subscribed to a topic in the Google Groups "discuss-webrtc" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/discuss-webrtc/fpbiC2HOgUY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CAHzHjDMxQYs5AZP7c3%3Dszn1nKszDBB%3D2iTAL9AkwpJhEoRgDMw%40mail.gmail.com.

[Emircan Uysaler]

What type of an embedded element in the DOM are you playing it in? This feature will support creating MediaSream from <video> and <canvas> only, if it is one of those elements it would work.

Sorry, I am still a little confused about your use case. AFAIU, you are using NaCl support to display contents. Is there a way you can put that one a WebGL canvas or video?

[Samson Timoner]

It looks like NaCl we are using is of type: application/x-pnacl

Since we are editing the stream in Nacl (using C++) and then WebGL, to get the edited stream back to Javascript. You have to stream it. That's pretty expensive in terms of delay introduced.

Thanks.

-- Samson

[Niklas Enbom]

Got it. Since <embed> elements aren't tied to the DOM I don't think there's any fast way to get the data into a canvas and then into a MediaStream. A crazy idea you might want to try:

- Create a canvas with the desired resolution (content of canvas should be irrelevant)

- Create a MediaStream from the canvas with desired frame rate using CaptureStream.

- Connect to that MediaStream in NaCl using https://developer.chrome.com/native-client/pepper_stable/cpp/classpp_1_1_media_stream_video_track

- Inject the NaCl generated images into each frame.

I have no idea if this will work or be efficient, but it could be worth testing.

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CAFJa7t6Yuof9grFyX3y8M7VgNKfCPiQsBvUvHg%3DqCmsr%2B3r0NQ%40mail.gmail.com.

[Mathieu Hofman]

canvas.drawImage combined with canvas.captureStream is unfortunately a bit of necessary hack to do this right now.

Is there any reason you cannot do your processing using WebGL in a canvas directly instead of NaCl? Hopefully with the upcoming OffscreenCanvas it shouldn't be to damaging on the main thread's perf.

Alternatively you could push Chrome to open up the API that allows a NaCl client to generate MediaStream content. Right now they're restricted to Hangouts. See: https://bugs.chromium.org/p/chromium/issues/detail?id=352205 and https://bugs.chromium.org/p/chromium/issues/detail?id=352219

FYI, there's work going on in the W3C Media Capture TF to handle processing of live video streams a little more naturally, i.e. without using canvas.drawImage and canvas.captureStream, but instead by creating a pipe that handles ImageBitmap objects for each new frame: https://github.com/w3c/mediacapture-worker

Mathieu

[Waikit Lau]

Niklas,

What is the timeline for the feature making it into a release?

Thx!

On Tuesday, January 26, 2016 at 3:24:41 PM UTC-5, Niklas Enbom wrote:

[Niklas Enbom]

We're targeting removing the flag soon, which would mean Chrome 51.

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/f65b3a60-582f-49fb-a09e-f88974915348%40googlegroups.com.

[Max Stoller]

Sorry to bump this thread, but is there an issue in the Chromium tracker about for "Canvas in background tabs does not update properly."? I ran into this bug and just discovered this thread.

Here's my issue:

https://bugs.chromium.org/p/chromium/issues/detail?id=639105

Thanks

[Bubble Goos]

Hello,

Weird thing is that captureStream works in canvas, but not for video elements.

Video.streamCapture work in firefox tho. What's up about that?

Thanks!

[Niklas Enbom]

This is not fully release yet, you have to enable the "Experimental Web Platform features" flag to make it work.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrtc+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/420d43eb-4141-413d-91c7-8a7a2e01cf1a%40googlegroups.com.

[Darren Guo]

Hello, Niklas,

Could you tell me how to run the sample https://cdn.rawgit.com/uysalere/js-demos/master/peerconnection2d.html? i tried it but i don't know how to make it work to see the stream in remote video element, could you give some detailed steps, thank you very much in advance.

Regards,

Darren

[PhistucK]

Did you enable the flag as instructed?

☆PhistucK

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrtc+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/ccc47cab-bc0a-42d4-97fa-c4523061a3fa%40googlegroups.com.

[Darren Guo]

Hi Phistuck,

Thank you for your reply. I enabled the flag, but i don't find how to setup server, i found a link from github, but the link can't be accessed to now, the link is https://code.google.com/searchframe#xSWYf0NTG_Q/trunk/peerconnection/README&q=README%20package:webrtc%5C.googlecode%5C.com. where could find a correct link to setup server, Thank you.


Regards,

Darren

On Wednesday, December 7, 2016 at 2:01:36 AM UTC-8, PhistucK wrote:

Did you enable the flag as instructed?

☆PhistucK

On Wed, Dec 7, 2016 at 11:06 AM, Darren Guo <guot...@gmail.com> wrote:

Hello, Niklas,

Could you tell me how to run the sample https://cdn.rawgit.com/uysalere/js-demos/master/peerconnection2d.html? i tried it but i don't know how to make it work to see the stream in remote video element, could you give some detailed steps, thank you very much in advance.

Regards,

Darren

On Tuesday, January 26, 2016 at 12:06:38 PM UTC-8, Niklas Enbom wrote:

Hi all,

You can now start experimenting with creating MediaStreams from <canvas> and <video> elements in Chrome. You need Chrome 50 (current canary, soon dev channel) and enable the Experimental Web Platform features flag under chrome://flags.

Some samples:  https://cdn.rawgit.com/uysalere/js-demos/master/intro.html

This is still work in progress but feel free to file bugs. Known issues:

- Audio is still not supported from <video> elements. 

- Canvas in background tabs does not update properly.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.

[Darren Guo]

Thank you Niklas for your reply, because Phistuck also followed this question, i followed this question in another thread. Just let you know, Thank you any way.

Regards,

Darren

On Friday, November 18, 2016 at 12:25:40 PM UTC-8, Niklas Enbom wrote:

This is not fully release yet, you have to enable the "Experimental Web Platform features" flag to make it work.

On Tue, Nov 8, 2016 at 7:40 AM, Bubble Goos <outdre...@gmail.com> wrote:

Hello,

Weird thing is that captureStream works in canvas, but not for video elements.

Video.streamCapture work in firefox tho. What's up about that?

Thanks!

On Tuesday, January 26, 2016 at 9:06:38 PM UTC+1, Niklas Enbom wrote:

Hi all,

You can now start experimenting with creating MediaStreams from <canvas> and <video> elements in Chrome. You need Chrome 50 (current canary, soon dev channel) and enable the Experimental Web Platform features flag under chrome://flags.

Some samples:  https://cdn.rawgit.com/uysalere/js-demos/master/intro.html

This is still work in progress but feel free to file bugs. Known issues:

- Audio is still not supported from <video> elements. 

- Canvas in background tabs does not update properly.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.