Adding a videoTrack to an existing mediaStream and play it in video element.

[indranil chandra]

HI,

I am upgrading an audio call to audio /video.
While new negotiation occurs successfully , and the remote client receives the upgraded video stream.

If i play the new stream  in a video element it runs fine.

But when I extract the video track and add it to an existing media-stream with an audio Track , then the video is not playing the video track , and only audio track runs properly.

Could anyone help me in this.?

The code is:-

 pc.onaddstream = function (evt) {
            console.log("New stream added");
           
            var remoteMediaStream=evt.stream;
            if(remoteMedia==null){
            remoteMedia=remoteMediaStream;
            }
            else
            {
                remoteMedia.addTrack(remoteMediaStream.getVideoTracks()[0]); //add the video track to the existing stream
            }
            if(remoteVideo!=null)  {
            //    $(remoteVideo).remove();
                //remoteVideo=null;
            }
            remoteVideo = document.getElementById('remotevideoCall_'+toUri);//avCaptureUtil.createVideoElement(toUri);
            remoteVideo.src = window.URL.createObjectURL(remoteMedia);


Here only the sound is playing.

Thanks ,
Indranil

[Dennis]

Just a thought (I haven't checked to see if this is the case), but is the new track muted by default?  I wonder if remoteMedia.getVideoTracks()[0].enabled is "false".  It might be according to http://dev.w3.org/2011/webrtc/editor/webrtc.html#events-on-mediastream in the event a remote peer adds a track or addtrack is called locally, it says "Initialize track’s readyState attribute to muted." and then fires the addtrack event.  

If that's true, try the following and see if it works (fyi didnt run to see if any typo bugs, nor do I even know if this is right, but this is what I would "try" on my first attempt):

remoteMedia.onaddtrack = function (evt) {

        console.log("New remote track added");
        // omitted: check to see if video track exists first before doing this
        remoteMedia.getVideoTracks()[0].enabled = true;

    };

pc.onaddstream = function (evt) {

    console.log("New stream added");

    var remoteMediaStream = evt.stream;

    if (remoteMedia == null) {

        remoteMedia     = remoteMediaStream;

        remoteVideo     = document.getElementById('remotevideoCall_'+toUri);//avCaptureUtil.createVideoElement(toUri);

        remoteVideo.src = window.URL.createObjectURL(remoteMedia);

    }

    else {

        remoteMedia.addTrack(remoteMediaStream.getVideoTracks()[0]); //add the video track to the existing stream

    }

I dont know if "remoteVideo.src = window.URL.createObjectURL(remoteMedia);" is necessary to do in the case that it's just being a track being added, so I moved that under the 'remoteMedia == null' case.

[Dennis]

actually, probably more like this:

function onRemoteTrackAdded() {

    console.log("New remote track added");
    // omitted: check to see if video track exists first before doing this
    remoteMedia.getVideoTracks()[0].enabled = true;

}

pc.onaddstream = function (evt) {

    console.log("New stream added");

    var remoteMediaStream = evt.stream;

    if (remoteMedia == null) {

        remoteMedia            = remoteMediaStream;

        remoteMedia.onaddtrack = onRemoteTrackAdded;  // add the handler here

[indranil chandra]

Hi Dennis,

Thanks a lot for your wonderful elaborate reply.
I checked the enabled state of the video track after it is added to existing remoteMedia stream and its value is true.


From inspect element:-

remoteMedia.getVideoTracks()[0]: MediaStreamTrack
enabled: true
id: "68cccbf7-cd73-4193-9ef0-bb36e0a37f70"
kind: "video"
label: "O2M4TOdnmKIRIGZ81WWB1OnnPIVtWGPZWDiRv0"
onended: null
onmute: null
onunmute: null
readyState: "live"
__proto__: MediaStreamTrack

Please suggest ..

Regards,
Indranil

[Vikas]

Hi,

Thanks for the feedback. This probably relates to issue 873 (  https://
code.google.com/p/webrtc/issues/detail?id=873 ).

/Vikas

On Apr 2, 11:46 am, indranil chandra <chandra.indra...@gmail.com>
wrote:

> Hi Dennis,
>
> Thanks a lot for your wonderful elaborate reply.
> I checked the enabled state of the video track after it is added to

> existing remoteMedia stream and its value is *true*.
>
> From inspect element:-
>
> > *remoteMedia.getVideoTracks()[0]: MediaStreamTrack

> > enabled: true
> > id: "68cccbf7-cd73-4193-9ef0-bb36e0a37f70"
> > kind: "video"
> > label: "O2M4TOdnmKIRIGZ81WWB1OnnPIVtWGPZWDiRv0"
> > onended: null
> > onmute: null
> > onunmute: null
> > readyState: "live"
> > __proto__: MediaStreamTrack

> > *

> >> "Initialize track’s readyState attribute to muted." and then fires the *
> >> addtrack* event.

[indranil chandra]

Thanks VIkas.

This issue has no fix or comments.
So is it resolved yet ?

[Vikas]

Hi,

I don't think it has been resolved yet.

/Vikas

On Apr 2, 9:28 pm, indranil chandra <chandra.indra...@gmail.com>
wrote:

[José Luis Millán]

Hi all,

I' m experiencing this same scenario; I establish an audio only session. Everything goes as expected. 

Now I want to add video to the session, and I can follow one of these steps:

1- Create a new MediaStream (Via getUserMedia) with video only, add the new stream to peerconnection and renegotiate.

2- Add a video track to the stream in peerconnection (A track from a different local MediaStream) and renegotiate.

In both cases, the remote stream in the remote peer is attached to a HTML5 element.

In the fist case, the other peer detects the new stream (pc.onaddstream) and displays the video correctly in the HTML5 video element.

In the second case, the other peer detects the new track (stream.onaddtrack) but does not display any video in the HTML5 video element. 

Regarding to the SDP Offer in the renegotiation, the only difference is that in the first case the new stream is represented in the 'a=msid-semantic' line, which states the new stream addition. 

Should it be possible to add a new track to a local mediaStream in the peerconnection without needing to add a full new stream?

The chrome version I' m using is: 28.0.1485.0 dev

 

Regards.

2013/4/3 Vikas <vikasm...@webrtc.org>

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
José Luis Millán

[Vikas]

Hi Jose,

I can recreate the problem described below. I think we need to wait
once issue 873 is implemented.

/Vikas

On May 2, 9:41 am, José Luis Millán <jmil...@aliax.net> wrote:
> Hi all,
>
> I' m experiencing this same scenario; I establish an audio only session.
> Everything goes as expected.
>
> Now I want to add video to the session, and I can follow one of these steps:
>
> 1- Create a new MediaStream (Via getUserMedia) with video only, add the new
> stream to peerconnection and renegotiate.
> 2- Add a video track to the stream in peerconnection (A track from a
> different local MediaStream) and renegotiate.
>
> In both cases, the remote stream in the remote peer is attached to a HTML5
> element.
>
> In the fist case, the other peer detects the new stream (pc.onaddstream)
> and displays the video correctly in the HTML5 video element.
> In the second case, the other peer detects the new track
> (stream.onaddtrack) but does not display any video in the HTML5 video
> element.
>
> Regarding to the SDP Offer in the renegotiation, the only difference is
> that in the first case the new stream is represented in the
> 'a=msid-semantic' line, which states the new stream addition.
>
> Should it be possible to add a new track to a local mediaStream in the
> peerconnection without needing to add a full new stream?
>
> The chrome version I' m using is: 28.0.1485.0 dev
>
> Regards.
>

> 2013/4/3 Vikas <vikasmarw...@webrtc.org>

>
>
>
>
>
>
>
>
>
> > Hi,
>
> > I don't think it has been resolved yet.
>
> > /Vikas
>
> > On Apr 2, 9:28 pm, indranil chandra <chandra.indra...@gmail.com>
> > wrote:
> > > Thanks VIkas.
>
> > > This issue has no fix or comments.
> > > So is it resolved yet ?
>
> > --
>
> > ---
> > You received this message because you are subscribed to the Google Groups
> > "discuss-webrtc" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to discuss-webrt...@googlegroups.com.

> > For more options, visithttps://groups.google.com/groups/opt_out.
>
> --
> José Luis Millán

[José Luis Millán]

Thank you for your response Vikas,

I understand the issue 873 as such, but I do not understand how is it related to the issue exposed in this thread; why the video track is not displayed on the remote HTML5 video element in spite of being present in the stream attached to it.

Any clarification would be much appreciated.

Regards

 

2013/5/2 Vikas <vikasm...@webrtc.org>

For more options, visit https://groups.google.com/groups/opt_out.

--
José Luis Millán

[Vikas]

Hi Jose,

Actually you are right, it is a different problem as i was able to get
it to work if i set the src for video tag after renegotiation. It
won't work currently if you set the video src before renogotiate. Will
file a separate bug to track this. I tried with Chrome 28 canary & it
seems to work.

/Vikas

On May 3, 11:23 am, José Luis Millán <jmil...@aliax.net> wrote:
> Thank you for your response Vikas,
>
> I understand the issue 873 as such, but I do not understand how is it
> related to the issue exposed in this thread; why the video track is not
> displayed on the remote HTML5 video element in spite of being present in
> the stream attached to it.
>
> Any clarification would be much appreciated.
>
> Regards
>

> 2013/5/2 Vikas <vikasmarw...@webrtc.org>

[Vikas Marwaha]

Filed issue https://code.google.com/p/webrtc/issues/detail?id=1736

/Vikas

[José Luis Millán]

Hi Vikas,

2013/5/7 Vikas <vikasm...@webrtc.org>

Hi Jose,

Actually you are right, it is a different problem as i was able to get
it to work if i set the src for video tag after renegotiation. It
won't work currently if you set the video src before renogotiate. Will
file a separate bug to track this. I tried with Chrome 28 canary & it
seems to work.

In my case it doesn't work even setting the src after renegotiation.

After firing onaddtrack() for stream `pc2.getRemoteStreams()[0]`:

```

vid2.src = webkitURL.createObjectURL(pc2.getRemoteStreams()[0]) 

```

The above does return a blob url but no media is displayed on the HTML5 video element. Could you please confirm that it works for you?

I'm testing with Chrome 28.0.1485.0 dev.

Thanks

For more options, visit https://groups.google.com/groups/opt_out.

--
José Luis Millán

[Vikas]

Hi Jose,

I tested on chrome 28.0.1499.0 canary on windows and it worked. Can
your share your test code so i can try?

/Vikas

On May 7, 5:09 am, José Luis Millán <jmil...@aliax.net> wrote:
> Hi Vikas,
>
> 2013/5/7 Vikas <vikasmarw...@webrtc.org>

[José Luis Millán]

Hi,

Please, find attached the test code to test wheter the HTML5 video element does reproduce the new added remote track.

Steps are:

1- Start

2- Call

3- Change Media

4- Call Again

Thanks for the given feedback in advance

Regards

2013/5/7 Vikas <vikasm...@webrtc.org>

For more options, visit https://groups.google.com/groups/opt_out.

--
José Luis Millán

[Vikas]

Hi Jose,

Thanks for the demo code, it seems to work fine on Mac with canary
build 28.0.1500.3. I do see the remote video and get the notification
for new track added. Can you try with latest canary?

/Vikas

On May 8, 8:36 am, José Luis Millán <jmil...@aliax.net> wrote:
> Hi,
>
> Please, find attached the test code to test wheter the HTML5 video element
> does reproduce the new added remote track.
>
> Steps are:
>
> 1- Start
> 2- Call
> 3- Change Media
> 4- Call Again
>
> Thanks for the given feedback in advance
>
> Regards
>

> 2013/5/7 Vikas <vikasmarw...@webrtc.org>

>  reoffer.html
> 5KViewDownload

[José Luis Millán]

Hi Vikas,

It's actually solved. Works in version 28.0.1500.5 dev. Until today, I have had several problems to install this new version due to the 'libc6' new version needed by chrome. I'm a Debian user.

Thank you very much

2013/5/9 Vikas <vikasm...@webrtc.org>

--
José Luis Millán