Screen sharing with Kurento Utils 6

3,212 views
Skip to first unread message

charli...@gmail.com

unread,
Jul 10, 2015, 8:32:33 AM7/10/15
to kur...@googlegroups.com
Hello,

We are currently using Kurento Media 6 and the last version of Kurento Utils. However, we would like to use screen sharing and as we saw on https://github.com/Kurento/kurento-utils-js, it is possible to do it with the module kurento-browser-extensions or by writing the getScreenConstraints function.

Is it possible to get the module kurento-browser-extensions? If not, is it possible to have some more documentation on the getScreenConstraints function (prototype, tasks...)?

Thanks in advance,
Charlie

Randall Bennett

unread,
Jul 10, 2015, 9:41:41 AM7/10/15
to kur...@googlegroups.com, charli...@gmail.com
Hey there,

Not a KMS team member, but I actually don't think it's possible without a browser extension. By design, screensharing should be a difficult task, which is why current browser vendors have chosen this route. You'll need to use extensions to pull it off, so your user has a bunch of chances to opt-out.

rb

charli...@gmail.com

unread,
Jul 13, 2015, 3:58:34 AM7/13/15
to kur...@googlegroups.com, charli...@gmail.com
Thank you for your reply. We are aware of the need of an extension. We saw that this extension can be useful: https://chrome.google.com/webstore/detail/screen-capturing/ajhifddimkapgcifgcodmmfdlknahffk

But we would like to know how to implement the getScreenConstraints (by using the extension functions for example or with kurento-browser-extensions if it exists).

Thanks in advance,
Charlie

Jan Moons

unread,
Jul 24, 2015, 7:19:47 AM7/24/15
to kurento, charli...@gmail.com
Have a look at my code, I think you know how to create the chrome extension right?

Now in the javascript on you web page at the following code, this will interact with the extension.

var screencastConstraints =
{
audio: false,
video: {
mandatory: {
maxWidth: 640,
maxHeight: 480,
maxFrameRate: 15,
minFrameRate: 15
}
}
};

window.addEventListener('message', function (msg) {
if (msg.data.state === 'completed') {
//alert('state: ' + msg.data.streamId);
if (msg.data.streamId != undefined) {
screencastConstraints.video.mandatory.chromeMediaSource = 'desktop';
screencastConstraints.video.mandatory.chromeMediaSourceId = msg.data.streamId;
self.streamScreen();
}
}
});

self.streamScreen = function () {
$log.debug('Stream screen');

kurentoUtils.WebRtcPeer.startSendOnly(null, function (offerSdp, wp) {
webRtcPeerScreencast = wp;
$log.debug('Invoking SDP offer callback function');
var message = {
id: 'screencast',
from: userID,
to: observerID,
sdpOffer: offerSdp
};
self.sendMessage(message);
}, function (error) {
$log.error('Stream screen error: ', error);
}, screencastConstraints);
};

Hope this helps!

Jan

eakaw...@gmail.com

unread,
Oct 8, 2015, 2:56:51 AM10/8/15
to kurento, charli...@gmail.com
Did you solve this problem? I also cannot find this library 'kurento-browser-extensions' and I don't know how to override getScreenConstraints to connect with this extension https://chrome.google.com/webstore/detail/screen-capturing/ajhifddimkapgcifgcodmmfdlknahffk

Ivan Gracia

unread,
Oct 8, 2015, 3:48:43 AM10/8/15
to Kurento Public, charli...@gmail.com
Hi,

Have a look at this extension. Maybe that one can help you! There is also some info about getScreenConstraints in this list, but can't find it right now.

Cheers,

Ivan Gracia



--
You received this message because you are subscribed to the Google Groups "kurento" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kurento+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

eakaw...@gmail.com

unread,
Oct 8, 2015, 10:29:23 AM10/8/15
to kurento, charli...@gmail.com
Thanks. I manage to get it to work now.

krask...@gmail.com

unread,
Mar 17, 2016, 8:14:30 PM3/17/16
to kurento, charli...@gmail.com, eakaw...@gmail.com
Could you please provide your solution? I use chrome extension from Muaz

Ivan Gracia

unread,
Apr 8, 2016, 2:34:48 AM4/8/16
to Kurento Public, charli...@gmail.com, eakaw...@gmail.com
Try this one.

Ivan Gracia


Message has been deleted
Message has been deleted

francis z

unread,
Feb 22, 2017, 7:00:12 AM2/22/17
to kurento, charli...@gmail.com, eakaw...@gmail.com
The link is broken.

Andy Savickij

unread,
Apr 4, 2017, 6:19:33 AM4/4/17
to kurento, charli...@gmail.com, eakaw...@gmail.com
Try https://webrtc.github.io/samples/src/content/extensions/desktopcapture/

среда, 22 февраля 2017 г., 15:00:12 UTC+3 пользователь drake написал:

vijay krishna

unread,
Apr 17, 2017, 2:03:34 PM4/17/17
to kurento, charli...@gmail.com
Hi All,

I found a kurento example url which consists of screensharing.Why didn't they provide code for that.I am assuming that consists of renegotiation because my screensharing code is too slow due to renegotiation

krishna...@gmail.com

unread,
Apr 19, 2017, 9:58:53 AM4/19/17
to kurento, charli...@gmail.com
Hi All, 
Can anybody help me out in making us of getScreenConstraints function properly. I am not able to get the screen on the other end. Please help if you have any suggestions. I am using the below code. 

getScreenConstraints(function(error, screen_constraints) {
    if (error) {
        return alert(error);
    }
    navigator.getUserMedia = navigator.webkitGetUserMedia || navigator.mozGetUserMedia;
navigator.getUserMedia({
        video: screen_constraints
    }, function(stream) {
        videoInput.src = URL.createObjectURL(stream);
shareScreen(stream);
    }, function(error) {
        alert(JSON.stringify(error, null, '\t'));
    });
});


Thank you,
Krishnakanth.P 


On Friday, July 10, 2015 at 6:02:33 PM UTC+5:30, charli...@gmail.com wrote:

Karthik Aiyer

unread,
May 8, 2017, 2:49:28 AM5/8/17
to kurento, charli...@gmail.com
Heyy...were you able to get this to work. I am facing the same issue. I am able to get the stream as local_video but when someone joins the room, they still see the webcam stream!

vijay krishna

unread,
May 8, 2017, 4:47:07 AM5/8/17
to kurento, charli...@gmail.com
Karthik,

Did you negotiate screen stream when you got that.?

If you didn't do that maybe you should do it. 

If you did do it just to be sure add few event listeners to endpoint to see ice candidates are negotiated successfully and connected and to check if media is flowing in and out of endpoint.Other thing you might be missing is to connect screen endpoint(publisher) to receivers endpoints(subscribers).Again i would advise you to add few event listeners to subscribers too.I was able to get it working. I might be posting a working prototype in community soon.

Karthik Aiyer

unread,
May 8, 2017, 5:01:16 AM5/8/17
to kurento, charli...@gmail.com
Heyy Vijay,

Thanks for the response. I will check out the re-negotiation thingy u mention about. Very possible it isnt going through successfully.
Really looking forward to that prototype u have developed. Would really appreciate it if you can post a few snippets of the offer-answer process that you have followed! Thanks.

vijay krishna

unread,
May 8, 2017, 4:14:16 PM5/8/17
to kurento, charli...@gmail.com
I can post those based on project and code base that you are working on...I am trying make a stable one on kurento-room. It has few glitches because it is being developed on a POC. When i am done with it and I am planning on posting it in community. 

Karthik Aiyer

unread,
May 8, 2017, 9:02:10 PM5/8/17
to kurento, charli...@gmail.com
The problem I seem to facing is that the video.src is not being set correctly by kurento-utils.js. I'm pretty sure the library is doing its job correctly, but then I have no idea to find out if I am sending the correct video for kurento-utils to set correctly.
Currently:
1st get the audio stream as:

function shareScreen(){
   var audioConstraints = {
       audio: false,  //turn this true in case you want to share this in a single stream
        video: true,
   };
   navigator.getUserMedia(audioConstraints, function(stream) {
       initiateScreenSharing(stream);
   }, function(error) {
       console.error("Could not get audio stream! " + error);
   });
}

Then I carry on to getting the video:
function initiateScreenSharing(audioStream){
   getScreenId(function (error, sourceId, screen_constraints) {
       navigator.getUserMedia = navigator.mozGetUserMedia || navigator.webkitGetUserMedia;
       navigator.getUserMedia(screen_constraints, function (stream) {
           var constraints = {
               audio: true,
               video: {
                   frameRate: {
                       min: 1, ideal: 15, max: 30
                   },
                   width: {
                       min: 32, ideal: 50, max: 320
                   },
                   height: {
                       min: 32, ideal: 50, max: 320
                   }
               }
           };
           
           var localParticipant = new Participant(sessionId);
           participants[sessionId] = localParticipant;
           localVideo = document.getElementById("local_video");
           var video = localVideo;

            // bind function so that calling 'this' in that function will receive the current instance
           var options = {
               localVideo: video,
               videoStream: stream,
               mediaConstraints: constraints,
               onicecandidate: localParticipant.onIceCandidate.bind(localParticipant),
               sendSource: 'desktop'
           };

            localParticipant.rtcPeer = new kurentoUtils.WebRtcPeer.WebRtcPeerSendrecv(options, function (error) {
               if (error) {
                   return console.error(error);
               }

                // Set localVideo to new object if on IE/Safari
               localVideo = document.getElementById("local_video");

                // initial main video to local first
               localVideoCurrentId = sessionId;
               localVideo.muted = true;

                console.log("local participant id : " + sessionId);
               this.generateOffer(localParticipant.offerToReceiveVideo.bind(localParticipant));
           });
       }, function (error) {
           console.error(error);
       });
   });
}

Is there any step m missing? I think the new offer being set should get the job done but I feel somehow "localParticipant" doesn't have the updated stream as its video and hence kurento-utils is not setting it up correctly.

vijay krishna

unread,
May 9, 2017, 4:26:00 PM5/9/17
to kurento, charli...@gmail.com
function shareScreen(){
getScreenId(function(error, sourceId, screen_constraints) {
        if(error == 'not-installed') {
          alert('Please install Chrome extension. See the link below.');
          return;
        }

        if(error == 'installed-disabled') {
          alert('Please install or enable Chrome extension. Please check "chrome://extensions" page.');
          return;
        }

        if(error == 'permission-denied') {
          alert('Please make sure you are using HTTPs. Because HTTPs is required.');
          return;
        }

        console.info('getScreenId callback \n(error, sourceId, screen_constraints) =>\n', error, sourceId, screen_constraints);

        var participant = participants[name];
        participant.screenShareConstraints =screen_constraints;
        
        participant.screenShare = true;
        var startTime = new Date();
        navigator.getUserMedia(screen_constraints,function(videoStream){
        console.info(new Date().getTime());
        participant.getVideoElement().src = URL.createObjectURL(videoStream);
        navigator.getUserMedia({video:false,audio:true}, function (audioStream) {
//         audioStream.addTrack(videoStream.getVideoTracks()[0])
        var options = {
//         localVideo: document.querySelector('video'),
        audioStream:audioStream,
        videoStream:videoStream,
        onicecandidate: participant.onIceCandidate.bind(participant)
        }
        participant.rtcPeer = new kurentoUtils.WebRtcPeer.WebRtcPeerSendonly(options,
        function (error) {
        if(error) {
        return console.error(error);
        }
        this.generateOffer (participant.offerToReceiveVideo.bind(participant));
        });
        },function (error) {
        console.error(error);
        });
        },function(error){
        console.error(error);
        })
    });
}


By looking at your code i am assuming you are working on kurento-group-call project. Add above code to get screen constraints and dispose previous webrtc peer and create a new peer with new audio and screen stream as above and continue the negotiation process by releasing old webrtc endpoint in your java code and creating a new webrtc endpoint. Gather ice candidates for that and connect this new webrtc endpoint to all objects available in incomingmedia map.

for (final UserSession participant : room.getParticipants()) {
       try {
        if(!participant.getName().equals(this.getName())){
        this.outgoingMedia.connect(participant.incomingMedia.get(this.getName()));
        }
       } catch (final Exception e) {
         log.debug("ROOM {}: participant {} could not be notified", name, participant.getName(), e);
       }
     }

add event listeners to check if media is flowing in and out of this new webrtc endpoint.

this.outgoingMedia.addOnIceGatheringDoneListener(new EventListener<OnIceGatheringDoneEvent>() {

@Override
public void onEvent(OnIceGatheringDoneEvent event) {
log.info("ice candidate gathering done after {} ms ",new Date().getTime()-startTime);
}
});
   
    this.outgoingMedia.addMediaFlowInStateChangeListener(new EventListener<MediaFlowInStateChangeEvent>() {

@Override
public void onEvent(MediaFlowInStateChangeEvent event) {
log.info("media flow in state changed : {}",event.getState());
}
});
   
    this.outgoingMedia.addIceComponentStateChangeListener(new EventListener<IceComponentStateChangeEvent>() {

@Override
public void onEvent(IceComponentStateChangeEvent event) {
log.info("ice candidate state changed : {}",event.getState());
}
});
   
    this.outgoingMedia.addMediaFlowOutStateChangeListener(new EventListener<MediaFlowOutStateChangeEvent>() {

@Override
public void onEvent(MediaFlowOutStateChangeEvent event) {
log.info("media flow out state changed : {}",event.getState());
}
});

Karthik Aiyer

unread,
May 10, 2017, 9:30:15 PM5/10/17
to kurento, charli...@gmail.com
Yes. I am using the group call code base.

By the way, the code u have provided seems very similar to what I have implemented but doesn't seem to be working for me.

When you say "dispose previous webrtc peer and create a new peer with new audio and screen stream" isn't it the code below? :
var options = {
               localVideo: video,
               videoStream: stream,
               mediaConstraints: constraints,
               onicecandidate: localParticipant.onIceCandidate.bind(localParticipant),
               sendSource: 'desktop'
           };

            localParticipant.rtcPeer = new kurentoUtils.WebRtcPeer.WebRtcPeerSendrecv(options, function (error)

Further, when you say "releasing old webrtc endpoint in your java code and creating a new webrtc endpoint", do you mean literally making the endpoint nil and then setting up a completely new end point? That would really not mean re-negotiation but rather kind of rejoining the room. Correct me if I'm wrong.

vijay krishna

unread,
May 11, 2017, 12:34:51 PM5/11/17
to kurento, charli...@gmail.com
I am afraid KMS WebRtcEndpoints that are provided to us don't support SDP renegotiation and i didn't mention anywhere about "my solution works with SDP renegotiation".When i said "release endpoint" i meant literally release endpoint not making it "nil". There is a method to release endpoints.

this.outgoingMedia.release();



Code that i have provided will work if you place it correctly.I just posted significant code that makes screen share work, calling it at right place is completely on your end.And Yes It is similar to rejoining a call.Only difference is my solution is faster because we won't be releasing any receiving endpoints .We will just pluck out publishing webrtcendpoint(outgoingMedia) of user sharing screen from mesh network , release it and negotiate a  new webrtcEndpoint with SDP offer generated based on screen stream , connect it back to the mesh network.


public void receiveVideoFrom(UserSession sender, String sdpOffer,JsonObject jsonMessage,Room room) throws IOException {

   log.info("USER {}: connecting with {} in room {}", this.name, sender.getName(), this.roomName);


    log.debug("USER {}: SdpOffer for {} is {}", this.name, sender.getName(), sdpOffer);

   

   final String ipSdpAnswer;

   long startTime=new Date().getTime();

   if(jsonMessage.has("screenShare")&&jsonMessage.get("screenShare")!=null){

          log.info("screen share");

      this.outgoingMedia.release();

          this.outgoingMedia = new WebRtcEndpoint.Builder(pipeline).build();

     this.outgoingMedia.addIceCandidateFoundListener(new EventListener<IceCandidateFoundEvent>() {


                @Override

              public void onEvent(IceCandidateFoundEvent event) {

              JsonObject response = new JsonObject();

                response.addProperty("id", "iceCandidate");

            response.addProperty("name", name);

            response.add("candidate", JsonUtils.toJsonObject(event.getCandidate()));

               try {

            synchronized (session) {

                 session.sendMessage(new TextMessage(response.toString()));

                   }

            } catch (IOException e) {

                log.debug(e.getMessage());

           }

            }

            });

     

        this.outgoingMedia.addOnIceGatheringDoneListener(new EventListener<OnIceGatheringDoneEvent>() {


                        @Override

                      public void onEvent(OnIceGatheringDoneEvent event) {

                           log.info("ice candidate gathering done after {} ms ",new Date().getTime()-startTime);

                  }

              });

   

        this.outgoingMedia.addMediaFlowInStateChangeListener(new EventListener<MediaFlowInStateChangeEvent>() {


                        @Override

                      public void onEvent(MediaFlowInStateChangeEvent event) {

                               log.info("media flow in state changed : {}",event.getState());

                 }

              });

   

        this.outgoingMedia.addIceComponentStateChangeListener(new EventListener<IceComponentStateChangeEvent>() {


                        @Override

                      public void onEvent(IceComponentStateChangeEvent event) {

                              log.info("ice candidate state changed : {}",event.getState());

                 }

              });

   

        this.outgoingMedia.addMediaFlowOutStateChangeListener(new EventListener<MediaFlowOutStateChangeEvent>() {


                        @Override

                      public void onEvent(MediaFlowOutStateChangeEvent event) {

                              log.info("media flow out state changed : {}",event.getState());

                }

              });

    ipSdpAnswer=this.outgoingMedia.processOffer(sdpOffer);

         this.outgoingMedia.gatherCandidates();

    for (final UserSession participant : room.getParticipants()) {

             try {

                  if(!participant.getName().equals(this.getName())){

                             this.outgoingMedia.connect(participant.incomingMedia.get(this.getName()));

                     }

              } catch (final Exception e) {

            log.debug("ROOM {}: participant {} could not be notified", name, participant.getName(), e);

          }

            }

   }

   else{

          ipSdpAnswer = this.getEndpointForUser(sender).processOffer(sdpOffer);

          this.getEndpointForUser(sender).gatherCandidates();

   }

   log.debug("ipSdpAnswer : "+ipSdpAnswer);

   final JsonObject scParams = new JsonObject();

   scParams.addProperty("id", "receiveVideoAnswer");

   scParams.addProperty("name", sender.getName());

   scParams.addProperty("sdpAnswer", ipSdpAnswer);


    log.debug("USER {}: SdpAnswer for {} is {}", this.name, sender.getName(), ipSdpAnswer);

   this.sendMessage(scParams);

   log.debug("gather candidates");

 }




And all webrtc peers created should be disposed else they will stay open with a connection to media server.

peer.dispose()


And Yes code i have provided is very similar to your code but my code is complete and works.
Reply all
Reply to author
Forward
0 new messages