Selfie segmentation / virtual background via Webrtc

[Elena]

I was able to implement Selfie segmentation on a videoconference project in Vue.js, but I cannot in any way send the result of the canvas to a remote peer.

I tried with 'insertable streams' and also with the captureStream() function, but to no avail.

After generating the canvas output, I try to add the track to the peer with a simple addTrack

this.signalClient.peers().forEach((p) => { p.removeTrack(old, this.localStream); p.addTrack(trackCanvas, this.localStream)})

Anyone know how this is done? I can't find any documentation about it, nor a sample anywhere

Thanks a lot

 

[Lorenzo Miniero]

You can check this demo I added to the Janus repo a few months ago: https://janus.conf.meetecho.com/virtualbg

It does indeed use the canvas as source of a PeerConnection, to send the encoded video to the EchoTest demo in Janus.

L.