Recommended way to implement a SFU with webrtc

[Zhipeng Teng]

Dear all,

I want to know which is the recommended way to implement a SFU server using webrtc.

The SFU server need to be high performance, so I want to implement in C++ using the webrtc c++ library.

After going through the source code, I think I can implment a class like VideoReceiveStream2 to implment the packet receive, nack send, fec resolve and so on...

and after receiving the packet, using the RTPSender to forward the packet to other peers.

Is that right?

Any sugguestions will be greatly appreciated.

[Tsahi Levent-Levi]

Hi,

The best recommendation is simply not to do it.

Use Janus, Jitsi or madiasoup.

If you're really after a C++ implementation then take Janus.

This would reduce your need to deal with writing an SFU altogether and let you focus on the actual use case you are after.

The challenge with your question, is that in most cases, I wouldn't be suggesting developing an SFU to someone who is asking it because... well... it requires too much knowledge and experience with VoIP and WebRTC. And having that much of experience usually means the question wouldn't be asked at all.

--

Regards,

Tsahi Levent-Levi

Analyst & Consultant

BlogGeek.me

Want to get more of your WebRTC sessions effectively connected? Enroll to my free video course: http://bit.ly/32Y3qZK

[Sean DuBois]

Tsahi is right that building a SFU for most applications isn't the right
choice. It is like building your own HTTP server to server static
HTML. It is pretty unlikely that you will do better then nginx or
Apache. I do recommend building an SFU for fun/learning though!

Building a MVP SFU isn't that hard and it is a great learning experience.
I would go even lower then libwebrtc though. Most WebRTC implementations
(client and server) are running the same code. OpenSSL, libsrtp,
usrsctp. The software gets different in the RTP/RTCP layer.

If you are ok with media only just do SDP, ICE, DTLS and SRTP. I
would recommend going with C/C++, Java, Go, Typescript, C# or Rust. All
these languages have the libraries you need today.

Think of your SFU as just 'RTP packet fanout' for your MVP. It is *way
more* complicated then this in a production application. But for your
MVP this is enough. These are the high level programming tasks.

* Create a 'PeerConnection' on your server.
- It consumes and distributes RTP packets.
- Each 'room' will be an array of 'PeerConnections' in your server.

* Implement the PeerConnection
- Generate a self-signed DTLS certificate
- Start a ICE-lite listener
- Generate a Offer that contains those details and two media sections
- Respond to STUN packets from remote ICE Agent
- Do DTLS Handshake
- Extract keying material that is used to encrypt/decrypt S(RTP)
packets

* Distribute your RTP packets
- On each incoming packets decrypt
- Then distribute to each 'PeerConnection'
- You will encrypt via SRTP
- Then send via ICE-lite socket

I glossed over lots of details. I think this is enough info to go
learn. Check out the book https://webrtcforthecurious.com/ it is an Open
Source book that teaches the protocols and might be helpful.

If you do decide to build your own server I would be careful about using
libwebrtc. I have helped some companies that went that route. They had
performance issues that made them end up switching. You also have to keep
up with API changes.

On Sat, Feb 27, 2021 at 06:12:15AM -0800, Tsahi Levent-Levi wrote:
> Hi,
>
> The best recommendation is simply not to do it.
> Use Janus, Jitsi or madiasoup.
> If you're really after a C++ implementation then take Janus.
>
> This would reduce your need to deal with writing an SFU altogether and let
> you focus on the actual use case you are after.
>
> The challenge with your question, is that in most cases, I wouldn't be
> suggesting developing an SFU to someone who is asking it because... well...
> it requires too much knowledge and experience with VoIP and WebRTC. And
> having that much of experience usually means the question wouldn't be asked
> at all.
>
> --
> Regards,

> *Tsahi Levent-Levi*
> Analyst & Consultant
> BlogGeek.me <https://bloggeek.me>

>
> Want to get more of your WebRTC sessions effectively connected? Enroll to
> my free video course: http://bit.ly/32Y3qZK
>
> On Friday, February 26, 2021 at 11:39:04 AM UTC+2 tengz...@gmail.com wrote:
>
> > Dear all,
> > I want to know which is the recommended way to implement a SFU server
> > using webrtc.
> > The SFU server need to be high performance, so I want to implement in C++
> > using the webrtc c++ library.
> > After going through the source code, I think I can implment a class like
> > VideoReceiveStream2 to implment the packet receive, nack send, fec resolve
> > and so on...
> > and after receiving the packet, using the RTPSender to forward the packet
> > to other peers.
> > Is that right?
> >
> > Any sugguestions will be greatly appreciated.
> >
>

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/aaf03dc4-b2fb-4388-8ef9-4053dfa08077n%40googlegroups.com.

[Jan Deinhard]

What you wrote is interesting. What is the alternative to use libwebrtc on the server? Are you suggesting to choose individual libraries per protocol?

Would you make the same suggestion for mobile apps?

Thanks

Jan

[Tsahi Levent-Levi]

Jan,

For the server, I'd use libwebrtc rather than every individual library separately. That said, I wouldn't build an SFU.

For mobile apps, only go with libwebrtc. That would be my suggestion. The main reason is that libwebrtc is working on mobile quite well and is well maintained there (ignoring for a second the challenging build process itself). The reason I wouldn't use anything else for a mobile client is the necessary investment in ongoing maintenance and improvements. Especially if you consider the work that needs to be done to keep track and interoperability working with the... libwebrtc implementation on the browser.

Regards,

Tsahi

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/91500d0f-a4ca-4c40-8dcd-27238faf3840n%40googlegroups.com.

--

Regards,

Tsahi Levent-Levi

Analyst & Consultant

BlogGeek.me

[Sean DuBois]

Yes! I think that building a SFU will teach you a lot. Even if the SFU
you wrote isn't the one you end up using. Those skills will really come
in handy when tough problems come up.

Pulling from this[0] list here is libwebrtc in Open Source WebRTC
servers

* licode - no libwebrtc usage
* Janus - No libwebrtc usage
* Kurento - No libwebrtc usage
* Jitsi - No libwebrtc usage
* Meedooze - Uses libwebrtc for VAD
* Mediasoup - Uses libwebrtc for Congestion Control temporarily [3]
* Pion - No libwebrtc usage

From my experience libwebrtc on a server makes sense if you are doing
1:1 experiences. Like having a user upload media via WebRTC. Or you are
serving a media source from the server and you are encoding live.

Projects that had problems because of libwebrtc were SFUs and protocol
bridging. libwebrtc has a lot of features you don't end up using, and they cost
you performance/complexity wise.

For mobile I think using libwebrtc is your best choice. The problem set is more
constrained. Check out flutter-webrtc[1] to remove some of the complexity of
shipping a client to multiple platforms.

For 'larger host/need more flexibility' I have seen some great success
stories with GStreamer's webrtcbin. You can tune/control things a lot
better.

If you want to embedded (mbedtls, minimal heap allocations) I worked on
KVS WebRTC [2]. This was for security cameras/IoT where things like
OpenSSL are too big.

I make mistakes though. I encourage people on the list to push back
where I am wrong :)

[0] https://stackoverflow.com/questions/24857637/current-state-of-javascript-webrtc-libraries/24879451#24879451
[1] https://github.com/flutter-webrtc/flutter-webrtc
[2] https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-c
[3] https://github.com/versatica/mediasoup/issues/344

> To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/91500d0f-a4ca-4c40-8dcd-27238faf3840n%40googlegroups.com.

[guest271314]

What is the use case?

In theory WebRTC is not necessary on the server.

Users can "upload" live raw data (WebRTC Insertable Streams; Media Capture Insertable Streams) to the server. The server can serve that live "raw" data to N requests.

FWIW a proof-of-concept of a user-defined Web radio station, a media server, that is the browser itself to serve an infinite (Opus) stream of the audio output to headphones or speakers, or the raw data without being output to speakers https://bugs.chromium.org/p/chromium/issues/detail?id=1161429#c97. 

[Michael Glenn Williams]

Thank you so much Tsahi, great advice!

--

[Michael Glenn Williams]

Tsahi,

Thank you for this wonderful advice!

--

Michael Williams CEO | CTO Smart Blonde Experiential 8054991994 michael.gle...@smartblondeco.com totalvu.tv 14060 Marquesas Way, Marina Del Rey, CA

[Philipp Hancke]

VideoReceiveStream2 is dealing with complete frames and you have a lot of elements in the pipeline in between it the socket and it that are not necessary in an SFU (which is generally packet-based, more similar to stopping after RtpVideoStreamReceiver::OnReceivedPayloadData).

Other than that, what Tsahi said.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/9d17edaa-da7b-4ef3-bcca-a1259920f37bn%40googlegroups.com.

[Tanzil Ahmad]

Thanks for your reply to my message. I would need your help to run the webrtc to my server with https.

Do you have any Skype I’d or WhatsApp number to continue discussion.

If you have any consultation payment, we can pay for that.

Thanks and Regards 

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/YDqA%2B3HfOkZJFqep%40SeanLaptop.sioBuD.com.