[erlang-questions] dtls error when used with chrome webrtc

[Joe K]

I'm trying to implement parts of webrtc stack with elixir/erlang and currently am stuck with setting up a dtls session.

The minimal example is, I think, the following (in console, erlang 20.2.2):

    2> ssl:start().

    ok

    3> {ok, ListenSocket} = ssl:listen(8090, [

    3>   binary,

    3>   {ip, {0, 0, 0, 0}},

    3>   {protocol, dtls},

    3>   {keyfile, <<"priv/server.key">>},

    3>   {certfile, <<"priv/server.pem">>},

    3>   {active, false}

    3> ]).

    {ok, ...}

    4> {ok, AcceptSocket} = ssl:transport_accept(ListenSocket).

    {ok,...}

    5> ssl:ssl_accept(AcceptSocket).

    {error,{tls_alert,"record overflow"}}

And js (with chrome canary): https://gist.github.com/idi-ot/a07b7330ff02f90373a2dcfe83883afa

After {error,{tls_alert,"record overflow"}} the RTCPeerConnection's iceConnectionState becomes "failed" and the connection itself "closed".

I wonder what I am doing wrong.

    openssl s_client -dtls1 -connect 127.0.0.1:8089 -debug

works fine with the code snippet above.

[Danil Zagoskin]

Hi!

What do you see in Wireshark?

Did you see handshake between two browsers?

Is your application ready to receive the packet sent by browser?

Do you use external STUN server?

Maybe browser sends STUN requests to your port when you expect DTLS hello?

Also try checking chrome://webrtc-internals and chrome://webrtc-logs for browser's view on what's going on.

_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

--

Danil Zagoskin | z...@gosk.in

[Danil Zagoskin]

> But now I don't know how to reply to both STUN binding request and then setup a DTLS session using erlang's ssl module.
Yes, dtls implementation lacks support of starting/accepting a handshake over existing socket.

It should be quite easy to implement and it would be consistent with ssl:connect/2 and ssl:ssl_accept for TCP sockets.

Also you may try using external STUN server (check RTCPeerConnection docs) and hope browser starts with DTLS hello.

If you try this, please share the results.

On Thu, Dec 28, 2017 at 3:26 PM, Joe K <goodj...@gmail.com> wrote:

Oops, I forgot to reply to the mailing list in my last email.

The response was

  > Maybe browser sends STUN requests to your port when you expect DTLS hello?

  You are absolutely right, Wireshark shows that there are lots of STUN binding requests being made, I didn't think of that.

  > Do you use external STUN server?

  I don't use external STUN servers ... For some reason, I didn't think I would need them.

  > Also try checking chrome://webrtc-internals and chrome://webrtc-logs for browser's view on what's going on.

  chrome://webrtc-logs is empty for the webrtc whole session.

But now I don't know how to reply to both STUN binding request and then setup a DTLS session using erlang's ssl module.

--

Danil Zagoskin | z...@gosk.in

[Joe K]

Oops, I forgot to reply to the mailing list in my last email.

The response was

  > Maybe browser sends STUN requests to your port when you expect DTLS hello?

  You are absolutely right, Wireshark shows that there are lots of STUN binding requests being made, I didn't think of that.

  > Do you use external STUN server?

  I don't use external STUN servers ... For some reason, I didn't think I would need them.

  > Also try checking chrome://webrtc-internals and chrome://webrtc-logs for browser's view on what's going on.

  chrome://webrtc-logs is empty for the webrtc whole session.

But now I don't know how to reply to both STUN binding request and then setup a DTLS session using erlang's ssl module.

On Thu, Dec 28, 2017 at 1:28 AM, Danil Zagoskin <z...@gosk.in> wrote:

[Joe K]

I don't think I can use `gen_udp:recv` on port returned in `ssl:transport_accept(ListenSocket)` (sorry for elixir terms):

  {:ok,

   {:sslsocket,

    {:gen_udp, {#PID<0.118.0>, {{{127, 0, 0, 1}, 54052}, #Port<0.1764>}},

     :dtls_connection}, #PID<0.143.0>}}

With elixir (`port` is the `#Port<...>` from above)

  :get_udp.recv(port, 0)

returns `{:error, :einval}`

On Thu, Dec 28, 2017 at 3:26 PM, Joe K <goodj...@gmail.com> wrote:

[Joe K]

> Also you may try using external STUN server (check RTCPeerConnection docs) and hope browser starts with DTLS hello.

I've tried that, but the browser still sends STUN binding requests to the DTLS process. And it uses the STUN server just to find out it's address.

> It should be quite easy to implement and it would be consistent with ssl:connect/2 and ssl:ssl_accept for TCP sockets.

Will try this now. Thank you.

[Joe K]

Tried this, hoped it would work, but it didn't ...

    1> {ok, Socket} = gen_udp:open(9090, [binary, {active, false}]).

    {ok,#Port<0.441>}

    2> dtls:connect(Socket, []).

    {error,{options,{not_supported,{packet,0}}}}

[Federico Carrone]

Joe,

We are creating an open source erlang webrtc server replacement for appear.in. You can check it here: https://github.com/lambdaclass/webrtc-server

We are using the processone stun library. I am not sure if this mail is of any help but might be interested in checking it since it is working fine.

Regards,

Federico.

[Danil Zagoskin]

Hi Federico!

Is it just signalling server?
E.g. do you handle all the DTLS+SRTP stuff or just build a full mesh of participants? 

--

Danil Zagoskin | z...@gosk.in

[Facundo Olano]

Hi Danil!

The server code is for signaling (using websockets), but it also includes processone/stun as a dependency, so it handles STUN/TURN as well. It also contains a couple of example applications that server javascript clients that connect to the server (both for signaling and ICE). The multiparty example uses a mesh.

To be honest I don't know what DTLS+SRTP is about :P

Thanks,

Facundo.

[Joe K]

Sorry for bothering you, Danil, but I was trying to make something like `dtls:ssl_accept` work on udp sockets and then thought I would get more STUN requests to keep the connection in NATs "alive" after I finally `sslaccept` the socket. Would I have to somehow downgrade the dtls session back to udp? Or is there some other way?

Right now I'm thinking about a hacky approach: forking erlang's ssl library and checking for STUN packets in every `handle_datagram` call in `dtls_udp_listener`.

And thank you again, you've been incredibly helpful.

[Joe K]

I've also been thinking about turning the server into a TURN server which would relay packets to itself, but for that I would still have to handle Allocate STUN requests.

[Max Lapshin]

Danil was asking, because we have spend enormous time in flussonic at making full webrtc _server_, not only a communication point.

DTLS + SRTP is only a beginning of problems =(

[Ingela Andin]

Hi!

2017-12-29 12:21 GMT+01:00 Joe K <goodj...@gmail.com>:

> Also you may try using external STUN server (check RTCPeerConnection docs) and hope browser starts with DTLS hello.

I've tried that, but the browser still sends STUN binding requests to the DTLS process. And it uses the STUN server just to find out it's address.

> It should be quite easy to implement and it would be consistent with ssl:connect/2 and ssl:ssl_accept for TCP sockets.

Will try this now. Thank you.

Pleas let us know if this is desirable functionality. So far we reasoned that as UDP is not connection oriented there is not the same interest to reuse

to underlying sockets as if there is an underlying connection.

Regards Ingela Erlang/OTP team - Ericsson AB

[Joe K]

Hi, Ingela!

I still don't know if that would actually solve my problem (STUN packets during DTLS session) ... So not particularly desirable right now.

[Joe K]

Hi, Max!

I wonder if you use ssl module in flussonic for dtls sessions. If you do, that would give me some hope ...

What other problems have you encountered while making flussonic work with webrtc besides dtls+srtp?

My use case was pretty simple, I want to receive media from the browser (h264+opus) and repackage it for hls (h264+aac), and then distribute it through some cdn. For some reason, I thought it would be simple ... If only browsers had websocket-like api for udp!

Anyway, right now I'm looking into janus gateway and how to write plugins for it.

[Andreas Schultz]

Spin off from: Re: [erlang-questions] dtls error when used with chrome webrtc

Ingela Andin <ingela...@gmail.com> schrieb am Di., 2. Jan. 2018 um 13:30 Uhr:

Hi!

2017-12-29 12:21 GMT+01:00 Joe K <goodj...@gmail.com>:

> Also you may try using external STUN server (check RTCPeerConnection docs) and hope browser starts with DTLS hello.

I've tried that, but the browser still sends STUN binding requests to the DTLS process. And it uses the STUN server just to find out it's address.

> It should be quite easy to implement and it would be consistent with ssl:connect/2 and ssl:ssl_accept for TCP sockets.

Will try this now. Thank you.

Pleas let us know if this is desirable functionality. So far we reasoned that as UDP is not connection oriented there is not the same interest to reuse

to underlying sockets as if there is an underlying connection.

I do have a use case that is even more complicated then simply upgrading UDP to DTLS.

CAPWAP is runnig unencrypted and DTLS traffic on the same socket. It distinguished between the traffic with a small header in front of the payload packet. I therefore need a demultiplexer on the UDP socket that removes the header and passes the encrypted payload to the DTLS stack.

There is somewhat similar problem when doing EAP-TLS over RADIUS or DIAMETER. The TLS traffic is encapsulated within RADIUS/DIAMETER requests and needs to be passed into the TLS stack and the replies need to encapsultated with RADIUS/DIAMETER.

The current socket abstraction in the SSL app is not prepared to handle this and would need invasive changes.

A simplistic workarround would be to forward the DTLS traffic on loopback UDP socket. Hwever that would incure additional latency, signaling overhead and would make detection of failed connections more difficult. I therefore don't want to go there.

A supported and documented API to pass (D)TLS traffic into the SSL app and get status change events and the decoded payload data back from the SSL app would IMHO help a lot.

Just some quick idea:

%% Create a new passive SSL connection of given type, return a opaque identifier.

ssl:create_connection(Protocol :: 'stream' | 'datagram', Opts) -> ssl_connection_id().

%% Pass received SSL traffic into the connection,

%% Return error, ok or Data to return on the connection.

ssl:recv(Connection :: ssl_connection_id(), EncData :: binary()) ->

   {error, Error} | ok | {ok, {send, Data :: binary()}}.

%% Pass unencrypted traffic into the SSL app

ssl: send(Connection :: ssl_connection_id(), PlainText :: binary()) ->

  {error, Error} | ok | {ok, {send, EncData :: binary()}}.

%% The owner of the connection is then getting messages like:

%% - send encrypted data:

%%     {ssl, Connection :: connection_id(), {send, EncData :: binary()}}

%% - got plaintext data:

%%     {ssl, Connection :: connection_id(), {recv, PlainText :: binary()}}

%% - connection event:

%%     {ssl, Connection :: connection_id(), Event :: ssl_connection_event()}

Wheter ssl:recv and ssl:send return data or use a message to the owner should depend on a mode setting (e.g. active, passive...)

What do you think? Comments?

Regards

Andreas

[...]

[Vance Shipley]

On Wed, Jan 3, 2018 at 2:39 PM, Andreas Schultz
<andreas...@travelping.com> wrote:
> I do have a use case that is even more complicated then simply upgrading UDP
> to DTLS.
> CAPWAP is runnig unencrypted and DTLS traffic on the same socket. It
> distinguished between the traffic with a small header in front of the
> payload packet. I therefore need a demultiplexer on the UDP socket that
> removes the header and passes the encrypted payload to the DTLS stack.

I think you're in luck.

> There is somewhat similar problem when doing EAP-TLS over RADIUS or
> DIAMETER. The TLS traffic is encapsulated within RADIUS/DIAMETER requests
> and needs to be passed into the TLS stack and the replies need to
> encapsultated with RADIUS/DIAMETER.

SigScale has a pure Erlang implementation of EAP-TTLS over RADIUS
using the SSL app in OTP in our open source Online Charging System
(OCS): https://github.com/sigscale/ocs

> The current socket abstraction in the SSL app is not prepared to handle this
> and would need invasive changes.

The existence of the API is hidden in this one sentence of the User Guide:

http://erlang.org/doc/apps/ssl/ssl_protocol.html
"By default SSL/TLS is run over the TCP/IP protocol even though you
can plug in any other reliable transport protocol with the same
Application Programming Interface (API) as the gen_tcp module in
Kernel."

Here is our SSL transport callback module:
https://github.com/sigscale/ocs/blob/master/src/ocs_eap_tls_transport.erl


--
-Vance

[Andreas Schultz]

Hi Vance,

Vance Shipley <van...@motivity.ca> schrieb am Mi., 3. Jan. 2018 um 13:44 Uhr:

On Wed, Jan 3, 2018 at 2:39 PM, Andreas Schultz
<andreas...@travelping.com> wrote:
> I do have a use case that is even more complicated then simply upgrading UDP
> to DTLS.
> CAPWAP is runnig unencrypted and DTLS traffic on the same socket. It
> distinguished between the traffic with a small header in front of the
> payload packet. I therefore need a demultiplexer on the UDP socket that
> removes the header and passes the encrypted payload to the DTLS stack.

I think you're in luck.

> There is somewhat similar problem when doing EAP-TLS over RADIUS or
> DIAMETER. The TLS traffic is encapsulated within RADIUS/DIAMETER requests
> and needs to be passed into the TLS stack and the replies need to
> encapsultated with RADIUS/DIAMETER.

SigScale has a pure Erlang implementation of EAP-TTLS over RADIUS
using the SSL app in OTP in our open source Online Charging System
(OCS): https://github.com/sigscale/ocs

I have seen that some time ago.

> The current socket abstraction in the SSL app is not prepared to handle this

> and would need invasive changes.

The existence of the API is hidden in this one sentence of the User Guide:

   http://erlang.org/doc/apps/ssl/ssl_protocol.html
  "By default SSL/TLS is run over the TCP/IP protocol even though you
can plug in any other reliable transport protocol with the same
Application Programming Interface (API) as the gen_tcp module in
Kernel."

Last time I looked at the SSL library in that depth (around 2014) it did not permit to use of a Pid. The actual socket had to be a Erlang port. Back then I needed this change to use a Pid as socket replacement: https://github.com/RoadRunnr/otp/commit/77b9256fc15fa2f4293bd84fd0bb8dc06da8ddbf

I also played with EAP based on Erlang SSL back then (https://github.com/travelping/eradius/commits/eap), but didn't have the time to properly finish it.

That SSL API restrictions to Erlang ports seem to have changed since then, at least for the TLS code.

The DTLS code still seems to have the hard coded assumtions that it always runs over UDP sockets:

https://github.com/erlang/otp/blob/master/lib/ssl/src/dtls_connection.erl#L142

The other major restriction was that it required two processes, one for the socket/transport side and another for the payload side of the SSL library. Using the same process for both sides would lead to a dead lock when calling ssl:send. Not sure if that restriction has been lifted.

Andreas

[Max Lapshin]

Your usecase is not simple, it is most complicated.

We use ssl, but use own implementation of dtls.

This configuration is working in flussonic at the moment and it is not simple.