Frustrating issue - How do you handle SLOW internet connections relay with TURN server?

2,849 views
Skip to first unread message

Sprogrammer

unread,
Mar 8, 2014, 3:04:39 AM3/8/14
to turn-server-project...@googlegroups.com
I have been testing with my team. they went to HOTEL and some PARKING space where you have HOTSPOT internet.
> using webRTC they place the call
> i see the call came in
> Turn server tried something

> instantly call is disconnect

It happen over and over. There internet is too slow and sometimes internet drops and comes back 
In such situation i have seen Skype works, Google Hangout works. But WebRTC with Turnserver is not working reliably like Skype/Google Hangout.

How can we it be resolved with Turn server? Any advise 

Thank you.

Best regards
/Sham

Oleg Moskalenko

unread,
Mar 8, 2014, 3:17:46 AM3/8/14
to turn-server-project...@googlegroups.com
Bad connections must not be a problem when UDP is used. But when you are using TCP, then the temporary network outage will inevitably result into TCP connection breakage. I do not know whther anything can be done about that. I'll think about that. But it may be a difficult thing to workaround.

Oleg

Gustavo Garcia

unread,
Mar 8, 2014, 3:31:28 AM3/8/14
to Oleg Moskalenko, turn-server-project...@googlegroups.com
I have seen people trying to use multiple TCP connections in parallel to avoid Head-of-Line blocking, but that's not part of TURN spec unfortunately.

Perhaps some changes at TCP level to partially disable retransmissions could be also done (acknowledging even lost packets), but probably require low level stack modifications.  What do you think?


--
You received this message because you are subscribed to the Google Groups "TURN Server (Open-Source project)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to turn-server-project-rfc57...@googlegroups.com.
To post to this group, send email to turn-server-project...@googlegroups.com.
Visit this group at http://groups.google.com/group/turn-server-project-rfc5766-turn-server.
For more options, visit https://groups.google.com/d/optout.

Oleg Moskalenko

unread,
Mar 8, 2014, 3:51:15 AM3/8/14
to turn-server-project...@googlegroups.com, Oleg Moskalenko
I am not sure that we do want to hack the TCP protocol. There may be unforeseen consequences. What you are suggesting is a protocol that looks like TCP, but behaves like UDP. I am not sure about that approach.

Of course if will ever be making a solution built on top a custom system, then we may think about something. But currently this is working on top of a standard TCP/IP stack.

Regards,
Oleg


On Saturday, March 8, 2014 12:31:28 AM UTC-8, Gustavo García wrote:
I have seen people trying to use multiple TCP connections in parallel to avoid Head-of-Line blocking, but that's not part of TURN spec unfortunately.

Perhaps some changes at TCP level to partially disable retransmissions could be also done (acknowledging even lost packets), but probably require low level stack modifications.  What do you think?
On Sat, Mar 8, 2014 at 12:17 AM, Oleg Moskalenko <mom0...@gmail.com> wrote:
Bad connections must not be a problem when UDP is used. But when you are using TCP, then the temporary network outage will inevitably result into TCP connection breakage. I do not know whther anything can be done about that. I'll think about that. But it may be a difficult thing to workaround.

Oleg



On Saturday, March 8, 2014 12:04:39 AM UTC-8, Sprogrammer wrote:
I have been testing with my team. they went to HOTEL and some PARKING space where you have HOTSPOT internet.
> using webRTC they place the call
> i see the call came in
> Turn server tried something

> instantly call is disconnect

It happen over and over. There internet is too slow and sometimes internet drops and comes back 
In such situation i have seen Skype works, Google Hangout works. But WebRTC with Turnserver is not working reliably like Skype/Google Hangout.

How can we it be resolved with Turn server? Any advise 

Thank you.

Best regards
/Sham

--
You received this message because you are subscribed to the Google Groups "TURN Server (Open-Source project)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to turn-server-project-rfc5766-turn-server+unsubscribe@googlegroups.com.
To post to this group, send email to turn-server-project-rfc5766-turn-...@googlegroups.com.

Oleg Moskalenko

unread,
Mar 8, 2014, 3:58:06 AM3/8/14
to turn-server-project...@googlegroups.com, Oleg Moskalenko
Of course, there is always a possibility to tune the TCP parameters in the system so it will better tolerate bad connections. But that's up to the system administrator how to do that.

Oleg

Oleg Moskalenko

unread,
Mar 8, 2014, 4:02:25 AM3/8/14
to Gustavo Garcia, turn-server-project...@googlegroups.com
as for the multiple TCP connections to avoid the "head of line" blocking, I see no problem using that with TURN. A client can create multiple TCP sessions with the TURN server, and each session will have its own UDP relay endpoint - but they all can have the same peer address. Then those several TCP connections can be used for the peer communications. You do not need to extend the TURN specs to do that.

Regards,
Oleg

Sprogrammer

unread,
Mar 8, 2014, 7:13:21 AM3/8/14
to turn-server-project...@googlegroups.com
@Oleg: Guru. Thank you

> i am using TCP which is causing it for sure, but i have also no choice without TCP (Because as you know behind Enterprise network/firewall we need to use 80 TCP, 443 TCP tunneling for turn relay to work)

Surprisingly Bria, Google Hangout, Skype works on such SLOW broken networks even!!!, ME and my team we are testing random networks:

> ME (sitting in my laboratory with public ip, fiber internet high speed bandwidth, all equip)
> my team members driving and finding random internet access places such is Train station wifi, hotspot in public places 
(Where internet is very poor, there main goal is to make WebRTC not work and compare WebRTC with Skype, Google Hangout, Bria in same network)

> my team from same networks, used Skype, Google Hangout, Bria soft phone all works with there ACME packet server. 
> But when they use WebRTC ( https://apprtc.appspot.com ) with our Turn relay server 
it does not work like connected and instantly call drops also when we use Bria with Turnserver relay it fails similar way




Gustavo Garcia

unread,
Mar 8, 2014, 10:30:57 AM3/8/14
to Oleg Moskalenko, turn-server-project...@googlegroups.com
Agree, what I really wanted to say is that I don't know any WebRTC spec with any reference to this way of using TURN and none of the existing implementation support it.   Thx for the clarification.

Oleg Moskalenko

unread,
Mar 8, 2014, 12:39:10 PM3/8/14
to Sprogrammer, turn-server-project...@googlegroups.com
I do not think that the problem is not the TURN server side. The TURN server is located in a reliable k, and it reacts to the network events it is receiving. If the TCP client connection is gone, then we hardly can do anything - we cannot recover the connection from the server side.

We use keep-alive option with default system parameters. So if the network is temporary broken then the server would NOT immediately close the connection - it takes a relatively long time for the system "keepalive" mechanism to give up and mark the connection broken. So there must be another reason for that immediate disconnect..

To eliminate "spurious" network events as a disconnection reason, please try this test build with a bad network:

http://turnserver.open-sys.org/downloads/v3.2.2.91test1/turnserver-3.2.2.91test1.tar.gz


Check the results, and post the log file output here, if there will be any outages.

Oleg



Oleg Moskalenko

unread,
Mar 9, 2014, 1:57:03 AM3/9/14
to turn-server-project...@googlegroups.com
I tried (one more time) to emulate the network situation described by Shamun: WiFi, slow connection, network overload with many retransmission and much of traffic loss.

For a clean picture, I used a dedicated test TURN client (to eliminate possible infractions from Chrome TURN implementation and ICE process).

The result:

1) UDP connections worked flawlessly (with of course the corresponding amount of loss).

2) TCP connections were VERY slow, as expected - with lots of retransmissions, etc - but I've not seen any spurious unexpected TCP disconnects. The default TCP/IP stack system settings are tolerant enough to allow the connections stay-alive even in those conditions.

I used Ubuntu 11.10 as the TURN server platform (real hardware, not a VM).

So, I do not think that the problems which you are observing have anything to do with the TURN server.

Regards,
Oleg

Sprogrammer

unread,
Mar 9, 2014, 5:29:39 AM3/9/14
to turn-server-project...@googlegroups.com, Sprogrammer
Just installed this version. Will let you know testing with Ubuntu 13.10 64-bit 
Now finding some SLOW internet to report you the the logs

Thank you

reg
/sham

Sprogrammer

unread,
Mar 10, 2014, 7:50:47 AM3/10/14
to turn-server-project...@googlegroups.com, Sprogrammer
@Oleg: UDP i cant use (cause my 99% users will be behind firewall/enterprise firewall etc), i have to do 100% TCP Turn relay. 

I tested with this Version Citrix-3.2.2.91 'Marshal West' several time over and over. Turnserver working perfectly as expected.
It is clear that Turn-server has nothing to do to recover it, from Client to Client the connection needs to be re-initialized and Turnserver works in that case.

Mar 10 12:43:11 sun-Alienware-X51-R2 turnserver: 94466: handle_udp_packet: New UDP endpoint: local addr 82.143.92.18:80, remote addr 82.143.92.22:62473
Mar 10 12:43:11 sun-Alienware-X51-R2 turnserver: 94466: session 003000000000000006: user <>: incoming packet BINDING processed, success
Mar 10 12:43:11 sun-Alienware-X51-R2 turnserver: 94466: IPv4. tcp or tls connected to: 82.143.92.22:61884
Mar 10 12:43:11 sun-Alienware-X51-R2 turnserver: 94466: session 004000000000000006: user <>: incoming packet message processed, error 401
Mar 10 12:43:11 sun-Alienware-X51-R2 turnserver: 94466: IPv4. Server relay addr: 82.143.92.18:80
Mar 10 12:43:11 sun-Alienware-X51-R2 turnserver: 94466: IPv4. Local relay addr: 82.143.92.18:57346
Mar 10 12:43:11 sun-Alienware-X51-R2 turnserver: 94466: session 004000000000000006: new, username=<root>, lifetime=600
Mar 10 12:43:11 sun-Alienware-X51-R2 turnserver: 94466: session 004000000000000006: user <root>: incoming packet ALLOCATE processed, success
Mar 10 12:43:12 sun-Alienware-X51-R2 turnserver: 94467: handle_udp_packet: New UDP endpoint: local addr 82.143.92.18:80, remote addr 82.143.92.17:52401
Mar 10 12:43:12 sun-Alienware-X51-R2 turnserver: 94467: session 004000000000000007: user <>: incoming packet BINDING processed, success
Mar 10 12:43:12 sun-Alienware-X51-R2 turnserver: 94467: IPv4. tcp or tls connected to: 82.143.92.17:34160
Mar 10 12:43:12 sun-Alienware-X51-R2 turnserver: 94467: session 007000000000000006: user <>: incoming packet message processed, error 401
Mar 10 12:43:12 sun-Alienware-X51-R2 turnserver: 94467: IPv4. Server relay addr: 82.143.92.18:80
Mar 10 12:43:12 sun-Alienware-X51-R2 turnserver: 94467: IPv4. Local relay addr: 82.143.92.18:55219
Mar 10 12:43:12 sun-Alienware-X51-R2 turnserver: 94467: session 007000000000000006: new, username=<root>, lifetime=600
Mar 10 12:43:12 sun-Alienware-X51-R2 turnserver: 94467: session 007000000000000006: user <root>: incoming packet ALLOCATE processed, success
Mar 10 12:43:12 sun-Alienware-X51-R2 turnserver: 94467: session 007000000000000006: user <root>: incoming packet CREATE_PERMISSION processed, success
Mar 10 12:43:12 sun-Alienware-X51-R2 turnserver: 94467: session 004000000000000006: user <root>: incoming packet CREATE_PERMISSION processed, success
Mar 10 12:43:12 sun-Alienware-X51-R2 turnserver: 94467: session 007000000000000006: user <root>: incoming packet CHANNEL_BIND processed, success
Mar 10 12:43:12 sun-Alienware-X51-R2 turnserver: 94467: session 004000000000000006: user <root>: incoming packet CHANNEL_BIND processed, success
Mar 10 12:43:21 sun-Alienware-X51-R2 turnserver: 94476: session 003000000000000006: user <>: incoming packet BINDING processed, success
Mar 10 12:43:22 sun-Alienware-X51-R2 turnserver: 94477: session 004000000000000007: user <>: incoming packet BINDING processed, success
Mar 10 12:43:24 sun-Alienware-X51-R2 turnserver: 94480: session 007000000000000006: usage: username=<root>, rp=641, rb=214663, sp=1407, sb=743564
Mar 10 12:43:24 sun-Alienware-X51-R2 turnserver: 94480: session 004000000000000006: usage: username=<root>, rp=1411, rb=743637, sp=637, sb=215096
Mar 10 12:43:31 sun-Alienware-X51-R2 turnserver: 94486: session 003000000000000006: user <>: incoming packet BINDING processed, success
Mar 10 12:43:32 sun-Alienware-X51-R2 turnserver: 94487: session 004000000000000007: user <>: incoming packet BINDING processed, success
Mar 10 12:43:34 sun-Alienware-X51-R2 turnserver: 94489: session 007000000000000006: usage: username=<root>, rp=599, rb=272691, sp=1449, sb=952596
Mar 10 12:43:34 sun-Alienware-X51-R2 turnserver: 94489: session 004000000000000006: usage: username=<root>, rp=1448, rb=950260, sp=600, sb=273596
Mar 10 12:43:41 sun-Alienware-X51-R2 turnserver: 94496: session 004000000000000006: usage: username=<root>, rp=1482, rb=1117684, sp=566, sb=315656
Mar 10 12:43:41 sun-Alienware-X51-R2 turnserver: 94496: session 007000000000000006: usage: username=<root>, rp=567, rb=314905, sp=1481, sb=1117688
Mar 10 12:43:48 sun-Alienware-X51-R2 turnserver: 94503: session 004000000000000006: socket closed remotely
Mar 10 12:43:48 sun-Alienware-X51-R2 turnserver: 94503: session 004000000000000006: closed (1st stage), user <root>, reason: TCP connection closed by peer (callback)


^^^^^^^^^^^^^^^^^^^^^^^ here the wireless connection dropped and clearly it shows


 
 
/<--- Here it was retrying that same session to re-establish the connection (but for some reason i could not make it complete yet)
/
|
Mar 10 12:44:11 sun-Alienware-X51-R2 turnserver: 94526: session 003000000000000006: closed (2nd stage), user <>, reason: allocation watchdog determined stale session state
Mar 10 12:44:11 sun-Alienware-X51-R2 turnserver: 94526: session 004000000000000006: closed (2nd stage), user <root>, reason: allocation watchdog determined stale session state
Mar 10 12:44:11 sun-Alienware-X51-R2 turnserver: 94526: session 004000000000000006: delete: username=<root>
Mar 10 12:44:12 sun-Alienware-X51-R2 turnserver: 94527: session 004000000000000007: closed (2nd stage), user <>, reason: allocation watchdog determined stale session state
Mar 10 12:44:26 sun-Alienware-X51-R2 turnserver: 94541: handle_udp_packet: New UDP endpoint: local addr 82.143.92.18:80, remote addr 82.143.92.22:63011
Mar 10 12:44:26 sun-Alienware-X51-R2 turnserver: 94541: session 006000000000000014: user <>: incoming packet BINDING processed, success
Mar 10 12:44:26 sun-Alienware-X51-R2 turnserver: 94541: IPv4. tcp or tls connected to: 82.143.92.22:62063
Mar 10 12:44:26 sun-Alienware-X51-R2 turnserver: 94541: session 003000000000000007: user <>: incoming packet message processed, error 401
Mar 10 12:44:26 sun-Alienware-X51-R2 turnserver: 94541: IPv4. Server relay addr: 82.143.92.18:80
Mar 10 12:44:26 sun-Alienware-X51-R2 turnserver: 94541: IPv4. Local relay addr: 82.143.92.18:59351
Mar 10 12:44:26 sun-Alienware-X51-R2 turnserver: 94541: session 003000000000000007: new, username=<root>, lifetime=600
Mar 10 12:44:26 sun-Alienware-X51-R2 turnserver: 94541: session 003000000000000007: user <root>: incoming packet ALLOCATE processed, success
Mar 10 12:44:36 sun-Alienware-X51-R2 turnserver: 94551: session 006000000000000014: user <>: incoming packet BINDING processed, success
Mar 10 12:44:46 sun-Alienware-X51-R2 turnserver: 94561: session 006000000000000014: user <>: incoming packet BINDING processed, success
Mar 10 12:44:56 sun-Alienware-X51-R2 turnserver: 94571: session 003000000000000007: socket closed remotely
Mar 10 12:44:56 sun-Alienware-X51-R2 turnserver: 94571: session 003000000000000007: closed (1st stage), user <root>, reason: TCP connection closed by peer (callback)


Thank you
Reg
/sham

Sprogrammer

unread,
Mar 10, 2014, 9:10:35 AM3/10/14
to turn-server-project...@googlegroups.com
OK - solved.  those who have similar problem like me TCP and SLOW internet connection. Here is how i resolved it

1. Client 1 and Client 2 connects and doing 100% relay via the turnserver

2. Client 1 has very slow internet it connects and then drops and again connects and drops internet or signal is too low

3. Now C1 and C2 session never ends unless they do P2P "disconnect" exchange till then 

4. if C1 internet drops, C1 start Denial TCP Attack towards TurnPort 80 unless it get "Reply"
once it get "Reply" C1 re-initiate the same session with C2 

It seems working (without anything to change in Turnserver)

But in WebRTC client algorithm i just made that looks stable now when ever internet comes back, it reconnects instantly.


Thanks!! 
reg
/sham

Silvia Pfeiffer

unread,
Mar 10, 2014, 4:06:56 PM3/10/14
to Sprogrammer, turn-server-project...@googlegroups.com

So did you did something in the apprtc code?

Best Regards,
Silvia.

--

Shamun Toha Md

unread,
Mar 10, 2014, 5:33:41 PM3/10/14
to Silvia Pfeiffer, turn-server-project...@googlegroups.com
The official Apprtc code i did not changed or modified anything (i keep it as it is in the repository), 
so we do not have to change anything there. 

i Just in my index.html file i have made a function to monitor the status "connecting" or "waiting" or "connected"

if "connected" or suddenly came into "waiting" or "disconnect" 
                  but did not received a final "disconnect" from C2 and C1 
                 did not exchanged it then that session is not considered as final disconnect. 

C1 and C2 has exchange and acknowledge that disconnect request. 
If that exchange is not approved then that connection must have been dropped by INTERNET or any physical interruption (logically)
so it will keep trying to re-connect that same session, by not accepting the fact that it was disconnected in a normal way.

Simple logic and simply working for me. 

I can disconnect the WiFi physically too, and wait for even 1 hour and then afterwards 
it reconnects on that same session means i can on the fly move from WiFi to 3G or 4G or lan and continue my same call (without getting frustrated)

that same session will be re-established like Bria softphone has done, similar concept i just did in my own customized way in the index.html file only.

i am doing more testing, if do not find any issue, and seems more reliable i will post a sample for the community.


reg
shamun




Dmitry Sobinov

unread,
Mar 12, 2014, 1:49:15 AM3/12/14
to turn-server-project...@googlegroups.com
Oleg, what about handling situations like this:

- 2 users, user_1 is using TURN/TCP, user_2 is UDP
- user_2 is sending heavy video stream
- user_1 has temporary problems with network connection (a few seconds) which would fine in UDP case (just freeze)
- user_2 doesn't know anything about network problem of user_1 and is keeping pushing data, which results in TCP outgoing packets queue growing on the TURN server

How does TURN server handles this currently? Is it dropping packets if queue is exceeding some threshold? It would make sense to do so, because not delivered in time packets are of no use to audio/video receiver. And more importantly, old packets can create burst in the network when connection is restored.


Regards,
Dmitry

Oleg Moskalenko

unread,
Mar 12, 2014, 2:44:50 AM3/12/14
to Dmitry Sobinov, turn-server-project...@googlegroups.com
Let me clarify your question: you are talking about the situation when both users1 and user2 are using TURN sessions for the communications, right ?

If that is just a very low quality connection that cannot deliver all traffic - then the TCP queue from server to user1 will grow until the buffer limit is exhausted, then the connection will be dropped - the user1 will be disconnected by the server.
 
I can change the implementation so that if the buffer is exhausted, the connection will stay alive.

Oleg




Dmitry Sobinov

unread,
Mar 12, 2014, 2:56:43 AM3/12/14
to turn-server-project...@googlegroups.com, Dmitry Sobinov
user2 is using UDP connection to TURN relay endpoint allocated for user1. So, if the TCP queue limit for user1 is set to some fixed value, time to drop connection will depend on user2 bandwidth used. Say, 100kbps video will drop connection in N seconds, but 2Mbps video will do it in N/20 seconds. If N is 10 seconds then even very short network failure will drop connection. Is it correct?

Regards,
Dmitry
To unsubscribe from this group and stop receiving emails from it, send an email to turn-server-project-rfc5766-turn-server+unsubscribe@googlegroups.com.
To post to this group, send email to turn-server-project-rfc5766-turn-...@googlegroups.com.

Oleg Moskalenko

unread,
Mar 12, 2014, 3:08:42 AM3/12/14
to Dmitry Sobinov, turn-server-project...@googlegroups.com
Yes, that's correct. The buffers can grow large (I have to check their sizes) but they are not unlimited.

Oleg


To unsubscribe from this group and stop receiving emails from it, send an email to turn-server-project-rfc57...@googlegroups.com.
To post to this group, send email to turn-server-project...@googlegroups.com.

Oleg Moskalenko

unread,
Mar 12, 2014, 3:58:32 AM3/12/14
to Dmitry Sobinov, turn-server-project...@googlegroups.com
... actually, Dmitry, that's not that simple. The TURN server itself is plugged into a very good network, and technically, from the server software point of view, the buffer is flushed once the application successfully sent the data through the socket. After that, that's the system kernel responsibility - not the TURN server application responsibility. The system kernel parameters are configurable - the size of the buffers, the window between the TCP send and the ACK, etc. So the TURN server system admin can set parameters so that it will tolerate the bad networks (somewhere after the local router) perfectly well.

Oleg



To unsubscribe from this group and stop receiving emails from it, send an email to turn-server-project-rfc57...@googlegroups.com.
To post to this group, send email to turn-server-project...@googlegroups.com.

Sprogrammer

unread,
Mar 12, 2014, 4:04:01 AM3/12/14
to turn-server-project...@googlegroups.com, Dmitry Sobinov
To me turn-server should do as below 2 thing with more priority to achieve quality of service out of it:

1) unlimited packet
* Peer 1 and Peer 2 - can send/receive any size of packet (1080p resolution + 60fps is a huge packets for real-time and zero-lipsync to have)
  i can generate with VLC and Gstreamer such amount of real-time packets and pass to turn-server, for making it to crash or do abnormal behaviour

* my suggestion would be to have in Turn-server configuration option where we can say: packets for in and out = unlimited (no limit) and also making higher priority to receive and send the relay as fast as we can. By allowing more options like debug mode to lower, or tiny so that there is more processing reservation for turn-server for doing heavy relay traffics

2) zero-latency packet exchange for relay:

* If Turn-server receives relay packets like a brute-force it should not get slow. What i have seen is that the packets become slow motion like when you move your "hands" or "clap your hands" peer1 and peer2 gets those packets very slowly and later. (you get the impression that its not real-time because of the 3 way relay)

3) session-initiation theory to be more reliable/fast/stable
when socket closed by remote and reconnected by remote (should behave as new session)
there are many scenario the whole session drops because of hardware/network/bandwidth related issue
allowing brute force of connections (without treating it as attack from turn-server), to protect brute-force attacks
someone should use third-party filtering or event watcher..


reg
/sham

Oleg Moskalenko

unread,
Mar 12, 2014, 4:14:48 AM3/12/14
to Sprogrammer, turn-server-project...@googlegroups.com, Dmitry Sobinov
I am not sure that I understand what you are saying.

As for unlimited packets, that's impossible - the format of TURN packets allows max 64K data packets. That's the protocol limitation. The larger packets will produce unintelligible connections and eventually a connection drop - but not a crash.

Do you really can produce a crash ? If yes, then describe it exactly and file a bug.

Oleg



Sprogrammer

unread,
Mar 12, 2014, 4:24:41 AM3/12/14
to turn-server-project...@googlegroups.com, Sprogrammer, Dmitry Sobinov
Guru No worries. 

YES i was able to make a crash and take the turnserver down with some tools i wrote for testing and learning purpose.
Crash tool does:

0) brute force tool with TCP connect and with UDP connect
1) connect to the turn server do all the primary handshake's
2) start sending flooding packets targetting to the turnserver ports and to the same port mapped ports
3) increase the session's as brute-force like DoS attack, at the same time flood the turn-server's other ports 

Then i see the system become ext ream slow and turnserver start behaving abnormal. I am trying to do some more testing i will report you with more details on those in a new topic.


reg
/sham


On Wednesday, March 12, 2014 9:14:48 AM UTC+1, Oleg Moskalenko wrote:
I am not sure that I understand what you are saying.

As for unlimited packets, that's impossible - the format of TURN packets allows max 64K data packets. That's the protocol limitation. The larger packets will produce unintelligible connections and eventually a connection drop - but not a crash.

Do you really can produce a crash ? If yes, then describe it exactly and file a bug.

Oleg

On Wed, Mar 12, 2014 at 1:04 AM, Sprogrammer <sha...@companysocia.com> wrote:
To me turn-server should do as below 2 thing with more priority to achieve quality of service out of it:

1) unlimited packet
* Peer 1 and Peer 2 - can send/receive any size of packet (1080p resolution + 60fps is a huge packets for real-time and zero-lipsync to have)
  i can generate with VLC and Gstreamer such amount of real-time packets and pass to turn-server, for making it to crash or do abnormal behaviour

* my suggestion would be to have in Turn-server configuration option where we can say: packets for in and out = unlimited (no limit) and also making higher priority to receive and send the relay as fast as we can. By allowing more options like debug mode to lower, or tiny so that there is more processing reservation for turn-server for doing heavy relay traffics

2) zero-latency packet exchange for relay:

* If Turn-server receives relay packets like a brute-force it should not get slow. What i have seen is that the packets become slow motion like when you move your "hands" or "clap your hands" peer1 and peer2 gets those packets very slowly and later. (you get the impression that its not real-time because of the 3 way relay)

3) session-initiation theory to be more reliable/fast/stable
when socket closed by remote and reconnected by remote (should behave as new session)
there are many scenario the whole session drops because of hardware/network/bandwidth related issue
allowing brute force of connections (without treating it as attack from turn-server), to protect brute-force attacks
someone should use third-party filtering or event watcher..


reg
/sham

--
You received this message because you are subscribed to the Google Groups "TURN Server (Open-Source project)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to turn-server-project-rfc5766-turn-server+unsubscribe@googlegroups.com.
To post to this group, send email to turn-server-project-rfc5766-turn-...@googlegroups.com.

Oleg Moskalenko

unread,
Mar 12, 2014, 4:41:06 AM3/12/14
to Sprogrammer, turn-server-project...@googlegroups.com, Dmitry Sobinov
If you overloaded the system then that's fine, there is no 100% defense against it.

But did you manage to crash the TURN server process ? That's what I was asking.

Thanks
Oleg



To unsubscribe from this group and stop receiving emails from it, send an email to turn-server-project-rfc57...@googlegroups.com.
To post to this group, send email to turn-server-project...@googlegroups.com.

Sprogrammer

unread,
Mar 12, 2014, 4:45:19 AM3/12/14
to turn-server-project...@googlegroups.com, Sprogrammer, Dmitry Sobinov
YES - i was able to crash the TURN server after doing a long duration of brute-force with it. It did not crashed in first few hours, but 
when i simulated many sessions and then did many brute-force randomly and more then 4 to 5 hours of brute-forcing. Suddenly i was success to see that
turnserver process was crashed. I tested this on Ubuntu 13.10 i was success and also i tested in RHEL 7 there it also happened 

But i am trying to reproduce it again and again which i failed and it takes a while to get it down. I will log everything and try to prepare something for you to have a look later.

reg
/sham

Oleg Moskalenko

unread,
Mar 12, 2014, 4:53:08 AM3/12/14
to Sprogrammer, turn-server-project...@googlegroups.com, Dmitry Sobinov
I did some TCP fixes in .910 build, that may help.

Oleg


--

Sprogrammer

unread,
Mar 12, 2014, 4:59:03 AM3/12/14
to turn-server-project...@googlegroups.com, Dmitry Sobinov
@Oleg: FYI

Let me clarify your question: you are talking about the situation when both users1 and user2 are using TURN sessions for the communications, right ?

If that is just a very low quality connection that cannot deliver all traffic - then the TCP queue from server to user1 will grow until the buffer limit is exhausted, then the connection will be dropped - the user1 will be disconnected by the server.
 
I can change the implementation so that if the buffer is exhausted, the connection will stay alive. 

> i have simulated what @Dimitry wanted to do
> When the user2 keep sending packets even user1 is disconnected, then TurnServer is doing correct, it disconnects that session because the socket was remotely cosed by peer1 (user1)

> i think as it is is perfect, how it should be

> when user1 is coming online, it auto re-initiating the session and turn-server is doing just fine as it is right now.
here is very tiny simple flow done by user1 (who is the curprit of internet slow) [nothing to do in turnserver its doing fine as it is now]

//
 
// SLOW - Internet
 
//
 
var frustrating = false; // true = when internet arrives reconnect
                           
// false = normal
 
var temp_timer;
 
var temp_counter = 0;
 
function temptimer() {
    temp_timer
= setTimeout(function() {
      console
.log("OK => internet is ON => Connecting... => " + mystatus + " => temptimer()temptimer()temptimer()temptimer()temptimer()temptimer()");                
     
if (tump==='Connected') {
       
if (mystatus==='busy') {
         
return false;
       
}
     
}        
      window
.location.reload();
      clearTimeout
(temp_timer);
   
}, 9000);
 
}
 
function internet() {  
    timer_internet
= setTimeout(function() {      
      $
.ajax({
        type    
: "GET",//POST
        dataType
: 'json',
        url    
: netlink,
        data    
: "username=" + roomKey + "&password=1234&language=EN&call=yes&idle=",
        async  
: true,
        beforeSend
: function()  {},
        complete
: function() { },
        success
: function(msg) {
         
if(msg.result=='success') {
           
           
if (tump==='Connecting...') {
              console
.log("OK ==> internet is ON ==> Connecting... => " + mystatus  + " ==> " + frustrating);
              callplaced
= false;
             
if (frustrating) {
                frustrating
= false;
                console
.log("OK => internet is ON => Connecting... => " + mystatus + " => Disconnect this call");
                console
.log("OK => internet is ON => Connecting... => " + mystatus + " => Request a new call");
                window
.location.reload();
             
}
               
             
if (mystatus ==='free') {
                console
.log("OK => internet is ON => Connecting... => " + mystatus + " => Reloaded the page once");                
                temp_counter
++;
               
if (temp_counter>10) {
                  temp_counter
=0 ;
                  mystatus
='busy';
               
}
             
} else if (mystatus ==='busy') {
                console
.log("OK => internet is ON => Connecting... => " + mystatus + " => if taking too long reload the page");                
                temp_counter
++;
               
if (temp_counter>120) {
                  console
.log("OK => internet is ON => Connecting... => " + mystatus + " => counter 10 !!!!!!!!!!!!!!!!!!!!!!!!");                
                  temp_counter
=0 ;
                  temptimer
();
               
}                
                mystatus
='busy';                
               
             
}
             
           
} else if(tump==='Waiting') {
              console
.log("OK ==> internet is ON ==> Waiting => " + mystatus + " Callplaced: " + callplaced + " Frustration: " + frustrating + ' Counter: ' + temp_counter);
             
             
if (mystatus==='free') {
               
// after call submit waiting and free should be considered as
               
// reload page again
               
if (callplaced) {
                 
if (frustrating) {
                    frustrating
= false;
                    console
.log("OK => internet is ON => Waiting => " + mystatus + " => Disconnect this call");
                    console
.log("OK => internet is ON => Waiting => " + mystatus + " => Request a new call");
                    window
.location.reload();                    
                 
}
                 
                  temp_counter
++;
                 
if (temp_counter>40) {
                    temp_counter
=0 ;
                    window
.location.reload();
                 
}                      
                 
               
}
             
}
             
           
} else if(tump==='Connected') {
              callplaced
= false;
              console
.log("OK ==> internet is ON ==> Connected => " + mystatus  + " ==> " + frustrating);
             
if (frustrating) {
                frustrating
= false;
                console
.log("OK => internet is ON => Connected => " + mystatus + " => Disconnect this call");
                console
.log("OK => internet is ON => Connected => " + mystatus + " => Request a new call");
                window
.location.reload();
             
}
             
             
if (mystatus ==='free') {
                console
.log("OK => internet is ON => Connected => " + mystatus + " => Reloaded the page once");
                mystatus
='busy';
             
}
             
           
} else {
              callplaced
= false;
              console
.log("OK ==> internet is ON ==> PPPPPEEPPP => " + mystatus);
           
}
           
            internet
();
         
}
       
},
        error
: function (xhr, ajaxOptions, thrownError) {
         
//console.log("FAIL ==> ERROR ==> " + xhr.status);
         
//console.log("FAIL ==> ERROR ==> " + thrownError);          
         
if (tump==='Connecting...') {
            callplaced
= false;
            console
.log("FAIL ==> internet is OFF ==> Connecting... => " + mystatus  + " ==> " + frustrating);
            frustrating
= true;
         
} else if(tump==='Waiting') {
            console
.log("FAIL ==> internet is OFF ==> Waiting => " + mystatus + " Callplaced: " + callplaced + " Frustration: "  + frustrating);            
           
if (mystatus==='free') {
             
// after call submit waiting and free should be considered as
             
// reload page again
             
if (callplaced) {
                frustrating
= true;
             
}
           
}            
         
} else if(tump==='Connected') {
            callplaced
= false;
            console
.log("FAIL ==> internet is OFF ==> Connected => " + mystatus + " ==> " + frustrating);
            frustrating
= true;
         
} else {
            callplaced
= false;
            console
.log("FAIL ==> internet is OFF ==> ELSE-ELSE => " + mystatus);
           
         
}            
          internet
();
       
}
     
     
});                  
   
} , 1000);    
 
}

To unsubscribe from this group and stop receiving emails from it, send an email to turn-server-project-rfc5766-turn-server+unsubscribe@googlegroups.com.
To post to this group, send email to turn-server-project-rfc5766-turn-...@googlegroups.com.

Sprogrammer

unread,
Mar 12, 2014, 5:02:55 AM3/12/14
to turn-server-project...@googlegroups.com, Sprogrammer, Dmitry Sobinov
@Oleg: Just checked i have been using http://turnserver.open-sys.org/downloads/v3.2.2.910/   YES, there i was able to create that crash.

reg
/sha

Oleg Moskalenko

unread,
Mar 12, 2014, 5:07:19 AM3/12/14
to Sprogrammer, turn-server-project...@googlegroups.com, Dmitry Sobinov
The problem with TCP packet dropping is that it is very difficult to drop TCP traffic consistently. If you ever drop a part of the message (not the whole message) then the TCP stream becomes broken and unrecoverable. So we are using the strategy as this:

1) Use TCP system and application buffering to keep the data as much as possible;
2) If we exhausted the buffers then we drop the connection.

I think that it must work fine in almost all situations.

Oleg

Oleg Moskalenko

unread,
Mar 12, 2014, 5:08:28 AM3/12/14
to Sprogrammer, turn-server-project...@googlegroups.com, Dmitry Sobinov
If you can compile it in debug mode (put option -g into the Makefile) and then produce a core dump and give it to me, then it would be great.

Thanks
Oleg



--

Dmitry Sobinov

unread,
Mar 12, 2014, 5:33:59 AM3/12/14
to turn-server-project...@googlegroups.com, Dmitry Sobinov
Oleg,

I'm concerned about the case when user's network (not the TURN's) fail for a few seconds. For example, short WiFi signal loss. In this case sending side of TURN server just buffer up incoming packets from another peer and send them at once as soon as network is OK. This is quite different from UDP connection case, when packets are just dropped (and it's fine for realtime media). Increasing socket buffers will just make following packet burst worse.

I understand that it's not happening often, but maybe it makes sense to explicit drop packets when queue is high or when queue is growing for N seconds without draining.

Regards,
Dmitry
To post to this group, send email to turn-server-project-rfc5766-turn-s...@googlegroups.com.

Oleg Moskalenko

unread,
Mar 12, 2014, 5:35:42 AM3/12/14
to Dmitry Sobinov, turn-server-project...@googlegroups.com
OK, I'll think what can be done.

Oleg



To unsubscribe from this group and stop receiving emails from it, send an email to turn-server-project-rfc57...@googlegroups.com.
To post to this group, send email to turn-server-project...@googlegroups.com.

Oleg Moskalenko

unread,
Mar 12, 2014, 6:10:51 AM3/12/14
to Dmitry Sobinov, turn-server-project...@googlegroups.com
I am not sure Dmitri that that can be done on the application level. Most of that processing is performed by the system level, and the application level buffers may be empty - but the system TCP/IP stack is still buffering the data.


To unsubscribe from this group and stop receiving emails from it, send an email to turn-server-project-rfc57...@googlegroups.com.
To post to this group, send email to turn-server-project...@googlegroups.com.

Oleg Moskalenko

unread,
Mar 12, 2014, 6:18:01 AM3/12/14
to Dmitry Sobinov, turn-server-project...@googlegroups.com
I guess I'll apply a similar strategy in .911 build.

Sprogrammer

unread,
Mar 12, 2014, 6:20:12 AM3/12/14
to turn-server-project...@googlegroups.com, Dmitry Sobinov
@Oleg: i will do that, Thank you.

@Dimitry: it should be optional but not default in the Turnserver. It works for me like it is now.
The thing you are asking is also make sense but like Oleg said TCP has to disconnect because it is designed like that but with UDP it can be applied.

Maybe with TCP its possible still, i have seen Gstreamer sync=true and false works with TCP stack too. for example:

$ gst-inspect-1.0 tcpserversink
Factory Details:
 
Rank                     none (0)
 
Long-name                TCP server sink
 
Klass                    Sink/Network
 
Description              Send data as a server over the network via TCP
 
Author                   Thomas Vander Stichele <thomas at apestaart dot org>


Plugin Details:
 
Name                     tcp
 
Description              transfer data over the network via TCP
 
Filename                 /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgsttcp.so
 
Version                  1.2.0
 
License                  LGPL
 
Source module            gst-plugins-base
 
Source release date      2013-09-24
 
Binary package           GStreamer Base Plugins (Ubuntu)
 
Origin URL               https://launchpad.net/distros/ubuntu/+source/gst-plugins-base1.0


GObject
 
+----GInitiallyUnowned
       
+----GstObject
             
+----GstElement
                   
+----GstBaseSink
                         
+----GstMultiHandleSink
                               
+----GstMultiSocketSink
                                     
+----GstTCPServerSink


Pad Templates:
  SINK
template: 'sink'
   
Availability: Always
   
Capabilities:
      ANY




Element Flags:
 
no flags set


Element Implementation:
 
Has change_state() function: gst_multi_handle_sink_change_state


Element has no clocking capabilities.
Element has no indexing capabilities.
Element has no URI handling capabilities.


Pads:
  SINK
: 'sink'
   
Implementation:
     
Has chainfunc(): gst_base_sink_chain
     
Has custom eventfunc(): gst_base_sink_event
     
Has custom queryfunc(): gst_base_sink_sink_query
     
Has custom iterintlinkfunc(): gst_pad_iterate_internal_links_default
   
Pad Template: 'sink'


Element Properties:
  name                
: The name of the object
                        flags
: readable, writable
                       
String. Default: "tcpserversink0"
  parent              
: The parent of the object
                        flags
: readable, writable
                       
Object of type "GstObject"
  sync                
: Sync on the clock
                        flags
: readable, writable
                       
Boolean. Default: true
  max
-lateness        : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited)
                        flags
: readable, writable
                       
Integer64. Range: -1 - 9223372036854775807 Default: -1
  qos                
: Generate Quality-of-Service events upstream
                        flags
: readable, writable
                       
Boolean. Default: false
  async              
: Go asynchronously to PAUSED
                        flags
: readable, writable
                       
Boolean. Default: true
  ts
-offset           : Timestamp offset in nanoseconds
                        flags
: readable, writable
                       
Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0
  enable
-last-sample  : Enable the last-sample property
                        flags
: readable, writable
                       
Boolean. Default: true
 
last-sample         : The last sample received in the sink
                        flags
: readable
                       
Boxed pointer of type "GstSample"
  blocksize          
: Size in bytes to pull per buffer (0 = default)
                        flags
: readable, writable
                       
Unsigned Integer. Range: 0 - 4294967295 Default: 4096
  render
-delay        : Additional render delay of the sink in nanoseconds
                        flags
: readable, writable
                       
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  throttle
-time       : The time to keep between rendered buffers (0 = disabled)
                        flags
: readable, writable
                       
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  max
-bitrate         : The maximum bits per second to render (0 = disabled)
                        flags
: readable, writable
                       
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  buffers
-queued      : Number of buffers currently queued
                        flags
: readable
                       
Unsigned Integer. Range: 0 - 4294967295 Default: 0
  unit
-format         : The unit to measure the max/soft-max/queued properties
                        flags
: readable, writable
                       
Enum "GstFormat" Default: 4, "buffers"
                           
(0): undefined        - GST_FORMAT_UNDEFINED
                           
(1): default          - GST_FORMAT_DEFAULT
                           
(2): bytes            - GST_FORMAT_BYTES
                           
(3): time             - GST_FORMAT_TIME
                           
(4): buffers          - GST_FORMAT_BUFFERS
                           
(5): percent          - GST_FORMAT_PERCENT
  units
-max           : max number of units to queue (-1 = no limit)
                        flags
: readable, writable
                       
Integer64. Range: -1 - 9223372036854775807 Default: -1
  units
-soft-max      : Recover client when going over this limit (-1 = no limit)
                        flags
: readable, writable
                       
Integer64. Range: -1 - 9223372036854775807 Default: -1
  buffers
-max         : max number of buffers to queue for a client (-1 = no limit)
                        flags
: readable, writable
                       
Integer. Range: -1 - 2147483647 Default: -1
  buffers
-soft-max    : Recover client when going over this limit (-1 = no limit)
                        flags
: readable, writable
                       
Integer. Range: -1 - 2147483647 Default: -1
  time
-min            : min number of time to queue (-1 = as little as possible)
                        flags
: readable, writable
                       
Integer64. Range: -1 - 9223372036854775807 Default: -1
  bytes
-min           : min number of bytes to queue (-1 = as little as possible)
                        flags
: readable, writable
                       
Integer. Range: -1 - 2147483647 Default: -1
  buffers
-min         : min number of buffers to queue (-1 = as few as possible)
                        flags
: readable, writable
                       
Integer. Range: -1 - 2147483647 Default: -1
  recover
-policy      : How to recover when client reaches the soft max
                        flags
: readable, writable
                       
Enum "GstMultiHandleSinkRecoverPolicy" Default: 0, "none"
                           
(0): none             - Do not try to recover
                           
(1): latest           - Resync client to latest buffer
                           
(2): soft-limit       - Resync client to soft limit
                           
(3): keyframe         - Resync client to most recent keyframe
  timeout            
: Maximum inactivity timeout in nanoseconds for a client (0 = no limit)
                        flags
: readable, writable
                       
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  sync
-method         : How to sync new clients to the stream
                        flags
: readable, writable
                       
Enum "GstMultiHandleSinkSyncMethod" Default: 0, "latest"
                           
(0): latest           - Serve starting from the latest buffer
                           
(1): next-keyframe    - Serve starting from the next keyframe
                           
(2): latest-keyframe  - Serve everything since the latest keyframe (burst)
                           
(3): burst            - Serve burst-value data to client
                           
(4): burst-keyframe   - Serve burst-value data starting on a keyframe
                           
(5): burst-with-keyframe - Serve burst-value data preferably starting on a keyframe
  bytes
-to-serve      : Number of bytes received to serve to clients
                        flags
: readable
                       
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  bytes
-served        : Total number of bytes send to all clients
                        flags
: readable
                       
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  burst
-format        : The format of the burst units (when sync-method is burst[[-with]-keyframe])
                        flags
: readable, writable
                       
Enum "GstFormat" Default: 0, "undefined"
                           
(0): undefined        - GST_FORMAT_UNDEFINED
                           
(1): default          - GST_FORMAT_DEFAULT
                           
(2): bytes            - GST_FORMAT_BYTES
                           
(3): time             - GST_FORMAT_TIME
                           
(4): buffers          - GST_FORMAT_BUFFERS
                           
(5): percent          - GST_FORMAT_PERCENT
  burst
-value         : The amount of burst expressed in burst-format
                        flags
: readable, writable
                       
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  qos
-dscp            : Quality of Service, differentiated services code point (-1 default)
                        flags
: readable, writable
                       
Integer. Range: -1 - 63 Default: -1
  resend
-streamheader : Resend the streamheader if it changes in the caps
                        flags
: readable, writable
                       
Boolean. Default: true
  num
-handles         : The current number of client handles
                        flags
: readable
                       
Unsigned Integer. Range: 0 - 4294967295 Default: 0
  host                
: The host/IP to listen on
                        flags
: readable, writable
                       
String. Default: "localhost"
  port                
: The port to listen to (0=random available port)
                        flags
: readable, writable
                       
Integer. Range: 0 - 65535 Default: 4953
  current
-port        : The port number the socket is currently bound to
                        flags
: readable
                       
Integer. Range: 0 - 65535 Default: 0


Element Signals:
 
"client-added" :  void user_function (GstElement* object,
                                       
GObject arg0,
                                        gpointer user_data
);
 
"client-removed" :  void user_function (GstElement* object,
                                          gint arg0
,
                                         
GstMultiHandleSinkClientStatus arg1,
                                          gpointer user_data
);
 
"client-socket-removed" :  void user_function (GstElement* object,
                                                 
GSocket* arg0,
                                                 gpointer user_data
);


Element Actions:
 
"add" :  void user_function (GstElement* object,
                               
GSocket* arg0);
 
"add-full" :  void user_function (GstElement* object,
                                   
GSocket* arg0,
                                   
GstMultiHandleSinkSyncMethod arg1,
                                   
GstFormat arg2,
                                    guint64 arg3
,
                                   
GstFormat arg4,
                                    guint64 arg5
);
 
"remove" :  void user_function (GstElement* object,
                                 
GSocket* arg0);
 
"remove-flush" :  void user_function (GstElement* object,
                                       
GSocket* arg0);
 
"get-stats" :  GstStructure * user_function (GstElement* object,
                                               
GSocket* arg0);
 
"clear" :  void user_function (GstElement* object);


$ gst
-inspect-1.0 udpsink
Factory Details:
 
Rank                     none (0)
 
Long-name                UDP packet sender
 
Klass                    Sink/Network
 
Description              Send data over the network via UDP
 
Author                   Wim Taymans <wim@fluendo.com>


Plugin Details:
 
Name                     udp
 
Description              transfer data via UDP
 
Filename                 /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstudp.so
 
Version                  1.2.0
 
License                  LGPL
 
Source module            gst-plugins-good
 
Source release date      2013-09-24
 
Binary package           GStreamer Good Plugins (Ubuntu)
 
Origin URL               https://launchpad.net/distros/ubuntu/+source/gst-plugins-good1.0


GObject
 
+----GInitiallyUnowned
       
+----GstObject
             
+----GstElement
                   
+----GstBaseSink
                         
+----GstMultiUDPSink
                               
+----GstUDPSink


Implemented Interfaces:
 
GstURIHandler


Pad Templates:
  SINK
template: 'sink'
   
Availability: Always
   
Capabilities:
      ANY




Element Flags:
 
no flags set


Element Implementation:
 
Has change_state() function: gst_base_sink_change_state


Element has no clocking capabilities.
Element has no indexing capabilities.


URI handling capabilities
:
 
Element can act as sink.
 
Supported URI protocols:
    udp


Pads:
  SINK
: 'sink'
   
Implementation:
     
Has chainfunc(): gst_base_sink_chain
     
Has custom eventfunc(): gst_base_sink_event
     
Has custom queryfunc(): gst_base_sink_sink_query
     
Has custom iterintlinkfunc(): gst_pad_iterate_internal_links_default
   
Pad Template: 'sink'


Element Properties:
  name                
: The name of the object
                        flags
: readable, writable
                       
String. Default: "udpsink0"
  parent              
: The parent of the object
                        flags
: readable, writable
                       
Object of type "GstObject"
  sync                
: Sync on the clock
                        flags
: readable, writable
                       
Boolean. Default: true
  max
-lateness        : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited)
                        flags
: readable, writable
                       
Integer64. Range: -1 - 9223372036854775807 Default: -1
  qos                
: Generate Quality-of-Service events upstream
                        flags
: readable, writable
                       
Boolean. Default: false
  async              
: Go asynchronously to PAUSED
                        flags
: readable, writable
                       
Boolean. Default: true
  ts
-offset           : Timestamp offset in nanoseconds
                        flags
: readable, writable
                       
Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0
  enable
-last-sample  : Enable the last-sample property
                        flags
: readable, writable
                       
Boolean. Default: true
 
last-sample         : The last sample received in the sink
                        flags
: readable
                       
Boxed pointer of type "GstSample"
  blocksize          
: Size in bytes to pull per buffer (0 = default)
                        flags
: readable, writable
                       
Unsigned Integer. Range: 0 - 4294967295 Default: 4096
  render
-delay        : Additional render delay of the sink in nanoseconds
                        flags
: readable, writable
                       
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  throttle
-time       : The time to keep between rendered buffers (0 = disabled)
                        flags
: readable, writable
                       
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  max
-bitrate         : The maximum bits per second to render (0 = disabled)
                        flags
: readable, writable
                       
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  bytes
-to-serve      : Number of bytes received to serve to clients
                        flags
: readable
                       
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  bytes
-served        : Total number of bytes sent to all clients
                        flags
: readable
                       
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  socket              
: Socket to use for UDP sending. (NULL == allocate)
                        flags
: readable, writable
                       
Object of type "GSocket"
  socket
-v6           : Socket to use for UDPv6 sending. (NULL == allocate)
                        flags
: readable, writable
                       
Object of type "GSocket"
  close
-socket        : Close socket if passed as property on state change
                        flags
: readable, writable
                       
Boolean. Default: true
  used
-socket         : Socket currently in use for UDP sending. (NULL == no socket)
                        flags
: readable
                       
Object of type "GSocket"
  used
-socket-v6      : Socket currently in use for UDPv6 sending. (NULL == no socket)
                        flags
: readable
                       
Object of type "GSocket"
  clients            
: A comma separated list of host:port pairs with destinations
                        flags
: readable, writable
                       
String. Default: "localhost:5004"
 
auto-multicast      : Automatically join/leave the multicast groups, FALSE means user has to do it himself
                        flags
: readable, writable
                       
Boolean. Default: true
  multicast
-iface     : The network interface on which to join the multicast group
                        flags
: readable, writable
                       
String. Default: null
  ttl                
: Used for setting the unicast TTL parameter
                        flags
: readable, writable
                       
Integer. Range: 0 - 255 Default: 64
  ttl
-mc              : Used for setting the multicast TTL parameter
                        flags
: readable, writable
                       
Integer. Range: 0 - 255 Default: 1
  loop                
: Used for setting the multicast loop parameter. TRUE = enable, FALSE = disable
                        flags
: readable, writable
                       
Boolean. Default: true
  force
-ipv4          : Forcing the use of an IPv4 socket (DEPRECATED, has no effect anymore)
                        flags
: readable, writable
                       
Boolean. Default: false
  qos
-dscp            : Quality of Service, differentiated services code point (-1 default)
                        flags
: readable, writable
                       
Integer. Range: -1 - 63 Default: -1
  send
-duplicates     : When a distination/port pair is added multiple times, send packets multiple times as well
                        flags
: readable, writable
                       
Boolean. Default: true
  buffer
-size         : Size of the kernel send buffer in bytes, 0=default
                        flags
: readable, writable
                       
Integer. Range: 0 - 2147483647 Default: 0
  bind
-address        : Address to bind the socket to
                        flags
: readable, writable
                       
String. Default: null
  bind
-port           : Port to bind the socket to
                        flags
: readable, writable
                       
Integer. Range: 0 - 65535 Default: 0
  host                
: The host/IP/Multicast group to send the packets to
                        flags
: readable, writable
                       
String. Default: "localhost"
  port                
: The port to send the packets to
                        flags
: readable, writable
                       
Integer. Range: 0 - 65535 Default: 5004


Element Signals:
 
"client-added" :  void user_function (GstElement* object,
                                        gchararray arg0
,
                                        gint arg1
,
                                        gpointer user_data
);
 
"client-removed" :  void user_function (GstElement* object,
                                          gchararray arg0
,
                                          gint arg1
,
                                          gpointer user_data
);


Element Actions:
 
"add" :  void user_function (GstElement* object,
                               gchararray arg0
,
                               gint arg1
);
 
"remove" :  void user_function (GstElement* object,
                                  gchararray arg0
,
                                  gint arg1
);
 
"clear" :  void user_function (GstElement* object);
 
"get-stats" :  GstStructure * user_function (GstElement* object,
                                               gchararray arg0
,
                                               gint arg1
);



Message has been deleted

Dmitry Sobinov

unread,
Mar 12, 2014, 6:28:59 AM3/12/14
to turn-server-project...@googlegroups.com, Dmitry Sobinov
Oleg, yes, I agree, if system/socket buffers (SO_SNDBUF) is too high, then applications buffers draining won't help much.

(my previous message marked as deleted somehow)

Regards,
Dmitry

Dmitry Sobinov

unread,
Mar 12, 2014, 6:32:44 AM3/12/14
to turn-server-project...@googlegroups.com, Dmitry Sobinov
Sprogrammer,

current TCP behavior also can potentially break rate adaptation for WebRTC video. For instance, if client <=> TURN/TCP link reduced bandwidth, but another peer keeps sending higher rate video, then the this media flow will increase queue size on the TURN server. I don't know all the details of rate adaptation algorithm, but I'm pretty sure it's based on measuring packet loss.

Regards,
Dmitry
GST_FORMAT_BUFFERS
                           
(5):<span style="color: #000;"
...

Sprogrammer

unread,
Mar 12, 2014, 6:46:41 AM3/12/14
to turn-server-project...@googlegroups.com, Dmitry Sobinov
@Dmity: i have done socat test, like you said and it works for relay over TCP by dropping packets but you have to re-initialize the connection to TCP relay port with UDP you do not have to. Like already Oleg said.

# socat TCP-LISTEN:80,reuseaddr,fork,su=nobody TCP:www.dmz.mydomain.org:80

- Consider this where you make Turn client connected on port 80
-- send 5 frames to 80
--- in the middle of 4 frame close the turn client
---- socat will also close that TCP session
----- but socat will delivery those received frames to www.dmz.mydomain.org:80
------ now turn client you can reconnect with socat and send the frame number 5
------- socat will send it to the destination


This flow of packets exchange, explains what Oleg was trying to say that TCP has to be disconnected but with UDP its opposite and more friendly

Oleg Moskalenko

unread,
Mar 12, 2014, 7:02:43 PM3/12/14
to turn-server-project...@googlegroups.com, Sprogrammer, Dmitry Sobinov
Shamun, I found a significant memory leak with TCP operations in .910 build. May be you are observing a crash due to that memory leak, when the memory is exhausted.

It will be fixed in .911.

Oleg

Sprogrammer

unread,
Mar 12, 2014, 7:17:17 PM3/12/14
to turn-server-project...@googlegroups.com, Sprogrammer, Dmitry Sobinov
Excellent!! looking forward, i will then start testing on .911
Thank you very much.

Reg
Shamun


Kevin Dempsey

unread,
Mar 13, 2014, 5:18:00 AM3/13/14
to turn-server-project...@googlegroups.com, Sprogrammer, Dmitry Sobinov
When relaying UDP over TCP, then if the UDP is arriving faster than the TCP connection can send data the incoming UDP packets can easily be dropped (for example when the TCP socket is not writable, drop any new UDP packets).

When relaying TCP, I would suggest not reading incoming TCP data when the outbound TCP socket is not writable, That way the originator of the TCP data will, eventually, notice the congestion.

Either way the TURN server should not disconnect because data is arriving faster than it can be sent out.

Oleg Moskalenko

unread,
Mar 13, 2014, 11:27:30 AM3/13/14
to Kevin Dempsey, turn-server-project...@googlegroups.com, Sprogrammer, Dmitry Sobinov
On Thu, Mar 13, 2014 at 2:18 AM, Kevin Dempsey <kevinde...@gmail.com> wrote:
When relaying UDP over TCP, then if the UDP is arriving faster than the TCP connection can send data the incoming UDP packets can easily be dropped (for example when the TCP socket is not writable, drop any new UDP packets).

"socket is not writable" is a too harsh condition, that's not that simple. But the general idea is going to be implemented in .911.
 

When relaying TCP, I would suggest not reading incoming TCP data when the outbound TCP socket is not writable, That way the originator of the TCP data will, eventually, notice the congestion.

That's more complex in RFC 6062 case. The reality is that this approach may affect the performance very badly.
 

Reply all
Reply to author
Forward
0 new messages