SCTP considerations.

[Witold Baryluk]

We all know what are the intentions to use TCP as transport in SPDY,
but i think
somebody should already define how SPDY will work on SCTP, to allow
quicker addoption both of SCTP and SPDY in future networks.

SCTP have many very important characteristics, noting:
- better congestion support
- multiplexing (which should nicly map to SPDY model)
- stateless connection establishing (no SYN DoS attack possible)
- message oriented
- losses of packets affects only streams which was in this packets,
allowing for a substantially smaller delays in others streams
- multihoming

the last two are quite important (i.e. in highly loaded, dynamic or
mobile network),
and impossible to do in single TCP connection, even with SPDY.

There are also many more, but i will just give few references:

Why is SCTP needed given TCP and UDP are widely available?
http://www.isoc.org/briefings/017/

RFC draft?
http://tools.ietf.org/html/draft-natarajan-http-over-sctp-02

HTTP over SCTP versus HTTP over TCP
http://www.cis.udel.edu/~leighton/

A comparison of TCP and SCTP performance using the HTTP protocol -
http://www.ict.kth.se/courses/IK1550/Sample-papers/Henrik_Oesterdahl-sctp-20050525.pdf

HTTP Transfer Latency over SCTP and TCP in Slow Start Phase
http://www.cs.ou.edu/~atiq/papers/07-ICC-Lee-HTTP-SCTP.pdf

And references there.

Hope people involved in spdy and interested parties will learn
somthing from
them.

[Bram Cohen]

On Nov 19, 3:58 pm, Witold Baryluk <witold.bary...@gmail.com> wrote:
> We all know what are the intentions to use TCP as transport in SPDY,
> but i think
> somebody should already define how SPDY will work on SCTP, to allow
> quicker addoption both of SCTP and SPDY in future networks.

The only real difference between TCP and SPDY is that SPDY can get
acks of individual messages. That wouldn't work too well with SPDY,
because SPDY's chunking would make receiving of an out of order chunk
a real problem.

My company is leading the LEDBAT initiative, which will get much
better congestion control into TCP itself, based on one way delays
rather than packet loss, which will cause the intervening packet
queues not to be filled, which will reduce latency very directly.

-Bram

[Mike Belshe]

I'm not against SCTP and would love to see some good perf comparisons of SPDY/SCTP and SPDY/TCP.

So far, SPDY only addresses the application layer, and not the transport.  We believe we can get many of the benefits of SCTP without having to switch out the transport on all systems.

At some point, we do need to look at the transport to verify that it is working as best it can.

Mike

[Paul Amer]

My group at the Univ of Delaware has been researching SCTP for
several years, and we are most interested in investigating which
of SCTP's innovative mechanisms might further enhance Chrome-SPDY.
We've demonstrated that in certain conditions, Firefox over SCTP's
multistreaming provides a significantly improved user experience,
particularly when
downloading multiple progressive images. We are planning now to
evaluate SPDY/SCTP vs SPDY/TCP...

Thanks,
Paul Amer

On Nov 19, 6:58 pm, Witold Baryluk <witold.bary...@gmail.com> wrote:
> We all know what are the intentions to use TCP as transport in SPDY,
> but i think
> somebody should already define how SPDY will work on SCTP, to allow
> quicker addoption both of SCTP and SPDY in future networks.
>
> SCTP have many very important characteristics, noting:
>   - better congestion support
>   - multiplexing  (which should nicly map to SPDY model)
>   - stateless connection establishing (no SYN DoS attack possible)
>   - message oriented
>   - losses of packets affects only streams which was in this packets,
>     allowing for a substantially smaller delays in others streams
>   - multihoming
>
> the last two are quite important (i.e. in highly loaded, dynamic or
> mobile network),
> and impossible to do in single TCP connection, even with SPDY.
>
> There are also many more, but i will just give few references:
>
> Why is SCTP needed given TCP and UDP are widely available?http://www.isoc.org/briefings/017/
>

> RFC draft?http://tools.ietf.org/html/draft-natarajan-http-over-sctp-02
>
> HTTP over SCTP versus HTTP over TCPhttp://www.cis.udel.edu/~leighton/
>
> A comparison of TCP and SCTP performance using the HTTP protocol -http://www.ict.kth.se/courses/IK1550/Sample-papers/Henrik_Oesterdahl-...
>
> HTTP Transfer Latency over SCTP and TCP in Slow Start Phasehttp://www.cs.ou.edu/~atiq/papers/07-ICC-Lee-HTTP-SCTP.pdf

[Bram Cohen]

On Nov 19, 5:23 pm, Bram Cohen <b...@bitconjurer.org> wrote:
>
> The only real difference between TCP and SPDY is that SPDY can get
> acks of individual messages.

Er, I meant 'difference between TCP and SCTP', sorry about that, too
many acronyms.

-Bram

[Witold Baryluk]

On 21 Lis, 02:20, Paul Amer <a...@cis.udel.edu> wrote:
> My group at the Univ of Delaware has been researching SCTP for
> several years, and we are most interested in investigating which
> of SCTP's innovative mechanisms might further enhance Chrome-SPDY.
> We've demonstrated that in certain conditions, Firefox over SCTP's
> multistreaming provides a significantly improved user experience,
> particularly when
> downloading multiple progressive images.  We are planning now to
> evaluate SPDY/SCTP vs SPDY/TCP...

I especially want to point to the feature of SCTP which is impossible
to implement in single TCP connection:

- losses of packets affects only streams which was in this packets,

allowing for a substantially smaller delays in others streams.

This way, on high loss-high latency connection, if we lost packet X
which only have date from stream A, and next packet Y have only data
from stream B,
this data Y from stream B can be immedietly delivered to receiving
application (ie. webbrowser),
and retransmission of X in A can be done asynchronously, without any
blocking.
In TCP (even with SPDY) we need to wait for receiving X, because we
can't deliver
Y befor X to application, so we wait, and then deliver X and Y. This
adds really big
latency to packet Y.

[Mike Belshe]

We should test this; but I don't believe you'll see a difference here.  When a network path is congested (a path is between the same two endpoints), a mux'd TCP connection (SPDY) or multiple streams on a SCTP session should have the same network congestion properties.  The only way one should outperform the other is if the congestion control algorithm is better.  If both SCTP and TCP implement similar congestion control, perf will be the same.  In other words - if the path is congested, all streams on that path feel the same pain; there is no reason to believe that SCTP can magically handle this case better.

But - let's test to find out!  Paul and his team may have more insights.

Mike

[Witold Baryluk]

> In other words - if the path is congested, all streams on that path
> feel the same pain; there is no reason to believe that SCTP can magically
> handle this case better.

There is situation when it is not completly true, packets losts are
not only
becuase network is congested and highly loaded. In mobile phones,
and other wifi-like connections, packets simply are droped/losts
becuase of noisy medium. In such situation it is quite possible
that we have relly big avaible bandwidth, but from time to time,
one packets is just lost.

I don't have big knowledge about congestion control algorithms,
but i know they are designed for many different conditions.

I'm now implementing Erlang implementation of server for SPDY, and
would like to add SCTP layer for it, and then experiment/benchmark
with it. Becuase it is probably only way we can see it is benefitiial
to use it,

[Roberto Peon]

Neat! I love seeing more implementations-- it means more people may send in data :)
-=R

On Nov 24, 2009 8:58 AM, "Witold Baryluk" <witold....@gmail.com> wrote:

> In other words - if the path is congested, all streams on that path > feel the same pain; there is...

[Mike Belshe]

On Tue, Nov 24, 2009 at 8:58 AM, Witold Baryluk <witold....@gmail.com> wrote:

> In other words - if the path is congested, all streams on that path
> feel the same pain; there is no reason to believe that SCTP can magically
> handle this case better.

There is situation when it is not completly true, packets losts are
not only
becuase network is congested and highly loaded. In mobile phones,
and other wifi-like connections, packets simply are droped/losts
becuase of noisy medium. In such situation it is quite possible
that we have relly big avaible bandwidth, but from time to time,
one packets is just lost.

I think random packet loss is rare, although I mostly lack data.   If you know anyone that has real world packet loss modeling data, that would be super.

I did hear from one wireless carrier that the IP-level packet loss is really low - almost zero because they have redundancy at lower layers.   It could be that for mobile networks it is exactly the opposite of intuition :-)

 

I don't have big knowledge about congestion control algorithms,
but i know they are designed for many different conditions.

I'm now implementing Erlang implementation of server for SPDY, and
would like to add SCTP layer for it, and then experiment/benchmark
with it. Becuase it is probably only way we can see it is benefitiial
to use it,

Nifty!  That sounds great

Mike

[Jon Leighton]

On Nov 24, 11:09 am, Mike Belshe <mbel...@google.com> wrote:

As Mike mentioned, we found no real difference here. In our work
running
Firefox and Apache over SCTP, we did find that SCTP tended to have a
very
slight advantage over TCP in terms of total transfer time. This
wasn't due
to SCTP's ability to deliver data independently on each stream. Most
likely it was due to SCTP's implementation of multiple fast
retransmit.
The key benefit of SCTP's logically independent streams was obtained
by
combining SCTP with progressive images. In this case, all images on a
page
were rendered in parallel, and a lost packet in one image did not
impact
the rendering of other images - in other words, head of line blocking
only
impacts the image which loses the packet. For image rich pages this
gives
the appearance of a much faster page load time when compared to
sequential
image rendering. For anyone familiar with our page loading comparison
movies (www.cis.udel.edu/~leighton/), look at for example the 4.3%
loss
with progressive images. While the screen on the right (SCTP) appears
to
load much faster than the screen on the left (TCP), note the blue
progress
bar in the bottom right corner of each screen. These progress bars
show
that the actual page load times are very similar, with SCTP finishing
slightly ahead of TCP.

- Jon

[Mike Belshe]

Thanks Jon -

Naive SCTP question:

Does SCTP prioritize its streams?

mike

[Jon Leighton]

On Nov 24, 2:39 pm, Mike Belshe <mbel...@google.com> wrote:
> Thanks Jon -
>
> Naive SCTP question:
>
> Does SCTP prioritize its streams?

No. I believe there's been some research in this area, but stream
prioritization is not currently a feature of SCTP. You can of course
apply prioritization at the application level, which I'm guessing
would suit most needs, but you do have to implement it yourself.

- Jon

[Bram Cohen]

Very interesting. This would seem to imply that it should be possible
to explicitly set the priority of several things as equal in a way
which means that they should be interleaved. That should be a SPDY
feature though, not a TCP-level feature.

[Mike Belshe]

On Tue, Nov 24, 2009 at 12:31 PM, Jon Leighton <leig...@cis.udel.edu> wrote:

On Nov 24, 2:39 pm, Mike Belshe <mbel...@google.com> wrote:
> Thanks Jon -
>
> Naive SCTP question:
>
> Does SCTP prioritize its streams?

No.  I believe there's been some research in this area, but stream
prioritization is not currently a feature of SCTP.  You can of course
apply prioritization at the application level, which I'm guessing
would suit most needs, but you do have to implement it yourself.

I would guess that works just fine.

SPDY over SCTP could have a few possibilities:

   a) Open many streams, and use one per concurrent request

   b) Open one stream per priority

   c) Open one stream and mux everything through it

In (a) and (b) the server would throttle where to send the request.  It would be interesting to see the performance characteristics of each.

Mike

[Robert Stupp]

Just a question regarding SCTP:
Has anyone experiences using SCTP "in the wild"? Means: Does SCTP work
with conventional routers and firewalls? Do all major operating
systems (beside Unix'es) work together using SCTP? Do all relevant
programming languages (beside C) support SCTP?
I do not want to argue against SCTP (I have no experiences with it) -
but does it really work in the wild?

I've searched for documents regarding SCTP and found some academical
dissertations. Some of them are "very old" and did use only lab
environments - noone used a realistic environment.

Robert

[sal]

Hi Robert,

a couple of year ago I implemented SIP UA on top of SCTP
and at the time it was almost impossible to configure firewalls to
allow SCTP traffic.
So I performed all the test among on the real Internet but only among
machines spread
around the world that weren't behind a Firewall.
My impression is that the situation is not changed.

all the major OS(s) support SCTP, not sure about interoperability
among them.


Salvatore

[Witold Baryluk]

Erlang programing language supports SCTP very well.

It is also avaible in most C based languages, notably C, C++, D, Java,
mayby others.

FreeBSD, Linux, Solaris supports SCTP, but unfortunetly
in most cases SCTP is slower than TCP (i think this
is because TCP have been highly tuned and optimized
over the years).

Problem is with Windows, there is SCTP implementation,
but it isn't free.

Main problem is not with implementstion but with network
infrastructure. Most routers will work,but:
- how with firewalls
- what with stateless routers
- what with NAT

I think SCTP support is very similar to the IPv6, mayby sligthly
in favor to IPv6, but anyway i long term SCTP will be more
and more supported.

I think that SPDY should consider SCTP as alternative transport protocol,
because IMHO it can provide few benefits which are impposible
to achive in TCP (but i can be wrong, or benefits can be
just to small to consider).

So:
- don't hard-code anything which will be highly depended on TCP
- allow mapping of SPDY stream to SCTP streams
- design low-latency SCTP / TCP discovery (so transition and
cooperation is possible)