SCTP in GO

[Olivier Van Acker]

Hi,

I'm currently writing my dissertation for my MSc in Computer Science and as subject of my paper I came up with the idea to extend Go with the SCTP protocol. As suggested on the Go website I'm posting this mail to discuss the idea before jumping in the deep. 

First a little bit of background; SCTP is a protocol in the same layer as TCP and UDP and can be seen as an improvement on these protocols. SCTP has message boundaries, is reliable, can have multiple data streams and much more. One of the main advantages is that with SCTP it is much easier to build a client-server app than with TCP because a lot of the complexity doesn't have to be resolved in the application layer anymore. (I do admit I'm generalizing a bit here because a lot depends on what type of application you build). 

With this in my mind I started reading about Go and soon discovered it also tries to make writing software less complicated. This got me to thinking: simple + simple = 'more simple' ;-)  So why not combine the two?

To see if this plan is feasible I did a bit of research and did some coding to see how difficult it would be, here are my findings so far:

• On of the main things to keep in mind is that SCTP only has a native implementation on FreeBSD, Solaris and Linux. For Mac OS X and Windows drivers are available but must ne installed manually.
• SCTP works with two models, one-to-one and one-to-many. The one-to-one model is for easy migration of existing TCP application, it maps one-to-one to the system calls TCP does to do a connection but doesn't have all the (cool) features of SCTP. One-to-many model is full on SCTP, significantly different from  TCP/UDP implementations.
• I created some C TCP/SCTP echo client server apps, rewrote them in GO (the TCP bit) and as a next step I checked out the release branch of Go and extended it  far enough to be able to write an SCTP Go server (one to one model, very similar to TCP). And hooray! I got the SCTP Go server working in less than a day :-)

The first issue (no native windows/osx support) can be tackled by implementing SCTP on top of UDP. An example implementation of this has been written by Randy Stewart (the main driver behind the protocol) and published in one of his books.  The second and third findings made me conclude that the one-to-one model is very easy to implement in Go, the TCP is an excellent template. The one-to-many model is more work and needs some extra thinking because of different way connections get established etc. But I don't think it would be to hard to do this. 

So what do you think, is this an idea worth implementing. If so what is the best way forward? Shall I integrate it in the language or separate it out a bit more? Are there other areas within Go which can benefit from this, e.g. channels?

Olivier

[Mikio Hara]

Hi Oliver,

On Sat, Jan 28, 2012 at 8:11 PM, Olivier Van Acker
<cyber...@gmail.com> wrote:

> So what do you think, is this an idea worth implementing. If so what is the
> best way forward? Shall I integrate it in the language or separate it out a
> bit more? Are there other areas within Go which can benefit from this, e.g.
> channels?

I'm happy to see SCTP stuff even though it works on freebsd, linux
only at first. I guess it would be a part of net package API like other
transport protocols such as TCP, UDP because there's no plan to
expose ProtocolConn's underlying sockets/FDs in the net package.

-- Mikio

[gta]

+1

A package for tunneling SCTP on top pf UDP could be nice to start play with.

[Ondřej Kupka]

Hi,

SCTP is definitely a nice protocol to have, +1 for implementing this. I was just wondering what is the status of the implementation. I would need to use it like yesterday, but I guess it's not ready yet :-)

It would be nice to have it in Go directly if it fits into net package. I found a repo containing Go sources extended with SCTP, but that is not very user-friendly, to install forked Go to get SCTP. I think that it should be either incorporated into mainline or implemented as a separate package...

Cheers,

Ondrej Kupka

[Alexandre Fiori]

I might be wrong but it could start as a separate package (even if it require a C companion lib in the beginning) and eventually get merged into the net package. The reason is to avoid maintaining the entire Go tree during the implementation - which can be hard to keep up with.

I'd also be happy to have SCTP and work on a Diameter implementation in pure Go. :-)

[Juan Batiz-Benet]

It would be really great to have SCTP (and have it work over UDP). Even if it does not fit into `net` package anytime soon, a library would be great!

The go fork SCTP implementation at https://bitbucket.org/cyberroadie/go-sctp is probably out of date with current Go. 

Question to the Go team: what's the plan with more protocols like SCTP, uTP, QUIC, etc? It's probably going to be painful for you to have to add them to stdlib (and that gating factor will probably mean lots of protocols are never implemented). At the same time, all the portable heavy lifting in the net pkg isn't exposed, so libs would be forced to copy it and diverge.

[Ian Taylor]

On Sat, Nov 8, 2014 at 7:21 AM, Juan Batiz-Benet <ju...@benet.ai> wrote:
>
> Question to the Go team: what's the plan with more protocols like SCTP, uTP,
> QUIC, etc? It's probably going to be painful for you to have to add them to
> stdlib (and that gating factor will probably mean lots of protocols are
> never implemented). At the same time, all the portable heavy lifting in the
> net pkg isn't exposed, so libs would be forced to copy it and diverge.

Many cases are handled by the go.net repository. I don't know whether
that approach can work with the protocols you mention, but we should
at least rule it out first.

Ian

[Juan Batiz-Benet]

> It would help bring webrtc data channels to go

Yes, it would. (( On that, i'm gathering people interested in ripping out the peer datachannel from libwebrtc into its own c lib for multi language friendliness here: https://github.com/jbenet/random-ideas/issues/13 I think it's easier to start with bindings to scope out the interface and then swap out a pure go impl over time ))

Related,  I needed a reliable-chan-on-udp-stopgap (i cant tunnel tcp on top of udp with standard go objects), so I wrote a wrapper around the C++ UDT impl (forked from @getlantern's start at it): 

- https://github.com/jbenet/go-udtwrapper/tree/master/udtcat

- https://github.com/jbenet/go-udtwrapper/

Still pretty raw, but it exposes net familiar constructs: Listener, Dialer, UDTConn, UDTAddr, etc. (yes, the names stutter, conventions clashing).

And have transferred 100s of GB with the udtcat tool reliably (test cases :) ). But, think it's slower than TCP-- will benchmark at some point to figure out why. (roughly, I'm seeing ~100MBps locally, and i think it should be closer to 1GBps.  Probably my sync crap (cause i can't use fd-mutex!) or clashing with the C++ impl's world.

Juan

On Mon, Nov 10, 2014 at 11:11 PM, <james....@gmail.com> wrote:

It would help bring webrtc data channels to go

It might be best to port the userland implementation into pure go

http://sctp.fh-muenster.de/sctp-user-land-stack.html

https://github.com/rillian/libusrsctp

http://goo.gl/10fQzq