Re: [go-nuts] Peer to peer communication

[David Anderson]

I wrote https://code.google.com/p/nat/ specifically to deal with the NAT traversal problem. It's poorly documented, so you'll have to figure stuff out from the source. Basically, it requires a low bandwidth proxied connection (via a publicly reachable server), and uses that to create a peer to peer connection. It's a slight variant on ICE, which is what most VOIP systems use to establish p2p sessions.

After that, you still have the problem of getting a reliable connection running on top of the UDP socket. My horrible linux-only hack for that is http://code.google.com/p/tcpoveranything . Again, you'll have to figure it out from the source (none of these libraries are really ready for widespread release - they work, but lack documentation).

An alternative would be to implement CurveCP (http://curvecp.org) for Go. agl very graciously implemented the NaCL crypto primitives needed, at http://code.google.com/p/go/source/browse?repo=crypto#hg%2Fnacl . You'll still have to write the rest, including Chicago congestion control, which is poorly documented (do you see a pattern here?).

Finally, another option to consider is to implement UPnP and NAT-PMP clients, so that you can discover consumer-grade routers on the local network and ask them to open a TCP pinhole back to your machine. This is what quite a few video games do these days, but I haven't implemented any of those protocols (warning: UPnP is a horrible, horrible protocol).

NAT traversal is one of my hobbies, as you can possibly tell. But be warned, it's a hard problem to solve in the general case, as you need to bring together a lot of different techniques and protocols. If it's at all possible, it's really simpler to have clients just connect to a publicly accessible server (or set of servers).

- Dave

(For bonus points, ask me why IPv6 won't solve any of this madness, even though it theoretically puts an end to the need for NAT)

On Mon, Oct 8, 2012 at 1:32 PM, ISockGo <amitab...@gmail.com> wrote:

Hi, 

     Can one suggest the best way to communicate and transfer data over peer to peer connections? And how to create for peer to peer sockets. Specially when clients don't have public ip.

Like Skype and flash does. 

Thanks,

Amitabh Arya

 

--
 
 

[Cole Mickens]

For reference, I'm using the mentioned 'nat' library. It has made establishing a UDP session very easy after David assisted me with some slice copying mistakes I had made. After that I have some logic that does some maintenance (heartbeat, timeouts) and another set of logic that has a list of supposed-peers and logic that resolves that list with the connections I really have established and active.

I don't know that this still works, as I extracted this from a private project and used it to debug with David (and has thus been neglected since I got it working last), but it may serve as a useful starting point for someone. https://github.com/colemickens/ezconnector This includes the server for sideband channel used to send the initial signaling messages as well as an example client that can be used to test that two peers can connect. Additionally, if anyone is curious about details I'd be happy to reply off-list.

It may be off topic but I am casually interested in more about CurveCP or why IPv6 won't help the NAT scene (aside from the fact that router manufacturers and corp networks will happily still use NAT).

Your other options besides doing this sort of STUN+signaling is to have your clients open a port to the outside world (via upnp or manually). Not super user-friendly.

[David Anderson]

On Mon, Oct 8, 2012 at 2:33 PM, Cole Mickens <cole.m...@gmail.com> wrote:

For reference, I'm using the mentioned 'nat' library. It has made establishing a UDP session very easy after David assisted me with some slice copying mistakes I had made. After that I have some logic that does some maintenance (heartbeat, timeouts) and another set of logic that has a list of supposed-peers and logic that resolves that list with the connections I really have established and active.

I don't know that this still works, as I extracted this from a private project and used it to debug with David (and has thus been neglected since I got it working last), but it may serve as a useful starting point for someone. https://github.com/colemickens/ezconnector This includes the server for sideband channel used to send the initial signaling messages as well as an example client that can be used to test that two peers can connect. Additionally, if anyone is curious about details I'd be happy to reply off-list.

It may be off topic but I am casually interested in more about CurveCP or why IPv6 won't help the NAT scene (aside from the fact that router manufacturers and corp networks will happily still use NAT).

[warning, rant ahead]

CurveCP, I'd rather let its creator talk about it: http://vimeo.com/18417770

The basic highlights is that it's meant as a replacement of TCP that offers high end to end security at decent performance. It's implemented on top of UDP and has a whole bunch of rather nice security properties (mutual authentication, confidentiality, integrity and better availability than TCP in the face of attacks).

The reality is slightly less rosy than the website makes it out to be. Notably, the congestion control algorithm (Chicago) still seems to fare poorly in real world networks dominated by hostile TCP algorithms, despite the original claims that it manages to wrangle a fair share of bandwidth. But still, it's a neat protocol. The hard bits of its implementation are the crypto primitives (an elliptic curve, a stream cipher and a couple other bits and pieces) and the congestion control algorithm (mostly because it's not specified other than in the code for the reference implementation).

Why won't IPv6 solve anything? Basically, "NAT traversal" is actually two different problems smushed together. One is conntrack traversal, i.e. tricking a firewall's connection tracking logic to let in outside traffic without a formal client-to-server handshake. NAT adds an extra layer of difficulty by making the address and port of the connection hard to figure out.

IPv6 will eliminate NAT, for the most part. However, it very likely won't eliminate connection tracking firewalls from the picture, especially in consumer-grade equipment where the expectation is that you run no servers, and thus only want to allow traffic related to LAN-initiated connections back into the LAN. In theory IPv6 would be all about end-host firewalling, but I sincerely doubt consumer grade equipment will ever give up and be just a dumb router. Dumb routers with no feature checklist don't sell.

Also, IPv6 won't eliminate NAT. There are several NAT mechanisms specified: for v4 to v6, for v6 to v4, and even v6 to v6 for enterprisey multihoming using "unique local addresses", which are the philosophical equivalent to the v4 private IP ranges, only without the risk of inter-LAN collisions when you merge networks. As long as one of these mechanisms still exists, a traversal library that wants to maximize the probability of success needs to handle NATs anyway.

So, while IPv6 will mostly eliminate NATs, it won't significantly ease establishment of peer to peer connections from networks you don't fully control. Fortunately, the ICE algorithm, stripped of all the VoIP-specific stuff, works just fine in IPv4 and IPv6, and is pretty damn close to working everywhere (it still fails in a few specific cases, typically involving two enterprise networks with symmetric NATs).

Your other options besides doing this sort of STUN+signaling is to have your clients open a port to the outside world (via upnp or manually). Not super user-friendly.

Note that UPnP and NAT-PMP can still fail in situations where ICE would succeed, even if the router on your network understands and complies with your request. If you have nested NATs (e.g. an ISP's modem+router with a consumer wifi router plugged into it, and machines hooked off of that), UPnP/NAT-PMP will only "see" the inner NAT device, and will only open a pinhole on it. The second NAT device will still scramble the address/port, making the connection fail.

ICE would succeed in this scenario, because it obtains NATed address information from the public internet, which is assumed (mostly correctly) to be the no-man's-land between all the layers of NATing. ICE doesn't really care about how many NATs are on the way to the target, as long as it can query a server on the internet for what the front-most NAT is doing.

But, of course, there's also a couple of cases where ICE would fail and UPnP/NAT-PMP would work. Sometimes, the nested NATs are intentional. This often happens on university dorm networks, where the dorm as a whole is NATed to the world, and each student has their own wifi router. In this case, if two students wanted to connect, using UPnP/NAT-PMP on their personal routers would allow them to connect through the "middle NAT", never touching the university's NAT device. ICE on the other hand would decide that the public address for both students is on the university's NAT, and the connection would very likely fail, because few NAT devices support "hairpinning" back into the LAN.

The most ideal scenario would be to use UPnP/NAT-PMP to obtain a pinhole, and provide that pinhole address in ICE's handshaking logic. That would enable ICE to connect two students, or a student and an external party. Even better, ICE would likely (depending on tuning params) pick the optimal network path for each type of connection, without having the user specify what their network topology is.

Did I mention how much hatred I have for all this crap that prevents two computers who want to talk from just talking? :-)

- Dave

 

On Monday, October 8, 2012 3:46:23 PM UTC-5, David Anderson wrote:

I wrote https://code.google.com/p/nat/ specifically to deal with the NAT traversal problem. It's poorly documented, so you'll have to figure stuff out from the source. Basically, it requires a low bandwidth proxied connection (via a publicly reachable server), and uses that to create a peer to peer connection. It's a slight variant on ICE, which is what most VOIP systems use to establish p2p sessions.

After that, you still have the problem of getting a reliable connection running on top of the UDP socket. My horrible linux-only hack for that is http://code.google.com/p/tcpoveranything . Again, you'll have to figure it out from the source (none of these libraries are really ready for widespread release - they work, but lack documentation).

An alternative would be to implement CurveCP (http://curvecp.org) for Go. agl very graciously implemented the NaCL crypto primitives needed, at http://code.google.com/p/go/source/browse?repo=crypto#hg%2Fnacl . You'll still have to write the rest, including Chicago congestion control, which is poorly documented (do you see a pattern here?).

Finally, another option to consider is to implement UPnP and NAT-PMP clients, so that you can discover consumer-grade routers on the local network and ask them to open a TCP pinhole back to your machine. This is what quite a few video games do these days, but I haven't implemented any of those protocols (warning: UPnP is a horrible, horrible protocol).

NAT traversal is one of my hobbies, as you can possibly tell. But be warned, it's a hard problem to solve in the general case, as you need to bring together a lot of different techniques and protocols. If it's at all possible, it's really simpler to have clients just connect to a publicly accessible server (or set of servers).

- Dave

(For bonus points, ask me why IPv6 won't solve any of this madness, even though it theoretically puts an end to the need for NAT)

On Mon, Oct 8, 2012 at 1:32 PM, ISockGo <amitab...@gmail.com> wrote:

Hi, 

     Can one suggest the best way to communicate and transfer data over peer to peer connections? And how to create for peer to peer sockets. Specially when clients don't have public ip.

Like Skype and flash does. 

Thanks,

Amitabh Arya

 

--
 
 

--
 
 

[Patrick Mylund Nielsen]

You could port pwnat to Go: http://samy.pl/pwnat/

[David Anderson]

A pwnat-like algorithm requires preexisting knowledge of one of the peer's addresses (so that you can send the ICMP error back to the correct address), and requires root/administrator privileges on both machines to forge and listen for ICMP packets. It's a neat hack, but comes at a significant user experience cost compared to more traditional rendezvous methods.

The algorithm pwnat uses to establish the connection once it learns the right addresses is a subset of the ICE algorithm that will fail in a number of situations where ICE would work fine: when one of the NATs munges the port number, when one of the NATs uses multiple egress IPs (e.g. enterprise NATs), when the two peers are actually on the same LAN and the NAT device doesn't support hairpinning. ICE handles all these situations, and additionally selects the best network path for the situation (e.g. completely bypassing the NAT device if direct communication is possible).

tl;dr: pwnat is a neat hack, but fails in many less than trivial situations, and has significant setup costs compared to alternatives like ICE. It's neat for one-off setups, but if it's one-off you might as well just open the pinholes on the NAT devices and use plain TCP.

- Dave

--

[ju...@benet.ai]

Update Survey. 

I want NAT Traversal in Go. Since this thread is from 2012, I looked around for newer things. I found:

- nat - UDP - (2 years ago) NAT traversal tools for Go. (started this thread).

  https://code.google.com/p/go-nat-pmp/

- GoNeedle - UDP - (4 years ago) is a system for punching reliable transport over UDP through NATs and Firewalls - does not seem maintained.

  https://github.com/petar/GoNeedle

- gomeet - TCP - (10 months ago) - Go server to punch NAT holes

  https://github.com/boppreh/gomeet

- go-nat-pmp - pmp - (1 year ago) - Go client for NAT-PMP

  https://code.google.com/p/go-nat-pmp/

Other possible options:

- make bindings for libjingle - https://developers.google.com/talk/libjingle/

- make bindings for libnice - http://nice.freedesktop.org/wiki/

- make bindings for pjnath - http://www.pjsip.org/pjnath/docs/html/

It looks like `nat` is still the best thing to date. David, are you still maintaining this?

---

I recommend -- to the Go community and anybody interested -- that we maintain one library that implements (standard) ICE, NAT-PMP, and UPnP. A clean, maintained, well documented library would be great to have for any potential p2p apps. Go is well suited for this type of app.

It seems that a fork of nat or GoNeedle, or bindings for the above libs, would be good places to start/continue. Will check these and report anything worth... reporting.

[cayma...@gmail.com]

Started working on bindings for PJNATH very recently, looking to collaborate with others who are well-versed (or not) using cgo.

https://github.com/WeMeetAgain/gopjnath

---
Getting a Go ICE implementation working seems even more useful now that browsers everywhere have WebRTC (which uses ICE to establish a p2p connection).

It seems like this would be a first step toward building p2p apps available to both browsers connecting to 'normal' websites and native clients. As small as that sounds, it would go a long way toward being able to popularize p2p applications for those who can't be bothered to 'download and install' anything without compromising other users' desire to not use the web browser.