金枪鱼之夜：RAIT - Poor man's SDWAN

[陈晟祺]

Hi Tunars,


有谁会不想要属于自己的大内网呢，或者用更加高端一点的说法：SDWAN。tinc，zerotier，我们也能找到许多开源的工具集为我们自动化这一过程，同时像 weave，flannel 这样的 CNI 也是全新的选择。But they do too much more than they are supposed to do. 他们常常会选择到并非最优的路径，性能上也不尽如人意，商业解决方案虽然可能会有更好的效果，价格自然也是 enterprice。而如今我们有了 wireguard，这一不仅安全，还有着极高性能的隧道协议，在它的基础上我们是否能够构建一个 Poor man’s SDWAN 呢？本次 tunight 就来讲讲 RAIT，这一切背后的控制平面，以及在开发及部署过程中的现实问题。

活动信息：
* 主讲人：曹隽诚
* 时间：2020/07/18 19:00 UTC +08:00
* 活动形式：线上会议 + 直播
    * Zoom：658 0847 2766
    * Zoom 直播：https://live.bilibili.com/699121

欢迎一起来玩！

P.S. 可查看 https://cloud.tsinghua.edu.cn/d/ca4dc48637e4432b88f0/ 了解本期主讲人 XD

-- 

Shengqi Chen

[Justin Wong]

赞！想去！

--

Justin Wong

--

您收到此邮件是因为您订阅了Google网上论坛上的“TUNA 主邮件列表”群组。

要退订此群组并停止接收此群组的电子邮件，请发送电子邮件到tuna-general...@googlegroups.com。

要在网络上查看此讨论，请访问https://groups.google.com/d/msgid/tuna-general/b5e8a68e-ff0a-4389-9c75-72bc5d2a464an%40googlegroups.com。

[Wang Kang]

好棒！ 看到了天空工场的小冰箱！(00:07:25开始)

(以及00:07:47的字幕组彩蛋)

[Wang Kang]

不知道是不是Toold，用以下的链接可以在网页里看，不用装Zoom客户端

https://zoom.com.cn/wc/65808472766/join

--
Wang Kang
Blog: http://scateu.me
Fingerprint: 011F 0492 97D6 5D75 8AC4 6458 D43F 3CE2 3353 B7BD
HAM Callsign: BH1RLW

On Thu, 16 Jul 2020, 陈晟祺 wrote:

[Nick Cao]

slides 在附件

项目地址：https://gitlab.com/NickCao/RAIT

陈晟祺 <chensh...@gmail.com> 于2020年7月16日周四 下午9:56写道：

--

[Wang Kang]

我来构成TXT:
RAIT - Poor man’s SDWAN
by Nick Cao

## 什么是沙雕网SDWAN
首先他是个WAN，然后它SD

## 什么是SDWAN
Software defined wide area network (SD-WAN) is a type of computer network that
enables bonding of multiple internet access resources – such as DSL, cable,
cellular or any other IP transport – to provide reliable high throughput data
channels.

ref: What is SD WAN? Software defined WAN (SDWAN) explained – GFI

## Scope of the problem
IP transport: tunnel over clearnet
Bonding of resource: out of scope, why not MPTCP
provide reliable high throughput data channels: dynamic routing

## 那我怎么搞一个

Zerotier ZeroTier – Global Area Networking
Tinc Tinc VPN
Weave Net Weave Net: Network Containers Across Environments
Flannel coreos/flannel: flannel is a network fabric for containers, designed for Kubernetes

## However……

大多有着糟糕的性能
甚至有自己的关不掉的IPAM
还可以硬依赖Docker
更别提诡异的选路

他们干的太多了！

Do One Thing and Do It Well

## Step 1: Link Local Connectivity ( IP transport )

VXLAN
GRE
IPIP
GRETAP
GENEVE

## Pros and Cons
Pro
1.
2.

standardized protocol ensures interoperability
BGP EPVN and other existing control plane eases deployment

Cons
1.
2.

the unneeded ethernet header adds to overhead
protocols other than TCP and UDP may have issue with middle boxes

## But we have: wireguard
1.
2.
3.
4.
5.

operates on layer 3
UDP encapsulated
built in roaming
formally verified cryptography, protocol and implementation
available natively in FreeBSD, OpenBSD and Linux

( it turns out to be a bad decision latter though)

## Step 2: Site Local Connectivity ( Routing Protocols )

RIP
BGP
OSPF
ISIS
EIGRP
OpenFabric

## Pros and Cons
Pros
1.

STAN DAR D （

Cons
1.
2.

mostly with static cost/metric
heavy implementation not suitable for restricted environments

## Still we have: babel

Babel — a loop-avoiding distance-vector routing protocol

optimized for wireless network or tunnels
updates link cost based on the RTT
and even other metrics
source specific routing !
have a stub implementation for embeded system
https://grafana.nichi.co/d/6td87mzGz/node-metrics?viewPanel=12&orgId=1

( I know, but the reference implementation sucks )

## Step 3: Global Connectivity
首先去RIPE注册一下ASN
然后Vultr全区开满
APNIC同款Anycast网络有了

## However……
Wireguard is broken, as intended.
AlllowedIPs is nothing but a routing table
And babeld requires multicast

## 解决办法
那我们创建114514个interface，每个peer一人一个不就好了
你说interface数量上限，不存在的（我试过了）
# for i in {1..1000000}; do ip l add $RANDOM$RANDOM type dummy; done
# ip l | wc -l
70604
至于端口，我们等下再说

## 关于端口
端口数量：1025-65535
端口分配：如何避免冲突？SendPort！

## ## RAIT

NickCao / RAIT · GitLab

（图片太大了塞不下）
https://pb.nichi.co/3f7b343d-5284-466c-b246-35133fd3594d

## What’s more
如何利用好多个上游？
https://github.com/FireflyTang/linux-wireguard-bind

## What’s more
如何省去这一打interface？
AF_WIREGUARD and mapped ethernet address

## Babeld
random-id true
export-table 254
local-path-readwrite /run/babeld.ctl
default type tunnel link-quality true split-horizon false rxcost 32 hello-interval 20
default max-rtt-penalty 1024 rtt-max 1024
interface foo
redistribute ip 2a0c:b641:69c::/48 ge 64 le 64 allow
redistribute ip ::/0 le 0 src-prefix 2a0c:b641:69c::/48 metric 4096
redistribute local deny

## See it in action
2a0c:b641:69c:99cc::1

## Thanks for watching!


--
Wang Kang

> > <https://groups.google.com/d/msgid/tuna-general/b5e8a68e-ff0a-4389-9c75-72bc5d2a464an%40googlegroups.com?utm_medium=email&utm_source=footer>
> > 。
> >
>
>

[Wang Kang]

(pdftotxt效果不太好，被pero指出来了，附上个好点的版本)

# RAIT - Poor man’s SDWAN

by Nick Cao

## 什么是沙雕网SDWAN

首先他是个WAN，然后它SD

## 什么是SDWAN

Software defined wide area network (SD-WAN) is a type of computer network that enables bonding of multiple internet access resources – such as DSL, cable, cellular or any other IP transport – to provide reliable high throughput data channels.

ref: [What is SD WAN? Software defined WAN (SDWAN) explained – GFI](https://www.gfi.com/sd-wan)

## Scope of the problem

- IP transport: tunnel over clearnet
- Bonding of resource: out of scope, why not MPTCP
- provide reliable high throughput data channels: dynamic routing

## 那我怎么搞一个

- Zerotier [ZeroTier – Global Area Networking](https://www.zerotier.com/)
- Tinc [Tinc VPN](https://www.tinc-vpn.org/)
- Weave Net [Weave Net: Network Containers Across Environments](https://www.weave.works/oss/net/)
- Flannel [coreos/flannel: flannel is a network fabric for containers, designed for Kubernetes](https://github.com/coreos/flannel)

## However……

- 大多有着糟糕的性能
- 甚至有自己的关不掉的IPAM
- 还可以硬依赖Docker
- 更别提诡异的选路

他们干的太多了！

Do One Thing and Do It Well

## Step 1: Link Local Connectivity ( IP transport )

- VXLAN
- GRE
- IPIP
- GRETAP
- GENEVE

## Pros and Cons

### Pro

1.standardized protocol ensures interoperability
2.BGP EPVN and other existing control plane eases deployment

### Cons

1.the unneeded ethernet header adds to overhead
2.protocols other than TCP and UDP may have issue with middle boxes

## But we have: wireguard

1.operates on layer 3
2.UDP encapsulated
3.built in roaming
4.formally verified cryptography, protocol and implementation
5.available natively in FreeBSD, OpenBSD and Linux

( it turns out to be a bad decision latter though)

## Step 2: Site Local Connectivity ( Routing Protocols )

- RIP
- BGP
- OSPF
- ISIS
- EIGRP
- OpenFabric

## Pros and Cons

### Pros

1.S T A N D A R D （

### Cons

1.mostly with static cost/metric
2.heavy implementation not suitable for restricted environments

## Still we have: babel

[Babel — a loop-avoiding distance-vector routing protocol](https://www.irif.fr/~jch/software/babel/)

- optimized for wireless network or tunnels
- updates link cost based on the RTT
- and even other metrics
- source specific routing !
- have a stub implementation for embeded system
- https://grafana.nichi.co/d/6td87mzGz/node-metrics?viewPanel=12&orgId=1

( I know, but the reference implementation sucks )

## Step 3: Global Connectivity

- ~~首先去RIPE注册一下ASN~~
- ~~然后Vultr全区开满~~
- ~~APNIC同款Anycast网络有了~~

## However……

- Wireguard is broken, as intended.
- AlllowedIPs is nothing but a routing table
- And babeld requires multicast

## 解决办法

那我们创建114514个interface，每个peer一人一个不就好了
你说interface数量上限，不存在的（我试过了）
# for i in {1..1000000}; do ip l add $RANDOM$RANDOM type dummy; done
# ip l | wc -l
70604
至于端口，我们等下再说

## 关于端口

端口数量：1025-65535
端口分配：如何避免冲突？SendPort！

## RAIT NickCao / [RAIT · GitLab](https://gitlab.com/NickCao/RAIT)