Skip to first unread message
Sonic Lee
May 18, 2017, 2:31:20 AM5/18/17
to Kubernetes user discussion and Q&A
I have set up kubeadm 1.6.1, and trying to use flannel.
I can set up kubeadm init and have another node join, so I have two nodes like this.
NAME STATUS AGE VERSION
master Ready 23m v1.6.1
node1 Ready 14m v1.6.1
I can set up flannel fine, and it will show that DNS is running and all,
but when I actually deploy a nodeJS with replication of 2 I just cannot curl slave's pod from master node PC.
Master node pod: 10.244.0.126
Slave node pod: 10.244.1.126
[master] curl 10.244.1.126:8080
This will hang and it will ONLY work if I curl from slave node PC.
Does anyone have any experience with this?
Thanks in advance.
Brandon Philips
May 18, 2017, 12:58:42 PM5/18/17
to kubernet...@googlegroups.com, Tom Denham
What instructions did you follow to setup flannel? Link?
Output from `ip addr`, `route -n`, `iptables -L -t nat`, and `iptables -L` is helpful as well.
This reminds me we need a better set of docs that lists these things out for filing issues.
Thanks,
Brandon
--
You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-use...@googlegroups.com.
To post to this group, send email to kubernet...@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.
Sonic Lee
May 19, 2017, 2:53:13 AM5/19/17
to Kubernetes user discussion and Q&A, t...@tigera.io
I followed pretty much the standard kubernetes 1.6 setup guide.
https://kubernetes.io/docs/getting-started-guides/kubeadm/#pod-network
So I just grabbed flannel.yaml here.
So I just grabbed flannel.yaml here.
-> Ver 0.7.1 seems to always deploy apps to guest node rather than the current master node. So I changed the ver to 0.7.0 and it deploys to BOTH master and guest nodes.
and flannel-rbac.yml
Now for your suggestions :)
$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 12:13:3a:ca:00:10 brd ff:ff:ff:ff:ff:ff
inet 50.1.111.181/24 brd 50.1.111.255 scope global eth0
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN
link/ether 02:42:4e:90:2d:24 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 scope global docker0
valid_lft forever preferred_lft forever
6: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1472 qdisc noqueue state UP qlen 1000
link/ether 0a:58:0a:f4:00:01 brd ff:ff:ff:ff:ff:ff
inet 10.244.0.1/24 scope global cni0
valid_lft forever preferred_lft forever
13: flannel0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1472 qdisc pfifo_fast state UNKNOWN qlen 500
link/none
inet 10.244.0.0/16 scope global flannel0
valid_lft forever preferred_lft forever
14: vetha45b5730@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1472 qdisc noqueue master cni0 state UP
link/ether 1a:b9:19:e4:21:58 brd ff:ff:ff:ff:ff:ff link-netnsid 0
15: vethcb19a152@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1472 qdisc noqueue master cni0 state UP
link/ether 36:4e:31:51:38:64 brd ff:ff:ff:ff:ff:ff link-netnsid 1
$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 50.1.111.1 0.0.0.0 UG 100 0 0 eth0
10.244.0.0 0.0.0.0 255.255.255.0 U 0 0 0 cni0
10.244.0.0 0.0.0.0 255.255.0.0 U 0 0 0 flannel0
10.244.0.0 0.0.0.0 255.255.0.0 U 2 0 0 flannel0
50.1.111.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
$ sudo iptables -L -t nat
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- anywhere anywhere /* kubernetes service portals */
PREROUTING_direct all -- anywhere anywhere
PREROUTING_ZONES_SOURCE all -- anywhere anywhere
PREROUTING_ZONES all -- anywhere anywhere
DOCKER all -- anywhere anywhere ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- anywhere anywhere /* kubernetes service portals */
OUTPUT_direct all -- anywhere anywhere
DOCKER all -- anywhere !loopback/8 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
KUBE-POSTROUTING all -- anywhere anywhere /* kubernetes postrouting rules */
MASQUERADE all -- 172.17.0.0/16 anywhere
POSTROUTING_direct all -- anywhere anywhere
POSTROUTING_ZONES_SOURCE all -- anywhere anywhere
POSTROUTING_ZONES all -- anywhere anywhere
RETURN all -- bef-dev-kubeadm-master/16 bef-dev-kubeadm-master/16
MASQUERADE all -- bef-dev-kubeadm-master/16 !base-address.mcast.net/4
MASQUERADE all -- !bef-dev-kubeadm-master/16 bef-dev-kubeadm-master/16
Chain DOCKER (2 references)
target prot opt source destination
RETURN all -- anywhere anywhere
Chain KUBE-MARK-DROP (0 references)
target prot opt source destination
MARK all -- anywhere anywhere MARK or 0x8000
Chain KUBE-MARK-MASQ (6 references)
target prot opt source destination
MARK all -- anywhere anywhere MARK or 0x4000
Chain KUBE-NODEPORTS (1 references)
target prot opt source destination
Chain KUBE-POSTROUTING (1 references)
target prot opt source destination
MASQUERADE all -- anywhere anywhere /* kubernetes service traffic requiring SNAT */ mark match 0x4000/0x4000
Chain KUBE-SEP-4IHXB76RSMVLITDK (1 references)
target prot opt source destination
KUBE-MARK-MASQ all -- 10.244.0.117 anywhere /* kube-system/kube-dns:dns */
DNAT udp -- anywhere anywhere /* kube-system/kube-dns:dns */ udp to:10.244.0.117:53
Chain KUBE-SEP-CZ6ETURPXDAIJEKZ (1 references)
target prot opt source destination
KUBE-MARK-MASQ all -- 10.244.0.117 anywhere /* kube-system/kube-dns:dns-tcp */
DNAT tcp -- anywhere anywhere /* kube-system/kube-dns:dns-tcp */ tcp to:10.244.0.117:53
Chain KUBE-SEP-ZGC6TZD6RICYJPAF (2 references)
target prot opt source destination
KUBE-MARK-MASQ all -- bef-dev-kubeadm-master anywhere /* default/kubernetes:https */
DNAT tcp -- anywhere anywhere /* default/kubernetes:https */ recent: SET name: KUBE-SEP-ZGC6TZD6RICYJPAF side: source mask: 255.255.255.255 tcp to:50.1.111.181:6443
Chain KUBE-SERVICES (2 references)
target prot opt source destination
KUBE-MARK-MASQ tcp -- !bef-dev-kubeadm-master/16 10.96.0.1 /* default/kubernetes:https cluster IP */ tcp dpt:https
KUBE-SVC-NPX46M4PTMTKRN6Y tcp -- anywhere 10.96.0.1 /* default/kubernetes:https cluster IP */ tcp dpt:https
KUBE-MARK-MASQ udp -- !bef-dev-kubeadm-master/16 10.96.0.10 /* kube-system/kube-dns:dns cluster IP */ udp dpt:domain
KUBE-SVC-TCOU7JCQXEZGVUNU udp -- anywhere 10.96.0.10 /* kube-system/kube-dns:dns cluster IP */ udp dpt:domain
KUBE-MARK-MASQ tcp -- !bef-dev-kubeadm-master/16 10.96.0.10 /* kube-system/kube-dns:dns-tcp cluster IP */ tcp dpt:domain
KUBE-SVC-ERIFXISQEP7F7OF4 tcp -- anywhere 10.96.0.10 /* kube-system/kube-dns:dns-tcp cluster IP */ tcp dpt:domain
KUBE-NODEPORTS all -- anywhere anywhere /* kubernetes service nodeports; NOTE: this must be the last rule in this chain */ ADDRTYPE match dst-type LOCAL
Chain KUBE-SVC-ERIFXISQEP7F7OF4 (1 references)
target prot opt source destination
KUBE-SEP-CZ6ETURPXDAIJEKZ all -- anywhere anywhere /* kube-system/kube-dns:dns-tcp */
Chain KUBE-SVC-NPX46M4PTMTKRN6Y (1 references)
target prot opt source destination
KUBE-SEP-ZGC6TZD6RICYJPAF all -- anywhere anywhere /* default/kubernetes:https */ recent: CHECK seconds: 10800 reap name: KUBE-SEP-ZGC6TZD6RICYJPAF side: source mask: 255.255.255.255
KUBE-SEP-ZGC6TZD6RICYJPAF all -- anywhere anywhere /* default/kubernetes:https */
Chain KUBE-SVC-TCOU7JCQXEZGVUNU (1 references)
target prot opt source destination
KUBE-SEP-4IHXB76RSMVLITDK all -- anywhere anywhere /* kube-system/kube-dns:dns */
Chain OUTPUT_direct (1 references)
target prot opt source destination
Chain POSTROUTING_ZONES (1 references)
target prot opt source destination
POST_public all -- anywhere anywhere [goto]
POST_public all -- anywhere anywhere [goto]
Chain POSTROUTING_ZONES_SOURCE (1 references)
target prot opt source destination
Chain POSTROUTING_direct (1 references)
target prot opt source destination
Chain POST_public (2 references)
target prot opt source destination
POST_public_log all -- anywhere anywhere
POST_public_deny all -- anywhere anywhere
POST_public_allow all -- anywhere anywhere
Chain POST_public_allow (1 references)
target prot opt source destination
Chain POST_public_deny (1 references)
target prot opt source destination
Chain POST_public_log (1 references)
target prot opt source destination
Chain PREROUTING_ZONES (1 references)
target prot opt source destination
PRE_public all -- anywhere anywhere [goto]
PRE_public all -- anywhere anywhere [goto]
Chain PREROUTING_ZONES_SOURCE (1 references)
target prot opt source destination
Chain PREROUTING_direct (1 references)
target prot opt source destination
Chain PRE_public (2 references)
target prot opt source destination
PRE_public_log all -- anywhere anywhere
PRE_public_deny all -- anywhere anywhere
PRE_public_allow all -- anywhere anywhere
Chain PRE_public_allow (1 references)
target prot opt source destination
Chain PRE_public_deny (1 references)
target prot opt source destination
Chain PRE_public_log (1 references)
target prot opt source destination
$ sudo iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
KUBE-FIREWALL all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere
INPUT_direct all -- anywhere anywhere
INPUT_ZONES_SOURCE all -- anywhere anywhere
INPUT_ZONES all -- anywhere anywhere
DROP all -- anywhere anywhere ctstate INVALID
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
Chain FORWARD (policy DROP)
target prot opt source destination
DOCKER-ISOLATION all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
DOCKER all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere
FORWARD_direct all -- anywhere anywhere
FORWARD_IN_ZONES_SOURCE all -- anywhere anywhere
FORWARD_IN_ZONES all -- anywhere anywhere
FORWARD_OUT_ZONES_SOURCE all -- anywhere anywhere
FORWARD_OUT_ZONES all -- anywhere anywhere
DROP all -- anywhere anywhere ctstate INVALID
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- anywhere anywhere /* kubernetes service portals */
KUBE-FIREWALL all -- anywhere anywhere
OUTPUT_direct all -- anywhere anywhere
Chain DOCKER (1 references)
target prot opt source destination
Chain DOCKER-ISOLATION (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere
Chain FORWARD_IN_ZONES (1 references)
target prot opt source destination
FWDI_public all -- anywhere anywhere [goto]
FWDI_public all -- anywhere anywhere [goto]
Chain FORWARD_IN_ZONES_SOURCE (1 references)
target prot opt source destination
Chain FORWARD_OUT_ZONES (1 references)
target prot opt source destination
FWDO_public all -- anywhere anywhere [goto]
FWDO_public all -- anywhere anywhere [goto]
Chain FORWARD_OUT_ZONES_SOURCE (1 references)
target prot opt source destination
Chain FORWARD_direct (1 references)
target prot opt source destination
Chain FWDI_public (2 references)
target prot opt source destination
FWDI_public_log all -- anywhere anywhere
FWDI_public_deny all -- anywhere anywhere
FWDI_public_allow all -- anywhere anywhere
ACCEPT icmp -- anywhere anywhere
Chain FWDI_public_allow (1 references)
target prot opt source destination
Chain FWDI_public_deny (1 references)
target prot opt source destination
Chain FWDI_public_log (1 references)
target prot opt source destination
Chain FWDO_public (2 references)
target prot opt source destination
FWDO_public_log all -- anywhere anywhere
FWDO_public_deny all -- anywhere anywhere
FWDO_public_allow all -- anywhere anywhere
Chain FWDO_public_allow (1 references)
target prot opt source destination
Chain FWDO_public_deny (1 references)
target prot opt source destination
Chain FWDO_public_log (1 references)
target prot opt source destination
Chain INPUT_ZONES (1 references)
target prot opt source destination
IN_public all -- anywhere anywhere [goto]
IN_public all -- anywhere anywhere [goto]
Chain INPUT_ZONES_SOURCE (1 references)
target prot opt source destination
Chain INPUT_direct (1 references)
target prot opt source destination
Chain IN_public (2 references)
target prot opt source destination
IN_public_log all -- anywhere anywhere
IN_public_deny all -- anywhere anywhere
IN_public_allow all -- anywhere anywhere
ACCEPT icmp -- anywhere anywhere
Chain IN_public_allow (1 references)
target prot opt source destination
ACCEPT tcp -- anywhere anywhere tcp dpt:ssh ctstate NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:sun-sr-https ctstate NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:https ctstate NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:31619 ctstate NEW
ACCEPT udp -- anywhere anywhere udp dpt:8285 ctstate NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:10250 ctstate NEW
ACCEPT udp -- anywhere anywhere udp dpt:otv ctstate NEW
Chain IN_public_deny (1 references)
target prot opt source destination
Chain IN_public_log (1 references)
target prot opt source destination
Chain KUBE-FIREWALL (2 references)
target prot opt source destination
DROP all -- anywhere anywhere /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000
Chain KUBE-SERVICES (1 references)
target prot opt source destination
Chain OUTPUT_direct (1 references)
target prot opt source destination
-------------------------- Thanks! ---------------------------------
2017년 5월 19일 금요일 오전 1시 58분 42초 UTC+9, Brandon Philips 님의 말:
2017년 5월 19일 금요일 오전 1시 58분 42초 UTC+9, Brandon Philips 님의 말:
Adieu
May 19, 2017, 4:44:27 AM5/19/17
to kubernet...@googlegroups.com
Hi Sonic，
I think the FORWARD chain might drop your packets sent to flannel.
>
> Chain FORWARD (policy DROP)
> target prot opt source destination
> DOCKER-ISOLATION all -- anywhere anywhere
> ACCEPT all -- anywhere anywhere ctstate
> RELATED,ESTABLISHED
> DOCKER all -- anywhere anywhere
> ACCEPT all -- anywhere anywhere
> ACCEPT all -- anywhere anywhere
> ACCEPT all -- anywhere anywhere ctstate
> RELATED,ESTABLISHED
> ACCEPT all -- anywhere anywhere
> FORWARD_direct all -- anywhere anywhere
> FORWARD_IN_ZONES_SOURCE all -- anywhere anywhere
> FORWARD_IN_ZONES all -- anywhere anywhere
> FORWARD_OUT_ZONES_SOURCE all -- anywhere anywhere
> FORWARD_OUT_ZONES all -- anywhere anywhere
> DROP all -- anywhere anywhere ctstate
> INVALID
> REJECT all -- anywhere anywhere reject-with
> icmp-host-prohibited
>
Try change the default rule to ACCEPT and see if it works.
Regards,
Adieu
I think the FORWARD chain might drop your packets sent to flannel.
>
> Chain FORWARD (policy DROP)
> target prot opt source destination
> DOCKER-ISOLATION all -- anywhere anywhere
> ACCEPT all -- anywhere anywhere ctstate
> RELATED,ESTABLISHED
> DOCKER all -- anywhere anywhere
> ACCEPT all -- anywhere anywhere
> ACCEPT all -- anywhere anywhere
> ACCEPT all -- anywhere anywhere ctstate
> RELATED,ESTABLISHED
> ACCEPT all -- anywhere anywhere
> FORWARD_direct all -- anywhere anywhere
> FORWARD_IN_ZONES_SOURCE all -- anywhere anywhere
> FORWARD_IN_ZONES all -- anywhere anywhere
> FORWARD_OUT_ZONES_SOURCE all -- anywhere anywhere
> FORWARD_OUT_ZONES all -- anywhere anywhere
> DROP all -- anywhere anywhere ctstate
> INVALID
> REJECT all -- anywhere anywhere reject-with
> icmp-host-prohibited
>
Regards,
Adieu
Sonic Lee
May 25, 2017, 12:18:37 AM5/25/17
to Kubernetes user discussion and Q&A
OK I see your point.
2017년 5월 19일 금요일 오후 5시 44분 27초 UTC+9, Ivan Diao 님의 말:
But since I'm using Centos 7 I'm using firewalld not iptables.
I noticed just now if I turn off firewalld, it works perfectly. It really must be the problem with having firewalld on.
Is it OK to just turn off firewalld and run kubernetes clusters?
2017년 5월 19일 금요일 오후 5시 44분 27초 UTC+9, Ivan Diao 님의 말:
Reply all
Reply to author
Forward
0 new messages