k8s Nodeport方式下service访问，iptables处理逻辑

2017年07月11日 13948点热度 0人点赞 0条评论
Warning: Array to string conversion in /home/wwwroot/myf5/wp-content/plugins/wpjam-basic/public/wpjam-utils.php on line 496
借用网上一张图，画的挺好~ 来自http://blog.csdn.net/xinghun_4/article/details/50492041
对应解释如下
[root@docker3 ~]#  iptables -nL -t nat -v
Chain PREROUTING (policy ACCEPT 2 packets, 1152 bytes)
 pkts bytes target     prot opt in     out     source               destination         
10837 3999K KUBE-SERVICES  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service portals */
====》1. 进入的请求，首先匹配PREROUTING 链里的该规则，并进入KUBE-SERVICES链
  238 27286 DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 2 packets, 1152 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 8 packets, 480 bytes)
 pkts bytes target     prot opt in     out     source               destination         
61563 3722K KUBE-SERVICES  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service portals */
    0     0 DOCKER     all  --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 8 packets, 480 bytes)
 pkts bytes target     prot opt in     out     source               destination         
61840 3742K KUBE-POSTROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes postrouting rules */
====》6. 需要路由到另一个主机的连接经由POSTROUTING该规则，进入KUBE-POSTROUTING 链
   62  4323 MASQUERADE  all  --  *      !docker0  10.2.39.0/24         0.0.0.0/0           
    0     0 MASQUERADE  all  --  *      !docker0  10.1.33.0/24         0.0.0.0/0           

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 RETURN     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0           

Chain KUBE-MARK-DROP (0 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            MARK or 0x8000

Chain KUBE-MARK-MASQ (6 references)
 pkts bytes target     prot opt in     out     source               destination         
    2   128 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            MARK or 0x4000

Chain KUBE-NODEPORTS (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    2   128 KUBE-MARK-MASQ  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/k8s-nginx: */ tcp dpt:30780
    2   128 KUBE-SVC-2RMP45C4XWDG5BGC  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/k8s-nginx: */ tcp dpt:30780
    ====》3.请求命中上述两个规则，一个进行进入KUBE-MARK-MASQ 链进行标记，一个进入KUBE-SVC-2RMP45C4XWDG5BGC链

Chain KUBE-POSTROUTING (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    1    64 MASQUERADE  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service traffic requiring SNAT */ mark match 0x4000/0x4000
    ====》7. 执行源地址转换(在flannel网络这里转换的地址是flannel.1即flannel在ifconfig里输出接口的地址），发往另一个node

Chain KUBE-SEP-D5T62RWZFFOCR77Q (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 KUBE-MARK-MASQ  all  --  *      *       10.2.39.3            0.0.0.0/0            /* default/k8s-nginx: */
    1    64 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/k8s-nginx: */ tcp to:10.2.39.3:80
    ====》5-2. DNAT到本机，交给INPUT

Chain KUBE-SEP-IK3IYR4STYKRJP77 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 KUBE-MARK-MASQ  all  --  *      *       10.2.39.2            0.0.0.0/0            /* kube-system/kube-dns:dns-tcp */
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns-tcp */ tcp to:10.2.39.2:53

Chain KUBE-SEP-WV6S37CDULKCYEVE (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 KUBE-MARK-MASQ  all  --  *      *       10.2.39.2            0.0.0.0/0            /* kube-system/kube-dns:dns */
    0     0 DNAT       udp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns */ udp to:10.2.39.2:53

Chain KUBE-SEP-X7YOSBI66WAQ7F6X (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 KUBE-MARK-MASQ  all  --  *      *       172.16.199.17        0.0.0.0/0            /* default/kubernetes:https */
    0     0 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/kubernetes:https */ recent: SET name: KUBE-SEP-X7YOSBI66WAQ7F6X side: source mask: 255.255.255.255 tcp to:172.16.199.17:6443

Chain KUBE-SEP-YXWG4KEJCDIRMCO5 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 KUBE-MARK-MASQ  all  --  *      *       10.2.4.2             0.0.0.0/0            /* default/k8s-nginx: */
    1    64 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/k8s-nginx: */ tcp to:10.2.4.2:80
    ====》5-1. DNAT到另一个node，随后执行postrouting

Chain KUBE-SERVICES (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 KUBE-SVC-2RMP45C4XWDG5BGC  tcp  --  *      *       0.0.0.0/0            169.169.148.143      /* default/k8s-nginx: cluster IP */ tcp dpt:80
    0     0 KUBE-SVC-NPX46M4PTMTKRN6Y  tcp  --  *      *       0.0.0.0/0            169.169.0.1          /* default/kubernetes:https cluster IP */ tcp dpt:443
    0     0 KUBE-SVC-TCOU7JCQXEZGVUNU  udp  --  *      *       0.0.0.0/0            169.169.0.53         /* kube-system/kube-dns:dns cluster IP */ udp dpt:53
    0     0 KUBE-SVC-ERIFXISQEP7F7OF4  tcp  --  *      *       0.0.0.0/0            169.169.0.53         /* kube-system/kube-dns:dns-tcp cluster IP */ tcp dpt:53
    2   128 KUBE-NODEPORTS  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service nodeports; NOTE: this must be the last rule in this chain */ ADDRTYPE match dst-type LOCAL
    ====》2.连接随后并该规则匹配，进入KUBE-NODEPORTS 链

Chain KUBE-SVC-2RMP45C4XWDG5BGC (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    1    64 KUBE-SEP-D5T62RWZFFOCR77Q  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/k8s-nginx: */ statistic mode random probability 0.50000000000
    1    64 KUBE-SEP-YXWG4KEJCDIRMCO5  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/k8s-nginx: */
====》4.请求被该链中两个规则匹配，这里只有两个规则，按照50% RR规则进行负载均衡分发，两个目标链都是进行DNAT，一个转到本node的pod IP上，一个转到另一台宿主机的pod上，因为该service下只有两个pod
==========》乱入的广告,www.myf5.net《==================
Chain KUBE-SVC-ERIFXISQEP7F7OF4 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 KUBE-SEP-IK3IYR4STYKRJP77  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns-tcp */

Chain KUBE-SVC-NPX46M4PTMTKRN6Y (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 KUBE-SEP-X7YOSBI66WAQ7F6X  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/kubernetes:https */ recent: CHECK seconds: 10800 reap name: KUBE-SEP-X7YOSBI66WAQ7F6X side: source mask: 255.255.255.255
    0     0 KUBE-SEP-X7YOSBI66WAQ7F6X  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* default/kubernetes:https */

Chain KUBE-SVC-TCOU7JCQXEZGVUNU (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 KUBE-SEP-WV6S37CDULKCYEVE  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-system/kube-dns:dns */
立即下载在被转发目标node的物理网口抓包（需decode as vxlan)
Array
本作品采用知识共享署名-非商业性使用 4.0 国际许可协议进行许可