在上一篇文章中分析了单租户在不同网络情形下的计算节点的网络结构，但未有描述当数据包到达网络节点时候，数据包是如何走向公网的，网络的结构如下：
通过前一篇文章我们知道，对于该租户的网络来说，相当于计算节点中的虚机实例通过GRE的隧道与网络节点的虚拟路由器打通了二层，逻辑结构就像这样：
那么这个二层究竟是如何相互联系起来的。
看下网络节点的ovs：
当数据包通过GRE tunnel到达br-tun后，br-tun中的 patch-int port与 br-int中的 patch-tun port互联，数据包被送到br-int桥里，由于tunnel的ID和br-int的vlan是关联起来的，因此数据包会被正确的tagged并送到正确的vlan segment中去，而每个对应的vlan都会有一个qr**接口与路由器连接起来，因此数据包可以到达路由器接口，该接口其实也就是该网络的gateway。随后虚拟路由器执行数据转发将数据路由到qg-***接口（路由器设置了其缺省路由指向外部实际网络的网关），该qg***接口在br-ex桥中与实际的物理网卡eth2进行桥接，从而数据包送出。
可以通过查看路由器的接口来对应上述各个接口：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
root@network:/home/mycisco# ip netns exec qrouter-040c1455-6096-4806-ba91-fec64cdaed81 ifconfig lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:25 errors:0 dropped:0 overruns:0 frame:0 TX packets:25 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:2464 (2.4 KB) TX bytes:2464 (2.4 KB) qg-6f726a6c-41 Link encap:Ethernet HWaddr fa:16:3e:fc:53:dc inet addr:192.168.0.2 Bcast:192.168.0.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fefc:53dc/64 Scope:Link UP BROADCAST RUNNING MTU:1500 Metric:1 RX packets:165307 errors:0 dropped:0 overruns:0 frame:0 TX packets:29 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:38076750 (38.0 MB) TX bytes:1662 (1.6 KB) qr-09208a89-0b Link encap:Ethernet HWaddr fa:16:3e:79:45:64 inet addr:10.10.20.1 Bcast:10.10.20.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fe79:4564/64 Scope:Link UP BROADCAST RUNNING MTU:1500 Metric:1 RX packets:358 errors:0 dropped:0 overruns:0 frame:0 TX packets:235 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:33858 (33.8 KB) TX bytes:31438 (31.4 KB) qr-4debe62b-e4 Link encap:Ethernet HWaddr fa:16:3e:75:b5:d4 inet addr:10.10.30.1 Bcast:10.10.30.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fe75:b5d4/64 Scope:Link UP BROADCAST RUNNING MTU:1500 Metric:1 RX packets:25 errors:0 dropped:0 overruns:0 frame:0 TX packets:11 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1878 (1.8 KB) TX bytes:906 (906.0 B) qr-d8fd27a6-d1 Link encap:Ethernet HWaddr fa:16:3e:2a:46:e0 inet addr:10.10.10.1 Bcast:10.10.10.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fe2a:46e0/64 Scope:Link UP BROADCAST RUNNING MTU:1500 Metric:1 RX packets:8422 errors:0 dropped:0 overruns:0 frame:0 TX packets:8243 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:799290 (799.2 KB) TX bytes:795846 (795.8 KB) |
那么这个虚拟路由器是如何转发数据包的：
|
1 2 |
root@network:/home/mycisco# ip netns exec qrouter-040c1455-6096-4806-ba91-fec64cdaed81 sysctl net.ipv4.ip_forward net.ipv4.ip_forward = 1 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
root@network:/home/mycisco# ip netns exec qrouter-040c1455-6096-4806-ba91-fec64cdaed81 iptables -L -n Chain INPUT (policy ACCEPT) target prot opt source destination neutron-l3-agent-INPUT all -- 0.0.0.0/0 0.0.0.0/0 Chain FORWARD (policy ACCEPT) target prot opt source destination neutron-filter-top all -- 0.0.0.0/0 0.0.0.0/0 neutron-l3-agent-FORWARD all -- 0.0.0.0/0 0.0.0.0/0 Chain OUTPUT (policy ACCEPT) target prot opt source destination neutron-filter-top all -- 0.0.0.0/0 0.0.0.0/0 neutron-l3-agent-OUTPUT all -- 0.0.0.0/0 0.0.0.0/0 Chain neutron-filter-top (2 references) target prot opt source destination neutron-l3-agent-local all -- 0.0.0.0/0 0.0.0.0/0 Chain neutron-l3-agent-FORWARD (1 references) target prot opt source destination Chain neutron-l3-agent-INPUT (1 references) target prot opt source destination ACCEPT tcp -- 0.0.0.0/0 127.0.0.1 tcp dpt:9697 Chain neutron-l3-agent-OUTPUT (1 references) target prot opt source destination Chain neutron-l3-agent-local (1 references) target prot opt source destination |
当数据包通过路由器去往外部网络时，虚拟路由器的的iptables会执行相关NAT操作：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
root@network:/home/mycisco# ip netns exec qrouter-040c1455-6096-4806-ba91-fec64cdaed81 iptables-save # Generated by iptables-save v1.4.21 on Wed Dec 24 00:21:13 2014 *filter :INPUT ACCEPT [67338:5340104] :FORWARD ACCEPT [133:10372] :OUTPUT ACCEPT [7966:691840] :neutron-filter-top - [0:0] :neutron-l3-agent-FORWARD - [0:0] :neutron-l3-agent-INPUT - [0:0] :neutron-l3-agent-OUTPUT - [0:0] :neutron-l3-agent-local - [0:0] -A INPUT -j neutron-l3-agent-INPUT -A FORWARD -j neutron-filter-top -A FORWARD -j neutron-l3-agent-FORWARD -A OUTPUT -j neutron-filter-top -A OUTPUT -j neutron-l3-agent-OUTPUT -A neutron-filter-top -j neutron-l3-agent-local -A neutron-l3-agent-INPUT -d 127.0.0.1/32 -p tcp -m tcp --dport 9697 -j ACCEPT COMMIT # Completed on Wed Dec 24 00:21:13 2014 # Generated by iptables-save v1.4.21 on Wed Dec 24 00:21:13 2014 *nat :PREROUTING ACCEPT [107285:31758348] :INPUT ACCEPT [625:72661] :OUTPUT ACCEPT [11:896] :POSTROUTING ACCEPT [11:764] :neutron-l3-agent-OUTPUT - [0:0] :neutron-l3-agent-POSTROUTING - [0:0] :neutron-l3-agent-PREROUTING - [0:0] :neutron-l3-agent-float-snat - [0:0] :neutron-l3-agent-snat - [0:0] :neutron-postrouting-bottom - [0:0] -A PREROUTING -j neutron-l3-agent-PREROUTING -A OUTPUT -j neutron-l3-agent-OUTPUT -A POSTROUTING -j neutron-l3-agent-POSTROUTING -A POSTROUTING -j neutron-postrouting-bottom -A neutron-l3-agent-OUTPUT -d 192.168.0.3/32 -j DNAT --to-destination 10.10.20.7 -A neutron-l3-agent-POSTROUTING ! -i qg-6f726a6c-41 ! -o qg-6f726a6c-41 -m conntrack ! --ctstate DNAT -j ACCEPT -A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697 -A neutron-l3-agent-PREROUTING -d 192.168.0.3/32 -j DNAT --to-destination 10.10.20.7 -A neutron-l3-agent-float-snat -s 10.10.20.7/32 -j SNAT --to-source 192.168.0.3 -A neutron-l3-agent-snat -j neutron-l3-agent-float-snat -A neutron-l3-agent-snat -s 10.10.20.0/24 -j SNAT --to-source 192.168.0.2 -A neutron-l3-agent-snat -s 10.10.10.0/24 -j SNAT --to-source 192.168.0.2 -A neutron-l3-agent-snat -s 10.10.30.0/24 -j SNAT --to-source 192.168.0.2 -A neutron-postrouting-bottom -j neutron-l3-agent-snat COMMIT |
从上表可以看出，系统还将 192.168.0.3和10.10.20.7做了一对一的static映射，因此当10.10.20.7机器的数据经过路由后将执行snat为192.168.0.3地址，而如果从外部直接访问192.168.0.3则直接可以访问到10.10.20.7，从而实现了从外部网络访问内部虚机。
![TDI`KHEFFSPZ8U9Z~S]1Z_V](http://www.cnadn.net/upload/2014/12/TDIKHEFFSPZ8U9ZS1Z_V.jpg)
在Horizon界面的access&security中增加一条入站，目的端口为22的tcp规则
![FMR3_B8KDV7]6QDUJK4JHB7](http://www.cnadn.net/upload/2014/12/FMR3_B8KDV76QDUJK4JHB7.png)
该规则将通过openvswtich agent写入计算节点的iptables中：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 |
root@compute:/home/mycisco# iptables -L -n Chain INPUT (policy ACCEPT) target prot opt source destination neutron-openvswi-INPUT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:53 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:53 ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:67 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:67 Chain FORWARD (policy ACCEPT) target prot opt source destination neutron-filter-top all -- 0.0.0.0/0 0.0.0.0/0 neutron-openvswi-FORWARD all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 192.168.122.0/24 ctstate RELATED,ESTABLISHED ACCEPT all -- 192.168.122.0/24 0.0.0.0/0 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable Chain OUTPUT (policy ACCEPT) target prot opt source destination neutron-filter-top all -- 0.0.0.0/0 0.0.0.0/0 neutron-openvswi-OUTPUT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:68 Chain neutron-filter-top (2 references) target prot opt source destination neutron-openvswi-local all -- 0.0.0.0/0 0.0.0.0/0 Chain neutron-openvswi-FORWARD (1 references) target prot opt source destination neutron-openvswi-sg-chain all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tapa6f975bb-60 --physdev-is-bridged neutron-openvswi-sg-chain all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tapa6f975bb-60 --physdev-is-bridged neutron-openvswi-sg-chain all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap8cbe7bb6-18 --physdev-is-bridged neutron-openvswi-sg-chain all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap8cbe7bb6-18 --physdev-is-bridged Chain neutron-openvswi-INPUT (1 references) target prot opt source destination neutron-openvswi-oa6f975bb-6 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tapa6f975bb-60 --physdev-is-bridged neutron-openvswi-o8cbe7bb6-1 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap8cbe7bb6-18 --physdev-is-bridged Chain neutron-openvswi-OUTPUT (1 references) target prot opt source destination Chain neutron-openvswi-i8cbe7bb6-1 (1 references) target prot opt source destination DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALID RETURN all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED RETURN tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:22 RETURN icmp -- 0.0.0.0/0 0.0.0.0/0 RETURN all -- 10.10.20.1 0.0.0.0/0 RETURN all -- 10.10.30.1 0.0.0.0/0 RETURN all -- 10.10.20.7 0.0.0.0/0 RETURN all -- 10.10.10.1 0.0.0.0/0 RETURN udp -- 10.10.10.2 0.0.0.0/0 udp spt:67 dpt:68 neutron-openvswi-sg-fallback all -- 0.0.0.0/0 0.0.0.0/0 Chain neutron-openvswi-ia6f975bb-6 (1 references) target prot opt source destination DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALID RETURN all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED RETURN tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:22 RETURN icmp -- 0.0.0.0/0 0.0.0.0/0 RETURN all -- 10.10.20.1 0.0.0.0/0 RETURN all -- 10.10.30.1 0.0.0.0/0 RETURN all -- 10.10.10.7 0.0.0.0/0 RETURN all -- 10.10.10.1 0.0.0.0/0 RETURN udp -- 10.10.20.2 0.0.0.0/0 udp spt:67 dpt:68 neutron-openvswi-sg-fallback all -- 0.0.0.0/0 0.0.0.0/0 Chain neutron-openvswi-local (1 references) target prot opt source destination Chain neutron-openvswi-o8cbe7bb6-1 (2 references) target prot opt source destination RETURN udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:68 dpt:67 neutron-openvswi-s8cbe7bb6-1 all -- 0.0.0.0/0 0.0.0.0/0 DROP udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:67 dpt:68 DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALID RETURN all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED RETURN icmp -- 0.0.0.0/0 0.0.0.0/0 RETURN all -- 0.0.0.0/0 0.0.0.0/0 neutron-openvswi-sg-fallback all -- 0.0.0.0/0 0.0.0.0/0 Chain neutron-openvswi-oa6f975bb-6 (2 references) target prot opt source destination RETURN udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:68 dpt:67 neutron-openvswi-sa6f975bb-6 all -- 0.0.0.0/0 0.0.0.0/0 DROP udp -- 0.0.0.0/0 0.0.0.0/0 udp spt:67 dpt:68 DROP all -- 0.0.0.0/0 0.0.0.0/0 state INVALID RETURN all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED RETURN icmp -- 0.0.0.0/0 0.0.0.0/0 RETURN all -- 0.0.0.0/0 0.0.0.0/0 neutron-openvswi-sg-fallback all -- 0.0.0.0/0 0.0.0.0/0 Chain neutron-openvswi-s8cbe7bb6-1 (1 references) target prot opt source destination RETURN all -- 10.10.10.7 0.0.0.0/0 MAC FA:16:3E:32:28:AF DROP all -- 0.0.0.0/0 0.0.0.0/0 Chain neutron-openvswi-sa6f975bb-6 (1 references) target prot opt source destination RETURN all -- 10.10.20.7 0.0.0.0/0 MAC FA:16:3E:CB:91:43 DROP all -- 0.0.0.0/0 0.0.0.0/0 Chain neutron-openvswi-sg-chain (4 references) target prot opt source destination neutron-openvswi-ia6f975bb-6 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tapa6f975bb-60 --physdev-is-bridged neutron-openvswi-oa6f975bb-6 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tapa6f975bb-60 --physdev-is-bridged neutron-openvswi-i8cbe7bb6-1 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-out tap8cbe7bb6-18 --physdev-is-bridged neutron-openvswi-o8cbe7bb6-1 all -- 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in tap8cbe7bb6-18 --physdev-is-bridged ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 Chain neutron-openvswi-sg-fallback (4 references) target prot opt source destination DROP all -- 0.0.0.0/0 0.0.0.0/0 |
文章评论