前面几篇博文都是基于单个计算节点情况下，单租户，多租户情形分析。本篇博文描述在两个计算节点情况下的以下情况：
-一个租户的两个同网段实例位于不同的计算节点
-一个租户的两个不同网段实例位于不同的计算节点
-两个租户，各自有一个实例，且位于不同的计算节点
实验拓扑变化为：
相比于之前的实验，本次增加了一个计算节点compute2. 由于使用的是GRE租户网络类型，当多个计算节点出现后，计算节点和网络几点之间成full mesh的tunnel，即 compute会分别和compute2、network建立tunnel，同样，compute2和compute、network建立tunnel，network和compute、compute2建立tunnel。
网络节点：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
Bridge br-tun Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Port "gre-0a0a648d" Interface "gre-0a0a648d" type: gre options: {in_key=flow, local_ip="10.10.100.139", out_key=flow, remote_ip="10.10.100.141"} Port "gre-0a0a648c" Interface "gre-0a0a648c" type: gre options: {in_key=flow, local_ip="10.10.100.139", out_key=flow, remote_ip="10.10.100.140"} Port br-tun Interface br-tun type: internal |
compute几点：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
Bridge br-tun Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Port "gre-0a0a648d" Interface "gre-0a0a648d" type: gre options: {in_key=flow, local_ip="10.10.100.140", out_key=flow, remote_ip="10.10.100.141"} Port "gre-0a0a648b" Interface "gre-0a0a648b" type: gre options: {in_key=flow, local_ip="10.10.100.140", out_key=flow, remote_ip="10.10.100.139"} Port br-tun Interface br-tun type: internal |
compute2几点：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
Bridge br-tun Port br-tun Interface br-tun type: internal Port patch-int Interface patch-int type: patch options: {peer=patch-tun} Port "gre-0a0a648b" Interface "gre-0a0a648b" type: gre options: {in_key=flow, local_ip="10.10.100.141", out_key=flow, remote_ip="10.10.100.139"} Port "gre-0a0a648c" Interface "gre-0a0a648c" type: gre options: {in_key=flow, local_ip="10.10.100.141", out_key=flow, remote_ip="10.10.100.140"} |
-一个租户的两个同网段实例位于不同的计算节点
首先在demo租户下创建两个两个实例，位于同一网络，nova-scheduler自动将该两个实例部署到了不同的hypervisor上
|
1 2 3 4 5 6 7 8 9 |
root@compute:/home/mycisco# nova hypervisor-servers compute +--------------------------------------+-------------------+---------------+---------------------+ | ID | Name | Hypervisor ID | Hypervisor Hostname | +--------------------------------------+-------------------+---------------+---------------------+ | 5dd22a7a-77de-4d87-98a0-a92e14b3de90 | instance-0000000b | 1 | compute | | 7243eadf-7ec2-4ad2-90a6-9a7d385898fe | instance-0000000d | 1 | compute | | e973e4bc-a59a-4af3-a2af-3465020663cf | instance-0000000e | 1 | compute | | 2727d71a-6ffc-453b-8df4-799d29f23ef3 | instance-0000000f | 2 | compute2 | +--------------------------------------+-------------------+---------------+---------------------+ |
根据不同的实例ID可以看出实例名e，f为上图中两个实例，且位于不同的计算节点上。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
root@compute:/home/mycisco# neutron port-list +--------------------------------------+------+-------------------+------------------------------------------------------------------------------------+ | id | name | mac_address | fixed_ips | +--------------------------------------+------+-------------------+------------------------------------------------------------------------------------+ | 08a68c60-a63d-40a9-951c-36385fcc18f6 | | fa:16:3e:5d:b3:6d | {"subnet_id": "f43edf85-594e-4eb1-bd02-adc90c47ada1", "ip_address": "10.90.10.4"} | | 09208a89-0b45-4b80-bf11-2aee68b30eb1 | | fa:16:3e:79:45:64 | {"subnet_id": "f380d778-d576-4179-96df-cb3a2618dbd3", "ip_address": "10.10.20.1"} | | 1b1b29de-97a8-4f9b-8ffd-556be72dcd20 | | fa:16:3e:c2:be:77 | {"subnet_id": "f43edf85-594e-4eb1-bd02-adc90c47ada1", "ip_address": "10.90.10.3"} | | 219fc331-d36d-4685-8bed-32fe363a87e3 | | fa:16:3e:32:d1:ce | {"subnet_id": "f43edf85-594e-4eb1-bd02-adc90c47ada1", "ip_address": "10.90.10.1"} | | 2e48b70e-e190-40cb-9dab-ff6ca2d831dd | | fa:16:3e:99:88:5a | {"subnet_id": "f43edf85-594e-4eb1-bd02-adc90c47ada1", "ip_address": "10.90.10.2"} | | 4debe62b-e44c-47c7-a5e5-b6a11da8d31c | | fa:16:3e:75:b5:d4 | {"subnet_id": "410a9d7c-6ae8-4059-954e-973529a933c5", "ip_address": "10.10.30.1"} | | 6f726a6c-412d-4d22-bcc7-eadedf407680 | | fa:16:3e:fc:53:dc | {"subnet_id": "7b2cd216-1251-4eaa-9f05-b8cadc82bc1e", "ip_address": "192.168.0.2"} | | 708ee330-0c8a-41bd-a2f5-271ca9662151 | | fa:16:3e:75:82:cd | {"subnet_id": "410a9d7c-6ae8-4059-954e-973529a933c5", "ip_address": "10.10.30.2"} | | 78dd7f1e-53e6-4a2c-b166-5eef9add12b7 | | fa:16:3e:3c:54:50 | {"subnet_id": "dcdd699a-c4b0-4360-9219-c62cb6ed0b3e", "ip_address": "10.10.10.2"} | | 8083e908-e3d5-4328-ad9f-102563a21751 | | fa:16:3e:6d:35:24 | {"subnet_id": "7b2cd216-1251-4eaa-9f05-b8cadc82bc1e", "ip_address": "192.168.0.3"} | | 814e9e39-998b-4829-9af3-2b075f35db7a | | fa:16:3e:23:10:a1 | {"subnet_id": "7b2cd216-1251-4eaa-9f05-b8cadc82bc1e", "ip_address": "192.168.0.4"} | | 8cbe7bb6-18d9-4455-abce-473ebe5a9591 | | fa:16:3e:32:28:af | {"subnet_id": "dcdd699a-c4b0-4360-9219-c62cb6ed0b3e", "ip_address": "10.10.10.7"} | | a6f975bb-605b-4f36-ad3f-669ccdab5fe5 | | fa:16:3e:cb:91:43 | {"subnet_id": "f380d778-d576-4179-96df-cb3a2618dbd3", "ip_address": "10.10.20.7"} | | d8fd27a6-d1cf-46f4-8e0e-4965001d547a | | fa:16:3e:2a:46:e0 | {"subnet_id": "dcdd699a-c4b0-4360-9219-c62cb6ed0b3e", "ip_address": "10.10.10.1"} | | ed034aae-1106-41b5-8b3f-2b8cb5776baf | | fa:16:3e:fb:f3:33 | {"subnet_id": "f380d778-d576-4179-96df-cb3a2618dbd3", "ip_address": "10.10.20.2"} | +--------------------------------------+------+-------------------+------------------------------------------------------------------------------------+ |
两个实例的IP 分别是10.90.10.2和10.90.10.4， 其port id分别是：
10.90.10.4： 08a68c60-a63d-40a9-951c-36385fcc18f6.
10.90.10.2： 2e48b70e-e190-40cb-9dab-ff6ca2d831dd
输出compute几点的br-int 看看：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
Bridge br-int fail_mode: secure Port br-int Interface br-int type: internal Port "qvoa6f975bb-60" tag: 2 Interface "qvoa6f975bb-60" Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} Port "qvo2e48b70e-e1" tag: 3 Interface "qvo2e48b70e-e1" Port "qvo8cbe7bb6-18" tag: 1 Interface "qvo8cbe7bb6-18" |
可以看出port qvo2e48b70e-e1对应于10.90.10.2实例的端口。而08a68c60不在节点上。 来输出compute2节点的br-int看看：
|
1 2 3 4 5 6 7 8 9 10 11 12 |
Bridge br-int fail_mode: secure Port "qvo08a68c60-a6" tag: 1 Interface "qvo08a68c60-a6" Port br-int Interface br-int type: internal Port patch-tun Interface patch-tun type: patch options: {peer=patch-int} |
compute2的br-int目前只有一个qvo接口，恰好是08a68c60对应的端口。
注意到两个节点上对应port的tag并不一致，compute上是tag 3， 而compute2是tag 1。
那么当这两台机器通信时候，实际的数据path是如何的。
首先，看一下compute节点上的open flow：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
root@compute:/home/mycisco# ovs-ofctl dump-flows br-tun NXST_FLOW reply (xid=0x4): cookie=0x0, duration=11591.902s, table=0, n_packets=397, n_bytes=37854, idle_age=3419, priority=1,in_port=3 actions=resubmit(,2) cookie=0x0, duration=25695.025s, table=0, n_packets=676, n_bytes=64180, idle_age=3374, priority=1,in_port=1 actions=resubmit(,1) cookie=0x0, duration=25694.154s, table=0, n_packets=168, n_bytes=26308, idle_age=3374, priority=1,in_port=2 actions=resubmit(,2) cookie=0x0, duration=25694.977s, table=0, n_packets=6, n_bytes=480, idle_age=25686, priority=0 actions=drop cookie=0x0, duration=25694.92s, table=1, n_packets=615, n_bytes=58470, idle_age=3374, priority=1,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20) cookie=0x0, duration=25694.871s, table=1, n_packets=61, n_bytes=5710, idle_age=3416, priority=1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,21) cookie=0x0, duration=25689.681s, table=2, n_packets=565, n_bytes=64162, idle_age=3374, priority=1,tun_id=0x5 actions=mod_vlan_vid:3,resubmit(,10) cookie=0x0, duration=25692.152s, table=2, n_packets=0, n_bytes=0, idle_age=25692, priority=1,tun_id=0x1 actions=mod_vlan_vid:1,resubmit(,10) cookie=0x0, duration=25690.144s, table=2, n_packets=0, n_bytes=0, idle_age=25690, priority=1,tun_id=0x2 actions=mod_vlan_vid:2,resubmit(,10) cookie=0x0, duration=25694.825s, table=2, n_packets=0, n_bytes=0, idle_age=25694, priority=0 actions=drop cookie=0x0, duration=25694.771s, table=3, n_packets=0, n_bytes=0, idle_age=25694, priority=0 actions=drop cookie=0x0, duration=25694.72s, table=10, n_packets=565, n_bytes=64162, idle_age=3374, priority=1 actions=learn(table=20,hard_timeout=300,priority=1,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1 cookie=0x0, duration=25694.668s, table=20, n_packets=0, n_bytes=0, idle_age=25694, priority=0 actions=resubmit(,21) cookie=0x0, duration=25689.763s, table=21, n_packets=23, n_bytes=2658, idle_age=3416, hard_age=11591, dl_vlan=3 actions=strip_vlan,set_tunnel:0x5,output:3,output:2 cookie=0x0, duration=25690.185s, table=21, n_packets=5, n_bytes=370, idle_age=25683, hard_age=11591, dl_vlan=2 actions=strip_vlan,set_tunnel:0x2,output:3,output:2 cookie=0x0, duration=25692.213s, table=21, n_packets=9, n_bytes=710, idle_age=25683, hard_age=11591, dl_vlan=1 actions=strip_vlan,set_tunnel:0x1,output:3,output:2 cookie=0x0, duration=25694.626s, table=21, n_packets=24, n_bytes=1972, idle_age=25688, priority=0 actions=drop |
在compute节点上，一共存在三个qvo** port， 每个port在本实验中其实就代表一个网络，从上述flow中可以看出，每个tag 都对应了一个tunnel id，也就是说每个网络都有一个唯一的tunnel ID与之对应，且注意到 dl_vlan 3对应着tunnle id 0x5. 从之前内容可以知道，vlan tag 3对应的恰好是demo租户的10.90.10.2机器。
再看compute2几点上open flow情况：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
root@compute2:/home/mycisco# ovs-ofctl dump-flows br-tun NXST_FLOW reply (xid=0x4): cookie=0x0, duration=11787.702s, table=0, n_packets=392, n_bytes=37532, idle_age=3612, priority=1,in_port=3 actions=resubmit(,2) cookie=0x0, duration=11788.931s, table=0, n_packets=656, n_bytes=62558, idle_age=3574, priority=1,in_port=1 actions=resubmit(,1) cookie=0x0, duration=11787.95s, table=0, n_packets=176, n_bytes=26902, idle_age=3574, priority=1,in_port=2 actions=resubmit(,2) cookie=0x0, duration=11788.887s, table=0, n_packets=6, n_bytes=480, idle_age=11780, priority=0 actions=drop cookie=0x0, duration=11788.843s, table=1, n_packets=623, n_bytes=59064, idle_age=3574, priority=1,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20) cookie=0x0, duration=11788.797s, table=1, n_packets=33, n_bytes=3494, idle_age=3616, priority=1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,21) cookie=0x0, duration=10845.785s, table=2, n_packets=559, n_bytes=63266, idle_age=3574, priority=1,tun_id=0x5 actions=mod_vlan_vid:1,resubmit(,10) cookie=0x0, duration=11788.753s, table=2, n_packets=9, n_bytes=1168, idle_age=10951, priority=0 actions=drop cookie=0x0, duration=11788.708s, table=3, n_packets=0, n_bytes=0, idle_age=11788, priority=0 actions=drop cookie=0x0, duration=11788.661s, table=10, n_packets=559, n_bytes=63266, idle_age=3574, priority=1 actions=learn(table=20,hard_timeout=300,priority=1,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1 cookie=0x0, duration=11788.617s, table=20, n_packets=0, n_bytes=0, idle_age=11788, priority=0 actions=resubmit(,21) cookie=0x0, duration=10845.855s, table=21, n_packets=23, n_bytes=2658, idle_age=3616, dl_vlan=1 actions=strip_vlan,set_tunnel:0x5,output:3,output:2 cookie=0x0, duration=11788.573s, table=21, n_packets=10, n_bytes=836, idle_age=10845, priority=0 actions=drop |
可以看出compute2上只有一个tunnel ID 0x5，因为该节点上目前只有一个实例的网络。vlan tag 1对应着tunnel id 0x5。
同一个网络在不同计算节点上有着相同的tunnel id，却对应的不同的vlan tag，这说明，在这种网络类型下，vlan tag实际上并不唯一代表租户的某个网络，所起的作用完全是隔离的作用。这句话反过来的意思也就是说，同一个网络在不同的计算节点上可能有着不同的vlan tag，唯一能代表该网络的是tunnel id，同一个网络不管在哪台计算（或网络）节点上，其tunnel id都是唯一且相同的。
那么当这两台虚机相互通信时，由于连个计算节点之间也建立了tunnel通道，因此数据包会直接在这两个虚机之间通过tunnel id 0x5进行通信，而无需通过网络节点中转：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
root@compute:/home/mycisco# ovs-ofctl dump-flows br-tun NXST_FLOW reply (xid=0x4): cookie=0x0, duration=12409.233s, table=0, n_packets=461, n_bytes=43902, idle_age=0, priority=1,in_port=3 actions=resubmit(,2) cookie=0x0, duration=26512.356s, table=0, n_packets=740, n_bytes=70228, idle_age=0, priority=1,in_port=1 actions=resubmit(,1) cookie=0x0, duration=26511.485s, table=0, n_packets=168, n_bytes=26308, idle_age=4191, priority=1,in_port=2 actions=resubmit(,2) cookie=0x0, duration=26512.308s, table=0, n_packets=6, n_bytes=480, idle_age=26503, priority=0 actions=drop cookie=0x0, duration=26512.251s, table=1, n_packets=678, n_bytes=64476, idle_age=0, priority=1,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20) cookie=0x0, duration=26512.202s, table=1, n_packets=62, n_bytes=5752, idle_age=59, priority=1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,21) cookie=0x0, duration=26507.012s, table=2, n_packets=629, n_bytes=70210, idle_age=0, priority=1,tun_id=0x5 actions=mod_vlan_vid:3,resubmit(,10) cookie=0x0, duration=26509.483s, table=2, n_packets=0, n_bytes=0, idle_age=26509, priority=1,tun_id=0x1 actions=mod_vlan_vid:1,resubmit(,10) cookie=0x0, duration=26507.475s, table=2, n_packets=0, n_bytes=0, idle_age=26507, priority=1,tun_id=0x2 actions=mod_vlan_vid:2,resubmit(,10) cookie=0x0, duration=26512.156s, table=2, n_packets=0, n_bytes=0, idle_age=26512, priority=0 actions=drop cookie=0x0, duration=26512.102s, table=3, n_packets=0, n_bytes=0, idle_age=26512, priority=0 actions=drop cookie=0x0, duration=26512.051s, table=10, n_packets=629, n_bytes=70210, idle_age=0, priority=1 actions=learn(table=20,hard_timeout=300,priority=1,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1 cookie=0x0, duration=59.632s, table=20, n_packets=63, n_bytes=6006, hard_timeout=300, idle_age=0, hard_age=0, priority=1,vlan_tci=0x0003/0x0fff,dl_dst=fa:16:3e:5d:b3:6d actions=load:0->NXM_OF_VLAN_TCI[],load:0x5->NXM_NX_TUN_ID[],output:3 cookie=0x0, duration=26511.999s, table=20, n_packets=0, n_bytes=0, idle_age=26511, priority=0 actions=resubmit(,21) cookie=0x0, duration=26507.094s, table=21, n_packets=24, n_bytes=2700, idle_age=59, hard_age=12409, dl_vlan=3 actions=strip_vlan,set_tunnel:0x5,output:3,output:2 cookie=0x0, duration=26507.516s, table=21, n_packets=5, n_bytes=370, idle_age=26500, hard_age=12408, dl_vlan=2 actions=strip_vlan,set_tunnel:0x2,output:3,output:2 cookie=0x0, duration=26509.544s, table=21, n_packets=9, n_bytes=710, idle_age=26500, hard_age=12409, dl_vlan=1 actions=strip_vlan,set_tunnel:0x1,output:3,output:2 cookie=0x0, duration=26511.957s, table=21, n_packets=24, n_bytes=1972, idle_age=26506, priority=0 actions=drop |
注意上述输出中的dl_dst的mac值
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
root@compute2:/home/mycisco# ovs-ofctl dump-flows br-tun NXST_FLOW reply (xid=0x4): cookie=0x0, duration=12579.256s, table=0, n_packets=636, n_bytes=60604, idle_age=1, priority=1,in_port=3 actions=resubmit(,2) cookie=0x0, duration=12580.485s, table=0, n_packets=900, n_bytes=85630, idle_age=1, priority=1,in_port=1 actions=resubmit(,1) cookie=0x0, duration=12579.504s, table=0, n_packets=176, n_bytes=26902, idle_age=4366, priority=1,in_port=2 actions=resubmit(,2) cookie=0x0, duration=12580.441s, table=0, n_packets=6, n_bytes=480, idle_age=12571, priority=0 actions=drop cookie=0x0, duration=12580.397s, table=1, n_packets=867, n_bytes=82136, idle_age=1, priority=1,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20) cookie=0x0, duration=12580.351s, table=1, n_packets=33, n_bytes=3494, idle_age=4407, priority=1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,21) cookie=0x0, duration=11637.339s, table=2, n_packets=803, n_bytes=86338, idle_age=1, priority=1,tun_id=0x5 actions=mod_vlan_vid:1,resubmit(,10) cookie=0x0, duration=12580.307s, table=2, n_packets=9, n_bytes=1168, idle_age=11742, priority=0 actions=drop cookie=0x0, duration=12580.262s, table=3, n_packets=0, n_bytes=0, idle_age=12580, priority=0 actions=drop cookie=0x0, duration=12580.215s, table=10, n_packets=803, n_bytes=86338, idle_age=1, priority=1 actions=learn(table=20,hard_timeout=300,priority=1,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1 cookie=0x0, duration=230.104s, table=20, n_packets=244, n_bytes=23072, hard_timeout=300, idle_age=1, hard_age=0, priority=1,vlan_tci=0x0001/0x0fff,dl_dst=fa:16:3e:99:88:5a actions=load:0->NXM_OF_VLAN_TCI[],load:0x5->NXM_NX_TUN_ID[],output:3 cookie=0x0, duration=12580.171s, table=20, n_packets=0, n_bytes=0, idle_age=12580, priority=0 actions=resubmit(,21) cookie=0x0, duration=11637.409s, table=21, n_packets=23, n_bytes=2658, idle_age=4407, dl_vlan=1 actions=strip_vlan,set_tunnel:0x5,output:3,output:2 cookie=0x0, duration=12580.127s, table=21, n_packets=10, n_bytes=836, idle_age=11637, priority=0 actions=drop |
注意上述输出中dl_dst的mac值， 对应查找下表，可以看出两个虚机之间直接形成了一个tunnel 通道
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
root@compute:/home/mycisco# neutron port-list +--------------------------------------+------+-------------------+------------------------------------------------------------------------------------+ | id | name | mac_address | fixed_ips | +--------------------------------------+------+-------------------+------------------------------------------------------------------------------------+ | 08a68c60-a63d-40a9-951c-36385fcc18f6 | | fa:16:3e:5d:b3:6d | {"subnet_id": "f43edf85-594e-4eb1-bd02-adc90c47ada1", "ip_address": "10.90.10.4"} | | 09208a89-0b45-4b80-bf11-2aee68b30eb1 | | fa:16:3e:79:45:64 | {"subnet_id": "f380d778-d576-4179-96df-cb3a2618dbd3", "ip_address": "10.10.20.1"} | | 1b1b29de-97a8-4f9b-8ffd-556be72dcd20 | | fa:16:3e:c2:be:77 | {"subnet_id": "f43edf85-594e-4eb1-bd02-adc90c47ada1", "ip_address": "10.90.10.3"} | | 219fc331-d36d-4685-8bed-32fe363a87e3 | | fa:16:3e:32:d1:ce | {"subnet_id": "f43edf85-594e-4eb1-bd02-adc90c47ada1", "ip_address": "10.90.10.1"} | | 2e48b70e-e190-40cb-9dab-ff6ca2d831dd | | fa:16:3e:99:88:5a | {"subnet_id": "f43edf85-594e-4eb1-bd02-adc90c47ada1", "ip_address": "10.90.10.2"} | | 4debe62b-e44c-47c7-a5e5-b6a11da8d31c | | fa:16:3e:75:b5:d4 | {"subnet_id": "410a9d7c-6ae8-4059-954e-973529a933c5", "ip_address": "10.10.30.1"} | | 6f726a6c-412d-4d22-bcc7-eadedf407680 | | fa:16:3e:fc:53:dc | {"subnet_id": "7b2cd216-1251-4eaa-9f05-b8cadc82bc1e", "ip_address": "192.168.0.2"} | | 708ee330-0c8a-41bd-a2f5-271ca9662151 | | fa:16:3e:75:82:cd | {"subnet_id": "410a9d7c-6ae8-4059-954e-973529a933c5", "ip_address": "10.10.30.2"} | | 78dd7f1e-53e6-4a2c-b166-5eef9add12b7 | | fa:16:3e:3c:54:50 | {"subnet_id": "dcdd699a-c4b0-4360-9219-c62cb6ed0b3e", "ip_address": "10.10.10.2"} | | 8083e908-e3d5-4328-ad9f-102563a21751 | | fa:16:3e:6d:35:24 | {"subnet_id": "7b2cd216-1251-4eaa-9f05-b8cadc82bc1e", "ip_address": "192.168.0.3"} | | 814e9e39-998b-4829-9af3-2b075f35db7a | | fa:16:3e:23:10:a1 | {"subnet_id": "7b2cd216-1251-4eaa-9f05-b8cadc82bc1e", "ip_address": "192.168.0.4"} | | 8cbe7bb6-18d9-4455-abce-473ebe5a9591 | | fa:16:3e:32:28:af | {"subnet_id": "dcdd699a-c4b0-4360-9219-c62cb6ed0b3e", "ip_address": "10.10.10.7"} | | a6f975bb-605b-4f36-ad3f-669ccdab5fe5 | | fa:16:3e:cb:91:43 | {"subnet_id": "f380d778-d576-4179-96df-cb3a2618dbd3", "ip_address": "10.10.20.7"} | | d8fd27a6-d1cf-46f4-8e0e-4965001d547a | | fa:16:3e:2a:46:e0 | {"subnet_id": "dcdd699a-c4b0-4360-9219-c62cb6ed0b3e", "ip_address": "10.10.10.1"} | | ed034aae-1106-41b5-8b3f-2b8cb5776baf | | fa:16:3e:fb:f3:33 | {"subnet_id": "f380d778-d576-4179-96df-cb3a2618dbd3", "ip_address": "10.10.20.2"} | +--------------------------------------+------+-------------------+------------------------------------------------------------------------------------+ |
此时网络节点上tunnel情况，可以看出网络节点不存在一个实时的通信tunnel（idle-age时间没有被持续刷新）：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
root@network:/home/mycisco# ovs-ofctl dump-flows br-tun NXST_FLOW reply (xid=0x4): cookie=0x0, duration=12946.88s, table=0, n_packets=272, n_bytes=26526, idle_age=4733, priority=1,in_port=3 actions=resubmit(,2) cookie=0x0, duration=39762.69s, table=0, n_packets=1695, n_bytes=192550, idle_age=4729, priority=1,in_port=1 actions=resubmit(,1) cookie=0x0, duration=27049.273s, table=0, n_packets=278, n_bytes=26916, idle_age=597, priority=1,in_port=2 actions=resubmit(,2) cookie=0x0, duration=39762.577s, table=0, n_packets=5, n_bytes=390, idle_age=39754, priority=0 actions=drop cookie=0x0, duration=39762.525s, table=1, n_packets=1623, n_bytes=186678, idle_age=4729, priority=1,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20) cookie=0x0, duration=39762.479s, table=1, n_packets=68, n_bytes=5524, idle_age=36531, priority=1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,21) cookie=0x0, duration=37307.346s, table=2, n_packets=1496, n_bytes=139410, idle_age=597, priority=1,tun_id=0x5 actions=mod_vlan_vid:4,resubmit(,10) cookie=0x0, duration=39760.354s, table=2, n_packets=436, n_bytes=41130, idle_age=27038, priority=1,tun_id=0x1 actions=mod_vlan_vid:2,resubmit(,10) cookie=0x0, duration=39759.932s, table=2, n_packets=0, n_bytes=0, idle_age=39759, priority=1,tun_id=0x4 actions=mod_vlan_vid:3,resubmit(,10) cookie=0x0, duration=39760.847s, table=2, n_packets=12, n_bytes=900, idle_age=27038, priority=1,tun_id=0x2 actions=mod_vlan_vid:1,resubmit(,10) cookie=0x0, duration=39762.391s, table=2, n_packets=12, n_bytes=1016, idle_age=27049, priority=0 actions=drop cookie=0x0, duration=39762.35s, table=3, n_packets=0, n_bytes=0, idle_age=39762, priority=0 actions=drop cookie=0x0, duration=39762.288s, table=10, n_packets=1944, n_bytes=181440, idle_age=597, priority=1 actions=learn(table=20,hard_timeout=300,priority=1,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1 cookie=0x0, duration=39762.23s, table=20, n_packets=2, n_bytes=196, idle_age=30992, priority=0 actions=resubmit(,21) cookie=0x0, duration=37307.446s, table=21, n_packets=8, n_bytes=648, idle_age=30992, hard_age=12946, dl_vlan=4 actions=strip_vlan,set_tunnel:0x5,output:3,output:2 cookie=0x0, duration=39759.994s, table=21, n_packets=4, n_bytes=280, idle_age=39749, hard_age=12946, dl_vlan=3 actions=strip_vlan,set_tunnel:0x4,output:3,output:2 cookie=0x0, duration=39760.43s, table=21, n_packets=5, n_bytes=378, idle_age=33697, hard_age=12946, dl_vlan=2 actions=strip_vlan,set_tunnel:0x1,output:3,output:2 cookie=0x0, duration=39760.895s, table=21, n_packets=5, n_bytes=370, idle_age=39749, hard_age=12946, dl_vlan=1 actions=strip_vlan,set_tunnel:0x2,output:3,output:2 cookie=0x0, duration=39762.176s, table=21, n_packets=47, n_bytes=3966, idle_age=36539, priority=0 actions=drop |
此时的数据path如下（蓝色instance2 和4 表示同一租户的同一网络上的两个虚机）
-一个租户的两个不同网段实例位于不同的计算节点
类似原理，同一个租户，不同网段位于两个计算节点，由于两个网段需要路由，所以数据必然通过网络节点进行路由，数据path将如下：、
-两个租户，各自有一个实例，且位于不同的计算节点
关于扩展的思考：
1.假设只有一个网络节点，那么一个租户内最大可容许的网络有多少个？
-由于一个租户的网络都要虚拟连接到一个虚拟路由器接口上，而虚拟路由器的每一个接口都对应于br-int桥上的一个端口，因此一个租户最大的网络数量取决于网络节点中br-int桥的接口数量。
2.一个openstack可以容许多少租户?
-假设一个租户只有一个网络，那么每个计算节点上可以创建的最大租户数是br-int桥的接口数量，但由于租户的网络可以落在不同的计算节点上，因此只要计算节点足够多，那么租户的数量也就可以足够多（当然租户网络的数量和gre隧道配置中的tunnel id区间有着关系，tunnel id不会超过配置文件中所定义的）
文章评论