Cloud Native应用交付
  • 首页
  • 关于本站
  • 个人介绍
  • Downloads
  • Repo
    • Github
    • Container
  • F5
    • F5 Python SDK
    • F5-container
    • F5-LBaaS
  • 社交
    • 联系我
    • 微信/微博
    • 公众号
    • 打赏赞助
行至水穷处 坐看云起时
☁️We are in new App Mesh era: imesh.club ☁️
  1. 首页
  2. 容器/k8s
  3. 正文

DOCKER容器跨宿主机通信方法(4):Flannel-Vxlan

2017年07月02日 3305点热度 0人点赞 0条评论

Vxlan模式

flannel的vxlan模式,数据包是通过kernal来转发的,而当使用udp模式时候是flannel自己转发。UDP模式下性能要差很多,性能差别可参考一些测试:

UDP模式:

60秒单线程测试

iperf3 -c x.x.x.x -t 60
结果:
[ 4] 0.00-60.00 sec 3.33 GBytes 477 Mbits/sec 15716 sender
[ 4] 0.00-60.00 sec 3.33 GBytes 476 Mbits/sec receiver

开启30个线程传输

iperf3 -c x.x.x.x -t 60 -P 30
结果:
[SUM] 0.00-60.00 sec 3.08 GBytes 441 Mbits/sec 205692 sender
[SUM] 0.00-60.00 sec 3.06 GBytes 438 Mbits/sec receiver
多线程并没有带宽的提升反而由于多线程调度影响,降低带宽。在万兆网的环境下UDP的性能损耗还是挺大的。

Vxlan模式:

60秒单线程测试

iperf3 -c x.x.x.x -t 60
结果:
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-60.00 sec 10.1 GBytes 1.44 Gbits/sec 6957 sender
[ 4] 0.00-60.00 sec 10.1 GBytes 1.44 Gbits/sec receiver

开启30个线程传输

iperf3 -c x.x.x.x -t 60 -P 30
结果:
[SUM] 0.00-60.00 sec 10.1 GBytes 1.45 Gbits/sec 90559 sender
[SUM] 0.00-60.00 sec 10.1 GBytes 1.44 Gbits/sec receiver
在vxlan下多线程也没有性能提升。

从上述两组数据可以明显看出flannel在vxlan下性能要明显高于udp。

从UDP模式配置转换为VXLAN模式

flannel及安装参考:flannel-UDP一文   ,如果新装,方法一致,只是在etcd中配置config key时候配置的value不一样。如果是从udp转换到vxlan,参考本实验。

在所有节点上停止docker及flanneld服务

1
2
systemctl stop flanneld
systemctl stop docker

删除etcd中已经建立的子网配置和修改etcd中的config key

1
2
3
etcdctl  rm --recursive /flannel/network/subnets
 
etcdctl set /flannel/network/config '{"Network": "10.2.0.0/16", "Backend": {"Type": "vxlan"}}'

重新启动所有节点的flanneld和docker,注意要先启flanneld

1
2
systemctl start flanneld
systemctl start docker.service

检查确认

1
2
3
4
[root@docker1 ~]# docker exec busybox-1 ping 10.2.101.2
PING 10.2.101.2 (10.2.101.2): 56 data bytes
64 bytes from 10.2.101.2: seq=0 ttl=62 time=0.701 ms
64 bytes from 10.2.101.2: seq=1 ttl=62 time=1.699 ms

1
2
3
4
5
6
7
8
[root@docker1 ~]# arp -a
? (10.2.101.0) at da:12:3c:1e:af:8c [ether] on flannel.1
? (172.16.199.37) at 00:0c:29:7d:f5:be [ether] on eno50332184
? (192.168.188.254) at 00:50:56:f7:07:54 [ether] on eno33554960
? (172.16.199.27) at 00:0c:29:ae:11:8d [ether] on eno50332184
? (10.2.79.2) at 02:42:0a:02:4f:02 [ether] on docker0
? (10.2.101.2) at da:12:3c:1e:af:8c [ether] on flannel.1
gateway (192.168.188.2) at 00:50:56:ee:c0:ec [ether] on eno33554960

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
[root@docker1 ~]# ifconfig
br-2aeddaf04b59: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.19.0.1  netmask 255.255.0.0  broadcast 0.0.0.0
        inet6 fe80::42:39ff:fee4:447d  prefixlen 64  scopeid 0x20<link>
        ether 02:42:39:e4:44:7d  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
 
docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 10.2.79.1  netmask 255.255.255.0  broadcast 0.0.0.0
        inet6 fe80::42:9cff:fe6c:62ef  prefixlen 64  scopeid 0x20<link>
        ether 02:42:9c:6c:62:ef  txqueuelen 0  (Ethernet)
        RX packets 5141  bytes 426300 (416.3 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 5152  bytes 499216 (487.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
 
eno16777736: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.0.228  netmask 255.255.255.0  broadcast 192.168.0.255
        inet6 fe80::20c:29ff:fe42:d98  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:42:0d:98  txqueuelen 1000  (Ethernet)
        RX packets 1913  bytes 324392 (316.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1039  bytes 321158 (313.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
 
eno33554960: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.188.131  netmask 255.255.255.0  broadcast 192.168.188.255
        inet6 fe80::20c:29ff:fe42:da2  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:42:0d:a2  txqueuelen 1000  (Ethernet)
        RX packets 167  bytes 15449 (15.0 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 194  bytes 16671 (16.2 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
 
eno50332184: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.16.199.17  netmask 255.255.255.0  broadcast 172.16.199.255
        inet6 fe80::20c:29ff:fe42:dac  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:42:0d:ac  txqueuelen 1000  (Ethernet)
        RX packets 181893  bytes 18071701 (17.2 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 181427  bytes 17875322 (17.0 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
 
flannel.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 10.2.79.0  netmask 255.255.255.255  broadcast 0.0.0.0
        ether ba:1e:bf:2f:cb:2f  txqueuelen 0  (Ethernet)
        RX packets 2400  bytes 199584 (194.9 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2508  bytes 202608 (197.8 KiB)
        TX errors 0  dropped 1 overruns 0  carrier 0  collisions 0
 
gre_sys: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 65490
        inet6 fe80::28da:26ff:fe25:9f69  prefixlen 64  scopeid 0x20<link>
        ether 2a:da:26:25:9f:69  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 8 overruns 0  carrier 0  collisions 0
 
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 4129  bytes 238096 (232.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4129  bytes 238096 (232.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
 
veth49e5bf6: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::acae:b3ff:fe70:3b0b  prefixlen 64  scopeid 0x20<link>
        ether ae:ae:b3:70:3b:0b  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 8  bytes 648 (648.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
 
vethe91eedb: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet6 fe80::7cbf:1cff:fe16:7e3c  prefixlen 64  scopeid 0x20<link>
        ether 7e:bf:1c:16:7e:3c  txqueuelen 0  (Ethernet)
        RX packets 2414  bytes 233772 (228.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2422  bytes 234420 (228.9 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

VXLAN封包:

1
2
3
4
5
6
7
8
9
10
11
12
Frame 1: 148 bytes on wire (1184 bits), 148 bytes captured (1184 bits)
Ethernet II, Src: Vmware_42:0d:ac (00:0c:29:42:0d:ac), Dst: Vmware_7d:f5:be (00:0c:29:7d:f5:be)
Internet Protocol Version 4, Src: 172.16.199.17, Dst: 172.16.199.37
User Datagram Protocol, Src Port: 47830, Dst Port: 8472
Virtual eXtensible Local Area Network
    Flags: 0x0800, VXLAN Network ID (VNI)
    Group Policy ID: 0
    VXLAN Network Identifier (VNI): 1
    Reserved: 0
Ethernet II, Src: ba:1e:bf:2f:cb:2f (ba:1e:bf:2f:cb:2f), Dst: da:12:3c:1e:af:8c (da:12:3c:1e:af:8c)
Internet Protocol Version 4, Src: 10.2.79.0, Dst: 10.2.101.2
Internet Control Message Protocol

flannel-vxlan.pcap抓包文件下载

本作品采用 知识共享署名 4.0 国际许可协议 进行许可
标签: docker flannel k8s vxlan
最后更新:2017年07月14日

纳米

http://linjing.io

打赏 点赞
< 上一篇
下一篇 >

文章评论

取消回复

纳米

http://linjing.io

☁️迈向Cloud Native ADC ☁️

认证获得:
Kubernetes: CKA #664
Microsoft: MCSE MCDBA
Cisco: CCNP
Juniper: JNCIS
F5:
F5 Certified Solution Expert, Security
F5 Certified Technology Specialist, LTM/GTM/APM/ASM
F5 Certified BIG-IP Administrator
  • 点击查看本博技术要素列表
  • 分类目录
    • Avi Networks (3)
    • Cisco ACI (1)
    • CISCO资源 (21)
    • F5 with ELK (8)
    • F5-Tech tips (38)
    • F5技术 (203)
    • Juniper (4)
    • Linux (7)
    • Nginx (18)
    • SDN (4)
    • ServiceMesh (19)
    • WEB编程 (8)
    • WINDOWS相关 (7)
    • 业界文章 (18)
    • 交换机技术 (20)
    • 化云为雨/Openstack (35)
    • 协议原理 (52)
    • 容器/k8s (64)
    • 我的工作 (19)
    • 我的生活 (70)
    • 网站技术 (19)
    • 路由器技术 (80)
    • 项目案例 (28)
    文章归档
    标签聚合
    F5 k8s openstack nginx istio DNS envoy gtm docker network flannel api irule bigip neutron cc kubernetes ELK vxlan BGP dhcp VPN IPSec lbaas ingress ingress controller nginx plus sidecar IPSec VPN NAT sql
    最新 热点 随机
    最新 热点 随机
    Say hello for 2021 二进制flannel部署,非cni网络模式下与k8s CIS结合方案 又是一年国庆 Service Account Token Volume Projection Istio ingressgateway 静态TLS证书加载与SDS发现方式配置区别 Istio里Gateway的port定义与实际ingressgateway的listener端口关系及规则 Helm 3 部署NGINX Ingress Controller 应用交付老兵眼中的Envoy, 云原生时代下的思考 Istio sidecar iptables以及流量控制分析 Istio 熔断策略及envoy配置
    Say hello for 2021
    kubernetes node TLS bootstrapping原理及配置 2009,我的这一年 2014 ADC Gartner magic quadrant(Oct. 2014) Istio Architecture session persistence 让VS直接访问另一个VS 做鸡 上班了 [原创]NSSA区域下LSA4的产生与传播 F5-LTM 替代WEB服务器执行SSL【问题解决】
    链接表
    • Jimmy Song‘s Blog
    • SDNap
    • SDNlab
    • SDN论坛
    • Service Mesh社区
    • 三斗室
    • 个人profile

    COPYRIGHT © 2020 Cloud Native应用交付. ALL RIGHTS RESERVED.

    THEME KRATOS MADE BY VTROIS

    京ICP备14048088号-1

    京公网安备 11010502041506号

    [ Placeholder content for popup link ] WordPress Download Manager - Best Download Management Plugin