思路:
k8s内使用headless service,service配置文件中配置selector,这样k8s内部的dns服务将把endpoint的IP作为 A记录到相关服务的名称下
让F5可以使用k8s的DNS 服务
在F5内使用FQDN方式配置pool,这样F5将自动获取所有k8s内的endpoint
优点:
不用关心容器内复杂的服务调度及iptables
无需使用api来创建F5配置
F5直接将流量送给endpoint效率高
endpoint内容器的健康性、扩展性由k8s负责,未被破坏,F5 monitor可以附加对应用的健康检查
pod漂移发生的地址变化,k8s内dns会自动更新,F5只需解析相关FQDN即可
F5只是从外部帮助引流。
无需F5 k8s-bigip-ctlr
要求:
endpoint可以直接通过宿主机外部访问到,比如flannel网络下,从一个k8s完全外部的client访问k8s创建的endpoint是可以的。
或者容器本身映射服务到宿主机,例如
1 2 3 4 5 6 7 8 9 10 11 12 13 |
apiVersion: v1 kind: Pod metadata: name: webapp labels: app: webapp spec: containers: - name: webapp image: tomcat ports: - containerPort: 8080 hostPort:8081 |
或者pod使用hostNetwork(这种网络模型最容易),例如
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
apiVersion: v1 kind: Pod metadata: name: webapp labels: app: webapp spec: hostNetwork: true containers: - name: webapp image: tomcat imagePullPolicy: Never ports: - containerPort: 8080 |
注意点:
F5 FQDN pool功能需要设置较小的dns查询价格,并设置忽略A记录的TTL时间
难点:
要确保kube-dns的对外服务IP,对F5来说能够恒定!
快速测试:
从flannel外部访问endpoint:
1.在外部机器上加入指向到endpoint网络的路由
BEI-ML-JLIN-:~ jlin$ sudo route add -net 10.2.4.0/24 172.16.199.27
2.验证路由
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
BEI-ML-JLIN-:~ jlin$ netstat -nr Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 192.168.0.1 UGSc 34 0 en0 default link#12 UCSI 1 0 bridge1 10.2.4/24 172.16.199.27 UGSc 0 0 vmnet3 127 127.0.0.1 UCS 0 0 lo0 127.0.0.1 127.0.0.1 UH 35 10657422 lo0 169.254 link#5 UCS 0 0 en0 172.16.150/24 link#18 UC 1 0 vmnet8 172.16.199/24 link#15 UC 3 0 vmnet3 172.16.199.17 0:c:29:42:d:ac UHLWIi 2 11146 vmnet3 835 172.16.199.27 0:c:29:ae:11:8d UHLWIi 2 1381 vmnet3 754 172.16.199.37 0:c:29:7d:f5:be UHLWIi 1 1148 vmnet3 806 |
3. k8s内设置
创建一个新的headless svc:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
kind: Service apiVersion: v1 metadata: labels: k8s-app: my-headless-service name: my-headless-service namespace: default spec: clusterIP: None ports: - port: 9999 targetPort: 80 selector: run: k8s-nginx |
此时kube-dns内的解析:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
[root@docker3 ~]# dig @10.2.39.2 my-headless-service.default.svc.cluster.local ; <<>> DiG 9.9.4-RedHat-9.9.4-38.el7_3.3 <<>> @10.2.39.2 my-headless-service.default.svc.cluster.local ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44048 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;my-headless-service.default.svc.cluster.local. IN A ;; ANSWER SECTION: my-headless-service.default.svc.cluster.local. 30 IN A 10.2.39.3 my-headless-service.default.svc.cluster.local. 30 IN A 10.2.4.2 ;; Query time: 0 msec ;; SERVER: 10.2.39.2#53(10.2.39.2) ;; WHEN: Wed Jul 12 09:23:28 CST 2017 ;; MSG SIZE rcvd: 95 |
1 2 3 4 5 6 7 8 9 |
[root@docker1 app]# kubectl exec busybox ping my-headless-service.default PING my-headless-service.default (10.2.39.3): 56 data bytes 64 bytes from 10.2.39.3: seq=0 ttl=62 time=1.186 ms 64 bytes from 10.2.39.3: seq=1 ttl=62 time=0.686 ms ^C [root@docker1 app]# kubectl exec busybox ping my-headless-service.default PING my-headless-service.default (10.2.4.2): 56 data bytes 64 bytes from 10.2.4.2: seq=0 ttl=64 time=0.462 ms 64 bytes from 10.2.4.2: seq=1 ttl=64 time=0.106 ms |
下面10.2.4.2实际是k8s创建的一个pod的IP,可以看到从外部电脑可以访问pod
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
BEI-ML-JLIN-:~ jlin$ curl http://10.2.4.2 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html> |
外部DNS解析:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
dig @10.2.39.2 my-headless-service.default.svc.cluster.local ; <<>> DiG 9.9.4-RedHat-9.9.4-38.el7_3.3 <<>> @10.2.39.2 my-headless-service.default.svc.cluster.local ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44048 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;my-headless-service.default.svc.cluster.local. IN A ;; ANSWER SECTION: my-headless-service.default.svc.cluster.local. 30 IN A 10.2.39.3 my-headless-service.default.svc.cluster.local. 30 IN A 10.2.4.2 ;; Query time: 0 msec ;; SERVER: 10.2.39.2#53(10.2.39.2) ;; WHEN: Wed Jul 12 09:23:28 CST 2017 ;; MSG SIZE rcvd: 95 |
F5 FQDN pool效果
附:
nginx plus作为容器运行在k8s内,并使用容器内DNS实现自动更新LB配置
文章评论