k8s satefulset 测试

1. 预先创建相关pv
   
   [root@k8s-master cka]# cat pv-5g.yaml kind: PersistentVolume apiVersion: v1 metadata: name: task-pv-volume-5g labels: type: local spec: storageClassName: manual capacity: storage: 5Gi accessModes: - ReadWriteOnce hostPath: path: "/root/cka/data" [root@k8s-master cka]# cat pv.yaml kind: PersistentVolume apiVersion: v1 metadata: name: task-pv-volume labels: type: local spec: storageClassName: manual capacity: storage: 2Gi accessModes: - ReadWriteOnce hostPath: path: "/root/cka/data"

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

   [root@k8s-master cka]# cat pv-5g.yaml kind: PersistentVolume apiVersion: v1 metadata:   name: task-pv-volume-5g   labels:     type: local spec:   storageClassName: manual   capacity:     storage: 5Gi   accessModes:     - ReadWriteOnce   hostPath:     path: "/root/cka/data"     [root@k8s-master cka]# cat pv.yaml kind: PersistentVolume apiVersion: v1 metadata:   name: task-pv-volume   labels:     type: local spec:   storageClassName: manual   capacity:     storage: 2Gi   accessModes:     - ReadWriteOnce   hostPath:     path: "/root/cka/data"

2. 执行以下yaml，创建headless servcie，statefulset，系统从statefulset模板中自动创建pvc
   
   [root@k8s-master cka]# cat statefulset.yaml apiVersion: v1 kind: Service metadata: name: nginx-stateful labels: app: nginx-stateful spec: ports: - port: 80 name: web clusterIP: None selector: appstateful: nginx <<<<这里要等于statefulset中的spec.selector.matchLabels和spec.template.metadata.labels --- apiVersion: apps/v1 kind: StatefulSet metadata: name: web spec: selector: matchLabels: appstateful: nginx # has to match .spec.template.metadata.labels serviceName: "nginx-stateful" replicas: 2 # by default is 1 template: metadata: labels: appstateful: nginx # has to match .spec.selector.matchLabels 应用到所有pods上 spec: terminationGracePeriodSeconds: 10 containers: - name: nginx image: nginx ports: - containerPort: 80 name: web volumeMounts: - name: www mountPath: /usr/share/nginx/html volumeClaimTemplates: - metadata: name: www spec: accessModes: [ "ReadWriteOnce" ] storageClassName: manual resources: requests: storage: 1Gi

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49

   [root@k8s-master cka]# cat statefulset.yaml apiVersion: v1 kind: Service metadata:   name: nginx-stateful   labels:     app: nginx-stateful spec:   ports:   - port: 80     name: web   clusterIP: None   selector:     appstateful: nginx  <<<<这里要等于statefulset中的spec.selector.matchLabels和spec.template.metadata.labels --- apiVersion: apps/v1 kind: StatefulSet metadata:   name: web spec:   selector:     matchLabels:       appstateful: nginx # has to match .spec.template.metadata.labels   serviceName: "nginx-stateful"   replicas: 2 # by default is 1   template:     metadata:       labels:         appstateful: nginx # has to match .spec.selector.matchLabels 应用到所有pods上     spec:       terminationGracePeriodSeconds: 10       containers:       - name: nginx         image: nginx         ports:         - containerPort: 80           name: web         volumeMounts:         - name: www           mountPath: /usr/share/nginx/html   volumeClaimTemplates:   - metadata:       name: www     spec:       accessModes: [ "ReadWriteOnce" ]       storageClassName: manual       resources:         requests:           storage: 1Gi

   
    
3. 产生的输出效果
   
   [root@k8s-master cka]# kubectl get statefulset web -o yaml apiVersion: apps/v1 kind: StatefulSet metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"apps/v1","kind":"StatefulSet","metadata":{"annotations":{},"name":"web","namespace":"default"},"spec":{"replicas":2,"selector":{"matchLabels":{"appstateful":"nginx"}},"serviceName":"nginx-stateful","template":{"metadata":{"labels":{"appstateful":"nginx"}},"spec":{"containers":[{"image":"nginx","name":"nginx","ports":[{"containerPort":80,"name":"web"}],"volumeMounts":[{"mountPath":"/usr/share/nginx/html","name":"www"}]}],"terminationGracePeriodSeconds":10}},"volumeClaimTemplates":[{"metadata":{"name":"www"},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"manual"}}]}} creationTimestamp: 2018-08-09T10:22:45Z generation: 5 name: web namespace: default resourceVersion: "245069" selfLink: /apis/apps/v1/namespaces/default/statefulsets/web uid: 282abb71-9bbe-11e8-a2f9-000c29850765 spec: podManagementPolicy: OrderedReady <<<如果是Parallel 则会同时启动删除pod而不按照强制顺序 replicas: 2 revisionHistoryLimit: 10 selector: matchLabels: appstateful: nginx serviceName: nginx-stateful template: metadata: creationTimestamp: null labels: appstateful: nginx spec: containers: - image: nginx imagePullPolicy: Always name: nginx ports: - containerPort: 80 name: web protocol: TCP resources: {} terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts: - mountPath: /usr/share/nginx/html name: www dnsPolicy: ClusterFirst restartPolicy: Always schedulerName: default-scheduler securityContext: {} terminationGracePeriodSeconds: 10 updateStrategy: rollingUpdate: partition: 0 <<<<<分区rollingupdate，pod编号大于等于这个数字的会被update，小于的不变。等于可以做到灰度发布 type: RollingUpdate volumeClaimTemplates: - metadata: creationTimestamp: null name: www spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi storageClassName: manual status: phase: Pending status: collisionCount: 0 currentReplicas: 2 currentRevision: web-7b4ff57f55 observedGeneration: 5 readyReplicas: 2 replicas: 2 updateRevision: web-7b4ff57f55

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72

   [root@k8s-master cka]# kubectl get statefulset web -o yaml apiVersion: apps/v1 kind: StatefulSet metadata:   annotations:     kubectl.kubernetes.io/last-applied-configuration: |       {"apiVersion":"apps/v1","kind":"StatefulSet","metadata":{"annotations":{},"name":"web","namespace":"default"},"spec":{"replicas":2,"selector":{"matchLabels":{"appstateful":"nginx"}},"serviceName":"nginx-stateful","template":{"metadata":{"labels":{"appstateful":"nginx"}},"spec":{"containers":[{"image":"nginx","name":"nginx","ports":[{"containerPort":80,"name":"web"}],"volumeMounts":[{"mountPath":"/usr/share/nginx/html","name":"www"}]}],"terminationGracePeriodSeconds":10}},"volumeClaimTemplates":[{"metadata":{"name":"www"},"spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"1Gi"}},"storageClassName":"manual"}}]}}   creationTimestamp: 2018-08-09T10:22:45Z   generation: 5   name: web   namespace: default   resourceVersion: "245069"   selfLink: /apis/apps/v1/namespaces/default/statefulsets/web   uid: 282abb71-9bbe-11e8-a2f9-000c29850765 spec:   podManagementPolicy: OrderedReady <<<如果是Parallel 则会同时启动删除pod而不按照强制顺序   replicas: 2   revisionHistoryLimit: 10   selector:     matchLabels:       appstateful: nginx   serviceName: nginx-stateful   template:     metadata:       creationTimestamp: null       labels:         appstateful: nginx     spec:       containers:       - image: nginx         imagePullPolicy: Always         name: nginx         ports:         - containerPort: 80           name: web           protocol: TCP         resources: {}         terminationMessagePath: /dev/termination-log         terminationMessagePolicy: File         volumeMounts:         - mountPath: /usr/share/nginx/html           name: www       dnsPolicy: ClusterFirst       restartPolicy: Always       schedulerName: default-scheduler       securityContext: {}       terminationGracePeriodSeconds: 10   updateStrategy:     rollingUpdate:       partition: 0 <<<<<分区rollingupdate，pod编号大于等于这个数字的会被update，小于的不变。等于可以做到灰度发布     type: RollingUpdate   volumeClaimTemplates:   - metadata:       creationTimestamp: null       name: www     spec:       accessModes:       - ReadWriteOnce       resources:         requests:           storage: 1Gi       storageClassName: manual     status:       phase: Pending status:   collisionCount: 0   currentReplicas: 2   currentRevision: web-7b4ff57f55   observedGeneration: 5   readyReplicas: 2   replicas: 2   updateRevision: web-7b4ff57f55

   
    
   
   [root@k8s-master cka]# kubectl get svc -o wide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR kubernetes ClusterIP 10.250.0.1 <none> 443/TCP 3d <none> nginx ClusterIP 10.250.0.157 <none> 80/TCP 7h app=nginx nginx-stateful ClusterIP None <none> 80/TCP 16m appstateful=nginx

   1 2 3 4 5

   [root@k8s-master cka]# kubectl get svc -o wide NAME             TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE       SELECTOR kubernetes       ClusterIP   10.250.0.1     <none>        443/TCP   3d        <none> nginx            ClusterIP   10.250.0.157   <none>        80/TCP    7h        app=nginx nginx-stateful   ClusterIP   None           <none>        80/TCP    16m       appstateful=nginx

   
   
   [root@k8s-master cka]# kubectl get statefulset -o wide NAME DESIRED CURRENT AGE CONTAINERS IMAGES web 2 2 16m nginx nginx

   1 2 3

   [root@k8s-master cka]# kubectl get statefulset -o wide NAME      DESIRED   CURRENT   AGE       CONTAINERS   IMAGES web       2         2         16m       nginx        nginx

   
   
   [root@k8s-master cka]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE busybox-6cdcdb6f5f-tfh84 1/1 Running 54 3d 10.244.0.4 k8s-master mysql 1/1 Running 0 1d 10.244.1.23 k8s-node1 nginx-56f766d96f-96vx7 1/1 Running 0 7h 10.244.1.24 k8s-node1 nginx-56f766d96f-ckxqj 1/1 Running 0 7h 10.244.0.9 k8s-master pi 0/1 Completed 0 2d 10.244.1.8 k8s-node1 pi-jkdfw 0/1 Completed 0 1d 10.244.1.14 k8s-node1 redis 1/1 Running 0 1d 10.244.1.19 k8s-node1 static-web-k8s-master 1/1 Running 0 3h 10.244.0.10 k8s-master web-0 1/1 Running 0 17m 10.244.1.25 k8s-node1 web-1 1/1 Running 0 17m 10.244.0.11 k8s-master

   1 2 3 4 5 6 7 8 9 10 11 12

   [root@k8s-master cka]# kubectl get pods -o wide NAME                       READY     STATUS      RESTARTS   AGE       IP            NODE busybox-6cdcdb6f5f-tfh84   1/1       Running     54         3d        10.244.0.4    k8s-master mysql                      1/1       Running     0          1d        10.244.1.23   k8s-node1 nginx-56f766d96f-96vx7     1/1       Running     0          7h        10.244.1.24   k8s-node1 nginx-56f766d96f-ckxqj     1/1       Running     0          7h        10.244.0.9    k8s-master pi                         0/1       Completed   0          2d        10.244.1.8    k8s-node1 pi-jkdfw                   0/1       Completed   0          1d        10.244.1.14   k8s-node1 redis                      1/1       Running     0          1d        10.244.1.19   k8s-node1 static-web-k8s-master      1/1       Running     0          3h        10.244.0.10   k8s-master web-0                      1/1       Running     0          17m       10.244.1.25   k8s-node1 web-1                      1/1       Running     0          17m       10.244.0.11   k8s-master

   
   
   [root@k8s-master cka]# kubectl get pvc -o wide NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE task-pv-claim Terminating task-pv-volume-4g 4Gi RWO manual 1d www-web-0 Bound task-pv-volume 2Gi RWO manual 18m www-web-1 Bound task-pv-volume-5g 5Gi RWO manual 18m

   1 2 3 4 5

   [root@k8s-master cka]# kubectl get pvc -o wide NAME            STATUS        VOLUME              CAPACITY   ACCESS MODES   STORAGECLASS   AGE task-pv-claim   Terminating   task-pv-volume-4g   4Gi        RWO            manual         1d www-web-0       Bound         task-pv-volume      2Gi        RWO            manual         18m www-web-1       Bound         task-pv-volume-5g   5Gi        RWO            manual         18m

   
    
4. dns解析
   
   [root@k8s-master cka]# kubectl exec -it busybox-6cdcdb6f5f-tfh84 sh / # nslookup web-0.nginx-stateful.default.svc.cluster.local Server: 10.250.0.53 Address 1: 10.250.0.53 Name: web-0.nginx-stateful.default.svc.cluster.local Address 1: 10.244.1.25 / # nslookup web-1.nginx-stateful.default.svc.cluster.local Server: 10.250.0.53 Address 1: 10.250.0.53 Name: web-1.nginx-stateful.default.svc.cluster.local Address 1: 10.244.0.11

   1 2 3 4 5 6 7 8 9 10 11 12 13

   [root@k8s-master cka]# kubectl exec -it busybox-6cdcdb6f5f-tfh84  sh / # nslookup web-0.nginx-stateful.default.svc.cluster.local Server:    10.250.0.53 Address 1: 10.250.0.53   Name:      web-0.nginx-stateful.default.svc.cluster.local Address 1: 10.244.1.25 / # nslookup web-1.nginx-stateful.default.svc.cluster.local Server:    10.250.0.53 Address 1: 10.250.0.53   Name:      web-1.nginx-stateful.default.svc.cluster.local Address 1: 10.244.0.11

   
   解析svc域名，直接解析出两个pod的ip
   
   / # nslookup nginx-stateful.default.svc.cluster.local Server: 10.250.0.53 Address 1: 10.250.0.53 Name: nginx-stateful.default.svc.cluster.local Address 1: 10.244.0.11 Address 2: 10.244.1.25

   1 2 3 4 5 6 7

   / # nslookup nginx-stateful.default.svc.cluster.local Server:    10.250.0.53 Address 1: 10.250.0.53   Name:      nginx-stateful.default.svc.cluster.local Address 1: 10.244.0.11 Address 2: 10.244.1.25

   
   修改image，产生rollingupdate：
   
   web-0 1/1 Running 0 52m web-1 1/1 Running 0 14m -------------开始rolling----- web-1 1/1 Terminating 0 14m web-1 0/1 Terminating 0 14m web-1 0/1 Terminating 0 15m web-1 0/1 Terminating 0 15m web-1 0/1 Pending 0 0s web-1 0/1 Pending 0 0s web-1 0/1 ContainerCreating 0 0s web-1 0/1 ContainerCreating 0 1s web-1 0/1 ContainerCreating 0 6s web-1 1/1 Running 0 15s ------以上终结web-1 pod,并重新启动一个新web-1 pod，成功后开始web0---- web-0 1/1 Terminating 0 53m web-0 0/1 Terminating 0 53m web-0 0/1 Terminating 0 53m web-0 0/1 Terminating 0 53m web-0 0/1 Pending 0 0s web-0 0/1 Pending 0 0s web-0 0/1 ContainerCreating 0 2s web-0 0/1 ContainerCreating 0 4s web-0 0/1 ContainerCreating 0 10s web-0 1/1 Running 0 34s ------以上web0终结及重启过程------------

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

   web-0                      1/1       Running     0          52m web-1                      1/1       Running     0          14m -------------开始rolling----- web-1     1/1       Terminating   0         14m web-1     0/1       Terminating   0         14m web-1     0/1       Terminating   0         15m web-1     0/1       Terminating   0         15m web-1     0/1       Pending   0         0s web-1     0/1       Pending   0         0s web-1     0/1       ContainerCreating   0         0s web-1     0/1       ContainerCreating   0         1s web-1     0/1       ContainerCreating   0         6s web-1     1/1       Running   0         15s ------以上终结web-1 pod,并重新启动一个新web-1 pod，成功后开始web0---- web-0     1/1       Terminating   0         53m web-0     0/1       Terminating   0         53m web-0     0/1       Terminating   0         53m web-0     0/1       Terminating   0         53m web-0     0/1       Pending   0         0s web-0     0/1       Pending   0         0s web-0     0/1       ContainerCreating   0         2s web-0     0/1       ContainerCreating   0         4s web-0     0/1       ContainerCreating   0         10s web-0     1/1       Running   0         34s ------以上web0终结及重启过程------------

   
    
5. [root@k8s-master cka]# kubectl patch statefulset web -p '{"spec":{"replicas":1}}'
   减少replicas，将从pod名字中数字编号最大的开始减。且被删除pod所用的pvc并不会被删除，必须手工干预。如果不删除这些pvc，则如果replicas恢复回来，对应的pod名字还会claim对应的pvc，这样保持一致。

   Note that, the PersistentVolumes associated with the Pods’ PersistentVolume Claims are not deleted when the Pods, or StatefulSet are deleted. This must be done manually.

    

Statefulset:

pod的名字按照 {statefulset名字}-{数字从0排序}，名字始终稳定，不论怎么扩展删除

自动按模板产生pvc， pvc名字 {pvc模板名}-{statefulset名}-{从0编号}，pod与pvc关联关系始终稳定不变

按顺序产生pod，按反序删除pod。

rollingupdate时候安装类似删除的动作，删一个，起一个。

https://v1-10.docs.kubernetes.io/docs/concepts/workloads/controllers/statefulset/