准备工作
为了学习以及理解k8s各个模块组件间关系，没有使用yum来安装，也没有使用kubeadm来做建立集群。采用全手工安装和配置方式,为了简化安装过程，集群内未使用证书来实现安全的接口通信。
环境
三台主机
172.16.199.17/24 Master
172.16.199.27/24 node
172.16.199.37/24 node
所有机器上都已安装好 docker，三机构成etcd集群，全部安装了flannel网络
系统版本Centos7
Master安装：
文件下载
本次测试安装时，k8s维护版本是1.6.7
|
1 |
wget https://github.com/kubernetes/kubernetes/releases/download/v1.6.7/kubernetes.tar.gz |
下载完毕后，当前k8s下载的包里不再含有k8s的相关二进制文件，需要执行里面的一个脚本下载：
|
1 2 3 |
[root@docker1 cluster]# pwd //root/kubernetes/cluster [root@docker1 cluster]# ./get-kube-binaries.sh |
下载完毕后，进入server目录，可以看到多出一个kubernetes-server-linux-amd64.tar.gz 压缩包
|
1 2 3 |
//root/kubernetes/server [root@docker1 server]# ls bin kubernetes-manifests.tar.gz kubernetes-salt.tar.gz kubernetes-server-linux-amd64.tar.gz README |
解压这个压缩包文件到/root/kubernetes/server/bin 目录下，随后将bin目录下的所有文件拷贝到/usr/bin下。 同时将这些文件一并拷贝到27和37两台机器的/usr/bin下
Master配置
apiserver
拷贝以下脚本，并执行
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 |
[root@docker1 scripts]# pwd /root/kubernetes/cluster/centos/master/scripts [root@docker1 scripts]# cat apiserver_jing.sh #!/bin/bash # Copyright 2014 The Kubernetes Authors. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. MASTER_ADDRESS=${1:-"172.16.199.17"} ETCD_SERVERS=${2:-"http://172.16.199.17:2379"} SERVICE_CLUSTER_IP_RANGE=${3:-"169.169.0.0/16"} ADMISSION_CONTROL=${4:-"AlwaysAdmit"} mkdir -p /etc/kubernetes cat <<EOF >/etc/kubernetes/kube-apiserver # --alsologtostderr=true: log to standard error as well as log file KUBE_ALSO_LOGTOSTDERR="--alsologtostderr=true" # --log-dir KUBE_LOG_DIR="--log-dir=/var/log/kubernetes" # --v=0: log level for V logs KUBE_LOG_LEVEL="--v=2" #stderrthreshold: only show log that over the threshold to stderr STDERR_THRESHOLD="--stderrthreshold=2" # --etcd-servers=[]: List of etcd servers to watch (http://ip:port), # comma separated. Mutually exclusive with -etcd-config KUBE_ETCD_SERVERS="--etcd-servers=${ETCD_SERVERS}" # --etcd-cafile="": SSL Certificate Authority file used to secure etcd communication. #KUBE_ETCD_CAFILE="--etcd-cafile=/srv/kubernetes/etcd/ca.pem" # --etcd-certfile="": SSL certification file used to secure etcd communication. #KUBE_ETCD_CERTFILE="--etcd-certfile=/srv/kubernetes/etcd/client.pem" # --etcd-keyfile="": key file used to secure etcd communication. #KUBE_ETCD_KEYFILE="--etcd-keyfile=/srv/kubernetes/etcd/client-key.pem" # --insecure-bind-address=127.0.0.1: The IP address on which to serve the --insecure-port. KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0" # --insecure-port=8080: The port on which to serve unsecured, unauthenticated access. KUBE_API_PORT="--insecure-port=8080" # --kubelet-port=10250: Kubelet port NODE_PORT="--kubelet-port=10250" # --advertise-address=<nil>: The IP address on which to advertise # the apiserver to members of the cluster. KUBE_ADVERTISE_ADDR="--advertise-address=${MASTER_ADDRESS}" # --allow-privileged=false: If true, allow privileged containers. KUBE_ALLOW_PRIV="--allow-privileged=false" # --service-cluster-ip-range=<nil>: A CIDR notation IP range from which to assign service cluster IPs. # This must not overlap with any IP ranges assigned to nodes for pods. KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=${SERVICE_CLUSTER_IP_RANGE}" # --admission-control="AlwaysAdmit": Ordered list of plug-ins # to do admission control of resources into cluster. # Comma-delimited list of: # LimitRanger, AlwaysDeny, SecurityContextDeny, NamespaceExists, # NamespaceLifecycle, NamespaceAutoProvision, AlwaysAdmit, # ServiceAccount, DefaultStorageClass, DefaultTolerationSeconds, ResourceQuota KUBE_ADMISSION_CONTROL="--admission-control=${ADMISSION_CONTROL}" # --client-ca-file="": If set, any request presenting a client certificate signed # by one of the authorities in the client-ca-file is authenticated with an identity # corresponding to the CommonName of the client certificate. #KUBE_API_CLIENT_CA_FILE="--client-ca-file=/srv/kubernetes/ca.crt" # --tls-cert-file="": File containing x509 Certificate for HTTPS. (CA cert, if any, # concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file # and --tls-private-key-file are not provided, a self-signed certificate and key are # generated for the public address and saved to /var/run/kubernetes. #KUBE_API_TLS_CERT_FILE="--tls-cert-file=/srv/kubernetes/server.cert" # --tls-private-key-file="": File containing x509 private key matching --tls-cert-file. #KUBE_API_TLS_PRIVATE_KEY_FILE="--tls-private-key-file=/srv/kubernetes/server.key" ##enable RBAC add-by-myf5.net KUBE_RBAC="--authorization-mode=RBAC" EOF KUBE_APISERVER_OPTS=" \${KUBE_ALSO_LOGTOSTDERR} \\ \${STDERR_THRESHOLD} \\ \${KUBE_LOG_DIR} \\ \${KUBE_LOG_LEVEL} \\ \${KUBE_ETCD_SERVERS} \\ \${KUBE_API_ADDRESS} \\ \${KUBE_API_PORT} \\ \${NODE_PORT} \\ \${KUBE_ADVERTISE_ADDR} \\ \${KUBE_ALLOW_PRIV} \\ \${KUBE_SERVICE_ADDRESSES} \\ \${KUBE_ADMISSION_CONTROL} \\ \${KUBE_RBAC}" cat <<EOF >/usr/lib/systemd/system/kube-apiserver.service [Unit] Description=Kubernetes API Server Documentation=https://github.com/kubernetes/kubernetes [Service] EnvironmentFile=-/etc/kubernetes/kube-apiserver ExecStart=/usr/bin/kube-apiserver ${KUBE_APISERVER_OPTS} Restart=on-failure [Install] WantedBy=multi-user.target EOF systemctl daemon-reload systemctl enable kube-apiserver systemctl restart kube-apiserver |
controller-manager
拷贝以下脚本并执行
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 |
[root@docker1 scripts]# cat controller-manager_jing.sh #!/bin/bash # Copyright 2014 The Kubernetes Authors. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. MASTER_ADDRESS=${1:-"172.16.199.17"} cat <<EOF >/etc/kubernetes/kube-controller-manager KUBE_LOGTOSTDERR="--logtostderr=false" KUBE_LOG_DIR="--log-dir=/var/log/kubernetes" KUBE_LOG_LEVEL="--v=4" KUBE_MASTER="--master=${MASTER_ADDRESS}:8080" KUBE_CLUSTER_CIDR="--cluster-cidr=10.1.0.0/16" # --root-ca-file="": If set, this root certificate authority will be included in # service account's token secret. This must be a valid PEM-encoded CA bundle. #KUBE_CONTROLLER_MANAGER_ROOT_CA_FILE="--root-ca-file=/srv/kubernetes/ca.crt" # --service-account-private-key-file="": Filename containing a PEM-encoded private # RSA key used to sign service account tokens. #KUBE_CONTROLLER_MANAGER_SERVICE_ACCOUNT_PRIVATE_KEY_FILE="--service-account-private-key-file=/srv/kubernetes/server.key" # --leader-elect KUBE_LEADER_ELECT="--leader-elect=false" EOF KUBE_CONTROLLER_MANAGER_OPTS=" \${KUBE_LOGTOSTDERR} \\ \${KUBE_LOG_DIR} \\ \${KUBE_LOG_LEVEL} \\ \${KUBE_MASTER} \\ \${KUBE_CLUSTER_CIDR}\\ \${KUBE_LEADER_ELECT}" cat <<EOF >/usr/lib/systemd/system/kube-controller-manager.service [Unit] Description=Kubernetes Controller Manager Documentation=https://github.com/kubernetes/kubernetes After=kube-apiserver.service After=network.target Requires=kube-apiserver.service [Service] EnvironmentFile=-/etc/kubernetes/kube-controller-manager ExecStart=/usr/bin/kube-controller-manager ${KUBE_CONTROLLER_MANAGER_OPTS} Restart=on-failure [Install] WantedBy=multi-user.target EOF systemctl daemon-reload systemctl enable kube-controller-manager systemctl restart kube-controller-manager |
scheduler
拷贝以下脚本并执行
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 |
#!/bin/bash # Copyright 2014 The Kubernetes Authors. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. MASTER_ADDRESS=${1:-"172.16.199.17"} cat <<EOF >/etc/kubernetes/kube-scheduler ### # kubernetes scheduler config # --logtostderr=true: log to standard error instead of files KUBE_LOGTOSTDERR="--logtostderr=false" # Log to file KUBE_LOG_DIR="--log-dir=/var/log/kubernetes" # --v=0: log level for V logs KUBE_LOG_LEVEL="--v=4" KUBE_MASTER="--master=${MASTER_ADDRESS}:8080" # --leader-elect KUBE_LEADER_ELECT="--leader-elect=false" # Add your own! KUBE_SCHEDULER_ARGS="" EOF KUBE_SCHEDULER_OPTS=" \${KUBE_LOGTOSTDERR} \\ \${KUBE_LOG_DIR} \\ \${KUBE_LOG_LEVEL} \\ \${KUBE_MASTER} \\ \${KUBE_LEADER_ELECT} \\ \${KUBE_SCHEDULER_ARGS}" cat <<EOF >/usr/lib/systemd/system/kube-scheduler.service [Unit] Description=Kubernetes Scheduler Documentation=https://github.com/kubernetes/kubernetes After=kube-apiserver.service Requires=kube-apiserver.service [Service] EnvironmentFile=-/etc/kubernetes/kube-scheduler ExecStart=/usr/bin/kube-scheduler ${KUBE_SCHEDULER_OPTS} Restart=on-failure [Install] WantedBy=multi-user.target EOF systemctl daemon-reload systemctl enable kube-scheduler systemctl restart kube-scheduler |
Node安装：
kubelet
拷贝以下脚本并执行，注意在每个node上修改对应node的IP. DNS地址为预留地址，dns服务还未安装。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 |
#!/bin/bash # Copyright 2014 The Kubernetes Authors. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. MASTER_ADDRESS=${1:-"172.16.199.17"} NODE_ADDRESS=${2:-"172.16.199.27"} DNS_SERVER_IP=${3:-"169.169.0.53"} DNS_DOMAIN=${4:-"cluster.local"} mkdir -p /etc/kubernetes cat <<EOF >/etc/kubernetes/kubelet # --logtostderr=true: log to standard error instead of files KUBE_LOGTOSTDERR="--logtostderr=false" #LOG TO FILE KUBE_LOG_DIR="--log-dir=/var/log/kubernetes" # --v=0: log level for V logs KUBE_LOG_LEVEL="--v=4" # --address=0.0.0.0: The IP address for the Kubelet to serve on (set to 0.0.0.0 for all interfaces) NODE_ADDRESS="--address=${NODE_ADDRESS}" # --port=10250: The port for the Kubelet to serve on. Note that "kubectl logs" will not work if you set this flag. NODE_PORT="" # --hostname-override="": If non-empty, will use this string as identification instead of the actual hostname. NODE_HOSTNAME="--hostname-override=${NODE_ADDRESS}" # --api-servers=[]: List of Kubernetes API servers for publishing events, # and reading pods and services. (ip:port), comma separated. KUBELET_API_SERVER="--api-servers=${MASTER_ADDRESS}:8080" # --allow-privileged=false: If true, allow containers to request privileged mode. [default=false] KUBE_ALLOW_PRIV="--allow-privileged=false" # DNS info KUBELET__DNS_IP="--cluster-dns=${DNS_SERVER_IP}" KUBELET_DNS_DOMAIN="--cluster-domain=${DNS_DOMAIN}" # Add your own! KUBELET_ARGS="" EOF KUBE_PROXY_OPTS=" \${KUBE_LOGTOSTDERR} \\ \${KUBE_LOG_DIR} \\ \${KUBE_LOG_LEVEL} \\ \${NODE_ADDRESS} \\ \${NODE_PORT} \\ \${NODE_HOSTNAME} \\ \${KUBELET_API_SERVER} \\ \${KUBE_ALLOW_PRIV} \\ \${KUBELET__DNS_IP} \\ \${KUBELET_DNS_DOMAIN} \\ \$KUBELET_ARGS" cat <<EOF >/usr/lib/systemd/system/kubelet.service [Unit] Description=Kubernetes Kubelet After=docker.service Requires=docker.service [Service] EnvironmentFile=-/etc/kubernetes/kubelet ExecStart=/usr/bin/kubelet ${KUBE_PROXY_OPTS} Restart=on-failure KillMode=process [Install] WantedBy=multi-user.target EOF systemctl daemon-reload systemctl enable kubelet systemctl restart kubelet |
kube-proxy
拷贝以下脚本并执行，注意在每个node上修改对应node的IP
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
#!/bin/bash # Copyright 2014 The Kubernetes Authors. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. MASTER_ADDRESS=${1:-"172.16.199.17"} NODE_ADDRESS=${2:-"172.16.199.27"} cat <<EOF >/etc/kubernetes/kube-proxy # --logtostderr=true: log to standard error instead of files KUBE_LOGTOSTDERR="--logtostderr=false" KUBE_LOG_DIR="--log-dir=/var/log/kubernetes" # --v=0: log level for V logs KUBE_LOG_LEVEL="--v=2" # --hostname-override="": If non-empty, will use this string as identification instead of the actual hostname. NODE_HOSTNAME="--hostname-override=${NODE_ADDRESS}" # --master="": The address of the Kubernetes API server (overrides any value in kubeconfig) KUBE_MASTER="--master=http://${MASTER_ADDRESS}:8080" EOF KUBE_PROXY_OPTS=" \${KUBE_LOGTOSTDERR} \\ \${KUBE_LOG_DIR} \\ \${KUBE_LOG_LEVEL} \\ \${NODE_HOSTNAME} \\ \${KUBE_MASTER}" cat <<EOF >/usr/lib/systemd/system/kube-proxy.service [Unit] Description=Kubernetes Proxy After=network.target [Service] EnvironmentFile=-/etc/kubernetes/kube-proxy ExecStart=/usr/bin/kube-proxy ${KUBE_PROXY_OPTS} Restart=on-failure [Install] WantedBy=multi-user.target EOF systemctl daemon-reload systemctl enable kube-proxy systemctl restart kube-proxy |
初步验证
在master上执行
|
1 2 3 |
[root@docker1 scripts]# kubectl get svc -o wide NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR kubernetes 169.169.0.1 <none> 443/TCP 17h <none> |
|
1 2 3 4 |
[root@docker1 scripts]# kubectl get node -o wide NAME STATUS AGE VERSION EXTERNAL-IP OS-IMAGE KERNEL-VERSION 172.16.199.27 Ready 40m v1.6.7 <none> CentOS Linux 7 (Core) 3.10.0-514.21.2.el7.x86_64 172.16.199.37 Ready 28m v1.6.7 <none> CentOS Linux 7 (Core) 3.10.0-514.21.2.el7.x86_64 |
文章评论