k8s安装部署:
安装规划:
安装方式:
部署工具:
使用批量部署工具如(ansible/l saltstack)、手动二进制、kubeadm、apt-get/yum等方式安装，以守护进程的方式启动在宿主机上，类似于是Nginx—样使用service脚本启动。
kubeadm:
https://kubernetes.io/zh-cn/docs/setup/production-environment/tools/kubeadm/high-availability/项目成熟度及维护周期。 使用k8s官方提供的部署工具kubeadm自动安装，需要在master和node节点上安装docker等组件，然后初始化，把管理端的控制服务和node上的服务都以pod的方式运行。
安装注意事项:
-
注意:
-
禁用swap
-
关闭selinux
-
关闭iptables,
-
优化内核参数及资源限制参数
net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 #二层的网桥在转发包时会被宿主机iptables的FORWARD规则匹配
-
部署过程:
组件规划及版本选择:
https://kubernetes.io/zh-cn/docs/setup/production-environment/container-runtimes/ #CRI运行时选择
具体步骤:
1、基础环境准备
2、部署harbor及haproxy高可用反向代理，实现控制节点的API反问入口高可用
3、在所有master节点安装指定版本的kubeadm , kubelet、kubectl、docker
4、在所有node节点安装指定版本的kubeadm 、 kubelet、docker，在node节点kubectl为可选安装，看是否需要在node执行kubectl命令进行集群管理及pod管理等操作。
5、master节点运行kubeadm init初始化命令
6、验证master节点状态
7、在node节点使用kubeadm命令将自己加入k8s master(需要使用master生成token认证)8、验证node节点状态
9、创建pod并测试网络通信
10、部署web服务Dashboard
11、k8s集群升级案例
基础环境准备:
服务器环境: 最小化安装基础系统，如果使用centos系统，则关闭防火墙selinux和swap，更新软件源、时间同步、安装常用命令，重启后验证基础centos推荐使用centos 7.5及以上的系统，ubuntu推荐18.04及以上稳定版。
| 角色 | 主机名 | IP地址 |
|---|---|---|
| k8s-master1 | kubeadm-master1 | 192.168.139.30 |
| k8s-master2 | kubeadm-masrer2 | 192.168.139.31 |
| k8s-master3 | kubeadm-masrer2 | 192.168.139.32 |
| haporxy | haporxy | 192.168.139.33 |
| harbor | harbor | 192.168.139.231 |
| node1 | node1 | 192.168.139.34 |
| node2 | node2 | 192.168.139.35 |
| node3 | node3 | 192.168.139.36 |
安装haproxy
[root@haporxy ~]# yum -y install haproxy
[root@haporxy ~]# tail -7 /etc/haproxy/haproxy.cfg
#`在配置文件最后添加`
listen k8s-6443
bind 192.168.139.33:6443
mode tcp
balance source
server 192.168.139.30 192.168.139.30:6443 check inter 3s fall 5s rise 5
# server 192.168.139.31 192.168.139.31:6443 check inter 3s fall 5s rise 5
# server 192.168.139.32 192.168.139.32:6443 check inter 3s fall 5s rise 5
[root@haporxy ~]# systemctl enable --now haproxy
[root@haporxy ~]# ss -tnl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:22 *:*
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 3000 *:5000 *:*
LISTEN 0 3000 192.168.139.33:6443 *:*
LISTEN 0 128 [::]:22 [::]:*
LISTEN 0 100 [::1]:25 [::]:*
安装harbor
#安装步骤略
[root@harbor harbor]# docker-compose ps
Name Command State Ports
---------------------------------------------------------------------------------------------
harbor-core /harbor/entrypoint.sh Up (healthy)
harbor-db /docker-entrypoint.sh 96 13 Up (healthy)
harbor-jobservice /harbor/entrypoint.sh Up (healthy)
harbor-log /bin/sh -c /usr/local/bin/ ... Up (healthy) 127.0.0.1:1514->10514/tcp
harbor-portal nginx -g daemon off; Up (healthy)
nginx nginx -g daemon off; Up (healthy) 0.0.0.0:80->8080/tcp
redis redis-server /etc/redis.conf Up (healthy)
registry /home/harbor/entrypoint.sh Up (healthy)
registryctl /home/harbor/start.sh Up (healthy)
trivy-adapter /home/scanner/entrypoint.sh Up (healthy)
安装kubeadm等组件:
-
在master和node节点安装kubeadm、kubelet、kubectl、docker等组件，负载均衡服务器不需要安装。
版本选择:
略
安装docker:
在master和node安装docker，然后都加入harbor私有仓库
root@kubeadm-master1:~# apt update
root@kubeadm-master1:~# apt -y install apt-transport-https ca-certificates curl software-properties-common
root@kubeadm-master1:~# mkdir /etc/apt/keyrings
root@kubeadm-master1:~# curl -fsSL https://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
root@kubeadm-master1:~# echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://mirrors.aliyun.com/docker-ce/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
root@kubeadm-master1:~# sudo apt-get update
root@kubeadm-master1:~# sudo apt-get install ca-certificates curl gnupg lsb-release
`查看镜像版本`
root@kubeadm-master1:~# apt-cache madison docker-ce docker-ce-cli
`安装docker`
root@kubeadm-master1:~# apt-get install docker-ce=5:19.03.15~3-0~ubuntu-focal docker-ce-cli=5:19.03.15~3-0~ubuntu-focal
`修改配置信任harbor`
root@kubeadm-master1:~# vim /lib/systemd/system/docker.service
.......
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --insecure-registry harbor.stangj.com
.......
root@kubeadm-master1:~# systemctl daemon-reload && systemctl restart docker.service
`加解析`
root@kubeadm-master1:~# vim /etc/hosts
.......
192.168.139.231 harbor.stangj.com
`登录验证`
root@kubeadm-master1:~# docker login harbor.stangj.com
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
所有节点安装kubelet kubeadm kubectl:
所有节点配置阿里云仓库地址并安装相关组件，node节点可选安装kubectl配置阿里云镜像的kubernetes源(用于安装kubelet kubeadm kubectl命令)
`centos`
[root@kubeadm-master1 src]# cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
exclude=kube*
EOF
[root@kubeadm-master1 src]# yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
[root@kubeadm-master1 src]# systemctl enable --now kubelet.service
`ubuntu`
apt update && apt-get install -y apt-transport-https
curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add -
cat >> /etc/apt/sources.list.d/kubernetes.list << EOF
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF
apt-get update
#`查看kubadm版本`
apt-cache madison kubeadm
#下载软件
apt install -y kubelet=1.20.5-00 kubeadm=1.20.5-00 kubectl=1.20.5-00
master节点运行kubeadm init初始化命令
-
在三台master中任意一台master进行集群初始化，而且集群初始化只需要初始化一次。
kubeadm命令使用:
Usage:
kubeadm [command]
Available Commands:
certs Commands related to handling kubernetes certificates
completion #bash命令补全，需要安装bash-completion
# mkdir /data/scripts -p
# kubeadm completion bash > /data/scripts/kubeadm_completion.sh
# source /data/scripts/kubeadm_completion.sh
# vim /etc/profile
......
source /data/scripts/kubeadm_completion.sh
config #管理kubeadm集群的配置,该配置保留在集群的configMap中
# kubeadm config print init-defaults
help Help about any command
init #初始化一个Kubernetes控制平面
join #将节点加入到已经存在的kBs master
kubeconfig Kubeconfig file utilities
reset #还原使用kubeadm init或者kubeadm join对系统产生的环境变化
token #管理token
upgrade #升级k8s版本
version #查看版本信息
kubeadm init命令简介:
https://kubernetes.io/zh-cn/docs/reference/setup-tools/kubeadm/ #命令使用
https://kubernetes.io/zh-cn/docs/reference/setup-tools/kubeadm/kubeadm-init/#集群初始化
root@kubeadm-master1:~# kubeadm init --help
--apiserver-advertise-address string #K8S API Server将要监听的监听的本机IP
--apiserver-bind-port int32 #API Server绑定的端口,默认为6443
--apiserver-cert-extra-sans strings #可选的证书额外信息，用于指定APIServer的服务器证书。可以是IP地址也可以是DNS名称。
--cert-dir string ##证书的存储路径，缺省路径为/etc/ kubernetes/pki
--certificate-key string ##定义一个用于加密kubeadm-certs secret中的控制平台证书的密钥
--config string ##kubeadm#配置文件的路径
--control-plane-endpoint string ##定义一个用于加密kubeadm-certs secret中的控制平台证书的密钥
--cri-socket string ##要连接的CRI(容器运行时接口，container Runtime Interface，简称CRI)套接字的路径，如果为空，则kubeadm将尝试自动检测此值，"仅当安装了多个CRI或具有非标准CRI插槽时，才使用此选项"
--dry-run ##不要应用任何更改，只是输出将要执行的揉作，其实就是测试运行。
--feature-gates string ##—组用来描述各种功能特性的键值(key=value)对，选项是:
IPv6Dualstack=true [ false ( ALPHA - default=false)
--ignore-preflight-errors strings ##可以忽略检查过程中出现的错误信息,比如忽珞swap，如果为all就忽略所有
--image-repository string ##设置一个镜像仓库,默认为k8s.gcr.io
--node-name string ##指定node节点名称
--kubernetes-version string ##指定安装k8s版本，默认为stable-1
--pod-network-cidr string ##设置pod ip地址范围
--service-cidr string ##设置service网络地址范围
--service-dns-domain string ##设置k8s内部域名，默认为cluster.local，会有相应的DNS服务(kube-dns/coredns)解析生成的域名记录
--skip-certificate-key-print ##不打印用于加密的key信息
--skip-phases strings ##要跳过哪些阶段
--skip-token-print ##跳过打印token信息
--token string ##指定token
--token-ttl ##指定token过期时间，默认为24小时，e为永不过期
--upload-certs ##更新证书
`全局可选`
--add-dir-header ##如果为true，在日志头部添加日志目录
--log-file string ##如果不为空，将使用此日志文件
--log-file-max-size uint ##设置日志文件的最大大小,单位为兆,默认为180e兆，e为没有限制
--rootfs string ##宿主机的根路径,也就是绝对路径
--skip-headers ##如果为true，在log日志里面不显示标题前缀
--skip-log-headers ##如果为true，在log日志里里不显示标题
"常用参数"
--apiserver-advertise-address
--apiserver-bind-port
--control-plane-endpoint 192.168.139.33
--ignore-preflight-errors=all
--image-repository
--kubernetes-version
--pod-network-cidr 10.100.0.0/16
--service-cidr 10.200.0.0/16
--service-dns-domain stangjedu.local
验证当前kubeadm版本:
root@kubeadm-master1:~# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.5", GitCommit:"6b1d87acf3c8253c123756b9e61dac642678305f", GitTreeState:"clean", BuildDate:"2021-03-18T01:08:27Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}
准备镜像:
root@kubeadm-master1:~# kubeadm config images list --kubernetes-version v1.20.5
k8s.gcr.io/kube-apiserver:v1.20.5
k8s.gcr.io/kube-controller-manager:v1.20.5
k8s.gcr.io/kube-scheduler:v1.20.5
k8s.gcr.io/kube-proxy:v1.20.5
k8s.gcr.io/pause:3.2
k8s.gcr.io/etcd:3.4.13-0
k8s.gcr.io/coredns:1.7.0
master节点下载镜像:
-
建议提前在master节点下载镜像以减少安装等待时间，但是镜像默认使用Google的镜像仓库，所以国内无法直接下载，但是可以通过阿里云的镜像仓库把镜像先提前下载下来，可以避免后期因镜像下载异常而导致k8s部署异常。
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.20.5
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.20.5
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.20.5
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.20.5
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.4.13-0
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.7.0
验证当前镜像:
root@kubeadm-master1:~# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy v1.20.5 5384b1650507 2 years ago 118MB
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver v1.20.5 d7e24aeb3b10 2 years ago 122MB
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager v1.20.5 6f0c3da8c99e 2 years ago 116MB
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler v1.20.5 8d13f1db8bfb 2 years ago 47.3MB
registry.cn-hangzhou.aliyuncs.com/google_containers/etcd 3.4.13-0 0369cf4303ff 3 years ago 253MB
registry.cn-hangzhou.aliyuncs.com/google_containers/coredns 1.7.0 bfe3a36ebd25 3 years ago 45.2MB
registry.cn-hangzhou.aliyuncs.com/google_containers/pause 3.2 80d28bedfe5d 3 years ago 683kB
单节点初始化:
root@kubeadm-master1:~# kubeadm init --apiserver-advertise-address=192.168.139.30 --apiserver-bind-port=6443 --kubernetes-version=v1.20.5 --pod-network-cidr=10.100.0.0/16 --service-cidr=10.200.0.0/16 --service-dns-domain=stangjedu.local --image-repository=registry.cn-hangzhou.aliyuncs.com/google_containers --ignore-preflight-errors=swap
"记住下面这些信息后面要用"
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.139.30:6443 --token kkn9pu.6mv4rv7punn8sepn \
--discovery-token-ca-cert-hash sha256:54264c3338ee813f0045a6d3a80bd037cd821d8a891f200b8b25a6d18132c527
root@kubeadm-master1:~# mkdir -p $HOME/.kube
root@kubeadm-master1:~# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
root@kubeadm-master1:~# sudo chown $(id -u):$(id -g) $HOME/.kube/config
root@kubeadm-master1:~# kubectl get node
NAME STATUS ROLES AGE VERSION
kubeadm-master1 NotReady control-plane,master 18m v1.20.5
root@kubeadm-master1:~# mkdir net_flannel
root@kubeadm-master1:~# cd net_flannel/
root@kubeadm-master1:~/net_flannel# wget https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
#flannel网址：https://quay.io/repository/coreos/flannel?tab=tags&tag=v0.14.0-rc1
"#修改kube-flannel.yml网络地址为 10.100.0.0/16"
root@kubeadm-master1:~/net_flannel# kubectl apply -f kube-flannel.yml
root@kubeadm-master1:~/net_flannel# kubectl get pod -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-flannel kube-flannel-ds-2mlkh 0/1 CrashLoopBackOff 3 2m28s
kube-system coredns-54d67798b7-qj7rw 0/1 ContainerCreating 0 50m
kube-system coredns-54d67798b7-qxptp 0/1 ContainerCreating 0 50m
kube-system etcd-kubeadm-master1 1/1 Running 0 51m
kube-system kube-apiserver-kubeadm-master1 1/1 Running 0 51m
kube-system kube-controller-manager-kubeadm-master1 1/1 Running 0 51m
kube-system kube-proxy-n7l2l 1/1 Running 0 50m
kube-system kube-scheduler-kubeadm-master1 1/1 Running 0 51m
root@kubeadm-master1:~/net_flannel# kubectl get node
NAME STATUS ROLES AGE VERSION
kubeadm-master1 Ready control-plane,master 51m v1.20.5
允许master节点部署pod:
增加node节点
root@node1:~# kubeadm join 192.168.139.30:6443 --token kkn9pu.6mv4rv7punn8sepn --discovery-token-ca-cert-hash sha256:54264c3338ee813f0045a6d3a80bd037cd821d8a891f200b8b25a6d18132c527
root@kubeadm-master1:~/net_flannel# kubectl get nodes
NAME STATUS ROLES AGE VERSION
kubeadm-master1 Ready control-plane,master 63m v1.20.5
node1 Ready <none> 110s v1.20.5
测试通信
root@kubeadm-master1:~# kubectl run test-pod1 --image=alpine sleep 400000
pod/test-pod1 created
root@kubeadm-master1:~# kubectl run test-pod2 --image=alpine sleep 400000
pod/test-pod2 created
root@kubeadm-master1:~# kubectl run test-pod3 --image=alpine sleep 400000
pod/test-pod3 created
root@kubeadm-master1:~/net_flannel# kubectl get pod
NAME READY STATUS RESTARTS AGE
test-pod1 1/1 Running 0 5m12s
test-pod2 1/1 Running 0 5m8s
test-pod3 1/1 Running 0 5m2s
root@kubeadm-master1:~# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
test-pod1 1/1 Running 0 6m29s 10.100.1.2 node1 <none> <none>
test-pod2 1/1 Running 0 6m25s 10.100.1.4 node1 <none> <none>
test-pod3 1/1 Running 0 6m19s 10.100.1.3 node1 <none> <none>
`进入test-pod1`
root@kubeadm-master1:~# kubectl exec -it test-pod1 sh
/ # ping 10.100.1.4
PING 10.100.1.4 (10.100.1.4): 56 data bytes
64 bytes from 10.100.1.4: seq=0 ttl=64 time=0.139 ms
^C
--- 10.100.1.4 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.139/0.139/0.139 ms
/ # ping 192.168.139.30
PING 192.168.139.30 (192.168.139.30): 56 data bytes
64 bytes from 192.168.139.30: seq=0 ttl=63 time=0.400 ms
^C
--- 192.168.139.30 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.400/0.400/0.400 ms
/ #
高可用master初始化:
-
基于keepalived实现高可用VIP,通过haproxy实现kube-apiserver的反向代理， 然后将对kube apiserver的管理请求转发至多台k8s master以实现管理端高可用。
2.8.1）基于命令初始化高可用master方式:
基于命令初始化高可用master方式:
root@kubeadm-master1:~# kubeadm init --apiserver-advertise-address=192.168.139.30 --control-plane-endpoint 192.168.139.33 --apiserver-bind-port=6443 --kubernetes-version=v1.20.5 --pod-network-cidr=10.100.0.0/16 --service-cidr=10.200.0.0/16 --service-dns-domain=stangjedu.local --image-repository=registry.cn-hangzhou.aliyuncs.com/google_containers --ignore-preflight-errors=swap
#保存下面信息
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
You can now join any number of control-plane nodes by copying certificate authorities
and service account keys on each node and then running the following as root:
kubeadm join 192.168.139.33:6443 --token 18r0za.ivh3ee0zir1fv5a7 \
--discovery-token-ca-cert-hash sha256:40bd3f5c3113405fd2678f5277c1588a851cc3b5743e5ec524d9cc00fb4bd561 \
--control-plane
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.139.33:6443 --token 18r0za.ivh3ee0zir1fv5a7 \
--discovery-token-ca-cert-hash sha256:40bd3f5c3113405fd2678f5277c1588a851cc3b5743e5ec524d9cc00fb4bd561
添加认证
root@kubeadm-master1:~# mkdir -p $HOME/.kube
root@kubeadm-master1:~# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
root@kubeadm-master1:~# sudo chown $(id -u):$(id -g) $HOME/.kube/config
配置kube-config文件及网络组件:
-
无论使用命令还是文件初始化的k8s环境，无论是单机还是集群，需要配置一下kube -config文件及网络组件。
root@kubeadm-master1:~# mkdir net_flannel
root@kubeadm-master1:~# cd net_flannel/
root@kubeadm-master1:~/net_flannel# wget https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
#flannel网址：https://quay.io/repository/coreos/flannel?tab=tags&tag=v0.14.0-rc1
"#修改kube-flannel.yml网络地址为 10.100.0.0/16"
root@kubeadm-master1:~/net_flannel# kubectl apply -f kube-flannel.yml
root@kubeadm-master1:~/net_flannel# kubectl get pod -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-flannel kube-flannel-ds-cnksq 1/1 Running 0 11s
kube-system coredns-54d67798b7-f56f4 0/1 Running 0 5m6s
kube-system coredns-54d67798b7-mdzq9 0/1 Running 0 5m6s
kube-system etcd-kubeadm-master1 1/1 Running 0 5m1s
kube-system kube-apiserver-kubeadm-master1 1/1 Running 1 5m1s
kube-system kube-controller-manager-kubeadm-master1 1/1 Running 0 5m1s
kube-system kube-proxy-cm4c7 1/1 Running 0 5m7s
kube-system kube-scheduler-kubeadm-master1 1/1 Running 0 5m1s
kube-config文件:
-
Kube-config文件中包含kube-apiserver地址及相关认证信息
2.9.2）当前maste生成证书用于添加新控制节点:
root@kubeadm-master1:~/net_flannel# kubeadm init phase upload-certs --upload-certs
................
afd8185d034e3d230e9d912cb1c906663ae718621d21a621d014aa5a453f3c99 #这个密钥用于添加master节点
添加节点到k8s集群:
-
将其他的maser节点及node节点分别添加到k8集群中。
master节 点2
root@kubeadm-master2:~# kubeadm join 192.168.139.33:6443 --token 18r0za.ivh3ee0zir1fv5a7 \
--discovery-token-ca-cert-hash sha256:40bd3f5c3113405fd2678f5277c1588a851cc3b5743e5ec524d9cc00fb4bd561 \
--control-plane --certificate-key afd8185d034e3d230e9d912cb1c906663ae718621d21a621d014aa5a453f3c99
添加node节点:
root@node1:~# kubeadm join 192.168.139.33:6443 --token 18r0za.ivh3ee0zir1fv5a7 \
--discovery-token-ca-cert-hash sha256:40bd3f5c3113405fd2678f5277c1588a851cc3b5743e5ec524d9cc00fb4bd561
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
You can now join any number of control-plane nodes by copying certificate authorities
and service account keys on each node and then running the following as root:
kubeadm join 192.168.139.33:6443 --token 42k5a2.w3d4qbstpenopstf \
--discovery-token-ca-cert-hash sha256:9ad76c06497c9bee93116b2102db5476042ded9d0712509c1b379993c44e3e02 \
--control-plane
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.139.33:6443 --token 42k5a2.w3d4qbstpenopstf \
--discovery-token-ca-cert-hash sha256:9ad76c06497c9bee93116b2102db5476042ded9d0712509c1b379993c44e3e02 \
--control-plane --certificate-key fe5dab310354348d8b545b55398661acad3ac9a00fff94dd1462d79c309b1cd7
验证当前node状态:
root@kubeadm-master1:~# kubectl get nodes
NAME STATUS ROLES AGE VERSION
kubeadm-master1 Ready control-plane,master 12m v1.20.5
kubeadm-master2 Ready control-plane,master 51s v1.20.5
node1 Ready <none> 8m23s v1.20.5
node2 Ready <none> 8m27s v1.20.5
验证当前证书状态:
root@kubeadm-master1:~# kubectl get csr
NAME AGE SIGNERNAME REQUESTOR CONDITION
csr-6r6qk 2m35s kubernetes.io/kube-apiserver-client-kubelet system:bootstrap:42k5a2 Approved,Issued
csr-vhhjs 10m kubernetes.io/kube-apiserver-client-kubelet system:bootstrap:42k5a2 Approved,Issued
csr-wscbw 10m kubernetes.io/kube-apiserver-client-kubelet system:bootstrap:42k5a2 Approved,Issued
k8s创建容器并测试内部网络:
root@kubeadm-master1:~# kubectl run test-pod1 --image=alpine sleep 400000
pod/test-pod1 created
root@kubeadm-master1:~# kubectl run test-pod2 --image=alpine sleep 400000
pod/test-pod2 created
root@kubeadm-master1:~# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
test-pod1 1/1 Running 0 26s 10.100.1.2 node2 <none> <none>
test-pod2 1/1 Running 0 21s 10.100.2.2 node1 <none> <none>
验证外部网络:
root@kubeadm-master1:~# kubectl exec -it test-pod1 sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ # ping 10.100.2.2
PING 10.100.2.2 (10.100.2.2): 56 data bytes
64 bytes from 10.100.2.2: seq=0 ttl=62 time=1.285 ms
^X^X64 bytes from 10.100.2.2: seq=1 ttl=62 time=0.716 ms
^[[A64 bytes from 10.100.2.2: seq=2 ttl=62 time=0.481 ms
/ # ping www.baidu.com
PING www.baidu.com (110.242.68.3): 56 data bytes
64 bytes from 110.242.68.3: seq=0 ttl=127 time=10.850 ms
kubeadm init创建k8s集群流程:
https://kubernetes.io/zh/docs/reference/setup-tools/kubeadm/kubeadm-init/
文章评论