当前位置：首页 > 行业知识 > CentOS7.9+Kubernetes1.29.2+Docker25.0.3高可用集群二进制部署

行业知识

CentOS7.9+Kubernetes1.29.2+Docker25.0.3高可用集群二进制部署

2026-11-03 20:54:52 来源：互联网转载

CentOS7.9+Kubernetes1.29.2+Docker25.0.3高可用集群二进制部署

Kubernetes高可用集群（Kubernetes1.29.2+Docker25.0.3）二进制部署
二进制软件部署flannel v0.22.3网络，使用的etcd是版本3，与之前使用版本2不同。查看官方文档进行了解。
截至北京时间2024年2月15日凌晨，k8s已经更新至1.29.2版。从v1.24起，Docker不能直接作为k8s的容器运行时。因为Docker庞大的生态和广泛的群众基础，我们可以通过补充安装cri-dockerd以满足容器运行时接口的条件。

��本关系

## 从kubernetes-server-linux-amd64.tar.gz解压后有kubeadm]# /opt/bin/kubernetes/kubeadm config images listregistry.k8s.io/kube-apiserver:v1.29.2registry.k8s.io/kube-controller-manager:v1.29.2registry.k8s.io/kube-scheduler:v1.29.2registry.k8s.io/kube-proxy:v1.29.2registry.k8s.io/coredns/coredns:v1.11.1registry.k8s.io/pause:3.9registry.k8s.io/etcd:3.5.10-0

一、总体规划

1.1 软件获取

所有软件均为开源软件，源文件从官方链接或官方镜像链接地址获取。
所有软件采用最新稳定版本（不包括主机操作系统）。

关于主机操作系统内核版本，参考相关说明指导进行升级。

1.1.1 centos 7.9

https://mirrors.tuna.tsinghua.edu.cn/centos/7.9.2009/isos/x86_64/

下载2009或2207其中任何一个版本都可以。

下载：https://mirrors.tuna.tsinghua.edu.cn/centos/7.9.2009/isos/x86_64/CentOS-7-x86_64-Minimal-2009.iso

下载：https://mirrors.tuna.tsinghua.edu.cn/centos/7.9.2009/isos/x86_64/CentOS-7-x86_64-Minimal-2207-02.iso

1.1.2 kubernetes v1.29.1

https://github.com/kubernetes/kubernetes/releases

下载：https://dl.k8s.io/v1.29.1/kubernetes-server-linux-amd64.tar.gz

1.1.3 docker 25.0.3

https://download.docker.com/linux/static/stable/x86_64/

下载：https://download.docker.com/linux/static/stable/x86_64/docker-25.0.3.tgz

1.1.4 cri-dockerd 0.3.9

https://github.com/Mirantis/cri-dockerd

下载：https://github.com/Mirantis/cri-dockerd/releases/download/v0.3.9/cri-dockerd-0.3.9.amd64.tgz

1.1.5 etcd v3.5.12

https://github.com/etcd-io/etcd/

下载：https://github.com/etcd-io/etcd/releases/download/v3.5.12/etcd-v3.5.12-linux-amd64.tar.gz

1.1.6 cfssl v1.6.4

https://github.com/cloudflare/cfssl/releases

下载：https://github.com/cloudflare/cfssl/releases/download/v1.6.4/cfssljson_1.6.4_linux_amd64

下载：https://github.com/cloudflare/cfssl/releases/download/v1.6.4/cfssl_1.6.4_linux_amd64

下载：https://github.com/cloudflare/cfssl/releases/download/v1.6.4/cfssl-certinfo_1.6.4_linux_amd64

1.1.7 flannel v0.24.2

https://github.com/flannel-io/flannel

下载：https://github.com/flannel-io/flannel/releases/download/v0.24.2/flannel-v0.24.2-linux-amd64.tar.gz

1.1.8 cni-plugins v1.4.0、cni-plugin v1.4.0

下载：https://github.com/containernetworking/plugins/releases/download/v1.4.0/cni-plugins-linux-amd64-v1.4.0.tgz

下载：https://github.com/flannel-io/cni-plugin/releases/download/v1.4.0-flannel1/cni-plugin-flannel-linux-amd64-v1.4.0-flannel1.tgz

1.19 nginx 1.24.0

http://nginx.org/en/download.html

下载：http://nginx.org/download/nginx-1.24.0.tar.gz

1.2 集群主机规划

1.2.1 虚拟机安装及主机设置

主机名与hosts本地解析

Hostname	Host IP	Docker IP	Role
k8s1.29.1-1	192.168.26.31	10.26.31.1/24	master&worker、etcd、docker、flannel
k8s1.29.1-2	192.168.26.32	10.26.32.1/24	master&worker、etcd、docker、flannel
k8s1.29.1-3	192.168.26.33	10.26.33.1/24	master&worker、etcd、docker、flannel

[root@vm51 ~]# hostname      ## 修改：# hostnamectl set-hostname k8s-31k8s-31src]# cat /etc/hosts127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4::1         localhost localhost.localdomain localhost6 localhost6.localdomain6192.168.26.31 k8s1.29.1-1 vm31 etcd_node1 k8s-31192.168.26.32 k8s1.29.1-2 vm32 etcd_node2 k8s-32192.168.26.33 k8s1.29.1-3 vm33 etcd_node3 k8s-33

关闭防火墙

 ~]# systemctl disable --now firewalld Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. ~]# firewall-cmd --state not running

关闭SELINUX

 ~]# sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux/config ~]# reboot ~]# sestatus SELinux status:                 disabled

修改SELinux配置需要重启操作系统。

关闭交换分区

 ~]# sed -ri 's/.*swap.*/#&/' /etc/fstab ~]# swapoff -a && sysctl -w vm.swappiness=0 ~]# cat /etc/fstab #UUID=2ee3ecc4-e9e9-47c5-9afe-7b7b46fd6bab swap                    swap    defaults        0 0 ~]# free -m              total        used        free      shared  buff/cache   availableMem:           3770         171        3460          19         138        3402Swap:             0           0           0

配置ulimit

 ~]# ulimit -SHn 65535 ~]# cat > /etc/security/limits.conf* soft nofile 655360* hard nofile 131072* soft nproc 655350* hard nproc 655350* soft memlock unlimited* hard memlock unlimitedEOF

1.2.2 ipvs管理工具安装及模块加载

获取离线安装rpm包，参考：附：安装ipvs工具

~]# yum install ipvsadm ipset sysstat conntrack libseccomp -y ...~]# cat >> /etc/modules-load.d/ipvs.conf   "data-root": "/data/docker",  "storage-driver": "overlay2",  "insecure-registries": ["harbor.oss.com:32402"],  "registry-mirrors": ["https://5gce61mx.mirror.aliyuncs.com"],  "bip": "10.26.31.1/24",  "exec-opts": ["native.cgroupdriver=systemd"],  "live-restore": true}## k8s-32：]# cat /etc/docker/daemon.json{  "data-root": "/data/docker",  "storage-driver": "overlay2",  "insecure-registries": ["harbor.oss.com:32402"],  "registry-mirrors": ["https://5gce61mx.mirror.aliyuncs.com"],  "bip": "10.26.32.1/24",  "exec-opts": ["native.cgroupdriver=systemd"],  "live-restore": true}## k8s-33：]# cat /etc/docker/daemon.json{  "data-root": "/data/docker",  "storage-driver": "overlay2",  "insecure-registries": ["harbor.oss.com:32402"],  "registry-mirrors": ["https://5gce61mx.mirror.aliyuncs.com"],  "bip": "10.26.33.1/24",  "exec-opts": ["native.cgroupdriver=systemd"],  "live-restore": true}  "CN": "kubernetes",  "key": {    "algo": "rsa",    "size": 2048  },  "names": [    {      "C": "CN",      "ST": "Beijing",      "L": "Beijing",      "O": "Kubernetes",      "OU": "Kubernetes-manual"    }  ],  "ca": {    "expiry": "876000h"  }}EOF  "signing": {    "default": {      "expiry": "876000h"    },    "profiles": {      "kubernetes": {        "usages": [            "signing",            "key encipherment",            "server auth",            "client auth"        ],        "expiry": "876000h"      }    }  }}EOF  "CN": "etcd",  "key": {    "algo": "rsa",    "size": 2048  },  "names": [    {      "C": "CN",      "ST": "Beijing",      "L": "Beijing",      "O": "etcd",      "OU": "Etcd Security"    }  ],  "ca": {    "expiry": "876000h"  }}EOF  "CN": "etcd",  "key": {    "algo": "rsa",    "size": 2048  },  "names": [    {      "C": "CN",      "ST": "Beijing",      "L": "Beijing",      "O": "etcd",      "OU": "Etcd Security"    }  ]}EOF  "CN": "kube-apiserver",  "key": {    "algo": "rsa",    "size": 2048  },  "names": [    {      "C": "CN",      "ST": "Beijing",      "L": "Beijing",      "O": "Kubernetes",      "OU": "Kubernetes-manual"    }  ]}EOF  "CN": "kubernetes",  "key": {     "algo": "rsa",     "size": 2048  },  "ca": {    "expiry": "876000h"  }}EOF  "CN": "front-proxy-client",  "key": {     "algo": "rsa",     "size": 2048  }}EOF  "CN": "system:kube-controller-manager",  "key": {    "algo": "rsa",    "size": 2048  },  "names": [    {      "C": "CN",      "ST": "Beijing",      "L": "Beijing",      "O": "system:kube-controller-manager",      "OU": "Kubernetes-manual"    }  ]}EOF  "CN": "admin",  "key": {    "algo": "rsa",    "size": 2048  },  "names": [    {      "C": "CN",      "ST": "Beijing",      "L": "Beijing",      "O": "system:masters",      "OU": "Kubernetes-manual"    }  ]}EOF  "CN": "system:kube-scheduler",  "key": {    "algo": "rsa",    "size": 2048  },  "names": [    {      "C": "CN",      "ST": "Beijing",      "L": "Beijing",      "O": "system:kube-scheduler",      "OU": "Kubernetes-manual"    }  ]}EOF  "CN": "system:kube-proxy",  "key": {    "algo": "rsa",    "size": 2048  },  "names": [    {      "C": "CN",      "ST": "Beijing",      "L": "Beijing",      "O": "system:kube-proxy",      "OU": "Kubernetes-manual"    }  ]}EOF"Network": "10.26.0.0/16", "Backend": {"Type": "host-gw"}}

h46.4、flannel解压、创建软连接/h4 blockquote pk8s-31：/p /blockquote pre ]# cd /opt/appk8s-31 app]# mkdir /opt/bin/flannel-v0.24.2-linux-amd64k8s-31 app]# tar zxvf flannel-v0.24.2-linux-amd64.tar.gz -C /opt/bin/flannel-v0.24.2-linux-amd64flanneldmk-docker-opts.shREADME.mdk8s-31 app]# scp -r /opt/bin/flannel-v0.24.2-linux-amd64 root@k8s-32:/opt/bin/.k8s-31 app]# scp -r /opt/bin/flannel-v0.24.2-linux-amd64 root@k8s-33:/opt/bin/.## 创建软连接]# ln -s /opt/bin/flannel-v0.24.2-linux-amd64/flanneld /opt/bin/flanneld/pre h46.5、添加配置文件/h4 pmarkk8s-31：--public-ip=192.168.26.31；k8s-32：--public-ip=192.168.26.32；k8s-33：--public-ip=192.168.26.33/mark/p pre cat /opt/cfg/kube-flanneld.conf 容器）：从k8s-31主机访问所有主机上的容器

k8s-31 ~]# ping 10.26.31.2 -c 3PING 10.26.31.2 (10.26.31.2) 56(84) bytes of data.64 bytes from 10.26.31.2: icmp_seq=1 ttl=64 time=0.034 ms64 bytes from 10.26.31.2: icmp_seq=2 ttl=64 time=0.031 ms64 bytes from 10.26.31.2: icmp_seq=3 ttl=64 time=0.027 ms--- 10.26.31.2 ping statistics ---3 packets transmitted, 3 received, 0% packet loss, time 1999msrtt min/avg/max/mdev = 0.027/0.030/0.034/0.007 msk8s-31 ~]# ping 10.26.32.2 -c 3PING 10.26.32.2 (10.26.32.2) 56(84) bytes of data.64 bytes from 10.26.32.2: icmp_seq=1 ttl=63 time=0.285 ms64 bytes from 10.26.32.2: icmp_seq=2 ttl=63 time=0.643 ms64 bytes from 10.26.32.2: icmp_seq=3 ttl=63 time=0.528 ms--- 10.26.32.2 ping statistics ---3 packets transmitted, 3 received, 0% packet loss, time 1999msrtt min/avg/max/mdev = 0.285/0.485/0.643/0.150 msk8s-31 ~]# ping 10.26.33.2 -c 3PING 10.26.33.2 (10.26.33.2) 56(84) bytes of data.64 bytes from 10.26.33.2: icmp_seq=1 ttl=63 time=0.457 ms64 bytes from 10.26.33.2: icmp_seq=2 ttl=63 time=0.522 ms64 bytes from 10.26.33.2: icmp_seq=3 ttl=63 time=0.513 ms--- 10.26.33.2 ping statistics ---3 packets transmitted, 3 received, 0% packet loss, time 2000msrtt min/avg/max/mdev = 0.457/0.497/0.522/0.034 ms

6.8.4 从容器访问百度(具备外网访问时测试)

[root@330e9de087a0 /]# ping www.baidu.com -c 3PING www.a.shifen.com (183.2.172.185) 56(84) bytes of data.64 bytes from www.baidu.com (183.2.172.185): icmp_seq=1 ttl=127 time=9.85 ms64 bytes from www.baidu.com (183.2.172.185): icmp_seq=2 ttl=127 time=8.76 ms64 bytes from www.baidu.com (183.2.172.185): icmp_seq=3 ttl=127 time=9.81 ms--- www.a.shifen.com ping statistics ---3 packets transmitted, 3 received, 0% packet loss, time 2003msrtt min/avg/max/mdev = 8.758/9.472/9.849/0.517 ms

6.8.5 k8s-32、k8s-33上的测试

步骤同上。

七、k8s核心组件

7.1 准备

 ## 方法一：解压软件包中指定的软件，并将运行软件放到/opt/bin目录k8s-31 app]# tar -xf kubernetes-server-linux-amd64.tar.gz  --strip-components=3 -C /opt/bin kubernetes/server/bin/kube{let,ctl,-apiserver,-controller-manager,-scheduler,-proxy}k8s-31 app]# ls/opt/bin/kube*kube-apiserverkube-controller-managerkubectlkubeletkube-proxykube-scheduler## 解压全部k8s-31 app]# tar -zxvf kubernetes-server-linux-amd64.tar.gzk8s-31 app]# mv kubernetes /opt/bin/kubernetes-v1.29.2k8s-31 app]# rm -rf /opt/bin/kubernetes-v1.29.2/{addons,kubernetes-src.tar.gz,LICENSES}k8s-31 app]# ll /opt/bin/kubernetes-v1.29.2总用量 0drwxr-xr-x 3 root root 17 1月  18 00:09 serverk8s-31 app]# rm -rf /opt/bin/kubernetes-v1.29.2/server/bin/*.tark8s-31 app]# rm -rf /opt/bin/kubernetes-v1.29.2/server/bin/*_tagk8s-31 app]# ll /opt/bin/kubernetes-v1.29.2/server/bin/总用量 717160-rwxr-xr-x 1 root root  61333504 2月  14 18:57 apiextensions-apiserver-rwxr-xr-x 1 root root  48246784 2月  14 18:57 kubeadm-rwxr-xr-x 1 root root  58761216 2月  14 18:57 kube-aggregator-rwxr-xr-x 1 root root 123719680 2月  14 18:57 kube-apiserver-rwxr-xr-x 1 root root 118349824 2月  14 18:57 kube-controller-manager-rwxr-xr-x 1 root root  49704960 2月  14 18:57 kubectl-rwxr-xr-x 1 root root  48087040 2月  14 18:57 kubectl-convert-rwxr-xr-x 1 root root 111812608 2月  14 18:57 kubelet-rwxr-xr-x 1 root root   1613824 2月  14 18:57 kube-log-runner-rwxr-xr-x 1 root root  55263232 2月  14 18:57 kube-proxy-rwxr-xr-x 1 root root  55943168 2月  14 18:57 kube-scheduler-rwxr-xr-x 1 root root   1536000 2月  14 18:57 mounterk8s-31 app]# ln -s /opt/bin/kubernetes-v1.29.2/server/bin /opt/bin/kubernetesk8s-31 app]# ln -s /opt/bin/kubernetes-v1.29.2/server/bin/kubectl /usr/local/bin/kubectlk8s-31 app]# /opt/bin/kubernetes/kubelet --versionKubernetes v1.29.2  ## 复制软件到其他节点，并创建软链接]# scp -r /opt/bin/kubernetes-v1.29.2/ root@k8s-32:/opt/bin/]# scp -r /opt/bin/kubernetes-v1.29.2/ root@k8s-33:/opt/bin/## 创建软链接同上

7.2 Nginx高可用方案

使用 nginx方案，kube-apiserver中的配置为： --server=https://127.0.0.1:8443

安装开发工具包（参考附件），编译安装nginx

## 在一台有开发环境的主机上进行编译安装，然后复制到集群节点app]# tar xvf nginx-1.24.0.tar.gzapp]# cd nginx-1.24.0nginx-1.24.0]# ./configure --with-stream --without-http --without-http_uwsgi_module --without-http_scgi_module --without-http_fastcgi_module......nginx-1.24.0]# make && make install......nginx-1.24.0]# ls -l /usr/local/nginxdrwxr-xr-x 2 root root 333 2月  15 09:08 confdrwxr-xr-x 2 root root  40 2月  15 09:08 htmldrwxr-xr-x 2 root root   6 2月  15 09:08 logsdrwxr-xr-x 2 root root  19 2月  15 09:08 sbin

nginx配置文件/usr/local/nginx/conf/kube-nginx.con

 # 写入nginx配置文件cat > /usr/local/nginx/conf/kube-nginx.conf   "kind": "KubeletConfiguration",  "apiVersion": "kubelet.config.k8s.io/v1beta1",  "authentication": {    "x509": {      "clientCAFile": "/opt/cert/ca.pem"    },    "webhook": {      "enabled": true,      "cacheTTL": "2m0s"    },    "anonymous": {      "enabled": false    }  },  "authorization": {    "mode": "Webhook",    "webhook": {      "cacheAuthorizedTTL": "5m0s",      "cacheUnauthorizedTTL": "30s"    }  },  "address": "192.168.26.31",  "port": 10250,  "readOnlyPort": 10255,  "cgroupDriver": "systemd",                      "hairpinMode": "promiscuous-bridge",  "serializeImagePulls": false,  "clusterDomain": "cluster.local.",  "clusterDNS": ["10.168.0.2"]}EOF  "name": "cbr0",  "cniVersion": "0.3.1",  "plugins": [    {      "type": "flannel",      "delegate": {        "hairpinMode": true,        "isDefaultGateway": true      }    },    {      "type": "portmap",      "capabilities": {        "portMappings": true      }    }  ]}}        name: tmp-dir      - name: ca-ssl        hostPath:          path: /opt/cert# 4 修改为image镜像image: registry.aliyuncs.com/google_containers/metrics-server:v0.7.0

上一篇：多人服务器怎么找人?

下一篇：前端实现水印效果的多种方案

行业知识

CentOS7.9+Kubernetes1.29.2+Docker25.0.3高可用集群二进制部署