怎么使用二进制搭建k8s想必大家一定听说过它的地狱难度

接下来 我来带领大家来搭建一下 来体验感受下有多难

etcd搭建

gitlab地址:   https://github.com/etcd-io/etcd

测试环境

192.168.32.11 master1 2C4G CentOS7.9 master kube-apiserver、kube-controller-manager、kube-scheduler、etcd
192.168.32.12 msater2 2C4G CentOS7.9 master kube-apiserver、kube-controller-manager、kube-scheduler、etcd
192.168.32.13 master3 2C4G CentOS7.9 master kube-apiserver、kube-controller-manager、kube-scheduler、etcd
192.168.32.14 node1 2C4G CentOS7.9 worker kubelet、kube-proxy
192.168.32.15 node2 2C4G CentOS7.9 worker kubelet、kube-proxy
192.168.32.16 node2 2C4G CentOS7.9 worker kubelet、kube-proxy
192.168.32.17 proxy1 2C4G CentOS7.9 keepalived haproxy
192.168.32.18 proxy2 2C4G CentOS7.9 keepalived haproxy

修改主机名

hostnamectl set-hostname master1
hostnamectl set-hostname node1
hostnamectl set-hostname proxy1

基础配置

配置hosts解析

cat >> /etc/hosts << EOF 
192.168.32.11 master1 
192.168.32.12 master2 
192.168.32.13 master3 
192.168.32.14 node1 
192.168.32.15 node2 
192.168.32.16 node3 
192.168.32.17 proxy1 
192.168.32.18 proxy2 
EOF

systemctl stop firewalld && setenforce 0 && sed -i 's/^SELINUX=.\*/SELINUX=disabled/' /etc/selinux/config && systemctl disable firewalld

sed -ri '/^[^#]*swap/s@^@#@' /etc/fstab && swapoff -a

时间同步

yum install -y chrony
systemctl start chronyd
systemctl enable chronyd
chronyc sources

修改内核参数

cat > /etc/sysctl.d/k8s.conf << EOF
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system

ipvs模块配置

modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
lsmod | grep ip_vs
lsmod | grep nf_conntrack_ipv4
yum install -y ipvsadm

创建etcd证书

工具下载

unzip oldboyedu-cfssl-v1.6.5.zip 
yum install rename
rename -v "s/_1.6.5_linux_amd64//g" cfssl*
mv cfssl* /usr/local/bin/
chmod +x /usr/local/bin/cfssl*
ll /usr/local/bin/cfssl*

配置ca请求文件

cd /data/work
cat > etcd-ca-csr.json <<EOF
{
"CN": "etcd",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "Beijing",
"L": "Beijing",
"O": "etcd",
"OU": "Etcd Security"
}
],
"ca": {
"expiry": "876000h"
}
}
EOF

生成证书

mkdir etcd 
cfssl gencert -initca ca-csr.json | cfssljson -bare etcd/ca

配置ca证书策略

vim ca-config.json
{
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"kubernetes": {
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
],
"expiry": "87600h"
}
}
}
}

配置etcd请求csr文件

vim etcd-csr.json
{
"CN": "etcd",
"hosts": [
"127.0.0.1",
"192.168.32.11",
"192.168.32.12",
"192.168.32.13"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [{
"C": "CN",
"ST": "Hubei",
"L": "Wuhan",
"O": "k8s",
"OU": "system"
}]
}

生成证书

 cfssl gencert -ca=etcd/ca.pem -ca-key=etcd/ca-key.pem -conetes etcd-csr.json | cfssljson -bare etcd

部署etcd集群

wget https://github.com/etcd-io/etcd/releases/download/v3.4.13/etcd-v3.4.13-linux-amd64.tar.gz
tar -xf etcd-v3.4.13-linux-amd64.tar.gz 
cp -p etcd-v3.4.13-linux-amd64/etcd* /usr/local/bin/
rsync -vaz etcd-v3.4.13-linux-amd64/etcd* master2:/usr/local/bin/
rsync -vaz etcd-v3.4.13-linux-amd64/etcd* master3:/usr/local/bin/

创建配置文件

#[Member]
ETCD_NAME="etcd1"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.32.11:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.32.11:2379,http://127.0.0.1:2379"

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.32.11:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.32.11:2379"
ETCD_INITIAL_CLUSTER="etcd1=https://192.168.32.11:2380,etcd2=https://192.168.32.12:2380,etcd3=https://192.168.32.1
3:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

ETCD_NAME：节点名称，集群中唯一
ETCD_DATA_DIR：数据目录
ETCD_LISTEN_PEER_URLS：集群通信监听地址
ETCD_LISTEN_CLIENT_URLS：客户端访问监听地址
ETCD_INITIAL_ADVERTISE_PEER_URLS：集群通告地址
ETCD_ADVERTISE_CLIENT_URLS：客户端通告地址
ETCD_INITIAL_CLUSTER：集群节点地址
ETCD_INITIAL_CLUSTER_TOKEN：集群Token
ETCD_INITIAL_CLUSTER_STATE：加入集群的当前状态，new是新集群，existing表示加入已有集群

创建启动服务文件

拷贝相关文件

cp etcd-key.pem /etc/etcd/ssl
cp etcd.pem /etc/etcd/ssl
cp etcd/* /etc/etcd/ssl
cp etcd.conf /etc/etcd
mkdir -p /var/lib/etcd/default.etcd
for i in master2 master3;do rsync -vaz /etc/etcd/etcd.conf $i:/etc/etcd/;done
for i in master2 master3;do rsync -vaz /etc/etcd/ssl/* $i:/etc/etcd/ssl/;done
for i in master2 master3;do rsync -vaz /usr/lib/systemd/system/etcd.service $i:/usr/lib/systemd/system/;done
for i in master2 master3;do rsync -vaz /var/lib/etcd/default.etcd $i:/var/lib/etcd/default.etcd;done

编写启动文件

[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=-/etc/etcd/etcd.conf
WorkingDirectory=/var/lib/etcd/
ExecStart=/usr/local/bin/etcd \
--cert-file=/etc/etcd/ssl/etcd.pem \
--key-file=/etc/etcd/ssl/etcd-key.pem \
--trusted-ca-file=/etc/etcd/ssl/ca.pem \
--peer-cert-file=/etc/etcd/ssl/etcd.pem \
--peer-key-file=/etc/etcd/ssl/etcd-key.pem \
--peer-trusted-ca-file=/etc/etcd/ssl/ca.pem \
--peer-client-cert-auth \
--client-cert-auth
Restart=on-failure
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

其他两个节点修改配置文件的节点名称和ip 并且创建 /var/lib/etcd/default.etcd

启动etcd集群

mkdir -p /var/lib/etcd/default.etcd
systemctl daemon-reload
systemctl enable etcd.service
systemctl start etcd.service
systemctl status etcd

查看节点状态

 ETCDCTL_API=3 /usr/local/bin/etcdctl --write-out=table --cacert=/etc/etcd/ssl/ca.pem --cert=/etc/etcd/ssl/etcd.pem --key=/etc/etcd/ssl/etcd-key.pem --endpoints=https://192.168.32.11:2379,https://192.168.32.12:2379,https://192.168.32.13:2379 endpoint health

 +----------------------------+--------+------------+-------+
| ENDPOINT | HEALTH | TOOK | ERROR |
+----------------------------+--------+------------+-------+
| https://192.168.32.11:2379 | true | 7.708613ms | |
| https://192.168.32.12:2379 | true | 7.790347ms | |
| https://192.168.32.13:2379 | true | 9.038279ms | |
+----------------------------+--------+------------+-------+

下一章将配置k8s的组件安装~