kubeadm 1.13 高可用

使用kubeadm安装配置kubernetes HA，etcd外放，使用VIP做故障转移，其中不同的是，这个VIP还做了域名解析。此前尝试使用keepalived+haproxy发现有一些问题。

恰巧内部有内部的DNS服务器，这样一来，两台master通过域名和VIP做转移，实现了kubernetes的高可用，如下图环境如下：

[root@linuxea.com ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.1",

[root@linuxea.com ~]# docker -v
Docker version 18.06.1-ce, build e68fc7a

先决条件

• hosts

cat >> /etc/hosts << EOF
172.25.50.13 master-0.k8s.org
172.25.50.14 master-1.k8s.org
127.0.0.1 www.linuxea.com
EOF

• hostname

[root@linuxea.com ~]# hostnamectl set-hostname  master-0.k8s.org
[root@host-172-25-50-13 ~]# echo "DHCP_HOSTNAME=master-0.k8s.org" >> /etc/sysconfig/network-scripts/ifcfg-eth0 
[root@linuxea.com ~]# systemctl restart network

修改后重启下，在重启前，关闭防火墙

[root@linuxea.com ~]# systemctl disable iptables firewalld.service 
[root@linuxea.com ~]# systemctl stop iptables firewalld.service 
[root@linuxea.com ~]# reboot

当然了，我这里此前安装的就是iptables

• swap

[root@master-0 ~]# swapoff -a

可以打开ipvs

cat << EOF > /etc/sysconfig/modules/ipvs.modules 
#!/bin/bash
ipvs_modules_dir="/usr/lib/modules/`uname -r`/kernel/net/netfilter/ipvs"
for i in `ls $ipvs_modules_dir | sed  -r 's#(.*).ko.*#1#'`; do
    /sbin/modinfo -F filename $i  &> /dev/null
    if [ $? -eq 0 ]; then
        /sbin/modprobe $i
    fi
done
EOF

chmod +x /etc/sysconfig/modules/ipvs.modules 
bash /etc/sysconfig/modules/ipvs.modules

echo "1" > /proc/sys/net/bridge/bridge-nf-call-iptables

确保模块安装,nf_nat_ipv4也是关键之一

[root@master-0 ~]# lsmod|grep ip_vs
ip_vs_wrr              16384  0 
ip_vs_wlc              16384  0 
ip_vs_sh               16384  0 
ip_vs_sed              16384  0 
ip_vs_rr               16384  0 
ip_vs_pe_sip           16384  0 
nf_conntrack_sip       28672  1 ip_vs_pe_sip
ip_vs_ovf              16384  0 
ip_vs_nq               16384  0 
ip_vs_mh               16384  0 
ip_vs_lc               16384  0 
ip_vs_lblcr            16384  0 
ip_vs_lblc             16384  0 
ip_vs_ftp              16384  0 
ip_vs_fo               16384  0 
ip_vs_dh               16384  0 
ip_vs                 151552  30 ip_vs_wlc,ip_vs_rr,ip_vs_dh,ip_vs_lblcr,ip_vs_sh,ip_vs_ovf,ip_vs_fo,ip_vs_nq,ip_vs_lblc,ip_vs_pe_sip,ip_vs_wrr,ip_vs_lc,ip_vs_mh,ip_vs_sedip_vs_ftp
nf_nat                 32768  2 nf_nat_ipv4,ip_vs_ftp
nf_conntrack          135168  8 xt_conntrack,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,nf_conntrack_sip,nf_conntrack_netlink,ip_vs
libcrc32c              16384  4 nf_conntrack,nf_nat,xfs,ip_vs

• 如果觉得上面的步骤太繁琐，可以参考这里的脚本：

curl -Lk https://raw.githubusercontent.com/marksugar/kubeadMHA/master/systeminit/chenage_hostname|bash
curl -Lk https://raw.githubusercontent.com/marksugar/kubeadMHA/master/systeminit/ip_vs_a_init|bash

keepalived

• install keepalived

 bash <(curl -s  https://raw.githubusercontent.com/marksugar/lvs/master/keepliaved/install.sh|more)

如下：

输入Master或者BACKUP和VIP

[root@master-0 ~]# bash <(curl -s  https://raw.githubusercontent.com/marksugar/lvs/master/keepliaved/install.sh|more)
You install role MASTER/BACKUP ?
         please enter(block letter):MASTER
Please enter the use VIP: 172.25.50.15

安装kubeadm

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
        https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
setenforce 0
yum install -y kubelet kubeadm
systemctl enable kubelet && systemctl start kubelet