Kubernetes 环境安装 面向生产:kubeadmin 环境准备
IP
主机名
用途
192.168.1.1
k8s-master
master、etcd
192.168.1.2
k8s-node-1
node1
192.168.1.3
k8s-node-2
node2
准备工作 设置三台机器的主机名 1 2 3 4 5 6 hostnamectl --static set-hostname k8s-master hostnamectl --static set-hostname k8s-node-1 hostnamectl --static set-hostname k8s-node-2
设置 hosts 在三台机器上设置 hosts,均执行如下命令:
1 2 3 4 echo '192.168.1.1 k8s-master 192.168.1.1 etcd 192.168.1.2 k8s-node-1 192.168.1.3 k8s-node-2' >> /etc/hosts
关闭 selinux 1 2 setenforce 0 sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux
网络配置 master 机
1 2 3 4 5 6 7 8 9 10 firewall-cmd --permanent --add-port=6443/tcp firewall-cmd --permanent --add-port=2379-2380/tcp firewall-cmd --permanent --add-port=10250/tcp firewall-cmd --permanent --add-port=10251/tcp firewall-cmd --permanent --add-port=10252/tcp firewall-cmd --permanent --add-port=10255/tcp firewall-cmd --reload modprobe br_netfilter echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptablessysctl -w net.ipv4.ip_forward=1
node 机
1 2 3 4 5 6 7 firewall-cmd --permanent --add-port=10250/tcp firewall-cmd --permanent --add-port=10255/tcp firewall-cmd --permanent --add-port=30000-32767/tcp firewall-cmd --permanent --add-port=6783/tcp firewall-cmd --reload echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptablessysctl -w net.ipv4.ip_forward=1
安装 kubelet kubeadm kubectl 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 cat <<EOF > /etc/yum .repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https:// packages.cloud.google.com/yum/ repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https:// packages.cloud.google.com/yum/ doc/yum-key.gpg https:/ /packages.cloud.google.com/yum /doc/ rpm-package-key.gpg exclude=kube* EOF setenforce 0 sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/ selinux/config yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes systemctl enable kubelet && systemctl start kubelet
参考:https://kubernetes.io/zh/docs/setup/independent/install-kubeadm/#%E5%AE%89%E8%A3%85-kubeadm-kubelet-%E5%92%8C-kubectl
安装 etcd 服务
更改配置
创建集群 面向试用:minikube 指引:https://kubernetes.io/docs/tasks/tools/install-kubectl/
安装 VM 驱动
腾讯云 CVM 未能启用驱动启动 minikube,安装驱动的过程可忽略。
- VirtualBox
CentOS:虚拟机不支持内部 VM,未在物理机上尝试
1 2 3 4 5 6 7 8 9 cd /etc/yum.repos.d/wget http://download.virtualbox.org/virtualbox/rpm/rhel/virtualbox.repo yum update rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm yum install -y binutils gcc make patch libgomp glibc-headers glibc-devel kernel-headers kernel-devel dkms yum install -y VirtualBox-5.2 service vboxdrv start
MAC 上可以直接下载安装:https://www.virtualbox.org/wiki/Downloads
- KVM2
安装成功,但 minikube start
时未能尝试成功
https://github.com/kubernetes/minikube/blob/master/docs/drivers.md#kvm2-driver
1 2 3 4 5 6 yum -y install libvirt-daemon-kvm qemu-kvm curl -Lo docker-machine-driver-kvm2 https://storage.googleapis.com/minikube/releases/latest/docker-machine-driver-kvm2 \ && chmod +x docker-machine-driver-kvm2 \ && sudo cp docker-machine-driver-kvm2 /usr/local /bin/ \ && rm docker-machine-driver-kvm2
附:查看虚拟机网络以及使用不同的网络启动 minikube
1 2 3 4 5 yum -y install libvirt virt-install bridge-utils virsh net-list --all minikube start --vm-driver kvm2 --kvm-network minikube-net
- 其他驱动 指引:https://github.com/kubernetes/minikube/blob/master/docs/drivers.md#kvm-driver
安装 kubectl 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 cat <<EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg EOF yum install -y kubectl brew install kubernetes-cli curl -Lo kubectl http://storage.googleapis.com/kubernetes-release/release/v1.5.1/bin/darwin/amd64/kubectl && chmod +x kubectl && sudo mv kubectl /usr/local /bin/ kubectl version systemctl enable kubelet.service systemctl start kubelet.service
安装 minikube 指引:https://github.com/kubernetes/minikube/releases
1 2 3 4 5 6 7 8 curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.29.0/minikube-linux-amd64 \ && chmod +x minikube \ && sudo cp minikube /usr/local /bin/ \ && rm minikube brew cask install minikube
运行 (使用 KVM2 驱动)运行之前
1 2 3 4 5 6 7 8 9 10 11 systemctl enable libvirtd && systemctl start libvirtd ls -l /var/run/libvirt/libvirt-sock vi /etc/libvirt/libvirtd.conf
拉取 k8s 相关镜像 原本在 minikube start
命令中会自动拉取镜像,但是因为众所周知的原因,我们无法成功拉取到镜像。这里可以使用阿里 docker 容器镜像服务:https://dev.aliyun.com/search.html
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 minikube ssh docker pull registry.cn-hangzhou.aliyuncs.com/k8sth/kube-apiserver-amd64:v1.10.0 docker pull registry.cn-hangzhou.aliyuncs.com/k8sth/kube-controller-manager-amd64:v1.10.0 docker pull registry.cn-hangzhou.aliyuncs.com/k8sth/kube-scheduler-amd64:v1.10.0 docker pull registry.cn-hangzhou.aliyuncs.com/k8sth/etcd-amd64:3.1.12 docker pull registry.cn-shenzhen.aliyuncs.com/kubernetes_google/kubernetes-dashboard-amd64:v1.10.0 docker tag registry.cn-hangzhou.aliyuncs.com/k8sth/kube-apiserver-amd64:v1.10.0 k8s.gcr.io/kube-apiserver-amd64:v1.10.0 docker tag registry.cn-hangzhou.aliyuncs.com/k8sth/kube-controller-manager-amd64:v1.10.0 k8s.gcr.io/kube-controller-manager-amd64:v1.10.0 docker tag registry.cn-hangzhou.aliyuncs.com/k8sth/kube-scheduler-amd64:v1.10.0 k8s.gcr.io/kube-scheduler-amd64:v1.10.0 docker tag registry.cn-hangzhou.aliyuncs.com/k8sth/etcd-amd64:3.1.12 k8s.gcr.io/etcd-amd64:3.1.12 docker tag registry.cn-shenzhen.aliyuncs.com/kubernetes_google/kubernetes-dashboard-amd64:v1.10.0 k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.0
启动 Kubenetes 指引:https://kubernetes.io/docs/setup/minikube/#quickstart
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 https_proxy=http://myproxy.com:8080 minikube start \ --docker-env http_proxy=http://myproxy.com:8080 \ --docker-env https_proxy=http://myproxy.com:8080 \ --docker-env no_proxy=192.168.99.0/24 --vm-driver=none minikube logs -f minikube start \ --network-plugin=cni \ --container-runtime=containerd \ --bootstrapper=kubeadm --vm-driver=none kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml kubectl proxy kubectl run hello-minikube --image=registry.cn-hangzhou.aliyuncs.com/acs/echoserver:1.4 --port=8080 kubectl expose deployment hello-minikube --type =NodePort kubectl get pod kubectl get pods --all-namespaces kubectl describe --namespace=kube-system po kubernetes-dashboard-6f4cfc5d87-d647l kubectl logs kubernetes-dashboard-6f4cfc5d87-x976v --namespace=kube-system kubectl cluster-info curl $(minikube service hello-minikube --url) kubectl delete services hello-minikube kubectl delete deployment hello-minikube minikube stop
向 kubernetes 添加账号 https://github.com/kubernetes/dashboard/wiki/Creating-sample-user
相关问题 failed to create kubelet: misconfiguration: kubelet cgroup driver: “cgroupfs” is different from docker cgr…driver: “systemd” 该问题因为 docker 的配置和 kubelet 的配置不一致导致。
使用 docker info 打印 docker 信息:
1 2 3 4 5 6 [root@VM_0_16_centos kubelet.service.d]# docker info | grep Driver WARNING : Usage of loopback devices is strongly discouraged for production use . Use ` WARNING : You're not using the default seccomp profile Storage Driver: devicemapper Logging Driver: journald Cgroup Driver: systemd
而查看 kubelet 服务的启动参数(--cgroup-driver
),其设置为 cgroupfs
:
1 2 3 4 5 6 7 8 9 10 11 [root@VM_0_16_centos kubelet.service.d]# more /etc/systemd/system/kubelet.service.d/10-kubeadm.conf [Unit] Wants =docker.socket[Service] ExecStart= ExecStart =/usr/bin/kubelet --hostname-override =minikube --cluster-domain =cluster.local --cgroup-driver =cgroupfs --authorization-mode =Webhook --client-ca-file =/var/lib/minikube/certs/ca.crt --fail-swap-on =false --kubeconfig =/etc/kubernetes/kubelet.conf --bootstrap-kubeconfig =/etc/kubernetes/bootstrap-kubelet.conf --pod-manifest-path =/etc/kubernetes/manifests --allow-privileged =true --cluster-dns =10.96.0.10 --cadvisor-port =0 [Install]
此时,修改 docker 的服务参数(vi /usr/lib/systemd/system/docker.service
),将其中的 --exec-opt native.cgroupdriver
参数值改为 cgroupfs
。
然后,重启 docker,重启 kubelet
1 systemctl daemon-reload && systemctl restart docker && systemctl restart kubelet
[ERROR FileContent–proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1 解决方案:
1 2 echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptablesecho 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables
Unable to update cni config: No networks found in /etc/cni/net.d 1 yum install -y kubernetes-cni
参考 https://www.jianshu.com/p/a42eeb66a19c