-
基于 kubernetes+docker构建高可用、高性能的 web 、CICD集群
文章目录
- 一、项目架构图
- 二 、项目描述
- 三、项目环境
- 四、环境准备
- 五、项目步骤
- 1、设计整个集群的架构，规划好服务器的IP地址，搭建集群
- 2、部署ansible完成相关软件的自动化运维工作，部署防火墙服务器，部署堡垒机
- 3、部署nfs服务器，为整个web集群提供数据，让所有的web业务pod都去访问，通过pv、pvc和卷挂载实现
- 4、构建CI/CD环境，部署gitlab，Jenkins，harbor实现相关的代码发布，镜像制作，数据备份等流水线工作
- 5、将自己用go开发的web接口系统制作成镜像，部署到k8s里作为web应用；采用HPA技术，当cpu使用率达到50%的时候，进行水平扩缩，最小20个业务pod，最多40个业务pod
- 6、启动mysql的pod，为web业务提供数据库服务
- 7、使用探针(liveness、readiness、startup)的（httpget、exec）方法对web业务pod进行监控，一旦出现问题马上重启，增强业务pod的可靠性
- 8、使用ingress给web业务做负载均衡，使用dashboard对整个集群资源进行掌控
- 9、使用dashboard对整个集群资源进行掌控
- 10、安装zabbix和promethues对整个集群资源（cpu，内存，网络带宽，web服务，数据库服务，磁盘IO等）进行监控
- 11、使用测试软件ab对整个k8s集群和相关的服务器进行压力测试
一、项目架构图
二 、项目描述
模拟公司的web业务，部署k8s，web，MySQL，nfs，harbor，zabbix，Prometheus，gitlab，Jenkins，ansible环境，保障web业务的高可用，达到一个高负载的生产环境。
三、项目环境
CentOS 7.9，ansible 2.9.27，Docker 20.10.6，Docker Compose 2.18.1，Kubernetes 1.20.6，Calico 3.23，Harbor 2.4.1，nfs v4，metrics-server 0.6.0，ingress-nginx-controllerv1.1.0，kube-webhook-certgen-v1.1.0，MySQL 5.7.42，Dashboard v2.5.0，Prometheus 2.34.0，zabbix 5.0，Grafana 10.0.0，jenkinsci/blueocean，Gitlab-16.0.4-jh。- 1
四、环境准备
10台全新的Linux服务器，关闭firewalld和seLinux，配置静态ip地址，修改主机名，添加hosts解析
1、IP地址规划
server ip k8smaster 192.168.2.104 k8snode1 192.168.2.111 k8snode2 192.168.2.112 ansibe 192.168.2.119 nfs 192.168.2.121 gitlab 192.168.2.124 harbor 192.168.2.106 zabbix 192.168.2.117 firewalld 192.168.2.141 Bastionhost 192.168.2.140 2、关闭selinux和firewall
# 防火墙并且设置防火墙开启不启动 service firewalld stop && systemctl disable firewalld # 临时关闭seLinux setenforce 0 # 永久关闭seLinux sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config [root@k8smaster ~]# service firewalld stop Redirecting to /bin/systemctl stop firewalld.service [root@k8smaster ~]# systemctl disable firewalld Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. [root@k8smaster ~]# reboot [root@k8smaster ~]# getenforce Disabled- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
3、配置静态ip地址
cd /etc/sysconfig/network-scripts/ vim ifcfg-ens33 TYPE="Ethernet" BOOTPROTO="static" DEVICE="ens33" NAME="ens33" ONBOOT="yes" IPADDR="192.168.2.104" PREFIX=24 GATEWAY="192.168.2.1" DNS1=114.114.114.114 TYPE="Ethernet" BOOTPROTO="static" DEVICE="ens33" NAME="ens33" ONBOOT="yes" IPADDR="192.168.2.111" PREFIX=24 GATEWAY="192.168.2.1" DNS1=114.114.114.114 TYPE="Ethernet" BOOTPROTO="static" DEVICE="ens33" NAME="ens33" ONBOOT="yes" IPADDR="192.168.2.112" PREFIX=24 GATEWAY="192.168.2.1" DNS1=114.114.114.114- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
4、修改主机名
hostnamcectl set-hostname k8smaster hostnamcectl set-hostname k8snode1 hostnamcectl set-hostname k8snode2 #切换用户，重新加载环境 su - root [root@k8smaster ~]# [root@k8snode1 ~]# [root@k8snode2 ~]#- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
5、升级系统（可做可不做）
yum update -y- 1
6、添加hosts解析
vim /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.2.104 k8smaster 192.168.2.111 k8snode1 192.168.2.112 k8snode2- 1
- 2
- 3
- 4
- 5
- 6
- 7
五、项目步骤
1、设计整个集群的架构，规划好服务器的IP地址，搭建集群
# 1.互相之间建立免密通道 ssh-keygen # 一路回车 ssh-copy-id k8smaster ssh-copy-id k8snode1 ssh-copy-id k8snode2 # 2.关闭交换分区（Kubeadm初始化的时候会检测） # 临时关闭：swapoff -a # 永久关闭：注释swap挂载，给swap这行开头加一下注释 [root@k8smaster ~]# cat /etc/fstab # # /etc/fstab # Created by anaconda on Thu Mar 23 15:22:20 2023 # # Accessible filesystems, by reference, are maintained under '/dev/disk' # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info # /dev/mapper/centos-root / xfs defaults 0 0 UUID=00236222-82bd-4c15-9c97-e55643144ff3 /boot xfs defaults 0 0 /dev/mapper/centos-home /home xfs defaults 0 0 #/dev/mapper/centos-swap swap swap defaults 0 0 # 3.加载相关内核模块 modprobe br_netfilter echo "modprobe br_netfilter" >> /etc/profile cat > /etc/sysctl.d/k8s.conf <.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 EOF #重新加载，使配置生效 sysctl -p /etc/sysctl.d/k8s.conf # 为什么要执行modprobe br_netfilter？ # "modprobe br_netfilter"命令用于在Linux系统中加载br_netfilter内核模块。这个模块是Linux内# 核中的一个网络桥接模块，它允许管理员使用iptables等工具对桥接到同一网卡的流量进行过滤和管理。 # 因为要使用Linux系统作为路由器或防火墙，并且需要对来自不同网卡的数据包进行过滤、转发或NAT操作。 # 为什么要开启net.ipv4.ip_forward = 1参数？ # 要让Linux系统具有路由转发功能，需要配置一个Linux的内核参数net.ipv4.ip_forward。这个参数指# 定了Linux系统当前对路由转发功能的支持情况；其值为0时表示禁止进行IP转发；如果是1,则说明IP转发# 功能已经打开。 # 4.配置阿里云的repo源 yum install -y yum-utils yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo yum install -y yum-utils device-mapper-persistent-data lvm2 wget net-tools nfs-utils lrzsz gcc gcc-c++ make cmake libxml2-devel openssl-devel curl curl-devel unzip sudo ntp libaio-devel wget vim ncurses-devel autoconf automake zlib-devel python-devel epel-release openssh-server socat ipvsadm conntrack ntpdate telnet ipvsadm # 5.配置安装k8s组件需要的阿里云的repo源 [root@k8smaster ~]# vim /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ enabled=1 gpgcheck=0 # 6.配置时间同步 [root@k8smaster ~]# crontab -e * */1 * * * /usr/sbin/ntpdate cn.pool.ntp.org #重启crond服务 [root@k8smaster ~]# service crond restart # 7.安装docker服务 yum install docker-ce-20.10.6 -y # 启动docker，设置开机自启 systemctl start docker && systemctl enable docker.service # 8.配置docker镜像加速器和驱动 vim /etc/docker/daemon.json { "registry-mirrors":["https://rsbud4vc.mirror.aliyuncs.com","https://registry.docker-cn.com","https://docker.mirrors.ustc.edu.cn","https://dockerhub.azk8s.cn","http://hub-mirror.c.163.com"], "exec-opts": ["native.cgroupdriver=systemd"] } # 重新加载配置，重启docker服务 systemctl daemon-reload && systemctl restart docker # 9.安装初始化k8s需要的软件包 yum install -y kubelet-1.20.6 kubeadm-1.20.6 kubectl-1.20.6 # 设置kubelet开机启动 systemctl enable kubelet #注：每个软件包的作用 #Kubeadm: kubeadm是一个工具，用来初始化k8s集群的 #kubelet: 安装在集群所有节点上，用于启动Pod的 #kubectl: 通过kubectl可以部署和管理应用，查看各种资源，创建、删除和更新各种组件 # 10.kubeadm初始化k8s集群 # 把初始化k8s集群需要的离线镜像包上传到k8smaster、k8snode1、k8snode2机器上，然后解压 docker load -i k8simage-1-20-6.tar.gz # 把文件远程拷贝到node节点 root@k8smaster ~]# scp k8simage-1-20-6.tar.gz root@k8snode1:/root root@k8smaster ~]# scp k8simage-1-20-6.tar.gz root@k8snode2:/root # 查看镜像 [root@k8snode1 ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE registry.aliyuncs.com/google_containers/kube-proxy v1.20.6 9a1ebfd8124d 2 years ago 118MB registry.aliyuncs.com/google_containers/kube-scheduler v1.20.6 b93ab2ec4475 2 years ago 47.3MB registry.aliyuncs.com/google_containers/kube-controller-manager v1.20.6 560dd11d4550 2 years ago 116MB registry.aliyuncs.com/google_containers/kube-apiserver v1.20.6 b05d611c1af9 2 years ago 122MB calico/pod2daemon-flexvol v3.18.0 2a22066e9588 2 years ago 21.7MB calico/node v3.18.0 5a7c4970fbc2 2 years ago 172MB calico/cni v3.18.0 727de170e4ce 2 years ago 131MB calico/kube-controllers v3.18.0 9a154323fbf7 2 years ago 53.4MB registry.aliyuncs.com/google_containers/etcd 3.4.13-0 0369cf4303ff 2 years ago 253MB registry.aliyuncs.com/google_containers/coredns 1.7.0 bfe3a36ebd25 3 years ago 45.2MB registry.aliyuncs.com/google_containers/pause 3.2 80d28bedfe5d 3 years ago 683kB # 11.使用kubeadm初始化k8s集群 kubeadm config print init-defaults > kubeadm.yaml [root@k8smaster ~]# vim kubeadm.yaml apiVersion: kubeadm.k8s.io/v1beta2 bootstrapTokens: - groups: - system:bootstrappers:kubeadm:default-node-token token: abcdef.0123456789abcdef ttl: 24h0m0s usages: - signing - authentication kind: InitConfiguration localAPIEndpoint: advertiseAddress: 192.168.2.104 #控制节点的ip bindPort: 6443 nodeRegistration: criSocket: /var/run/dockershim.sock name: k8smaster #控制节点主机名 taints: - effect: NoSchedule key: node-role.kubernetes.io/master --- apiServer: timeoutForControlPlane: 4m0s apiVersion: kubeadm.k8s.io/v1beta2 certificatesDir: /etc/kubernetes/pki clusterName: kubernetes controllerManager: {} dns: type: CoreDNS etcd: local: dataDir: /var/lib/etcd imageRepository: registry.aliyuncs.com/google_containers # 需要修改为阿里云的仓库 kind: ClusterConfiguration kubernetesVersion: v1.20.6 networking: dnsDomain: cluster.local serviceSubnet: 10.96.0.0/12 podSubnet: 10.244.0.0/16 #指定pod网段，需要新增加这个 scheduler: {} #追加如下几行 --- apiVersion: kubeproxy.config.k8s.io/v1alpha1 kind: KubeProxyConfiguration mode: ipvs --- apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration cgroupDriver: systemd # 12.基于kubeadm.yaml文件初始化k8s [root@k8smaster ~]# kubeadm init --config=kubeadm.yaml --ignore-preflight-errors=SystemVerification mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config kubeadm join 192.168.2.104:6443 --token abcdef.0123456789abcdef \ --discovery-token-ca-cert-hash sha256:83421a7d1baa62269508259b33e6563e45fbeb9139a9c214cbe9fc107f07cb4c # 13.扩容k8s集群-添加工作节点 [root@k8snode1 ~]# kubeadm join 192.168.2.104:6443 --token abcdef.0123456789abcdef \ --discovery-token-ca-cert-hash sha256:83421a7d1baa62269508259b33e6563e45fbeb9139a9c214cbe9fc107f07cb4c [root@k8snode2 ~]# kubeadm join 192.168.2.104:6443 --token abcdef.0123456789abcdef \ --discovery-token-ca-cert-hash sha256:83421a7d1baa62269508259b33e6563e45fbeb9139a9c214cbe9fc107f07cb4c # 14.在k8smaster上查看集群节点状况 [root@k8smaster ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION k8smaster NotReady control-plane,master 2m49s v1.20.6 k8snode1 NotReady 19s v1.20.6 k8snode2 NotReady 14s v1.20.6 # 15.k8snode1,k8snode2的ROLES角色为空， 就表示这个节点是工作节点。 可以把k8snode1,k8snode2的ROLES变成work [root@k8smaster ~]# kubectl label node k8snode1 node-role.kubernetes.io/worker=worker node/k8snode1 labeled [root@k8smaster ~]# kubectl label node k8snode2 node-role.kubernetes.io/worker=worker node/k8snode2 labeled [root@k8smaster ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION k8smaster NotReady control-plane,master 2m43s v1.20.6 k8snode1 NotReady worker 2m15s v1.20.6 k8snode2 NotReady worker 2m11s v1.20.6 # 注意：上面状态都是NotReady状态，说明没有安装网络插件 # 16.安装kubernetes网络组件-Calico # 上传calico.yaml到k8smaster上，使用yaml文件安装calico网络插件 。 wget https://docs.projectcalico.org/v3.23/manifests/calico.yaml --no-check-certificate [root@k8smaster ~]# kubectl apply -f calico.yaml configmap/calico-config created customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created clusterrole.rbac.authorization.k8s.io/calico-kube-controllers created clusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers created clusterrole.rbac.authorization.k8s.io/calico-node created clusterrolebinding.rbac.authorization.k8s.io/calico-node created daemonset.apps/calico-node created serviceaccount/calico-node created deployment.apps/calico-kube-controllers created serviceaccount/calico-kube-controllers created poddisruptionbudget.policy/calico-kube-controllers created # 再次查看集群状态 [root@k8smaster ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION k8smaster Ready control-plane,master 5m57s v1.20.6 k8snode1 Ready worker 3m27s v1.20.6 k8snode2 Ready worker 3m22s v1.20.6 # STATUS状态是Ready，说明k8s集群正常运行了- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
- 111
- 112
- 113
- 114
- 115
- 116
- 117
- 118
- 119
- 120
- 121
- 122
- 123
- 124
- 125
- 126
- 127
- 128
- 129
- 130
- 131
- 132
- 133
- 134
- 135
- 136
- 137
- 138
- 139
- 140
- 141
- 142
- 143
- 144
- 145
- 146
- 147
- 148
- 149
- 150
- 151
- 152
- 153
- 154
- 155
- 156
- 157
- 158
- 159
- 160
- 161
- 162
- 163
- 164
- 165
- 166
- 167
- 168
- 169
- 170
- 171
- 172
- 173
- 174
- 175
- 176
- 177
- 178
- 179
- 180
- 181
- 182
- 183
- 184
- 185
- 186
- 187
- 188
- 189
- 190
- 191
- 192
- 193
- 194
- 195
- 196
- 197
- 198
- 199
- 200
- 201
- 202
- 203
- 204
- 205
- 206
- 207
- 208
- 209
- 210
- 211
- 212
- 213
- 214
- 215
- 216
- 217
- 218
- 219
- 220
- 221
- 222
- 223
- 224
- 225
- 226
- 227
- 228
- 229
- 230
- 231
- 232
- 233
- 234
- 235
- 236
- 237
- 238
- 239
- 240
- 241
- 242
- 243
- 244
- 245
- 246
- 247
- 248
2、部署ansible完成相关软件的自动化运维工作，部署防火墙服务器，部署堡垒机
# 1.建立免密通道 在ansible主机上生成密钥对 [root@ansible ~]# ssh-keygen -t ecdsa Generating public/private ecdsa key pair. Enter file in which to save the key (/root/.ssh/id_ecdsa): Created directory '/root/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_ecdsa. Your public key has been saved in /root/.ssh/id_ecdsa.pub. The key fingerprint is: SHA256:FNgCSDVk6i3foP88MfekA2UzwNn6x3kyi7V+mLdoxYE root@ansible The key's randomart image is: +---[ECDSA 256]---+ | ..+*o =. | | .o .* o. | | . +. . | | . . ..= E . | | o o +S+ o . | | + o+ o O + | | . . .= B X | | . .. + B.o | | ..o. +oo.. | +----[SHA256]-----+ [root@ansible ~]# cd /root/.ssh [root@ansible .ssh]# ls id_ecdsa id_ecdsa.pub # 2.上传公钥到所有服务器的root用户家目录下 # 所有服务器上开启ssh服务 ，开放22号端口，允许root用户登录 # 上传公钥到k8smaster [root@ansible .ssh]# ssh-copy-id -i id_ecdsa.pub root@192.168.2.104 /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "id_ecdsa.pub" The authenticity of host '192.168.2.104 (192.168.2.104)' can't be established. ECDSA key fingerprint is SHA256:l7LRfACELrI6mU2XvYaCz+sDBWiGkYnAecPgnxJxdvE. ECDSA key fingerprint is MD5:b6:f7:e1:c5:23:24:5c:16:1f:66:42:ba:80:a6:3c:fd. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@192.168.2.104's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'root@192.168.2.104'" and check to make sure that only the key(s) you wanted were added. # 上传公钥到k8snode [root@ansible .ssh]# ssh-copy-id -i id_ecdsa.pub root@192.168.2.111 /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "id_ecdsa.pub" The authenticity of host '192.168.2.111 (192.168.2.111)' can't be established. ECDSA key fingerprint is SHA256:l7LRfACELrI6mU2XvYaCz+sDBWiGkYnAecPgnxJxdvE. ECDSA key fingerprint is MD5:b6:f7:e1:c5:23:24:5c:16:1f:66:42:ba:80:a6:3c:fd. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@192.168.2.111's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'root@192.168.2.111'" and check to make sure that only the key(s) you wanted were added. [root@ansible .ssh]# ssh-copy-id -i id_ecdsa.pub root@192.168.2.112 /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "id_ecdsa.pub" The authenticity of host '192.168.2.112 (192.168.2.112)' can't be established. ECDSA key fingerprint is SHA256:l7LRfACELrI6mU2XvYaCz+sDBWiGkYnAecPgnxJxdvE. ECDSA key fingerprint is MD5:b6:f7:e1:c5:23:24:5c:16:1f:66:42:ba:80:a6:3c:fd. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@192.168.2.112's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'root@192.168.2.112'" and check to make sure that only the key(s) you wanted were added. # 验证是否实现免密码密钥认证 [root@ansible .ssh]# ssh root@192.168.2.121 Last login: Tue Jun 20 10:33:33 2023 from 192.168.2.240 [root@nfs ~]# exit 登出 Connection to 192.168.2.121 closed. [root@ansible .ssh]# ssh root@192.168.2.112 Last login: Tue Jun 20 10:34:18 2023 from 192.168.2.240 [root@k8snode2 ~]# exit 登出 Connection to 192.168.2.112 closed. [root@ansible .ssh]# # 3.安装ansible，在管理节点上 # 目前,只要机器上安装了 Python 2.6 或 Python 2.7 (windows系统不可以做控制主机),都可以运行Ansible. [root@ansible .ssh]# yum install epel-release -y [root@ansible .ssh]# yum install ansible -y [root@ansible ~]# ansible --version ansible 2.9.27 config file = /etc/ansible/ansible.cfg configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python2.7/site-packages/ansible executable location = /usr/bin/ansible python version = 2.7.5 (default, Oct 14 2020, 14:45:30) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] # 4.编写主机清单 [root@ansible .ssh]# cd /etc/ansible [root@ansible ansible]# ls ansible.cfg hosts roles [root@ansible ansible]# vim hosts ## 192.168.1.110 [k8smaster] 192.168.2.104 [k8snode] 192.168.2.111 192.168.2.112 [nfs] 192.168.2.121 [gitlab] 192.168.2.124 [harbor] 192.168.2.106 [zabbix] 192.168.2.117 # 测试 [root@ansible ansible]# ansible all -m shell -a "ip add"- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
a、部署堡垒机
仅需两步快速安装 JumpServer：
准备一台 2核4G （最低）且可以访问互联网的 64 位 Linux 主机；
以 root 用户执行如下命令一键安装 JumpServer。curl -sSL https://resource.fit2cloud.com/jumpserver/jumpserver/releases/latest/download/quick_start.sh | bash- 1
b、部署firewall服务器
# 关闭虚拟机，增加一块网卡（ens37） # 编写脚本实现SNAT_DNAT功能 [root@firewalld ~]# cat snat_dnat.sh #!/bin/bash # open route echo 1 >/proc/sys/net/ipv4/ip_forward # stop firewall systemctl stop firewalld systemctl disable firewalld # clear iptables rule iptables -F iptables -t nat -F # enable snat iptables -t nat -A POSTROUTING -s 192.168.2.0/24 -o ens33 -j MASQUERADE #内网来的192.168.2.0网段过来的ip地址全部伪装（替换）为ens33接口的公网ip地址,好处就是不需要考虑ens33接口的ip地址是多少，你是哪个ip地址，我就伪装成哪个ip地址 # enable dnat iptables -t nat -A PREROUTING -d 192.168.0.169 -i ens33 -p tcp --dport 2233 -j DNAT --to-destination 192.168.2.104:22 # open web 80 iptables -t nat -A PREROUTING -d 192.168.0.169 -i ens33 -p tcp --dport 80 -j DNAT --to-destination 192.168.2.104:80 # web服务器上操作 [root@k8smaster ~]# cat open_app.sh #!/bin/bash # open ssh iptables -t filter -A INPUT -p tcp --dport 22 -j ACCEPT # open dns iptables -t filter -A INPUT -p udp --dport 53 -s 192.168.2.0/24 -j ACCEPT # open dhcp iptables -t filter -A INPUT -p udp --dport 67 -j ACCEPT # open http/https iptables -t filter -A INPUT -p tcp --dport 80 -j ACCEPT iptables -t filter -A INPUT -p tcp --dport 443 -j ACCEPT # open mysql iptables -t filter -A INPUT -p tcp --dport 3306 -j ACCEPT # default policy DROP iptables -t filter -P INPUT DROP # drop icmp request iptables -t filter -A INPUT -p icmp --icmp-type 8 -j DROP- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
3、部署nfs服务器，为整个web集群提供数据，让所有的web业务pod都去访问，通过pv、pvc和卷挂载实现
# 1.搭建好nfs服务器 [root@nfs ~]# yum install nfs-utils -y # 建议k8s集群内的所有的节点都安装nfs-utils软件，因为节点服务器里创建卷需要支持nfs网络文件系统 [root@k8smaster ~]# yum install nfs-utils -y [root@k8smaster ~]# service nfs restart Redirecting to /bin/systemctl restart nfs.service [root@k8smaster ~]# ps aux |grep nfs root 87368 0.0 0.0 0 0 ? S< 16:49 0:00 [nfsd4_callbacks] root 87374 0.0 0.0 0 0 ? S 16:49 0:00 [nfsd] root 87375 0.0 0.0 0 0 ? S 16:49 0:00 [nfsd] root 87376 0.0 0.0 0 0 ? S 16:49 0:00 [nfsd] root 87377 0.0 0.0 0 0 ? S 16:49 0:00 [nfsd] root 87378 0.0 0.0 0 0 ? S 16:49 0:00 [nfsd] root 87379 0.0 0.0 0 0 ? S 16:49 0:00 [nfsd] root 87380 0.0 0.0 0 0 ? S 16:49 0:00 [nfsd] root 87381 0.0 0.0 0 0 ? S 16:49 0:00 [nfsd] root 96648 0.0 0.0 112824 988 pts/0 S+ 17:02 0:00 grep --color=auto nfs # 2.设置共享目录 [root@nfs ~]# vim /etc/exports [root@nfs ~]# cat /etc/exports /web 192.168.2.0/24(rw,no_root_squash,sync) # 3.新建共享目录和index.html [root@nfs ~]# mkdir /web [root@nfs ~]# cd /web [root@nfs web]# echo "welcome to changsha" >index.html [root@nfs web]# ls index.html [root@nfs web]# ll -d /web drwxr-xr-x. 2 root root 24 6月 18 16:46 /web # 4.刷新nfs或者重新输出共享目录 [root@nfs ~]# exportfs -r #输出所有共享目录 [root@nfs ~]# exportfs -v #显示输出的共享目录 /web 192.168.2.0/24(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash) # 5.重启nfs服务并且设置nfs开机自启 [root@nfs web]# systemctl restart nfs && systemctl enable nfs Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service. # 6.在k8s集群里的任意一个节点服务器上测试能否挂载nfs服务器共享的目录 [root@k8snode1 ~]# mkdir /node1_nfs [root@k8snode1 ~]# mount 192.168.2.121:/web /node1_nfs 您在 /var/spool/mail/root 中有新邮件 [root@k8snode1 ~]# df -Th|grep nfs 192.168.2.121:/web nfs4 17G 1.5G 16G 9% /node1_nfs # 7.取消挂载 [root@k8snode1 ~]# umount /node1_nfs # 8.创建pv使用nfs服务器上的共享目录 [root@k8smaster pv]# vim nfs-pv.yml [root@k8smaster pv]# cat nfs-pv.yml apiVersion: v1 kind: PersistentVolume metadata: name: pv-web labels: type: pv-web spec: capacity: storage: 10Gi accessModes: - ReadWriteMany storageClassName: nfs # pv对应的名字 nfs: path: "/web" # nfs共享的目录 server: 192.168.2.121 # nfs服务器的ip地址 readOnly: false # 访问模式 [root@k8smaster pv]# kubectl apply -f nfs-pv.yml persistentvolume/pv-web created [root@k8smaster pv]# kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pv-web 10Gi RWX Retain Available nfs 5s # 9.创建pvc使用pv [root@k8smaster pv]# vim nfs-pvc.yml [root@k8smaster pv]# cat nfs-pvc.yml apiVersion: v1 kind: PersistentVolumeClaim metadata: name: pvc-web spec: accessModes: - ReadWriteMany resources: requests: storage: 1Gi storageClassName: nfs #使用nfs类型的pv [root@k8smaster pv]# kubectl apply -f pvc-nfs.yaml persistentvolumeclaim/sc-nginx-pvc created [root@k8smaster pv]# kubectl apply -f nfs-pvc.yml persistentvolumeclaim/pvc-web created [root@k8smaster pv]# kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE pvc-web Bound pv-web 10Gi RWX nfs 6s # 10.创建pod使用pvc [root@k8smaster pv]# vim nginx-deployment.yaml [root@k8smaster pv]# cat nginx-deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: volumes: - name: sc-pv-storage-nfs persistentVolumeClaim: claimName: pvc-web containers: - name: sc-pv-container-nfs image: nginx imagePullPolicy: IfNotPresent ports: - containerPort: 80 name: "http-server" volumeMounts: - mountPath: "/usr/share/nginx/html" name: sc-pv-storage-nfs [root@k8smaster pv]# kubectl apply -f nginx-deployment.yaml deployment.apps/nginx-deployment created [root@k8smaster pv]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-deployment-76855d4d79-2q4vh 1/1 Running 0 42s 10.244.185.194 k8snode2nginx-deployment-76855d4d79-mvgq7 1/1 Running 0 42s 10.244.185.195 k8snode2 nginx-deployment-76855d4d79-zm8v4 1/1 Running 0 42s 10.244.249.3 k8snode1 # 11.测试访问 [root@k8smaster pv]# curl 10.244.185.194 welcome to changsha [root@k8smaster pv]# curl 10.244.185.195 welcome to changsha [root@k8smaster pv]# curl 10.244.249.3 welcome to changsha [root@k8snode1 ~]# curl 10.244.185.194 welcome to changsha [root@k8snode1 ~]# curl 10.244.185.195 welcome to changsha [root@k8snode1 ~]# curl 10.244.249.3 welcome to changsha [root@k8snode2 ~]# curl 10.244.185.194 welcome to changsha [root@k8snode2 ~]# curl 10.244.185.195 welcome to changsha [root@k8snode2 ~]# curl 10.244.249.3 welcome to changsha # 12.修改内容 [root@nfs web]# echo "hello,world" >> index.html [root@nfs web]# cat index.html welcome to changsha hello,world # 13.再次访问 [root@k8snode1 ~]# curl 10.244.249.3 welcome to changsha hello,world - 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
- 111
- 112
- 113
- 114
- 115
- 116
- 117
- 118
- 119
- 120
- 121
- 122
- 123
- 124
- 125
- 126
- 127
- 128
- 129
- 130
- 131
- 132
- 133
- 134
- 135
- 136
- 137
- 138
- 139
- 140
- 141
- 142
- 143
- 144
- 145
- 146
- 147
- 148
- 149
- 150
- 151
- 152
- 153
- 154
- 155
- 156
- 157
- 158
- 159
- 160
- 161
- 162
- 163
- 164
- 165
- 166
- 167
- 168
- 169
- 170
- 171
- 172
- 173
- 174
- 175
- 176
- 177
- 178
- 179
4、构建CI/CD环境，部署gitlab，Jenkins，harbor实现相关的代码发布，镜像制作，数据备份等流水线工作
a、部署gitlab
# 部署gitlab https://gitlab.cn/install/ [root@localhost ~]# hostnamectl set-hostname gitlab [root@localhost ~]# su - root su - root 上一次登录：日 6月 18 18:28:08 CST 2023从 192.168.2.240pts/0 上 [root@gitlab ~]# cd /etc/sysconfig/network-scripts/ [root@gitlab network-scripts]# vim ifcfg-ens33 [root@gitlab network-scripts]# service network restart Restarting network (via systemctl): [ 确定 ] [root@gitlab network-scripts]# sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config [root@gitlab network-scripts]# service firewalld stop && systemctl disable firewalld Redirecting to /bin/systemctl stop firewalld.service Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service. Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. [root@gitlab network-scripts]# reboot [root@gitlab ~]# getenforce Disabled # 1.安装和配置必须的依赖项 yum install -y curl policycoreutils-python openssh-server perl # 2.配置极狐GitLab 软件源镜像 [root@gitlab ~]# curl -fsSL https://packages.gitlab.cn/repository/raw/scripts/setup.sh | /bin/bash ==> Detected OS centos ==> Add yum repo file to /etc/yum.repos.d/gitlab-jh.repo [gitlab-jh] name=JiHu GitLab baseurl=https://packages.gitlab.cn/repository/el/$releasever/ gpgcheck=0 gpgkey=https://packages.gitlab.cn/repository/raw/gpg/public.gpg.key priority=1 enabled=1 ==> Generate yum cache for gitlab-jh ==> Successfully added gitlab-jh repo. To install JiHu GitLab, run "sudo yum/dnf install gitlab-jh". [root@gitlab ~]# yum install gitlab-jh -y Thank you for installing JiHu GitLab! GitLab was unable to detect a valid hostname for your instance. Please configure a URL for your JiHu GitLab instance by setting `external_url` configuration in /etc/gitlab/gitlab.rb file. Then, you can start your JiHu GitLab instance by running the following command: sudo gitlab-ctl reconfigure For a comprehensive list of configuration options please see the Omnibus GitLab readme https://jihulab.com/gitlab-cn/omnibus-gitlab/-/blob/main-jh/README.md Help us improve the installation experience, let us know how we did with a 1 minute survey: https://wj.qq.com/s2/10068464/dc66 [root@gitlab ~]# vim /etc/gitlab/gitlab.rb external_url 'http://myweb.first.com' [root@gitlab ~]# gitlab-ctl reconfigure Notes: Default admin account has been configured with following details: Username: root Password: You didn't opt-in to print initial root password to STDOUT. Password stored to /etc/gitlab/initial_root_password. This file will be cleaned up in first reconfigure run after 24 hours. NOTE: Because these credentials might be present in your log files in plain text, it is highly recommended to reset the password following https://docs.gitlab.com/ee/security/reset_user_password.html#reset-your-root-password. gitlab Reconfigured! # 查看密码 [root@gitlab ~]# cat /etc/gitlab/initial_root_password # WARNING: This value is valid only in the following conditions # 1. If provided manually (either via `GITLAB_ROOT_PASSWORD` environment variable or via `gitlab_rails['initial_root_password']` setting in `gitlab.rb`, it was provided before database was seeded for the first time (usually, the first reconfigure run). # 2. Password hasn't been changed manually, either via UI or via command line. # # If the password shown here doesn't work, you must reset the admin password following https://docs.gitlab.com/ee/security/reset_user_password.html#reset-your-root-password. Password: Al5rgYomhXDz5kNfDl3y8qunrSX334aZZxX5vONJ05s= # NOTE: This file will be automatically deleted in the first reconfigure run after 24 hours. # 可以登录后修改语言为中文 # 用户的profile/preferences # 修改密码 [root@gitlab ~]# gitlab-rake gitlab:env:info System information System: Proxy: no Current User: git Using RVM: no Ruby Version: 3.0.6p216 Gem Version: 3.4.13 Bundler Version:2.4.13 Rake Version: 13.0.6 Redis Version: 6.2.11 Sidekiq Version:6.5.7 Go Version: unknown GitLab information Version: 16.0.4-jh Revision: c2ed99db36f Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: PostgreSQL DB Version: 13.11 URL: http://myweb.first.com HTTP Clone URL: http://myweb.first.com/some-group/some-project.git SSH Clone URL: git@myweb.first.com:some-group/some-project.git Elasticsearch: no Geo: no Using LDAP: no Using Omniauth: yes Omniauth Providers: GitLab Shell Version: 14.20.0 Repository storages: - default: unix:/var/opt/gitlab/gitaly/gitaly.socket GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
- 111
- 112
- 113
- 114
- 115
- 116
- 117
- 118
b、部署Jenkins
# Jenkins部署到k8s里 # 1.安装git软件 [root@k8smaster jenkins]# yum install git -y # 2.下载相关的yaml文件 [root@k8smaster jenkins]# git clone https://github.com/scriptcamp/kubernetes-jenkins 正克隆到 'kubernetes-jenkins'... remote: Enumerating objects: 16, done. remote: Counting objects: 100% (7/7), done. remote: Compressing objects: 100% (7/7), done. remote: Total 16 (delta 1), reused 0 (delta 0), pack-reused 9 Unpacking objects: 100% (16/16), done. [root@k8smaster jenkins]# ls kubernetes-jenkins [root@k8smaster jenkins]# cd kubernetes-jenkins/ [root@k8smaster kubernetes-jenkins]# ls deployment.yaml namespace.yaml README.md serviceAccount.yaml service.yaml volume.yaml # 3.创建命名空间 [root@k8smaster kubernetes-jenkins]# cat namespace.yaml apiVersion: v1 kind: Namespace metadata: name: devops-tools [root@k8smaster kubernetes-jenkins]# kubectl apply -f namespace.yaml namespace/devops-tools created [root@k8smaster kubernetes-jenkins]# kubectl get ns NAME STATUS AGE default Active 22h devops-tools Active 19s ingress-nginx Active 139m kube-node-lease Active 22h kube-public Active 22h kube-system Active 22h # 4.创建服务账号，集群角色，绑定 [root@k8smaster kubernetes-jenkins]# cat serviceAccount.yaml --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: jenkins-admin rules: - apiGroups: [""] resources: ["*"] verbs: ["*"] --- apiVersion: v1 kind: ServiceAccount metadata: name: jenkins-admin namespace: devops-tools --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: jenkins-admin roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: jenkins-admin subjects: - kind: ServiceAccount name: jenkins-admin [root@k8smaster kubernetes-jenkins]# kubectl apply -f serviceAccount.yaml clusterrole.rbac.authorization.k8s.io/jenkins-admin created serviceaccount/jenkins-admin created clusterrolebinding.rbac.authorization.k8s.io/jenkins-admin created # 5.创建卷，用来存放数据 [root@k8smaster kubernetes-jenkins]# cat volume.yaml kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: local-storage provisioner: kubernetes.io/no-provisioner volumeBindingMode: WaitForFirstConsumer --- apiVersion: v1 kind: PersistentVolume metadata: name: jenkins-pv-volume labels: type: local spec: storageClassName: local-storage claimRef: name: jenkins-pv-claim namespace: devops-tools capacity: storage: 10Gi accessModes: - ReadWriteOnce local: path: /mnt nodeAffinity: required: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/hostname operator: In values: - k8snode1 # 需要修改为k8s里的node节点的名字 --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: jenkins-pv-claim namespace: devops-tools spec: storageClassName: local-storage accessModes: - ReadWriteOnce resources: requests: storage: 3Gi [root@k8smaster kubernetes-jenkins]# kubectl apply -f volume.yaml storageclass.storage.k8s.io/local-storage created persistentvolume/jenkins-pv-volume created persistentvolumeclaim/jenkins-pv-claim created [root@k8smaster kubernetes-jenkins]# kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE jenkins-pv-volume 10Gi RWO Retain Bound devops-tools/jenkins-pv-claim local-storage 33s pv-web 10Gi RWX Retain Bound default/pvc-web nfs 21h [root@k8smaster kubernetes-jenkins]# kubectl describe pv jenkins-pv-volume Name: jenkins-pv-volume Labels: type=local Annotations:Finalizers: [kubernetes.io/pv-protection] StorageClass: local-storage Status: Bound Claim: devops-tools/jenkins-pv-claim Reclaim Policy: Retain Access Modes: RWO VolumeMode: Filesystem Capacity: 10Gi Node Affinity: Required Terms: Term 0: kubernetes.io/hostname in [k8snode1] Message: Source: Type: LocalVolume (a persistent volume backed by local storage on a node) Path: /mnt Events: # 6.部署Jenkins [root@k8smaster kubernetes-jenkins]# cat deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: jenkins namespace: devops-tools spec: replicas: 1 selector: matchLabels: app: jenkins-server template: metadata: labels: app: jenkins-server spec: securityContext: fsGroup: 1000 runAsUser: 1000 serviceAccountName: jenkins-admin containers: - name: jenkins image: jenkins/jenkins:lts imagePullPolicy: IfNotPresent resources: limits: memory: "2Gi" cpu: "1000m" requests: memory: "500Mi" cpu: "500m" ports: - name: httpport containerPort: 8080 - name: jnlpport containerPort: 50000 livenessProbe: httpGet: path: "/login" port: 8080 initialDelaySeconds: 90 periodSeconds: 10 timeoutSeconds: 5 failureThreshold: 5 readinessProbe: httpGet: path: "/login" port: 8080 initialDelaySeconds: 60 periodSeconds: 10 timeoutSeconds: 5 failureThreshold: 3 volumeMounts: - name: jenkins-data mountPath: /var/jenkins_home volumes: - name: jenkins-data persistentVolumeClaim: claimName: jenkins-pv-claim [root@k8smaster kubernetes-jenkins]# kubectl apply -f deployment.yaml deployment.apps/jenkins created [root@k8smaster kubernetes-jenkins]# kubectl get deploy -n devops-tools NAME READY UP-TO-DATE AVAILABLE AGE jenkins 1/1 1 1 5m36s [root@k8smaster kubernetes-jenkins]# kubectl get pod -n devops-tools NAME READY STATUS RESTARTS AGE jenkins-7fdc8dd5fd-bg66q 1/1 Running 0 19s # 7.启动服务发布Jenkins的pod [root@k8smaster kubernetes-jenkins]# cat service.yaml apiVersion: v1 kind: Service metadata: name: jenkins-service namespace: devops-tools annotations: prometheus.io/scrape: 'true' prometheus.io/path: / prometheus.io/port: '8080' spec: selector: app: jenkins-server type: NodePort ports: - port: 8080 targetPort: 8080 nodePort: 32000 [root@k8smaster kubernetes-jenkins]# kubectl apply -f service.yaml service/jenkins-service created [root@k8smaster kubernetes-jenkins]# kubectl get svc -n devops-tools NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE jenkins-service NodePort 10.104.76.252 8080:32000/TCP 24s # 8.在Windows机器上访问Jenkins，宿主机ip+端口号 http://192.168.2.104:32000/login?from=%2F # 9.进入pod里获取登录的密码 [root@k8smaster kubernetes-jenkins]# kubectl exec -it jenkins-7fdc8dd5fd-bg66q -n devops-tools -- bash bash-5.1$ cat /var/jenkins_home/secrets/initialAdminPassword b0232e2dad164f89ad2221e4c46b0d46 # 修改密码 [root@k8smaster kubernetes-jenkins]# kubectl get pod -n devops-tools NAME READY STATUS RESTARTS AGE jenkins-7fdc8dd5fd-5nn7m 1/1 Running 0 91s - 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
- 111
- 112
- 113
- 114
- 115
- 116
- 117
- 118
- 119
- 120
- 121
- 122
- 123
- 124
- 125
- 126
- 127
- 128
- 129
- 130
- 131
- 132
- 133
- 134
- 135
- 136
- 137
- 138
- 139
- 140
- 141
- 142
- 143
- 144
- 145
- 146
- 147
- 148
- 149
- 150
- 151
- 152
- 153
- 154
- 155
- 156
- 157
- 158
- 159
- 160
- 161
- 162
- 163
- 164
- 165
- 166
- 167
- 168
- 169
- 170
- 171
- 172
- 173
- 174
- 175
- 176
- 177
- 178
- 179
- 180
- 181
- 182
- 183
- 184
- 185
- 186
- 187
- 188
- 189
- 190
- 191
- 192
- 193
- 194
- 195
- 196
- 197
- 198
- 199
- 200
- 201
- 202
- 203
- 204
- 205
- 206
- 207
- 208
- 209
- 210
- 211
- 212
- 213
- 214
- 215
- 216
- 217
- 218
- 219
- 220
- 221
- 222
- 223
- 224
- 225
- 226
- 227
- 228
- 229
- 230
- 231
- 232
- 233
- 234
- 235
- 236
- 237
- 238
- 239
- 240
- 241
- 242
- 243
- 244
- 245
- 246
- 247
- 248
- 249
- 250
- 251
- 252
- 253
- 254
- 255
- 256
- 257
- 258
- 259
- 260
- 261
- 262
- 263
- 264
- 265
- 266
c、部署harbor
# 前提是安装好 docker 和 docker compose # 1.配置阿里云的repo源 yum install -y yum-utils yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo # 2.安装docker服务 yum install docker-ce-20.10.6 -y # 启动docker，设置开机自启 systemctl start docker && systemctl enable docker.service # 3.查看docker版本，docker compose版本 [root@harbor ~]# docker version Client: Docker Engine - Community Version: 24.0.2 API version: 1.41 (downgraded from 1.43) Go version: go1.20.4 Git commit: cb74dfc Built: Thu May 25 21:55:21 2023 OS/Arch: linux/amd64 Context: default Server: Docker Engine - Community Engine: Version: 20.10.6 API version: 1.41 (minimum version 1.12) Go version: go1.13.15 Git commit: 8728dd2 Built: Fri Apr 9 22:43:57 2021 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.6.21 GitCommit: 3dce8eb055cbb6872793272b4f20ed16117344f8 runc: Version: 1.1.7 GitCommit: v1.1.7-0-g860f061 docker-init: Version: 0.19.0 GitCommit: de40ad0 [root@harbor ~]# docker compose version Docker Compose version v2.18.1 # 4.安装 docker-compose [root@harbor ~]# ls anaconda-ks.cfg docker-compose-linux-x86_64 harbor [root@harbor ~]# chmod +x docker-compose-linux-x86_64 [root@harbor ~]# mv docker-compose-linux-x86_64 /usr/local/sbin/docker-compose # 5.安装 harbor，到 harbor 官网或者 github 下载harbor源码包 [root@harbor harbor]# ls harbor-offline-installer-v2.4.1.tgz # 6.解压 [root@harbor harbor]# tar xf harbor-offline-installer-v2.4.1.tgz [root@harbor harbor]# ls harbor harbor-offline-installer-v2.4.1.tgz [root@harbor harbor]# cd harbor [root@harbor harbor]# ls common.sh harbor.v2.4.1.tar.gz harbor.yml.tmpl install.sh LICENSE prepare [root@harbor harbor]# pwd /root/harbor/harbor # 7.修改配置文件 [root@harbor harbor]# cat harbor.yml # Configuration file of Harbor # The IP address or hostname to access admin UI and registry service. # DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients. hostname: 192.168.2.106 # 修改为主机ip地址 # http related config http: # port for http, default is 80. If https enabled, this port will redirect to https port port: 5000 # 修改成其他端口号 #https可以全关闭 # https related config #https: # https port for harbor, default is 443 #port: 443 # The path of cert and key files for nginx #certificate: /your/certificate/path #private_key: /your/private/key/path # # Uncomment following will enable tls communication between all harbor components # internal_tls: # # set enabled to true means internal tls is enabled # enabled: true # # put your cert and key files on dir # dir: /etc/harbor/tls/internal # Uncomment external_url if you want to enable external proxy # And when it enabled the hostname will no longer used # external_url: https://reg.mydomain.com:8433 # The initial password of Harbor admin # It only works in first time to install harbor # Remember Change the admin password from UI after launching Harbor. harbor_admin_password: Harbor12345 #登录密码 # Harbor DB configuration database: # The password for the root user of Harbor DB. Change this before any production use. password: root123 # The maximum number of connections in the idle connection pool. If it <=0, no idle connections are retained. max_idle_conns: 100 # The maximum number of open connections to the database. If it <= 0, then there is no limit on the number of open connections. # Note: the default number of connections is 1024 for postgres of harbor. max_open_conns: 900 # The default data volume data_volume: /data # 8.执行部署脚本 [root@harbor harbor]# ./install.sh [Step 0]: checking if docker is installed ... Note: docker version: 24.0.2 [Step 1]: checking docker-compose is installed ... ✖ Need to install docker-compose(1.18.0+) by yourself first and run this script again. [root@harbor harbor]# ./install.sh [+] Running 10/10 ⠿ Network harbor_harbor Created 0.7s ⠿ Container harbor-log Started 1.6s ⠿ Container registry Started 5.2s ⠿ Container harbor-db Started 4.9s ⠿ Container harbor-portal Started 5.1s ⠿ Container registryctl Started 4.8s ⠿ Container redis Started 3.9s ⠿ Container harbor-core Started 6.5s ⠿ Container harbor-jobservice Started 9.0s ⠿ Container nginx Started 9.1s ✔ ----Harbor has been installed and started successfully.---- # 9.配置开机自启 [root@harbor harbor]# vim /etc/rc.local [root@harbor harbor]# cat /etc/rc.local #!/bin/bash # THIS FILE IS ADDED FOR COMPATIBILITY PURPOSES # # It is highly advisable to create own systemd services or udev rules # to run scripts during boot instead of using this file. # # In contrast to previous versions due to parallel execution during boot # this script will NOT be run after all other services. # # Please note that you must run 'chmod +x /etc/rc.d/rc.local' to ensure # that this script will be executed during boot. touch /var/lock/subsys/local /usr/local/sbin/docker-compose -f /root/harbor/harbor/docker-compose.yml up -d # 10.设置权限 [root@harbor harbor]# chmod +x /etc/rc.local /etc/rc.d/rc.local # 11.登录 http://192.168.2.106:5000/ # 账号：admin # 密码：Harbor12345 # 新建一个项目 # 测试（以nginx为例进行推送到harbor上） [root@harbor harbor]# docker image ls | grep nginx nginx latest 605c77e624dd 17 months ago 141MB goharbor/nginx-photon v2.4.1 78aad8c8ef41 18 months ago 45.7MB [root@harbor harbor]# docker tag nginx:latest 192.168.2.106:5000/test/nginx1:v1 [root@harbor harbor]# docker image ls | grep nginx 192.168.2.106:5000/test/nginx1 v1 605c77e624dd 17 months ago 141MB nginx latest 605c77e624dd 17 months ago 141MB goharbor/nginx-photon v2.4.1 78aad8c8ef41 18 months ago 45.7MB [root@harbor harbor]# docker push 192.168.2.106:5000/test/nginx1:v1 The push refers to repository [192.168.2.106:5000/test/nginx1] Get https://192.168.2.106:5000/v2/: http: server gave HTTP response to HTTPS client [root@harbor harbor]# vim /etc/docker/daemon.json { "insecure-registries":["192.168.2.106:5000"] } [root@harbor harbor]# docker login 192.168.2.106:5000 Username: admin Password: WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded [root@harbor harbor]# docker push 192.168.2.106:5000/test/nginx1:v1 The push refers to repository [192.168.2.106:5000/test/nginx1] d874fd2bc83b: Pushed 32ce5f6a5106: Pushed f1db227348d0: Pushed b8d6e692a25e: Pushed e379e8aedd4d: Pushed 2edcec3590a4: Pushed v1: digest: sha256:ee89b00528ff4f02f2405e4ee221743ebc3f8e8dd0bfd5c4c20a2fa2aaa7ede3 size: 1570 [root@harbor harbor]# cat /etc/docker/daemon.json { "insecure-registries":["192.168.2.106:5000"] }- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
- 111
- 112
- 113
- 114
- 115
- 116
- 117
- 118
- 119
- 120
- 121
- 122
- 123
- 124
- 125
- 126
- 127
- 128
- 129
- 130
- 131
- 132
- 133
- 134
- 135
- 136
- 137
- 138
- 139
- 140
- 141
- 142
- 143
- 144
- 145
- 146
- 147
- 148
- 149
- 150
- 151
- 152
- 153
- 154
- 155
- 156
- 157
- 158
- 159
- 160
- 161
- 162
- 163
- 164
- 165
- 166
- 167
- 168
- 169
- 170
- 171
- 172
- 173
- 174
- 175
- 176
- 177
- 178
- 179
- 180
- 181
- 182
- 183
- 184
- 185
- 186
- 187
- 188
- 189
- 190
- 191
- 192
- 193
- 194
- 195
- 196
- 197
- 198
- 199
- 200
- 201
- 202
- 203
- 204
- 205
- 206
- 207
- 208
- 209
- 210
- 211
5、将自己用go开发的web接口系统制作成镜像，部署到k8s里作为web应用；采用HPA技术，当cpu使用率达到50%的时候，进行水平扩缩，最小20个业务pod，最多40个业务pod
# k8s集群每个节点都登入到harbor中，以便于从harbor中拉回镜像。 [root@k8snode2 ~]# cat /etc/docker/daemon.json { "registry-mirrors":["https://rsbud4vc.mirror.aliyuncs.com","https://registry.docker-cn.com","https://docker.mirrors.ustc.edu.cn","https://dockerhub.azk8s.cn","http://hub-mirror.c.163.com"], "insecure-registries":["192.168.2.106:5000"], "exec-opts": ["native.cgroupdriver=systemd"] } # 重新加载配置，重启docker服务 systemctl daemon-reload && systemctl restart docker # 登录harbor [root@k8smaster mysql]# docker login 192.168.2.106:5000 Username: admin Password: WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded [root@k8snode1 ~]# docker login 192.168.2.106:5000 Username: admin Password: WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded [root@k8snode2 ~]# docker login 192.168.2.106:5000 Username: admin Password: WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded # 测试：从harbor拉取nginx镜像 [root@k8snode1 ~]# docker pull 192.168.2.106:5000/test/nginx1:v1 [root@k8snode1 ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE mysql 5.7.42 2be84dd575ee 5 days ago 569MB nginx latest 605c77e624dd 17 months ago 141MB 192.168.2.106:5000/test/nginx1 v1 605c77e624dd 17 months ago 141MB # 制作镜像 [root@harbor ~]# cd go [root@harbor go]# ls scweb Dockerfile [root@harbor go]# cat Dockerfile FROM centos:7 WORKDIR /go COPY . /go RUN ls /go && pwd ENTRYPOINT ["/go/scweb"] [root@harbor go]# docker build -t scmyweb:1.1 . [root@harbor go]# docker image ls | grep scweb scweb 1.1 f845e97e9dfd 4 hours ago 214MB [root@harbor go]# docker tag scweb:1.1 192.168.2.106:5000/test/web:v2 [root@harbor go]# docker image ls | grep web 192.168.2.106:5000/test/web v2 00900ace4935 4 minutes ago 214MB scweb 1.1 00900ace4935 4 minutes ago 214MB [root@harbor go]# docker push 192.168.2.106:5000/test/web:v2 The push refers to repository [192.168.2.106:5000/test/web] 3e252407b5c2: Pushed 193a27e04097: Pushed b13a87e7576f: Pushed 174f56854903: Pushed v1: digest: sha256:a723c83407c49e6fcf9aa67a041a4b6241cf9856170c1703014a61dec3726b29 size: 1153 [root@k8snode1 ~]# docker login 192.168.2.106:5000 Authenticating with existing credentials... WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded [root@k8snode1 ~]# docker pull 192.168.2.106:5000/test/web:v2 v1: Pulling from test/web 2d473b07cdd5: Pull complete bc5e56dd1476: Pull complete 694440c745ce: Pull complete 78694d1cffbb: Pull complete Digest: sha256:a723c83407c49e6fcf9aa67a041a4b6241cf9856170c1703014a61dec3726b29 Status: Downloaded newer image for 192.168.2.106:5000/test/web:v2 192.168.2.106:5000/test/web:v1 [root@k8snode1 ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE 192.168.2.106:5000/test/web v2 f845e97e9dfd 4 hours ago 214MB [root@k8snode2 ~]# docker login 192.168.2.106:5000 Authenticating with existing credentials... WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded [root@k8snode2 ~]# docker pull 192.168.2.106:5000/test/web:v2 v1: Pulling from test/web 2d473b07cdd5: Pull complete bc5e56dd1476: Pull complete 694440c745ce: Pull complete 78694d1cffbb: Pull complete Digest: sha256:a723c83407c49e6fcf9aa67a041a4b6241cf9856170c1703014a61dec3726b29 Status: Downloaded newer image for 192.168.2.106:5000/test/web:v2 192.168.2.106:5000/test/web:v1 [root@k8snode2 ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE 192.168.2.106:5000/test/web v2 f845e97e9dfd 4 hours ago 214MB # 采用HPA技术，当cpu使用率达到50%的时候，进行水平扩缩，最小1个，最多10个pod # HorizontalPodAutoscaler（简称 HPA ）自动更新工作负载资源（例如Deployment），目的是自动扩缩# 工作负载以满足需求。 https://kubernetes.io/zh-cn/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/ # 1.安装metrics server # 下载components.yaml配置文件 wget https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml # 替换image image: registry.aliyuncs.com/google_containers/metrics-server:v0.6.0 imagePullPolicy: IfNotPresent args: # // 新增下面两行参数 - --kubelet-insecure-tls - --kubelet-preferred-address-types=InternalDNS,InternalIP,ExternalDNS,ExternalIP,Hostname # 修改components.yaml配置文件 [root@k8smaster ~]# cat components.yaml spec: containers: - args: - --kubelet-insecure-tls - --kubelet-preferred-address-types=InternalIP - --cert-dir=/tmp - --secure-port=4443 - --kubelet-preferred-address-types=InternalDNS,InternalIP,ExternalIP,Hostname - --kubelet-use-node-status-port - --metric-resolution=15s image: registry.aliyuncs.com/google_containers/metrics-server:v0.6.0 imagePullPolicy: IfNotPresent # 执行安装命令 [root@k8smaster metrics]# kubectl apply -f components.yaml serviceaccount/metrics-server created clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created clusterrole.rbac.authorization.k8s.io/system:metrics-server created rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created service/metrics-server created deployment.apps/metrics-server created apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created # 查看效果 [root@k8smaster metrics]# kubectl get pod -n kube-system NAME READY STATUS RESTARTS AGE calico-kube-controllers-6949477b58-xdk88 1/1 Running 1 22h calico-node-4knc8 1/1 Running 4 22h calico-node-8jzrn 1/1 Running 1 22h calico-node-9d7pt 1/1 Running 2 22h coredns-7f89b7bc75-52c4x 1/1 Running 2 22h coredns-7f89b7bc75-82jrx 1/1 Running 1 22h etcd-k8smaster 1/1 Running 1 22h kube-apiserver-k8smaster 1/1 Running 1 22h kube-controller-manager-k8smaster 1/1 Running 1 22h kube-proxy-8wp9c 1/1 Running 2 22h kube-proxy-d46jp 1/1 Running 1 22h kube-proxy-whg4f 1/1 Running 1 22h kube-scheduler-k8smaster 1/1 Running 1 22h metrics-server-6c75959ddf-hw7cs 1/1 Running 0 61s # 能够使用下面的命令查看到pod的效果，说明metrics server已经安装成功 [root@k8smaster metrics]# kubectl top node NAME CPU(cores) CPU% MEMORY(bytes) MEMORY% k8smaster 322m 16% 1226Mi 71% k8snode1 215m 10% 874Mi 50% k8snode2 190m 9% 711Mi 41% # 确保metrics-server安装好 # 查看pod、apiservice验证metrics-server安装好了 [root@k8smaster HPA]# kubectl get pod -n kube-system|grep metrics metrics-server-6c75959ddf-hw7cs 1/1 Running 4 6h35m [root@k8smaster HPA]# kubectl get apiservice |grep metrics v1beta1.metrics.k8s.io kube-system/metrics-server True 6h35m [root@k8smaster HPA]# kubectl top node NAME CPU(cores) CPU% MEMORY(bytes) MEMORY% k8smaster 349m 17% 1160Mi 67% k8snode1 271m 13% 1074Mi 62% k8snode2 226m 11% 1224Mi 71% [root@k8snode1 ~]# docker images|grep metrics registry.aliyuncs.com/google_containers/metrics-server v0.6.0 5787924fe1d8 14 months ago 68.8MB 您在 /var/spool/mail/root 中有新邮件 # node节点上查看 [root@k8snode1 ~]# docker images|grep metrics registry.aliyuncs.com/google_containers/metrics-server v0.6.0 5787924fe1d8 17 months ago 68.8MB kubernetesui/metrics-scraper v1.0.7 7801cfc6d5c0 2 years ago 34.4MB # 2.以yaml文件启动web并暴露服务 [root@k8smaster hpa]# cat my-web.yaml apiVersion: apps/v1 kind: Deployment metadata: labels: app: myweb name: myweb spec: replicas: 3 selector: matchLabels: app: myweb template: metadata: labels: app: myweb spec: containers: - name: myweb image: 192.168.2.106:5000/test/web:v2 imagePullPolicy: IfNotPresent ports: - containerPort: 8000 resources: limits: cpu: 300m requests: cpu: 100m --- apiVersion: v1 kind: Service metadata: labels: app: myweb-svc name: myweb-svc spec: selector: app: myweb type: NodePort ports: - port: 8000 protocol: TCP targetPort: 8000 nodePort: 30001 [root@k8smaster HPA]# kubectl apply -f my-web.yaml deployment.apps/myweb created service/myweb-svc created # 3.创建HPA功能 [root@k8smaster HPA]# kubectl autoscale deployment myweb --cpu-percent=50 --min=1 --max=10 horizontalpodautoscaler.autoscaling/myweb autoscaled [root@k8smaster HPA]# kubectl get pod NAME READY STATUS RESTARTS AGE myweb-6dc7b4dfcb-9q85g 1/1 Running 0 9s myweb-6dc7b4dfcb-ddq82 1/1 Running 0 9s myweb-6dc7b4dfcb-l7sw7 1/1 Running 0 9s [root@k8smaster HPA]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1443/TCP 3d2h myweb-svc NodePort 10.102.83.168 8000:30001/TCP 15s [root@k8smaster HPA]# kubectl get hpa NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE myweb Deployment/myweb /50% 1 10 3 16s # 4.访问 http://192.168.2.112:30001/ [root@k8smaster HPA]# kubectl get hpa NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE myweb Deployment/myweb 1%/50% 1 10 1 11m [root@k8smaster HPA]# kubectl get pod NAME READY STATUS RESTARTS AGE myweb-6dc7b4dfcb-ddq82 1/1 Running 0 10m # 5.删除hpa [root@k8smaster HPA]# kubectl delete hpa myweb-svc - 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
- 111
- 112
- 113
- 114
- 115
- 116
- 117
- 118
- 119
- 120
- 121
- 122
- 123
- 124
- 125
- 126
- 127
- 128
- 129
- 130
- 131
- 132
- 133
- 134
- 135
- 136
- 137
- 138
- 139
- 140
- 141
- 142
- 143
- 144
- 145
- 146
- 147
- 148
- 149
- 150
- 151
- 152
- 153
- 154
- 155
- 156
- 157
- 158
- 159
- 160
- 161
- 162
- 163
- 164
- 165
- 166
- 167
- 168
- 169
- 170
- 171
- 172
- 173
- 174
- 175
- 176
- 177
- 178
- 179
- 180
- 181
- 182
- 183
- 184
- 185
- 186
- 187
- 188
- 189
- 190
- 191
- 192
- 193
- 194
- 195
- 196
- 197
- 198
- 199
- 200
- 201
- 202
- 203
- 204
- 205
- 206
- 207
- 208
- 209
- 210
- 211
- 212
- 213
- 214
- 215
- 216
- 217
- 218
- 219
- 220
- 221
- 222
- 223
- 224
- 225
- 226
- 227
- 228
- 229
- 230
- 231
- 232
- 233
- 234
- 235
- 236
- 237
- 238
- 239
- 240
- 241
- 242
- 243
- 244
- 245
- 246
- 247
- 248
- 249
- 250
- 251
- 252
- 253
- 254
- 255
- 256
- 257
- 258
- 259
- 260
- 261
- 262
- 263
- 264
- 265
- 266
- 267
- 268
- 269
- 270
- 271
- 272
- 273
- 274
- 275
- 276
- 277
- 278
- 279
- 280
- 281
- 282
- 283
- 284
- 285
- 286
- 287
- 288
- 289
- 290
- 291
- 292
- 293
- 294
6、启动mysql的pod，为web业务提供数据库服务
[root@k8smaster mysql]# cat mysql-deployment.yaml # 定义mysql的Deployment apiVersion: apps/v1 kind: Deployment metadata: labels: app: mysql name: mysql spec: replicas: 1 selector: matchLabels: app: mysql template: metadata: labels: app: mysql spec: containers: - image: mysql:5.7.42 name: mysql imagePullPolicy: IfNotPresent env: - name: MYSQL_ROOT_PASSWORD value: "123456" ports: - containerPort: 3306 --- #定义mysql的Service apiVersion: v1 kind: Service metadata: labels: app: svc-mysql name: svc-mysql spec: selector: app: mysql type: NodePort ports: - port: 3306 protocol: TCP targetPort: 3306 nodePort: 30007 [root@k8smaster mysql]# kubectl apply -f mysql-deployment.yaml deployment.apps/mysql created service/svc-mysql created [root@k8smaster mysql]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1443/TCP 28h svc-mysql NodePort 10.105.96.217 3306:30007/TCP 10m [root@k8smaster mysql]# kubectl get pod NAME READY STATUS RESTARTS AGE mysql-5f9bccd855-6kglf 1/1 Running 0 8m59s [root@k8smaster mysql]# kubectl exec -it mysql-5f9bccd855-6kglf -- bash bash-4.2# mysql -uroot -p123456 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.7.42 MySQL Community Server (GPL) Copyright (c) 2000, 2023, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | +--------------------+ 4 rows in set (0.01 sec) mysql> exit Bye bash-4.2# exit exit [root@k8smaster mysql]# # Web服务和MySQL数据库结合起来 # 第一种：在mysql的service中增加以下内容 ports: - name: mysql protocol: TCP port: 3306 targetPort: 3306 # 在web的pod中增加以下内容 env: - name: MYSQL_HOST value: mysql - name: MYSQL_PORT value: "3306" # 第二种：安装MySQL驱动程序，在 Go 代码中引入并初始化该驱动程序。 # 1.导入必要的包和驱动程序import ( "database/sql" "fmt" _ "github.com/go-sql-driver/mysql" # 导入 MySQL 驱动程序 ) # 2.建立数据库连接db, err := sql.Open("mysql", "username:password@tcp(hostname:port)/dbname") if err != nil { fmt.Println("Failed to connect to database:", err) return } defer db.Close() # 记得关闭数据库连接 - 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
- 111
- 112
- 113
- 114
- 115
- 116
- 117
- 118
a、尝试：k8s部署有状态的MySQL
# 1.创建 ConfigMap [root@k8smaster mysql]# cat mysql-configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: mysql labels: app: mysql data: primary.cnf: | # 仅在主服务器上应用此配置 [mysqld] log-bin replica.cnf: | # 仅在副本服务器上应用此配置 [mysqld] super-read-only [root@k8smaster mysql]# kubectl apply -f mysql-configmap.yaml configmap/mysql created [root@k8smaster mysql]# kubectl get cm NAME DATA AGE kube-root-ca.crt 1 6d22h mysql 2 5s # 2.创建服务 [root@k8smaster mysql]# cat mysql-services.yaml # 为 StatefulSet 成员提供稳定的 DNS 表项的无头服务（Headless Service） apiVersion: v1 kind: Service metadata: name: mysql labels: app: mysql app.kubernetes.io/name: mysql spec: ports: - name: mysql port: 3306 clusterIP: None selector: app: mysql --- # 用于连接到任一 MySQL 实例执行读操作的客户端服务 # 对于写操作，你必须连接到主服务器：mysql-0.mysql apiVersion: v1 kind: Service metadata: name: mysql-read labels: app: mysql app.kubernetes.io/name: mysql readonly: "true" spec: ports: - name: mysql port: 3306 selector: app: mysql [root@k8smaster mysql]# kubectl apply -f mysql-services.yaml service/mysql created service/mysql-read created [root@k8smaster mysql]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1443/TCP 6d22h mysql ClusterIP None 3306/TCP 7s mysql-read ClusterIP 10.102.31.144 3306/TCP 7s # 3.创建 StatefulSet [root@k8smaster mysql]# cat mysql-statefulset.yaml apiVersion: apps/v1 kind: StatefulSet metadata: name: mysql spec: selector: matchLabels: app: mysql app.kubernetes.io/name: mysql serviceName: mysql replicas: 3 template: metadata: labels: app: mysql app.kubernetes.io/name: mysql spec: initContainers: - name: init-mysql image: mysql:5.7.42 imagePullPolicy: IfNotPresent command: - bash - "-c" - | set -ex # 基于 Pod 序号生成 MySQL 服务器的 ID。 [[ $HOSTNAME =~ -([0-9]+)$ ]] || exit 1 ordinal=${BASH_REMATCH[1]} echo [mysqld] > /mnt/conf.d/server-id.cnf # 添加偏移量以避免使用 server-id=0 这一保留值。 echo server-id=$((100 + $ordinal)) >> /mnt/conf.d/server-id.cnf # 将合适的 conf.d 文件从 config-map 复制到 emptyDir。 if [[ $ordinal -eq 0 ]]; then cp /mnt/config-map/primary.cnf /mnt/conf.d/ else cp /mnt/config-map/replica.cnf /mnt/conf.d/ fi volumeMounts: - name: conf mountPath: /mnt/conf.d - name: config-map mountPath: /mnt/config-map - name: clone-mysql image: registry.cn-hangzhou.aliyuncs.com/google_samples_thepoy/xtrabackup:1.0 command: - bash - "-c" - | set -ex # 如果已有数据，则跳过克隆。 [[ -d /var/lib/mysql/mysql ]] && exit 0 # 跳过主实例（序号索引 0）的克隆。 [[ `hostname` =~ -([0-9]+)$ ]] || exit 1 ordinal=${BASH_REMATCH[1]} [[ $ordinal -eq 0 ]] && exit 0 # 从原来的对等节点克隆数据。 ncat --recv-only mysql-$(($ordinal-1)).mysql 3307 | xbstream -x -C /var/lib/mysql # 准备备份。 xtrabackup --prepare --target-dir=/var/lib/mysql volumeMounts: - name: data mountPath: /var/lib/mysql subPath: mysql - name: conf mountPath: /etc/mysql/conf.d containers: - name: mysql image: mysql:5.7.42 imagePullPolicy: IfNotPresent env: - name: MYSQL_ALLOW_EMPTY_PASSWORD value: "1" ports: - name: mysql containerPort: 3306 volumeMounts: - name: data mountPath: /var/lib/mysql subPath: mysql - name: conf mountPath: /etc/mysql/conf.d resources: requests: cpu: 500m memory: 1Gi livenessProbe: exec: command: ["mysqladmin", "ping"] initialDelaySeconds: 30 periodSeconds: 10 timeoutSeconds: 5 readinessProbe: exec: # 检查我们是否可以通过 TCP 执行查询（skip-networking 是关闭的）。 command: ["mysql", "-h", "127.0.0.1", "-e", "SELECT 1"] initialDelaySeconds: 5 periodSeconds: 2 timeoutSeconds: 1 - name: xtrabackup image: registry.cn-hangzhou.aliyuncs.com/google_samples_thepoy/xtrabackup:1.0 ports: - name: xtrabackup containerPort: 3307 command: - bash - "-c" - | set -ex cd /var/lib/mysql # 确定克隆数据的 binlog 位置（如果有的话）。 if [[ -f xtrabackup_slave_info && "x$( - 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
- 111
- 112
- 113
- 114
- 115
- 116
- 117
- 118
- 119
- 120
- 121
- 122
- 123
- 124
- 125
- 126
- 127
- 128
- 129
- 130
- 131
- 132
- 133
- 134
- 135
- 136
- 137
- 138
- 139
- 140
- 141
- 142
- 143
- 144
- 145
- 146
- 147
- 148
- 149
- 150
- 151
- 152
- 153
- 154
- 155
- 156
- 157
- 158
- 159
- 160
- 161
- 162
- 163
- 164
- 165
- 166
- 167
- 168
- 169
- 170
- 171
- 172
- 173
- 174
- 175
- 176
- 177
- 178
- 179
- 180
- 181
- 182
- 183
- 184
- 185
- 186
- 187
- 188
- 189
- 190
- 191
- 192
- 193
- 194
- 195
- 196
- 197
- 198
- 199
- 200
- 201
- 202
- 203
- 204
- 205
- 206
- 207
- 208
- 209
- 210
- 211
- 212
- 213
- 214
- 215
- 216
- 217
- 218
- 219
- 220
- 221
- 222
- 223
- 224
- 225
- 226
- 227
- 228
- 229
- 230
- 231
- 232
- 233
- 234
- 235
- 236
- 237
- 238
- 239
- 240
- 241
- 242
- 243
- 244
- 245
- 246
- 247
- 248
- 249
- 250
- 251
- 252
- 253
- 254
- 255
- 256
- 257
- 258
- 259
- 260
- 261
- 262
- 263
- 264
- 265
- 266
- 267
- 268
- 269
- 270
- 271
- 272
- 273
- 274
- 275
- 276
- 277
- 278
- 279
- 280
- 281
- 282
- 283
- 284
- 285
- 286
- 287
- 288
- 289
- 290
- 291
- 292
- 293
- 294
- 295
- 296
- 297
- 298
- 299
- 300
- 301
- 302
- 303
- 304
- 305
- 306
- 307
- 308
- 309
- 310
- 311
- 312
- 313
- 314
- 315
- 316
- 317
- 318
- 319
- 320
- 321
- 322
- 323
- 324
- 325
- 326
- 327
- 328
- 329
- 330
- 331
- 332
- 333
- 334
- 335
- 336
- 337
- 338
- 339
- 340
- 341
- 342
- 343
- 344
- 345
- 346
- 347
- 348
- 349
- 350
- 351
- 352
- 353
- 354
- 355
- 356
- 357
- 358
- 359
- 360
- 361
- 362
- 363
- 364
- 365
- 366
- 367
- 368
- 369
- 370
- 371
- 372
- 373
- 374
- 375
- 376
- 377
- 378
- 379
- 380
- 381
- 382
- 383
- 384
- 385
- 386
7、使用探针(liveness、readiness、startup)的（httpget、exec）方法对web业务pod进行监控，一旦出现问题马上重启，增强业务pod的可靠性
livenessProbe: exec: command: - ls - /tmp initialDelaySeconds: 5 periodSeconds: 5 readinessProbe: exec: command: - ls - /tmp initialDelaySeconds: 5 periodSeconds: 5 startupProbe: httpGet: path: / port: 8000 failureThreshold: 30 periodSeconds: 10 [root@k8smaster probe]# vim my-web.yaml apiVersion: apps/v1 kind: Deployment metadata: labels: app: myweb name: myweb spec: replicas: 3 selector: matchLabels: app: myweb template: metadata: labels: app: myweb spec: containers: - name: myweb image: 192.168.2.106:5000/test/web:v2 imagePullPolicy: IfNotPresent ports: - containerPort: 8000 resources: limits: cpu: 300m requests: cpu: 100m livenessProbe: exec: command: - ls - /tmp initialDelaySeconds: 5 periodSeconds: 5 readinessProbe: exec: command: - ls - /tmp initialDelaySeconds: 5 periodSeconds: 5 startupProbe: httpGet: path: / port: 8000 failureThreshold: 30 periodSeconds: 10 --- apiVersion: v1 kind: Service metadata: labels: app: myweb-svc name: myweb-svc spec: selector: app: myweb type: NodePort ports: - port: 8000 protocol: TCP targetPort: 8000 nodePort: 30001 [root@k8smaster probe]# kubectl apply -f my-web.yaml deployment.apps/myweb created service/myweb-svc created [root@k8smaster probe]# kubectl get pod NAME READY STATUS RESTARTS AGE myweb-6b89fb9c7b-4cdh9 1/1 Running 0 53s myweb-6b89fb9c7b-dh87w 1/1 Running 0 53s myweb-6b89fb9c7b-zvc52 1/1 Running 0 53s [root@k8smaster probe]# kubectl describe pod myweb-6b89fb9c7b-4cdh9 Name: myweb-6b89fb9c7b-4cdh9 Namespace: default Priority: 0 Node: k8snode2/192.168.2.112 Start Time: Thu, 22 Jun 2023 16:47:20 +0800 Labels: app=myweb pod-template-hash=6b89fb9c7b Annotations: cni.projectcalico.org/podIP: 10.244.185.219/32 cni.projectcalico.org/podIPs: 10.244.185.219/32 Status: Running IP: 10.244.185.219 IPs: IP: 10.244.185.219 Controlled By: ReplicaSet/myweb-6b89fb9c7b Containers: myweb: Container ID: docker://8c55c0c825483f86e4b3c87413984415b2ccf5cad78ed005eed8bedb4252c130 Image: 192.168.2.106:5000/test/web:v2 Image ID: docker-pullable://192.168.2.106:5000/test/web@sha256:3bef039aa5c13103365a6868c9f052a000de376a45eaffcbad27d6ddb1f6e354 Port: 8000/TCP Host Port: 0/TCP State: Running Started: Thu, 22 Jun 2023 16:47:23 +0800 Ready: True Restart Count: 0 Limits: cpu: 300m Requests: cpu: 100m Liveness: exec [ls /tmp] delay=5s timeout=1s period=5s #success=1 #failure=3 Readiness: exec [ls /tmp] delay=5s timeout=1s period=5s #success=1 #failure=3 Startup: http-get http://:8000/ delay=0s timeout=1s period=10s #success=1 #failure=30 Environment:Mounts: /var/run/secrets/kubernetes.io/serviceaccount from default-token-24tkk (ro) Conditions: Type Status Initialized True Ready True ContainersReady True PodScheduled True Volumes: default-token-24tkk: Type: Secret (a volume populated by a Secret) SecretName: default-token-24tkk Optional: false QoS Class: Burstable Node-Selectors: Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 55s default-scheduler Successfully assigned default/myweb-6b89fb9c7b-4cdh9 to k8snode2 Normal Pulled 52s kubelet Container image "192.168.2.106:5000/test/web:v2" already present on machine Normal Created 52s kubelet Created container myweb Normal Started 52s kubelet Started container myweb - 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
- 111
- 112
- 113
- 114
- 115
- 116
- 117
- 118
- 119
- 120
- 121
- 122
- 123
- 124
- 125
- 126
- 127
- 128
- 129
- 130
- 131
- 132
- 133
- 134
- 135
- 136
- 137
- 138
- 139
- 140
- 141
- 142
- 143
- 144
- 145
- 146
- 147
- 148
- 149
- 150
- 151
- 152
- 153
- 154
- 155
- 156
8、使用ingress给web业务做负载均衡，使用dashboard对整个集群资源进行掌控
# ingress controller 本质上是一个nginx软件，用来做负载均衡。 # ingress 是k8s内部管理nginx配置（nginx.conf）的组件，用来给ingress controller传参。 [root@k8smaster ingress]# ls ingress-controller-deploy.yaml kube-webhook-certgen-v1.1.0.tar.gz sc-nginx-svc-1.yaml ingress-nginx-controllerv1.1.0.tar.gz sc-ingress.yaml ingress-controller-deploy.yaml 是部署ingress controller使用的yaml文件 ingress-nginx-controllerv1.1.0.tar.gz ingress-nginx-controller镜像 kube-webhook-certgen-v1.1.0.tar.gz kube-webhook-certgen镜像 sc-ingress.yaml 创建ingress的配置文件 sc-nginx-svc-1.yaml 启动sc-nginx-svc-1服务和相关pod的yaml nginx-deployment-nginx-svc-2.yaml 启动nginx-deployment-nginx-svc-2服务和相关pod的yaml # 第1大步骤：安装ingress controller # 1.将镜像scp到所有的node节点服务器上 [root@k8smaster ingress]# scp ingress-nginx-controllerv1.1.0.tar.gz k8snode1:/root ingress-nginx-controllerv1.1.0.tar.gz 100% 276MB 101.1MB/s 00:02 [root@k8smaster ingress]# scp ingress-nginx-controllerv1.1.0.tar.gz k8snode2:/root ingress-nginx-controllerv1.1.0.tar.gz 100% 276MB 98.1MB/s 00:02 [root@k8smaster ingress]# scp kube-webhook-certgen-v1.1.0.tar.gz k8snode1:/root kube-webhook-certgen-v1.1.0.tar.gz 100% 47MB 93.3MB/s 00:00 [root@k8smaster ingress]# scp kube-webhook-certgen-v1.1.0.tar.gz k8snode2:/root kube-webhook-certgen-v1.1.0.tar.gz 100% 47MB 39.3MB/s 00:01 # 2.导入镜像，在所有的节点服务器上进行 [root@k8snode1 ~]# docker load -i ingress-nginx-controllerv1.1.0.tar.gz [root@k8snode1 ~]# docker load -i kube-webhook-certgen-v1.1.0.tar.gz [root@k8snode2 ~]# docker load -i ingress-nginx-controllerv1.1.0.tar.gz [root@k8snode2 ~]# docker load -i kube-webhook-certgen-v1.1.0.tar.gz [root@k8snode1 ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE nginx latest 605c77e624dd 17 months ago 141MB registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller v1.1.0 ae1a7201ec95 19 months ago 285MB registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen v1.1.1 c41e9fcadf5a 20 months ago 47.7MB [root@k8snode2 ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE nginx latest 605c77e624dd 17 months ago 141MB registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller v1.1.0 ae1a7201ec95 19 months ago 285MB registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen v1.1.1 c41e9fcadf5a 20 months ago 47.7MB # 3.执行yaml文件去创建ingres controller [root@k8smaster ingress]# kubectl apply -f ingress-controller-deploy.yaml namespace/ingress-nginx created serviceaccount/ingress-nginx created configmap/ingress-nginx-controller created clusterrole.rbac.authorization.k8s.io/ingress-nginx created clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created role.rbac.authorization.k8s.io/ingress-nginx created rolebinding.rbac.authorization.k8s.io/ingress-nginx created service/ingress-nginx-controller-admission created service/ingress-nginx-controller created deployment.apps/ingress-nginx-controller created ingressclass.networking.k8s.io/nginx created validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created serviceaccount/ingress-nginx-admission created clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created role.rbac.authorization.k8s.io/ingress-nginx-admission created rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created job.batch/ingress-nginx-admission-create created job.batch/ingress-nginx-admission-patch created # 4.查看ingress controller的相关命名空间 [root@k8smaster ingress]# kubectl get ns NAME STATUS AGE default Active 20h ingress-nginx Active 30s kube-node-lease Active 20h kube-public Active 20h kube-system Active 20h # 5.查看ingress controller的相关service [root@k8smaster ingress]# kubectl get svc -n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ingress-nginx-controller NodePort 10.105.213.9580:31457/TCP,443:32569/TCP 64s ingress-nginx-controller-admission ClusterIP 10.98.225.196 443/TCP 64s # 6.查看ingress controller的相关pod [root@k8smaster ingress]# kubectl get pod -n ingress-nginx NAME READY STATUS RESTARTS AGE ingress-nginx-admission-create-9sg56 0/1 Completed 0 80s ingress-nginx-admission-patch-8sctb 0/1 Completed 1 80s ingress-nginx-controller-6c8ffbbfcf-bmdj9 1/1 Running 0 80s ingress-nginx-controller-6c8ffbbfcf-j576v 1/1 Running 0 80s # 第2大步骤：创建pod和暴露pod的服务 [root@k8smaster new]# cat sc-nginx-svc-1.yaml apiVersion: apps/v1 kind: Deployment metadata: name: sc-nginx-deploy labels: app: sc-nginx-feng spec: replicas: 3 selector: matchLabels: app: sc-nginx-feng template: metadata: labels: app: sc-nginx-feng spec: containers: - name: sc-nginx-feng image: nginx imagePullPolicy: IfNotPresent ports: - containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: sc-nginx-svc labels: app: sc-nginx-svc spec: selector: app: sc-nginx-feng ports: - name: name-of-service-port protocol: TCP port: 80 targetPort: 80 [root@k8smaster new]# kubectl apply -f sc-nginx-svc-1.yaml deployment.apps/sc-nginx-deploy created service/sc-nginx-svc created [root@k8smaster ingress]# kubectl get pod NAME READY STATUS RESTARTS AGE sc-nginx-deploy-7bb895f9f5-hmf2n 1/1 Running 0 7s sc-nginx-deploy-7bb895f9f5-mczzg 1/1 Running 0 7s sc-nginx-deploy-7bb895f9f5-zzndv 1/1 Running 0 7s [root@k8smaster ingress]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 443/TCP 20h sc-nginx-svc ClusterIP 10.96.76.55 80/TCP 26s # 查看服务器的详细信息，查看Endpoints对应的pod的ip和端口是否正常 [root@k8smaster ingress]# kubectl describe svc sc-nginx-svc Name: sc-nginx-svc Namespace: default Labels: app=sc-nginx-svc Annotations: Selector: app=sc-nginx-feng Type: ClusterIP IP Families: IP: 10.96.76.55 IPs: 10.96.76.55 Port: name-of-service-port 80/TCP TargetPort: 80/TCP Endpoints: 10.244.185.209:80,10.244.185.210:80,10.244.249.16:80 Session Affinity: None Events: # 访问服务暴露的ip [root@k8smaster ingress]# curl 10.96.76.55 <!DOCTYPE html> Welcome to nginx!<<span class="token operator">/</span>title> <style> html <span class="token punctuation">{<!-- --></span> color-scheme: light dark<span class="token punctuation">;</span> <span class="token punctuation">}</span> body <span class="token punctuation">{<!-- --></span> width: 35em<span class="token punctuation">;</span> margin: 0 auto<span class="token punctuation">;</span> font-family: Tahoma<span class="token punctuation">,</span> Verdana<span class="token punctuation">,</span> Arial<span class="token punctuation">,</span> sans-serif<span class="token punctuation">;</span> <span class="token punctuation">}</span> <<span class="token operator">/</span>style> <<span class="token operator">/</span>head> <body> <h1>Welcome to nginx!<<span class="token operator">/</span>h1> <p><span class="token keyword">If</span> you see this page<span class="token punctuation">,</span> the nginx web server is successfully installed and working<span class="token punctuation">.</span> Further configuration is required<span class="token punctuation">.</span><<span class="token operator">/</span>p> <p><span class="token keyword">For</span> online documentation and support please refer to <a href=<span class="token string">"http://nginx.org/"</span>>nginx<span class="token punctuation">.</span>org<<span class="token operator">/</span>a><span class="token punctuation">.</span><br/> Commercial support is available at <a href=<span class="token string">"http://nginx.com/"</span>>nginx<span class="token punctuation">.</span>com<<span class="token operator">/</span>a><span class="token punctuation">.</span><<span class="token operator">/</span>p> <p><em>Thank you <span class="token keyword">for</span> <span class="token keyword">using</span> nginx<span class="token punctuation">.</span><<span class="token operator">/</span>em><<span class="token operator">/</span>p> <<span class="token operator">/</span>body> <<span class="token operator">/</span>html> <span class="token comment"># 第3大步骤：启用ingress关联ingress controller 和service</span> <span class="token comment"># 创建一个yaml文件，去启动ingress</span> <span class="token namespace">[root@k8smaster ingress]</span><span class="token comment"># cat sc-ingress.yaml </span> apiVersion: networking<span class="token punctuation">.</span>k8s<span class="token punctuation">.</span>io/v1 kind: Ingress metadata: name: <span class="token function">sc</span><span class="token operator">-</span>ingress annotations: kubernets<span class="token punctuation">.</span>io/ingress<span class="token punctuation">.</span><span class="token keyword">class</span>: nginx <span class="token comment">#注释 这个ingress 是关联ingress controller的</span> spec: ingressClassName: nginx <span class="token comment">#关联ingress controller</span> rules: <span class="token operator">-</span> host: www<span class="token punctuation">.</span>feng<span class="token punctuation">.</span>com http: paths: <span class="token operator">-</span> pathType: Prefix path: <span class="token operator">/</span> backend: service: name: <span class="token function">sc</span><span class="token operator">-</span>nginx-svc port: number: 80 <span class="token operator">-</span> host: www<span class="token punctuation">.</span>zhang<span class="token punctuation">.</span>com http: paths: <span class="token operator">-</span> pathType: Prefix path: <span class="token operator">/</span> backend: service: name: <span class="token function">sc</span><span class="token operator">-</span>nginx-svc-2 port: number: 80 <span class="token namespace">[root@k8smaster ingress]</span><span class="token comment"># kubectl apply -f my-ingress.yaml </span> ingress<span class="token punctuation">.</span>networking<span class="token punctuation">.</span>k8s<span class="token punctuation">.</span>io/my-ingress created <span class="token comment"># 查看ingress</span> <span class="token namespace">[root@k8smaster ingress]</span><span class="token comment"># kubectl get ingress</span> NAME <span class="token keyword">CLASS</span> HOSTS ADDRESS PORTS AGE <span class="token function">sc</span><span class="token operator">-</span>ingress nginx www<span class="token punctuation">.</span>feng<span class="token punctuation">.</span>com<span class="token punctuation">,</span>www<span class="token punctuation">.</span>zhang<span class="token punctuation">.</span>com 192<span class="token punctuation">.</span>168<span class="token punctuation">.</span>2<span class="token punctuation">.</span>111<span class="token punctuation">,</span>192<span class="token punctuation">.</span>168<span class="token punctuation">.</span>2<span class="token punctuation">.</span>112 80 52s <span class="token comment"># 第4大步骤：查看ingress controller 里的nginx.conf 文件里是否有ingress对应的规则</span> <span class="token namespace">[root@k8smaster ingress]</span><span class="token comment"># kubectl get pod -n ingress-nginx</span> NAME READY STATUS RESTARTS AGE ingress-nginx-admission-create-9sg56 0/1 Completed 0 6m53s ingress-nginx-admission-patch-8sctb 0/1 Completed 1 6m53s ingress-nginx-controller-6c8ffbbfcf-bmdj9 1/1 Running 0 6m53s ingress-nginx-controller-6c8ffbbfcf-j576v 1/1 Running 0 6m53s <span class="token namespace">[root@k8smaster ingress]</span><span class="token comment"># kubectl exec -n ingress-nginx -it ingress-nginx-controller-6c8ffbbfcf-bmdj9 -- bash</span> bash-5<span class="token punctuation">.</span>1$ <span class="token function">cat</span> nginx<span class="token punctuation">.</span>conf <span class="token punctuation">|</span>grep feng<span class="token punctuation">.</span>com <span class="token comment">## start server www.feng.com</span> server_name www<span class="token punctuation">.</span>feng<span class="token punctuation">.</span>com <span class="token punctuation">;</span> <span class="token comment">## end server www.feng.com</span> bash-5<span class="token punctuation">.</span>1$ <span class="token function">cat</span> nginx<span class="token punctuation">.</span>conf <span class="token punctuation">|</span>grep zhang<span class="token punctuation">.</span>com <span class="token comment">## start server www.zhang.com</span> server_name www<span class="token punctuation">.</span>zhang<span class="token punctuation">.</span>com <span class="token punctuation">;</span> <span class="token comment">## end server www.zhang.com</span> bash-5<span class="token punctuation">.</span>1$ <span class="token function">cat</span> nginx<span class="token punctuation">.</span>conf<span class="token punctuation">|</span>grep <span class="token operator">-</span>C3 upstream_balancer error_log <span class="token operator">/</span><span class="token keyword">var</span><span class="token operator">/</span>log/nginx/error<span class="token punctuation">.</span>log notice<span class="token punctuation">;</span> upstream upstream_balancer <span class="token punctuation">{<!-- --></span> server 0<span class="token punctuation">.</span>0<span class="token punctuation">.</span>0<span class="token punctuation">.</span>1:1234<span class="token punctuation">;</span> <span class="token comment"># placeholder</span> <span class="token comment"># 获取ingress controller对应的service暴露宿主机的端口，访问宿主机和相关端口，就可以验证ingress controller是否能进行负载均衡</span> <span class="token namespace">[root@k8smaster ingress]</span><span class="token comment"># kubectl get svc -n ingress-nginx</span> NAME <span class="token function">TYPE</span> CLUSTER-IP EXTERNAL-IP PORT<span class="token punctuation">(</span>S<span class="token punctuation">)</span> AGE ingress-nginx-controller NodePort 10<span class="token punctuation">.</span>105<span class="token punctuation">.</span>213<span class="token punctuation">.</span>95 <none> 80:31457/TCP<span class="token punctuation">,</span>443:32569/TCP 8m12s ingress-nginx-controller-admission ClusterIP 10<span class="token punctuation">.</span>98<span class="token punctuation">.</span>225<span class="token punctuation">.</span>196 <none> 443/TCP 8m12s <span class="token comment"># 在其他的宿主机或者windows机器上使用域名进行访问</span> <span class="token namespace">[root@zabbix ~]</span><span class="token comment"># vim /etc/hosts</span> <span class="token namespace">[root@zabbix ~]</span><span class="token comment"># cat /etc/hosts</span> 127<span class="token punctuation">.</span>0<span class="token punctuation">.</span>0<span class="token punctuation">.</span>1 localhost localhost<span class="token punctuation">.</span>localdomain localhost4 localhost4<span class="token punctuation">.</span>localdomain4 ::1 localhost localhost<span class="token punctuation">.</span>localdomain localhost6 localhost6<span class="token punctuation">.</span>localdomain6 192<span class="token punctuation">.</span>168<span class="token punctuation">.</span>2<span class="token punctuation">.</span>111 www<span class="token punctuation">.</span>feng<span class="token punctuation">.</span>com 192<span class="token punctuation">.</span>168<span class="token punctuation">.</span>2<span class="token punctuation">.</span>112 www<span class="token punctuation">.</span>zhang<span class="token punctuation">.</span>com <span class="token comment"># 因为我们是基于域名做的负载均衡的配置，所以必须要在浏览器里使用域名去访问，不能使用ip地址</span> <span class="token comment"># 同时ingress controller做负载均衡的时候是基于http协议的，7层负载均衡。</span> <span class="token namespace">[root@zabbix ~]</span><span class="token comment"># curl www.feng.com</span> <<span class="token operator">!</span>DOCTYPE html> <html> <head> <title>Welcome to nginx!<<span class="token operator">/</span>title> <style> html <span class="token punctuation">{<!-- --></span> color-scheme: light dark<span class="token punctuation">;</span> <span class="token punctuation">}</span> body <span class="token punctuation">{<!-- --></span> width: 35em<span class="token punctuation">;</span> margin: 0 auto<span class="token punctuation">;</span> font-family: Tahoma<span class="token punctuation">,</span> Verdana<span class="token punctuation">,</span> Arial<span class="token punctuation">,</span> sans-serif<span class="token punctuation">;</span> <span class="token punctuation">}</span> <<span class="token operator">/</span>style> <<span class="token operator">/</span>head> <body> <h1>Welcome to nginx!<<span class="token operator">/</span>h1> <p><span class="token keyword">If</span> you see this page<span class="token punctuation">,</span> the nginx web server is successfully installed and working<span class="token punctuation">.</span> Further configuration is required<span class="token punctuation">.</span><<span class="token operator">/</span>p> <p><span class="token keyword">For</span> online documentation and support please refer to <a href=<span class="token string">"http://nginx.org/"</span>>nginx<span class="token punctuation">.</span>org<<span class="token operator">/</span>a><span class="token punctuation">.</span><br/> Commercial support is available at <a href=<span class="token string">"http://nginx.com/"</span>>nginx<span class="token punctuation">.</span>com<<span class="token operator">/</span>a><span class="token punctuation">.</span><<span class="token operator">/</span>p> <p><em>Thank you <span class="token keyword">for</span> <span class="token keyword">using</span> nginx<span class="token punctuation">.</span><<span class="token operator">/</span>em><<span class="token operator">/</span>p> <<span class="token operator">/</span>body> <<span class="token operator">/</span>html> <span class="token comment"># 访问www.zhang.com出现异常，503错误，是nginx内部错误</span> <span class="token namespace">[root@zabbix ~]</span><span class="token comment"># curl www.zhang.com</span> <html> <head><title>503 Service Temporarily Unavailable<<span class="token operator">/</span>title><<span class="token operator">/</span>head> <body> <center><h1>503 Service Temporarily Unavailable<<span class="token operator">/</span>h1><<span class="token operator">/</span>center> <hr><center>nginx<<span class="token operator">/</span>center> <<span class="token operator">/</span>body> <<span class="token operator">/</span>html> <span class="token comment"># 第5大步骤：启动第2个服务和pod，使用了pv+pvc+nfs</span> <span class="token comment"># 需要提前准备好nfs服务器+创建pv和pvc</span> <span class="token namespace">[root@k8smaster pv]</span><span class="token comment"># pwd</span> <span class="token operator">/</span>root/pv <span class="token namespace">[root@k8smaster pv]</span><span class="token comment"># ls</span> nfs-pvc<span class="token punctuation">.</span>yml nfs-pv<span class="token punctuation">.</span>yml nginx-deployment<span class="token punctuation">.</span>yml <span class="token namespace">[root@k8smaster pv]</span><span class="token comment"># cat nfs-pv.yml </span> apiVersion: v1 kind: PersistentVolume metadata: name: pv-web labels: <span class="token function">type</span>: pv-web spec: capacity: storage: 10Gi accessModes: <span class="token operator">-</span> ReadWriteMany storageClassName: nfs <span class="token comment"># pv对应的名字</span> nfs: path: <span class="token string">"/web"</span> <span class="token comment"># nfs共享的目录</span> server: 192<span class="token punctuation">.</span>168<span class="token punctuation">.</span>2<span class="token punctuation">.</span>121 <span class="token comment"># nfs服务器的ip地址</span> readOnly: false <span class="token comment"># 访问模式</span> <span class="token namespace">[root@k8smaster pv]</span><span class="token comment"># kubectl apply -f nfs-pv.yaml</span> <span class="token namespace">[root@k8smaster pv]</span><span class="token comment"># kubectl apply -f nfs-pvc.yaml</span> <span class="token namespace">[root@k8smaster pv]</span><span class="token comment"># kubectl get pv</span> NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pv-web 10Gi RWX Retain Bound default/pvc-web nfs 19h <span class="token namespace">[root@k8smaster pv]</span><span class="token comment"># kubectl get pvc</span> NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE pvc-web Bound pv-web 10Gi RWX nfs 19h <span class="token namespace">[root@k8smaster ingress]</span><span class="token comment"># cat nginx-deployment-nginx-svc-2.yaml </span> apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: <span class="token function">sc</span><span class="token operator">-</span>nginx-feng-2 template: metadata: labels: app: <span class="token function">sc</span><span class="token operator">-</span>nginx-feng-2 spec: volumes: <span class="token operator">-</span> name: <span class="token function">sc</span><span class="token operator">-</span>pv-storage-nfs persistentVolumeClaim: claimName: pvc-web containers: <span class="token operator">-</span> name: <span class="token function">sc</span><span class="token operator">-</span>pv-container-nfs image: nginx imagePullPolicy: IfNotPresent ports: <span class="token operator">-</span> containerPort: 80 name: <span class="token string">"http-server"</span> volumeMounts: <span class="token operator">-</span> mountPath: <span class="token string">"/usr/share/nginx/html"</span> name: <span class="token function">sc</span><span class="token operator">-</span>pv-storage-nfs <span class="token operator">--</span><span class="token operator">-</span> apiVersion: v1 kind: Service metadata: name: <span class="token function">sc</span><span class="token operator">-</span>nginx-svc-2 labels: app: <span class="token function">sc</span><span class="token operator">-</span>nginx-svc-2 spec: selector: app: <span class="token function">sc</span><span class="token operator">-</span>nginx-feng-2 ports: <span class="token operator">-</span> name: name-of-service-port protocol: TCP port: 80 targetPort: 80 <span class="token namespace">[root@k8smaster ingress]</span><span class="token comment"># kubectl apply -f nginx-deployment-nginx-svc-2.yaml </span> deployment<span class="token punctuation">.</span>apps/nginx-deployment created service/<span class="token function">sc</span><span class="token operator">-</span>nginx-svc-2 created <span class="token namespace">[root@k8smaster ingress]</span><span class="token comment"># kubectl get svc -n ingress-nginx</span> NAME <span class="token function">TYPE</span> CLUSTER-IP EXTERNAL-IP PORT<span class="token punctuation">(</span>S<span class="token punctuation">)</span> AGE ingress-nginx-controller NodePort 10<span class="token punctuation">.</span>105<span class="token punctuation">.</span>213<span class="token punctuation">.</span>95 <none> 80:31457/TCP<span class="token punctuation">,</span>443:32569/TCP 24m ingress-nginx-controller-admission ClusterIP 10<span class="token punctuation">.</span>98<span class="token punctuation">.</span>225<span class="token punctuation">.</span>196 <none> 443/TCP 24m <span class="token namespace">[root@k8smaster ingress]</span><span class="token comment"># kubectl get ingress</span> NAME <span class="token keyword">CLASS</span> HOSTS ADDRESS PORTS AGE <span class="token function">sc</span><span class="token operator">-</span>ingress nginx www<span class="token punctuation">.</span>feng<span class="token punctuation">.</span>com<span class="token punctuation">,</span>www<span class="token punctuation">.</span>zhang<span class="token punctuation">.</span>com 192<span class="token punctuation">.</span>168<span class="token punctuation">.</span>2<span class="token punctuation">.</span>111<span class="token punctuation">,</span>192<span class="token punctuation">.</span>168<span class="token punctuation">.</span>2<span class="token punctuation">.</span>112 80 18m <span class="token comment"># 访问宿主机暴露的端口号30092或者80都可以</span> <span class="token comment"># 使用ingress controller暴露服务，感觉不需要使用30000以上的端口访问，可以直接访问80或者443</span> 比使用service 暴露服务还是有点优势 <span class="token namespace">[root@zabbix ~]</span><span class="token comment"># curl www.zhang.com</span> welcome to changsha hello<span class="token punctuation">,</span>world <span class="token namespace">[root@zabbix ~]</span><span class="token comment"># curl www.feng.com</span> <<span class="token operator">!</span>DOCTYPE html> <html> <head> <title>Welcome to nginx!<<span class="token operator">/</span>title> <style> html <span class="token punctuation">{<!-- --></span> color-scheme: light dark<span class="token punctuation">;</span> <span class="token punctuation">}</span> body <span class="token punctuation">{<!-- --></span> width: 35em<span class="token punctuation">;</span> margin: 0 auto<span class="token punctuation">;</span> font-family: Tahoma<span class="token punctuation">,</span> Verdana<span class="token punctuation">,</span> Arial<span class="token punctuation">,</span> sans-serif<span class="token punctuation">;</span> <span class="token punctuation">}</span> <<span class="token operator">/</span>style> <<span class="token operator">/</span>head> <body> <h1>Welcome to nginx!<<span class="token operator">/</span>h1> <p><span class="token keyword">If</span> you see this page<span class="token punctuation">,</span> the nginx web server is successfully installed and working<span class="token punctuation">.</span> Further configuration is required<span class="token punctuation">.</span><<span class="token operator">/</span>p> <p><span class="token keyword">For</span> online documentation and support please refer to <a href=<span class="token string">"http://nginx.org/"</span>>nginx<span class="token punctuation">.</span>org<<span class="token operator">/</span>a><span class="token punctuation">.</span><br/> Commercial support is available at <a href=<span class="token string">"http://nginx.com/"</span>>nginx<span class="token punctuation">.</span>com<<span class="token operator">/</span>a><span class="token punctuation">.</span><<span class="token operator">/</span>p> <p><em>Thank you <span class="token keyword">for</span> <span class="token keyword">using</span> nginx<span class="token punctuation">.</span><<span class="token operator">/</span>em><<span class="token operator">/</span>p> <<span class="token operator">/</span>body> <<span class="token operator">/</span>html> <div class="hljs-button signin active" data-title="登录复制" data-report-click="{"spm":"1001.2101.3001.4334"}"></div></code><div class="hide-preCode-box"><span class="hide-preCode-bt" data-report-view="{"spm":"1001.2101.3001.7365"}"><img class="look-more-preCode contentImg-no-view" src="https://1000bd.com/contentImg/2022/06/27/191644837.png" alt="" title=""></span></div><ul class="pre-numbering" style=""><li style="color: rgb(153, 153, 153);">1</li><li style="color: rgb(153, 153, 153);">2</li><li style="color: rgb(153, 153, 153);">3</li><li style="color: rgb(153, 153, 153);">4</li><li style="color: rgb(153, 153, 153);">5</li><li style="color: rgb(153, 153, 153);">6</li><li style="color: rgb(153, 153, 153);">7</li><li style="color: rgb(153, 153, 153);">8</li><li style="color: rgb(153, 153, 153);">9</li><li style="color: rgb(153, 153, 153);">10</li><li style="color: rgb(153, 153, 153);">11</li><li style="color: rgb(153, 153, 153);">12</li><li style="color: rgb(153, 153, 153);">13</li><li style="color: rgb(153, 153, 153);">14</li><li style="color: rgb(153, 153, 153);">15</li><li style="color: rgb(153, 153, 153);">16</li><li style="color: rgb(153, 153, 153);">17</li><li style="color: rgb(153, 153, 153);">18</li><li style="color: rgb(153, 153, 153);">19</li><li style="color: rgb(153, 153, 153);">20</li><li style="color: rgb(153, 153, 153);">21</li><li style="color: rgb(153, 153, 153);">22</li><li style="color: rgb(153, 153, 153);">23</li><li style="color: rgb(153, 153, 153);">24</li><li style="color: rgb(153, 153, 153);">25</li><li style="color: rgb(153, 153, 153);">26</li><li style="color: rgb(153, 153, 153);">27</li><li style="color: rgb(153, 153, 153);">28</li><li style="color: rgb(153, 153, 153);">29</li><li style="color: rgb(153, 153, 153);">30</li><li style="color: rgb(153, 153, 153);">31</li><li style="color: rgb(153, 153, 153);">32</li><li style="color: rgb(153, 153, 153);">33</li><li style="color: rgb(153, 153, 153);">34</li><li style="color: rgb(153, 153, 153);">35</li><li style="color: rgb(153, 153, 153);">36</li><li style="color: rgb(153, 153, 153);">37</li><li style="color: rgb(153, 153, 153);">38</li><li style="color: rgb(153, 153, 153);">39</li><li style="color: rgb(153, 153, 153);">40</li><li style="color: rgb(153, 153, 153);">41</li><li style="color: rgb(153, 153, 153);">42</li><li style="color: rgb(153, 153, 153);">43</li><li style="color: rgb(153, 153, 153);">44</li><li style="color: rgb(153, 153, 153);">45</li><li style="color: rgb(153, 153, 153);">46</li><li style="color: rgb(153, 153, 153);">47</li><li style="color: rgb(153, 153, 153);">48</li><li style="color: rgb(153, 153, 153);">49</li><li style="color: rgb(153, 153, 153);">50</li><li style="color: rgb(153, 153, 153);">51</li><li style="color: rgb(153, 153, 153);">52</li><li style="color: rgb(153, 153, 153);">53</li><li style="color: rgb(153, 153, 153);">54</li><li style="color: rgb(153, 153, 153);">55</li><li style="color: rgb(153, 153, 153);">56</li><li style="color: rgb(153, 153, 153);">57</li><li style="color: rgb(153, 153, 153);">58</li><li style="color: rgb(153, 153, 153);">59</li><li style="color: rgb(153, 153, 153);">60</li><li style="color: rgb(153, 153, 153);">61</li><li style="color: rgb(153, 153, 153);">62</li><li style="color: rgb(153, 153, 153);">63</li><li style="color: rgb(153, 153, 153);">64</li><li style="color: rgb(153, 153, 153);">65</li><li style="color: rgb(153, 153, 153);">66</li><li style="color: rgb(153, 153, 153);">67</li><li style="color: rgb(153, 153, 153);">68</li><li style="color: rgb(153, 153, 153);">69</li><li style="color: rgb(153, 153, 153);">70</li><li style="color: rgb(153, 153, 153);">71</li><li style="color: rgb(153, 153, 153);">72</li><li style="color: rgb(153, 153, 153);">73</li><li style="color: rgb(153, 153, 153);">74</li><li style="color: rgb(153, 153, 153);">75</li><li style="color: rgb(153, 153, 153);">76</li><li style="color: rgb(153, 153, 153);">77</li><li style="color: rgb(153, 153, 153);">78</li><li style="color: rgb(153, 153, 153);">79</li><li style="color: rgb(153, 153, 153);">80</li><li style="color: rgb(153, 153, 153);">81</li><li style="color: rgb(153, 153, 153);">82</li><li style="color: rgb(153, 153, 153);">83</li><li style="color: rgb(153, 153, 153);">84</li><li style="color: rgb(153, 153, 153);">85</li><li style="color: rgb(153, 153, 153);">86</li><li style="color: rgb(153, 153, 153);">87</li><li style="color: rgb(153, 153, 153);">88</li><li style="color: rgb(153, 153, 153);">89</li><li style="color: rgb(153, 153, 153);">90</li><li style="color: rgb(153, 153, 153);">91</li><li style="color: rgb(153, 153, 153);">92</li><li style="color: rgb(153, 153, 153);">93</li><li style="color: rgb(153, 153, 153);">94</li><li style="color: rgb(153, 153, 153);">95</li><li style="color: rgb(153, 153, 153);">96</li><li style="color: rgb(153, 153, 153);">97</li><li style="color: rgb(153, 153, 153);">98</li><li style="color: rgb(153, 153, 153);">99</li><li style="color: rgb(153, 153, 153);">100</li><li style="color: rgb(153, 153, 153);">101</li><li style="color: rgb(153, 153, 153);">102</li><li style="color: rgb(153, 153, 153);">103</li><li style="color: rgb(153, 153, 153);">104</li><li style="color: rgb(153, 153, 153);">105</li><li style="color: rgb(153, 153, 153);">106</li><li style="color: rgb(153, 153, 153);">107</li><li style="color: rgb(153, 153, 153);">108</li><li style="color: rgb(153, 153, 153);">109</li><li style="color: rgb(153, 153, 153);">110</li><li style="color: rgb(153, 153, 153);">111</li><li style="color: rgb(153, 153, 153);">112</li><li style="color: rgb(153, 153, 153);">113</li><li style="color: rgb(153, 153, 153);">114</li><li style="color: rgb(153, 153, 153);">115</li><li style="color: rgb(153, 153, 153);">116</li><li style="color: rgb(153, 153, 153);">117</li><li style="color: rgb(153, 153, 153);">118</li><li style="color: rgb(153, 153, 153);">119</li><li style="color: rgb(153, 153, 153);">120</li><li style="color: rgb(153, 153, 153);">121</li><li style="color: rgb(153, 153, 153);">122</li><li style="color: rgb(153, 153, 153);">123</li><li style="color: rgb(153, 153, 153);">124</li><li style="color: rgb(153, 153, 153);">125</li><li style="color: rgb(153, 153, 153);">126</li><li style="color: rgb(153, 153, 153);">127</li><li style="color: rgb(153, 153, 153);">128</li><li style="color: rgb(153, 153, 153);">129</li><li style="color: rgb(153, 153, 153);">130</li><li style="color: rgb(153, 153, 153);">131</li><li style="color: rgb(153, 153, 153);">132</li><li style="color: rgb(153, 153, 153);">133</li><li style="color: rgb(153, 153, 153);">134</li><li style="color: rgb(153, 153, 153);">135</li><li style="color: rgb(153, 153, 153);">136</li><li style="color: rgb(153, 153, 153);">137</li><li style="color: rgb(153, 153, 153);">138</li><li style="color: rgb(153, 153, 153);">139</li><li style="color: rgb(153, 153, 153);">140</li><li style="color: rgb(153, 153, 153);">141</li><li style="color: rgb(153, 153, 153);">142</li><li style="color: rgb(153, 153, 153);">143</li><li style="color: rgb(153, 153, 153);">144</li><li style="color: rgb(153, 153, 153);">145</li><li style="color: rgb(153, 153, 153);">146</li><li style="color: rgb(153, 153, 153);">147</li><li style="color: rgb(153, 153, 153);">148</li><li style="color: rgb(153, 153, 153);">149</li><li style="color: rgb(153, 153, 153);">150</li><li style="color: rgb(153, 153, 153);">151</li><li style="color: rgb(153, 153, 153);">152</li><li style="color: rgb(153, 153, 153);">153</li><li style="color: rgb(153, 153, 153);">154</li><li style="color: rgb(153, 153, 153);">155</li><li style="color: rgb(153, 153, 153);">156</li><li style="color: rgb(153, 153, 153);">157</li><li style="color: rgb(153, 153, 153);">158</li><li style="color: rgb(153, 153, 153);">159</li><li style="color: rgb(153, 153, 153);">160</li><li style="color: rgb(153, 153, 153);">161</li><li style="color: rgb(153, 153, 153);">162</li><li style="color: rgb(153, 153, 153);">163</li><li style="color: rgb(153, 153, 153);">164</li><li style="color: rgb(153, 153, 153);">165</li><li style="color: rgb(153, 153, 153);">166</li><li style="color: rgb(153, 153, 153);">167</li><li style="color: rgb(153, 153, 153);">168</li><li style="color: rgb(153, 153, 153);">169</li><li style="color: rgb(153, 153, 153);">170</li><li style="color: rgb(153, 153, 153);">171</li><li style="color: rgb(153, 153, 153);">172</li><li style="color: rgb(153, 153, 153);">173</li><li style="color: rgb(153, 153, 153);">174</li><li style="color: rgb(153, 153, 153);">175</li><li style="color: rgb(153, 153, 153);">176</li><li style="color: rgb(153, 153, 153);">177</li><li style="color: rgb(153, 153, 153);">178</li><li style="color: rgb(153, 153, 153);">179</li><li style="color: rgb(153, 153, 153);">180</li><li style="color: rgb(153, 153, 153);">181</li><li style="color: rgb(153, 153, 153);">182</li><li style="color: rgb(153, 153, 153);">183</li><li style="color: rgb(153, 153, 153);">184</li><li style="color: rgb(153, 153, 153);">185</li><li style="color: rgb(153, 153, 153);">186</li><li style="color: rgb(153, 153, 153);">187</li><li style="color: rgb(153, 153, 153);">188</li><li style="color: rgb(153, 153, 153);">189</li><li style="color: rgb(153, 153, 153);">190</li><li style="color: rgb(153, 153, 153);">191</li><li style="color: rgb(153, 153, 153);">192</li><li style="color: rgb(153, 153, 153);">193</li><li style="color: rgb(153, 153, 153);">194</li><li style="color: rgb(153, 153, 153);">195</li><li style="color: rgb(153, 153, 153);">196</li><li style="color: rgb(153, 153, 153);">197</li><li style="color: rgb(153, 153, 153);">198</li><li style="color: rgb(153, 153, 153);">199</li><li style="color: rgb(153, 153, 153);">200</li><li style="color: rgb(153, 153, 153);">201</li><li style="color: rgb(153, 153, 153);">202</li><li style="color: rgb(153, 153, 153);">203</li><li style="color: rgb(153, 153, 153);">204</li><li style="color: rgb(153, 153, 153);">205</li><li style="color: rgb(153, 153, 153);">206</li><li style="color: rgb(153, 153, 153);">207</li><li style="color: rgb(153, 153, 153);">208</li><li style="color: rgb(153, 153, 153);">209</li><li style="color: rgb(153, 153, 153);">210</li><li style="color: rgb(153, 153, 153);">211</li><li style="color: rgb(153, 153, 153);">212</li><li style="color: rgb(153, 153, 153);">213</li><li style="color: rgb(153, 153, 153);">214</li><li style="color: rgb(153, 153, 153);">215</li><li style="color: rgb(153, 153, 153);">216</li><li style="color: rgb(153, 153, 153);">217</li><li style="color: rgb(153, 153, 153);">218</li><li style="color: rgb(153, 153, 153);">219</li><li style="color: rgb(153, 153, 153);">220</li><li style="color: rgb(153, 153, 153);">221</li><li style="color: rgb(153, 153, 153);">222</li><li style="color: rgb(153, 153, 153);">223</li><li style="color: rgb(153, 153, 153);">224</li><li style="color: rgb(153, 153, 153);">225</li><li style="color: rgb(153, 153, 153);">226</li><li style="color: rgb(153, 153, 153);">227</li><li style="color: rgb(153, 153, 153);">228</li><li style="color: rgb(153, 153, 153);">229</li><li style="color: rgb(153, 153, 153);">230</li><li style="color: rgb(153, 153, 153);">231</li><li style="color: rgb(153, 153, 153);">232</li><li style="color: rgb(153, 153, 153);">233</li><li style="color: rgb(153, 153, 153);">234</li><li style="color: rgb(153, 153, 153);">235</li><li style="color: rgb(153, 153, 153);">236</li><li style="color: rgb(153, 153, 153);">237</li><li style="color: rgb(153, 153, 153);">238</li><li style="color: rgb(153, 153, 153);">239</li><li style="color: rgb(153, 153, 153);">240</li><li style="color: rgb(153, 153, 153);">241</li><li style="color: rgb(153, 153, 153);">242</li><li style="color: rgb(153, 153, 153);">243</li><li style="color: rgb(153, 153, 153);">244</li><li style="color: rgb(153, 153, 153);">245</li><li style="color: rgb(153, 153, 153);">246</li><li style="color: rgb(153, 153, 153);">247</li><li style="color: rgb(153, 153, 153);">248</li><li style="color: rgb(153, 153, 153);">249</li><li style="color: rgb(153, 153, 153);">250</li><li style="color: rgb(153, 153, 153);">251</li><li style="color: rgb(153, 153, 153);">252</li><li style="color: rgb(153, 153, 153);">253</li><li style="color: rgb(153, 153, 153);">254</li><li style="color: rgb(153, 153, 153);">255</li><li style="color: rgb(153, 153, 153);">256</li><li style="color: rgb(153, 153, 153);">257</li><li style="color: rgb(153, 153, 153);">258</li><li style="color: rgb(153, 153, 153);">259</li><li style="color: rgb(153, 153, 153);">260</li><li style="color: rgb(153, 153, 153);">261</li><li style="color: rgb(153, 153, 153);">262</li><li style="color: rgb(153, 153, 153);">263</li><li style="color: rgb(153, 153, 153);">264</li><li style="color: rgb(153, 153, 153);">265</li><li style="color: rgb(153, 153, 153);">266</li><li style="color: rgb(153, 153, 153);">267</li><li style="color: rgb(153, 153, 153);">268</li><li style="color: rgb(153, 153, 153);">269</li><li style="color: rgb(153, 153, 153);">270</li><li style="color: rgb(153, 153, 153);">271</li><li style="color: rgb(153, 153, 153);">272</li><li style="color: rgb(153, 153, 153);">273</li><li style="color: rgb(153, 153, 153);">274</li><li style="color: rgb(153, 153, 153);">275</li><li style="color: rgb(153, 153, 153);">276</li><li style="color: rgb(153, 153, 153);">277</li><li style="color: rgb(153, 153, 153);">278</li><li style="color: rgb(153, 153, 153);">279</li><li style="color: rgb(153, 153, 153);">280</li><li style="color: rgb(153, 153, 153);">281</li><li style="color: rgb(153, 153, 153);">282</li><li style="color: rgb(153, 153, 153);">283</li><li style="color: rgb(153, 153, 153);">284</li><li style="color: rgb(153, 153, 153);">285</li><li style="color: rgb(153, 153, 153);">286</li><li style="color: rgb(153, 153, 153);">287</li><li style="color: rgb(153, 153, 153);">288</li><li style="color: rgb(153, 153, 153);">289</li><li style="color: rgb(153, 153, 153);">290</li><li style="color: rgb(153, 153, 153);">291</li><li style="color: rgb(153, 153, 153);">292</li><li style="color: rgb(153, 153, 153);">293</li><li style="color: rgb(153, 153, 153);">294</li><li style="color: rgb(153, 153, 153);">295</li><li style="color: rgb(153, 153, 153);">296</li><li style="color: rgb(153, 153, 153);">297</li><li style="color: rgb(153, 153, 153);">298</li><li style="color: rgb(153, 153, 153);">299</li><li style="color: rgb(153, 153, 153);">300</li><li style="color: rgb(153, 153, 153);">301</li><li style="color: rgb(153, 153, 153);">302</li><li style="color: rgb(153, 153, 153);">303</li><li style="color: rgb(153, 153, 153);">304</li><li style="color: rgb(153, 153, 153);">305</li><li style="color: rgb(153, 153, 153);">306</li><li style="color: rgb(153, 153, 153);">307</li><li style="color: rgb(153, 153, 153);">308</li><li style="color: rgb(153, 153, 153);">309</li><li style="color: rgb(153, 153, 153);">310</li><li style="color: rgb(153, 153, 153);">311</li><li style="color: rgb(153, 153, 153);">312</li><li style="color: rgb(153, 153, 153);">313</li><li style="color: rgb(153, 153, 153);">314</li><li style="color: rgb(153, 153, 153);">315</li><li style="color: rgb(153, 153, 153);">316</li><li style="color: rgb(153, 153, 153);">317</li><li style="color: rgb(153, 153, 153);">318</li><li style="color: rgb(153, 153, 153);">319</li><li style="color: rgb(153, 153, 153);">320</li><li style="color: rgb(153, 153, 153);">321</li><li style="color: rgb(153, 153, 153);">322</li><li style="color: rgb(153, 153, 153);">323</li><li style="color: rgb(153, 153, 153);">324</li><li style="color: rgb(153, 153, 153);">325</li><li style="color: rgb(153, 153, 153);">326</li><li style="color: rgb(153, 153, 153);">327</li><li style="color: rgb(153, 153, 153);">328</li><li style="color: rgb(153, 153, 153);">329</li><li style="color: rgb(153, 153, 153);">330</li><li style="color: rgb(153, 153, 153);">331</li><li style="color: rgb(153, 153, 153);">332</li><li style="color: rgb(153, 153, 153);">333</li><li style="color: rgb(153, 153, 153);">334</li><li style="color: rgb(153, 153, 153);">335</li><li style="color: rgb(153, 153, 153);">336</li><li style="color: rgb(153, 153, 153);">337</li><li style="color: rgb(153, 153, 153);">338</li><li style="color: rgb(153, 153, 153);">339</li><li style="color: rgb(153, 153, 153);">340</li><li style="color: rgb(153, 153, 153);">341</li><li style="color: rgb(153, 153, 153);">342</li><li style="color: rgb(153, 153, 153);">343</li><li style="color: rgb(153, 153, 153);">344</li><li style="color: rgb(153, 153, 153);">345</li><li style="color: rgb(153, 153, 153);">346</li><li style="color: rgb(153, 153, 153);">347</li><li style="color: rgb(153, 153, 153);">348</li><li style="color: rgb(153, 153, 153);">349</li><li style="color: rgb(153, 153, 153);">350</li><li style="color: rgb(153, 153, 153);">351</li><li style="color: rgb(153, 153, 153);">352</li><li style="color: rgb(153, 153, 153);">353</li><li style="color: rgb(153, 153, 153);">354</li><li style="color: rgb(153, 153, 153);">355</li><li style="color: rgb(153, 153, 153);">356</li><li style="color: rgb(153, 153, 153);">357</li><li style="color: rgb(153, 153, 153);">358</li><li style="color: rgb(153, 153, 153);">359</li><li style="color: rgb(153, 153, 153);">360</li><li style="color: rgb(153, 153, 153);">361</li><li style="color: rgb(153, 153, 153);">362</li><li style="color: rgb(153, 153, 153);">363</li><li style="color: rgb(153, 153, 153);">364</li><li style="color: rgb(153, 153, 153);">365</li><li style="color: rgb(153, 153, 153);">366</li><li style="color: rgb(153, 153, 153);">367</li><li style="color: rgb(153, 153, 153);">368</li><li style="color: rgb(153, 153, 153);">369</li><li style="color: rgb(153, 153, 153);">370</li><li style="color: rgb(153, 153, 153);">371</li><li style="color: rgb(153, 153, 153);">372</li><li style="color: rgb(153, 153, 153);">373</li><li style="color: rgb(153, 153, 153);">374</li><li style="color: rgb(153, 153, 153);">375</li><li style="color: rgb(153, 153, 153);">376</li><li style="color: rgb(153, 153, 153);">377</li><li style="color: rgb(153, 153, 153);">378</li><li style="color: rgb(153, 153, 153);">379</li><li style="color: rgb(153, 153, 153);">380</li><li style="color: rgb(153, 153, 153);">381</li><li style="color: rgb(153, 153, 153);">382</li><li style="color: rgb(153, 153, 153);">383</li><li style="color: rgb(153, 153, 153);">384</li><li style="color: rgb(153, 153, 153);">385</li><li style="color: rgb(153, 153, 153);">386</li><li style="color: rgb(153, 153, 153);">387</li><li style="color: rgb(153, 153, 153);">388</li><li style="color: rgb(153, 153, 153);">389</li><li style="color: rgb(153, 153, 153);">390</li><li style="color: rgb(153, 153, 153);">391</li><li style="color: rgb(153, 153, 153);">392</li><li style="color: rgb(153, 153, 153);">393</li><li style="color: rgb(153, 153, 153);">394</li><li style="color: rgb(153, 153, 153);">395</li><li style="color: rgb(153, 153, 153);">396</li><li style="color: rgb(153, 153, 153);">397</li><li style="color: rgb(153, 153, 153);">398</li><li style="color: rgb(153, 153, 153);">399</li><li style="color: rgb(153, 153, 153);">400</li><li style="color: rgb(153, 153, 153);">401</li><li style="color: rgb(153, 153, 153);">402</li><li style="color: rgb(153, 153, 153);">403</li><li style="color: rgb(153, 153, 153);">404</li><li style="color: rgb(153, 153, 153);">405</li><li style="color: rgb(153, 153, 153);">406</li><li style="color: rgb(153, 153, 153);">407</li><li style="color: rgb(153, 153, 153);">408</li><li style="color: rgb(153, 153, 153);">409</li><li style="color: rgb(153, 153, 153);">410</li><li style="color: rgb(153, 153, 153);">411</li><li style="color: rgb(153, 153, 153);">412</li><li style="color: rgb(153, 153, 153);">413</li><li style="color: rgb(153, 153, 153);">414</li><li style="color: rgb(153, 153, 153);">415</li><li style="color: rgb(153, 153, 153);">416</li><li style="color: rgb(153, 153, 153);">417</li><li style="color: rgb(153, 153, 153);">418</li><li style="color: rgb(153, 153, 153);">419</li><li style="color: rgb(153, 153, 153);">420</li><li style="color: rgb(153, 153, 153);">421</li><li style="color: rgb(153, 153, 153);">422</li><li style="color: rgb(153, 153, 153);">423</li><li style="color: rgb(153, 153, 153);">424</li><li style="color: rgb(153, 153, 153);">425</li><li style="color: rgb(153, 153, 153);">426</li><li style="color: rgb(153, 153, 153);">427</li><li style="color: rgb(153, 153, 153);">428</li><li style="color: rgb(153, 153, 153);">429</li><li style="color: rgb(153, 153, 153);">430</li><li style="color: rgb(153, 153, 153);">431</li><li style="color: rgb(153, 153, 153);">432</li></ul></pre> <h4><a name="t20"></a><a id="9dashboard_2771"></a>9、使用dashboard对整个集群资源进行掌控</h4> <pre data-index="19" class="set-code-hide prettyprint"><code class="prism language-powershell has-numbering" onclick="mdcp.signin(event)" style="position: unset;"><span class="token comment"># 1.先下载recommended.yaml文件</span> <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.5.0/aio/deploy/recommended.yaml</span> <span class="token operator">--</span>2023-06-19 10:18:50-<span class="token operator">-</span> https:<span class="token operator">/</span><span class="token operator">/</span>raw<span class="token punctuation">.</span>githubusercontent<span class="token punctuation">.</span>com/kubernetes/dashboard/v2<span class="token punctuation">.</span>5<span class="token punctuation">.</span>0/aio/deploy/recommended<span class="token punctuation">.</span>yaml 正在解析主机 raw<span class="token punctuation">.</span>githubusercontent<span class="token punctuation">.</span>com <span class="token punctuation">(</span>raw<span class="token punctuation">.</span>githubusercontent<span class="token punctuation">.</span>com<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span> 185<span class="token punctuation">.</span>199<span class="token punctuation">.</span>110<span class="token punctuation">.</span>133<span class="token punctuation">,</span> 185<span class="token punctuation">.</span>199<span class="token punctuation">.</span>108<span class="token punctuation">.</span>133<span class="token punctuation">,</span> 185<span class="token punctuation">.</span>199<span class="token punctuation">.</span>111<span class="token punctuation">.</span>133<span class="token punctuation">,</span> <span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span> 正在连接 raw<span class="token punctuation">.</span>githubusercontent<span class="token punctuation">.</span>com <span class="token punctuation">(</span>raw<span class="token punctuation">.</span>githubusercontent<span class="token punctuation">.</span>com<span class="token punctuation">)</span><span class="token punctuation">|</span>185<span class="token punctuation">.</span>199<span class="token punctuation">.</span>110<span class="token punctuation">.</span>133<span class="token punctuation">|</span>:443<span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span> 已连接。 已发出 HTTP 请求，正在等待回应<span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span> 200 OK 长度：7621 <span class="token punctuation">(</span>7<span class="token punctuation">.</span>4K<span class="token punctuation">)</span> <span class="token namespace">[text/plain]</span> 正在保存至: “recommended<span class="token punctuation">.</span>yaml” 100%<span class="token punctuation">[</span>=============================================================================><span class="token punctuation">]</span> 7<span class="token punctuation">,</span>621 <span class="token operator">--</span><span class="token punctuation">.</span><span class="token operator">-</span>K/s 用时 0s 2023-06-19 10:18:52 <span class="token punctuation">(</span>23<span class="token punctuation">.</span>6 MB/s<span class="token punctuation">)</span> <span class="token operator">-</span> 已保存 “recommended<span class="token punctuation">.</span>yaml” <span class="token punctuation">[</span>7621/7621<span class="token punctuation">]</span><span class="token punctuation">)</span> <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># ls</span> recommended<span class="token punctuation">.</span>yaml <span class="token comment"># 2.启动</span> <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># kubectl apply -f recommended.yaml </span> namespace/kubernetes-dashboard created serviceaccount/kubernetes-dashboard created service/kubernetes-dashboard created secret/kubernetes-dashboard-certs created secret/kubernetes-dashboard-csrf created secret/kubernetes-dashboard-key-holder created configmap/kubernetes-dashboard-settings created role<span class="token punctuation">.</span>rbac<span class="token punctuation">.</span>authorization<span class="token punctuation">.</span>k8s<span class="token punctuation">.</span>io/kubernetes-dashboard created clusterrole<span class="token punctuation">.</span>rbac<span class="token punctuation">.</span>authorization<span class="token punctuation">.</span>k8s<span class="token punctuation">.</span>io/kubernetes-dashboard created rolebinding<span class="token punctuation">.</span>rbac<span class="token punctuation">.</span>authorization<span class="token punctuation">.</span>k8s<span class="token punctuation">.</span>io/kubernetes-dashboard created clusterrolebinding<span class="token punctuation">.</span>rbac<span class="token punctuation">.</span>authorization<span class="token punctuation">.</span>k8s<span class="token punctuation">.</span>io/kubernetes-dashboard created deployment<span class="token punctuation">.</span>apps/kubernetes-dashboard created service/dashboard-metrics-scraper created deployment<span class="token punctuation">.</span>apps/dashboard-metrics-scraper created <span class="token comment"># 3.查看是否启动dashboard的pod</span> <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># kubectl get ns</span> NAME STATUS AGE default Active 18h ingress-nginx Active 13h kube-node-lease Active 18h kube-public Active 18h kube-system Active 18h kubernetes-dashboard Active 9s <span class="token comment"># kubernetes-dashboard 是dashboard自己的命名空间</span> <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># kubectl get pod -n kubernetes-dashboard</span> NAME READY STATUS RESTARTS AGE dashboard-metrics-scraper-5b8896d7fc-6kjlr 1/1 Running 0 4m56s kubernetes-dashboard-cb988587b-s2f6z 1/1 Running 0 4m57s <span class="token comment"># 4.查看dashboard对应的服务，因为发布服务的类型是ClusterIP ，外面的机器不能访问，不便于我们通过浏览器访问，因此需要改成NodePort</span> <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># kubectl get svc -n kubernetes-dashboard</span> NAME <span class="token function">TYPE</span> CLUSTER-IP EXTERNAL-IP PORT<span class="token punctuation">(</span>S<span class="token punctuation">)</span> AGE dashboard-metrics-scraper ClusterIP 10<span class="token punctuation">.</span>110<span class="token punctuation">.</span>32<span class="token punctuation">.</span>41 <none> 8000/TCP 4m24s kubernetes-dashboard ClusterIP 10<span class="token punctuation">.</span>106<span class="token punctuation">.</span>104<span class="token punctuation">.</span>124 <none> 443/TCP 4m24s <span class="token comment"># 5.删除已经创建的dashboard 的服务</span> <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># kubectl delete svc kubernetes-dashboard -n kubernetes-dashboard</span> service <span class="token string">"kubernetes-dashboard"</span> deleted <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># kubectl get svc -n kubernetes-dashboard</span> NAME <span class="token function">TYPE</span> CLUSTER-IP EXTERNAL-IP PORT<span class="token punctuation">(</span>S<span class="token punctuation">)</span> AGE dashboard-metrics-scraper ClusterIP 10<span class="token punctuation">.</span>110<span class="token punctuation">.</span>32<span class="token punctuation">.</span>41 <none> 8000/TCP 5m39s <span class="token comment"># 6.创建一个nodeport的service</span> <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># vim dashboard-svc.yml</span> <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># cat dashboard-svc.yml</span> kind: Service apiVersion: v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard spec: <span class="token function">type</span>: NodePort ports: <span class="token operator">-</span> port: 443 targetPort: 8443 selector: k8s-app: kubernetes-dashboard <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># kubectl apply -f dashboard-svc.yml</span> service/kubernetes-dashboard created <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># kubectl get svc -n kubernetes-dashboard</span> NAME <span class="token function">TYPE</span> CLUSTER-IP EXTERNAL-IP PORT<span class="token punctuation">(</span>S<span class="token punctuation">)</span> AGE dashboard-metrics-scraper ClusterIP 10<span class="token punctuation">.</span>110<span class="token punctuation">.</span>32<span class="token punctuation">.</span>41 <none> 8000/TCP 8m11s kubernetes-dashboard NodePort 10<span class="token punctuation">.</span>103<span class="token punctuation">.</span>185<span class="token punctuation">.</span>254 <none> 443:32571/TCP 37s <span class="token comment"># 7.想要访问dashboard服务，就要有访问权限，创建kubernetes-dashboard管理员角色</span> <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># vim dashboard-svc-account.yaml</span> <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># cat dashboard-svc-account.yaml </span> apiVersion: v1 kind: ServiceAccount metadata: name: dashboard-admin namespace: kube-system <span class="token operator">--</span><span class="token operator">-</span> kind: ClusterRoleBinding apiVersion: rbac<span class="token punctuation">.</span>authorization<span class="token punctuation">.</span>k8s<span class="token punctuation">.</span>io/v1 metadata: name: dashboard-admin subjects: <span class="token operator">-</span> kind: ServiceAccount name: dashboard-admin namespace: kube-system roleRef: kind: ClusterRole name: cluster-admin apiGroup: rbac<span class="token punctuation">.</span>authorization<span class="token punctuation">.</span>k8s<span class="token punctuation">.</span>io <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># kubectl apply -f dashboard-svc-account.yaml </span> serviceaccount/dashboard-admin created clusterrolebinding<span class="token punctuation">.</span>rbac<span class="token punctuation">.</span>authorization<span class="token punctuation">.</span>k8s<span class="token punctuation">.</span>io/dashboard-admin created <span class="token comment"># 8.获取dashboard的secret对象的名字</span> <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># kubectl get secret -n kube-system|grep admin|awk '{print $1}'</span> dashboard-admin-token-hd2nl <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># kubectl describe secret dashboard-admin-token-hd2nl -n kube-system</span> Name: dashboard-admin-token-hd2nl Namespace: kube-system Labels: <none> Annotations: kubernetes<span class="token punctuation">.</span>io/service-account<span class="token punctuation">.</span>name: dashboard-admin kubernetes<span class="token punctuation">.</span>io/service-account<span class="token punctuation">.</span>uid: 4e42ca6a-e5eb-4672-bf3e-ae22935417ef <span class="token function">Type</span>: kubernetes<span class="token punctuation">.</span>io/service-account-token <span class="token keyword">Data</span> ==== ca<span class="token punctuation">.</span>crt: 1066 bytes namespace: 11 bytes token: eyJhbGciOiJSUzI1NiIsImtpZCI6InBBckJ2U051Y3J4NjVPY2VxOVZzRjBIdzdjNzgycFppcVZ5WWFnQlNsS00ifQ<span class="token punctuation">.</span>eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4taGQybmwiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiNGU0MmNhNmEtZTVlYi00NjcyLWJmM2UtYWUyMjkzNTQxN2VmIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9<span class="token punctuation">.</span>EAVV-s6OnS4htu4kvv3UvlZpqzg5Ei1_tNiBLr08GquUxKX09JGvQhsZQYgluNmS2yqad_lxK_Ie_RgwayqfBdXYtugQPM8m9gZHScsUdo_3b8b4ZEUz7KlDzJVBdBvDFSJjz-7cJhtj-HtazRuLluJbeoQV4zXMXvfhDhYt0k126eiqKzvbHhJmNM8U5XViAUmpUPCUjqFHm8tS1Su7aW75R-qXH6aGjGOv7kTpQdOjFeVO-AbFRIcbDOcqYRrKMyZu0yuH9QZGL35L1Lj3HgePsDbwd3jm2ZS05BjuacSFGle6CdZTOB0b5haeUlFrZ6FWsU-2qoQ67ysOwB0xKQ <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># </span> <span class="token comment"># 9.获取secret里的token的内容--》token理解为认证的密码</span> <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># kubectl describe secret dashboard-admin-token-hd2nl -n kube-system|awk '/^token/ {print $2}'</span> eyJhbGciOiJSUzI1NiIsImtpZCI6InBBckJ2U051Y3J4NjVPY2VxOVZzRjBIdzdjNzgycFppcVZ5WWFnQlNsS00ifQ<span class="token punctuation">.</span>eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4taGQybmwiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiNGU0MmNhNmEtZTVlYi00NjcyLWJmM2UtYWUyMjkzNTQxN2VmIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9<span class="token punctuation">.</span>EAVV-s6OnS4htu4kvv3UvlZpqzg5Ei1_tNiBLr08GquUxKX09JGvQhsZQYgluNmS2yqad_lxK_Ie_RgwayqfBdXYtugQPM8m9gZHScsUdo_3b8b4ZEUz7KlDzJVBdBvDFSJjz-7cJhtj-HtazRuLluJbeoQV4zXMXvfhDhYt0k126eiqKzvbHhJmNM8U5XViAUmpUPCUjqFHm8tS1Su7aW75R-qXH6aGjGOv7kTpQdOjFeVO-AbFRIcbDOcqYRrKMyZu0yuH9QZGL35L1Lj3HgePsDbwd3jm2ZS05BjuacSFGle6CdZTOB0b5haeUlFrZ6FWsU-2qoQ67ysOwB0xKQ <span class="token comment"># 10.浏览器里访问</span> <span class="token namespace">[root@k8smaster dashboard]</span><span class="token comment"># kubectl get svc -n kubernetes-dashboard</span> NAME <span class="token function">TYPE</span> CLUSTER-IP EXTERNAL-IP PORT<span class="token punctuation">(</span>S<span class="token punctuation">)</span> AGE dashboard-metrics-scraper ClusterIP 10<span class="token punctuation">.</span>110<span class="token punctuation">.</span>32<span class="token punctuation">.</span>41 <none> 8000/TCP 11m kubernetes-dashboard NodePort 10<span class="token punctuation">.</span>103<span class="token punctuation">.</span>185<span class="token punctuation">.</span>254 <none> 443:32571/TCP 4m4s <span class="token comment"># 访问宿主机的ip+端口号</span> https:<span class="token operator">/</span><span class="token operator">/</span>192<span class="token punctuation">.</span>168<span class="token punctuation">.</span>2<span class="token punctuation">.</span>104:32571/<span class="token comment">#/login</span> <span class="token comment"># 11.输入上面获得的token，登录。</span> thisisunsafe https:<span class="token operator">/</span><span class="token operator">/</span>192<span class="token punctuation">.</span>168<span class="token punctuation">.</span>2<span class="token punctuation">.</span>104:32571/<span class="token comment">#/workloads?namespace=default</span> <div class="hljs-button signin active" data-title="登录复制" data-report-click="{"spm":"1001.2101.3001.4334"}"></div></code><div class="hide-preCode-box"><span class="hide-preCode-bt" data-report-view="{"spm":"1001.2101.3001.7365"}"><img class="look-more-preCode contentImg-no-view" src="https://1000bd.com/contentImg/2022/06/27/191644837.png" alt="" title=""></span></div><ul class="pre-numbering" style=""><li style="color: rgb(153, 153, 153);">1</li><li style="color: rgb(153, 153, 153);">2</li><li style="color: rgb(153, 153, 153);">3</li><li style="color: rgb(153, 153, 153);">4</li><li style="color: rgb(153, 153, 153);">5</li><li style="color: rgb(153, 153, 153);">6</li><li style="color: rgb(153, 153, 153);">7</li><li style="color: rgb(153, 153, 153);">8</li><li style="color: rgb(153, 153, 153);">9</li><li style="color: rgb(153, 153, 153);">10</li><li style="color: rgb(153, 153, 153);">11</li><li style="color: rgb(153, 153, 153);">12</li><li style="color: rgb(153, 153, 153);">13</li><li style="color: rgb(153, 153, 153);">14</li><li style="color: rgb(153, 153, 153);">15</li><li style="color: rgb(153, 153, 153);">16</li><li style="color: rgb(153, 153, 153);">17</li><li style="color: rgb(153, 153, 153);">18</li><li style="color: rgb(153, 153, 153);">19</li><li style="color: rgb(153, 153, 153);">20</li><li style="color: rgb(153, 153, 153);">21</li><li style="color: rgb(153, 153, 153);">22</li><li style="color: rgb(153, 153, 153);">23</li><li style="color: rgb(153, 153, 153);">24</li><li style="color: rgb(153, 153, 153);">25</li><li style="color: rgb(153, 153, 153);">26</li><li style="color: rgb(153, 153, 153);">27</li><li style="color: rgb(153, 153, 153);">28</li><li style="color: rgb(153, 153, 153);">29</li><li style="color: rgb(153, 153, 153);">30</li><li style="color: rgb(153, 153, 153);">31</li><li style="color: rgb(153, 153, 153);">32</li><li style="color: rgb(153, 153, 153);">33</li><li style="color: rgb(153, 153, 153);">34</li><li style="color: rgb(153, 153, 153);">35</li><li style="color: rgb(153, 153, 153);">36</li><li style="color: rgb(153, 153, 153);">37</li><li style="color: rgb(153, 153, 153);">38</li><li style="color: rgb(153, 153, 153);">39</li><li style="color: rgb(153, 153, 153);">40</li><li style="color: rgb(153, 153, 153);">41</li><li style="color: rgb(153, 153, 153);">42</li><li style="color: rgb(153, 153, 153);">43</li><li style="color: rgb(153, 153, 153);">44</li><li style="color: rgb(153, 153, 153);">45</li><li style="color: rgb(153, 153, 153);">46</li><li style="color: rgb(153, 153, 153);">47</li><li style="color: rgb(153, 153, 153);">48</li><li style="color: rgb(153, 153, 153);">49</li><li style="color: rgb(153, 153, 153);">50</li><li style="color: rgb(153, 153, 153);">51</li><li style="color: rgb(153, 153, 153);">52</li><li style="color: rgb(153, 153, 153);">53</li><li style="color: rgb(153, 153, 153);">54</li><li style="color: rgb(153, 153, 153);">55</li><li style="color: rgb(153, 153, 153);">56</li><li style="color: rgb(153, 153, 153);">57</li><li style="color: rgb(153, 153, 153);">58</li><li style="color: rgb(153, 153, 153);">59</li><li style="color: rgb(153, 153, 153);">60</li><li style="color: rgb(153, 153, 153);">61</li><li style="color: rgb(153, 153, 153);">62</li><li style="color: rgb(153, 153, 153);">63</li><li style="color: rgb(153, 153, 153);">64</li><li style="color: rgb(153, 153, 153);">65</li><li style="color: rgb(153, 153, 153);">66</li><li style="color: rgb(153, 153, 153);">67</li><li style="color: rgb(153, 153, 153);">68</li><li style="color: rgb(153, 153, 153);">69</li><li style="color: rgb(153, 153, 153);">70</li><li style="color: rgb(153, 153, 153);">71</li><li style="color: rgb(153, 153, 153);">72</li><li style="color: rgb(153, 153, 153);">73</li><li style="color: rgb(153, 153, 153);">74</li><li style="color: rgb(153, 153, 153);">75</li><li style="color: rgb(153, 153, 153);">76</li><li style="color: rgb(153, 153, 153);">77</li><li style="color: rgb(153, 153, 153);">78</li><li style="color: rgb(153, 153, 153);">79</li><li style="color: rgb(153, 153, 153);">80</li><li style="color: rgb(153, 153, 153);">81</li><li style="color: rgb(153, 153, 153);">82</li><li style="color: rgb(153, 153, 153);">83</li><li style="color: rgb(153, 153, 153);">84</li><li style="color: rgb(153, 153, 153);">85</li><li style="color: rgb(153, 153, 153);">86</li><li style="color: rgb(153, 153, 153);">87</li><li style="color: rgb(153, 153, 153);">88</li><li style="color: rgb(153, 153, 153);">89</li><li style="color: rgb(153, 153, 153);">90</li><li style="color: rgb(153, 153, 153);">91</li><li style="color: rgb(153, 153, 153);">92</li><li style="color: rgb(153, 153, 153);">93</li><li style="color: rgb(153, 153, 153);">94</li><li style="color: rgb(153, 153, 153);">95</li><li style="color: rgb(153, 153, 153);">96</li><li style="color: rgb(153, 153, 153);">97</li><li style="color: rgb(153, 153, 153);">98</li><li style="color: rgb(153, 153, 153);">99</li><li style="color: rgb(153, 153, 153);">100</li><li style="color: rgb(153, 153, 153);">101</li><li style="color: rgb(153, 153, 153);">102</li><li style="color: rgb(153, 153, 153);">103</li><li style="color: rgb(153, 153, 153);">104</li><li style="color: rgb(153, 153, 153);">105</li><li style="color: rgb(153, 153, 153);">106</li><li style="color: rgb(153, 153, 153);">107</li><li style="color: rgb(153, 153, 153);">108</li><li style="color: rgb(153, 153, 153);">109</li><li style="color: rgb(153, 153, 153);">110</li><li style="color: rgb(153, 153, 153);">111</li><li style="color: rgb(153, 153, 153);">112</li><li style="color: rgb(153, 153, 153);">113</li><li style="color: rgb(153, 153, 153);">114</li><li style="color: rgb(153, 153, 153);">115</li><li style="color: rgb(153, 153, 153);">116</li><li style="color: rgb(153, 153, 153);">117</li><li style="color: rgb(153, 153, 153);">118</li><li style="color: rgb(153, 153, 153);">119</li><li style="color: rgb(153, 153, 153);">120</li><li style="color: rgb(153, 153, 153);">121</li><li style="color: rgb(153, 153, 153);">122</li><li style="color: rgb(153, 153, 153);">123</li><li style="color: rgb(153, 153, 153);">124</li><li style="color: rgb(153, 153, 153);">125</li><li style="color: rgb(153, 153, 153);">126</li><li style="color: rgb(153, 153, 153);">127</li><li style="color: rgb(153, 153, 153);">128</li><li style="color: rgb(153, 153, 153);">129</li><li style="color: rgb(153, 153, 153);">130</li><li style="color: rgb(153, 153, 153);">131</li><li style="color: rgb(153, 153, 153);">132</li><li style="color: rgb(153, 153, 153);">133</li><li style="color: rgb(153, 153, 153);">134</li><li style="color: rgb(153, 153, 153);">135</li><li style="color: rgb(153, 153, 153);">136</li><li style="color: rgb(153, 153, 153);">137</li><li style="color: rgb(153, 153, 153);">138</li><li style="color: rgb(153, 153, 153);">139</li><li style="color: rgb(153, 153, 153);">140</li><li style="color: rgb(153, 153, 153);">141</li><li style="color: rgb(153, 153, 153);">142</li><li style="color: rgb(153, 153, 153);">143</li><li style="color: rgb(153, 153, 153);">144</li><li style="color: rgb(153, 153, 153);">145</li><li style="color: rgb(153, 153, 153);">146</li><li style="color: rgb(153, 153, 153);">147</li><li style="color: rgb(153, 153, 153);">148</li><li style="color: rgb(153, 153, 153);">149</li><li style="color: rgb(153, 153, 153);">150</li><li style="color: rgb(153, 153, 153);">151</li></ul></pre> <h4><a name="t21"></a><a id="10zabbixpromethuescpuwebIO_2927"></a>10、安装zabbix和promethues对整个集群资源（cpu，内存，网络带宽，web服务，数据库服务，磁盘IO等）进行监控</h4> <pre data-index="20" class="set-code-hide prettyprint"><code class="prism language-powershell has-numbering" onclick="mdcp.signin(event)" style="position: unset;"><span class="token comment"># 部署zabbix</span> <span class="token comment"># 1.安装zabbix服务器的源</span> 源：repository 软件仓库，用来找到zabbix官方网站提供的软件，可以下载软件的地方 <span class="token namespace">[root@zabbix ~]</span><span class="token comment"># rpm -Uvh https://repo.zabbix.com/zabbix/5.0/rhel/7/x86_64/zabbix-release-5.0-1.el7.noarch.rpm</span> 获取https:<span class="token operator">/</span><span class="token operator">/</span>repo<span class="token punctuation">.</span>zabbix<span class="token punctuation">.</span>com/zabbix/5<span class="token punctuation">.</span>0/rhel/7/x86_64/zabbix-release-5<span class="token punctuation">.</span>0-1<span class="token punctuation">.</span>el7<span class="token punctuation">.</span>noarch<span class="token punctuation">.</span>rpm 警告：<span class="token operator">/</span><span class="token keyword">var</span><span class="token operator">/</span>tmp/rpm-tmp<span class="token punctuation">.</span>lL96Rw: 头V4 RSA/SHA512 Signature<span class="token punctuation">,</span> 密钥 ID a14fe591: NOKEY 准备中<span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span> <span class="token comment">################################# [100%]</span> 正在升级<span class="token operator">/</span>安装<span class="token punctuation">.</span><span class="token punctuation">.</span><span class="token punctuation">.</span> 1:zabbix-release-5<span class="token punctuation">.</span>0-1<span class="token punctuation">.</span>el7 <span class="token comment">################################# [100%]</span> <span class="token namespace">[root@zabbix ~]</span><span class="token comment"># cd /etc/yum.repos.d/</span> <span class="token namespace">[root@zabbix yum.repos.d]</span><span class="token comment"># ls</span> CentOS-Base<span class="token punctuation">.</span>repo CentOS-Debuginfo<span class="token punctuation">.</span>repo CentOS-Media<span class="token punctuation">.</span>repo CentOS-Vault<span class="token punctuation">.</span>repo zabbix<span class="token punctuation">.</span>repo CentOS-CR<span class="token punctuation">.</span>repo CentOS-fasttrack<span class="token punctuation">.</span>repo CentOS-Sources<span class="token punctuation">.</span>repo CentOS-x86_64-kernel<span class="token punctuation">.</span>repo CentOS-Base<span class="token punctuation">.</span>repo 仓库文件： 用来找到centos官方提供的下载软件的地方的文件 Base 存放centos官方基本软件的仓库 zabbix<span class="token punctuation">.</span>repo 帮助我们找到zabbix官方提供的软件下载地方的文件 <span class="token namespace">[root@zabbix yum.repos.d]</span><span class="token comment"># cat zabbix.repo</span> <span class="token namespace">[zabbix]</span> 源的名字 name=Zabbix Official Repository <span class="token operator">-</span> <span class="token variable">$basearch</span> 对这个源的介绍 baseurl=http:<span class="token operator">/</span><span class="token operator">/</span>repo<span class="token punctuation">.</span>zabbix<span class="token punctuation">.</span>com/zabbix/5<span class="token punctuation">.</span>0/rhel/7/<span class="token variable">$basearch</span><span class="token operator">/</span> 具体源的位置 enabled=1 表示这个源可以使用 gpgcheck=1 操作系统会对下载的软件进行gpg检验码的检查，防止软件不是正版的 gpgkey=file:<span class="token operator">/</span><span class="token operator">/</span><span class="token operator">/</span>etc/pki/rpm-gpg/RPM-GPG-KEY-ZABBIX-A14FE591 <span class="token operator">--</span>》防伪码 <span class="token comment"># 2.安装zabbix相关的软件</span> <span class="token namespace">[root@zabbix yum.repos.d]</span><span class="token comment"># yum install zabbix-server-mysql zabbix-agent -y</span> zabbix-server-mysql 安装zabbix server和连接mysql功能的软件 zabbix-agent zabbix的代理软件 <span class="token comment"># 3.安装Zabbix前端</span> <span class="token namespace">[root@zabbix yum.repos.d]</span><span class="token comment"># yum install centos-release-scl -y </span> <span class="token comment"># 修改仓库文件，启用前端的源</span> <span class="token namespace">[root@zabbix yum.repos.d]</span><span class="token comment"># vim zabbix.repo</span> <span class="token namespace">[zabbix-frontend]</span> name=Zabbix Official Repository frontend <span class="token operator">-</span> <span class="token variable">$basearch</span> baseurl=http:<span class="token operator">/</span><span class="token operator">/</span>repo<span class="token punctuation">.</span>zabbix<span class="token punctuation">.</span>com/zabbix/5<span class="token punctuation">.</span>0/rhel/7/<span class="token variable">$basearch</span><span class="token operator">/</span>frontend enabled=1 <span class="token comment"># 修改为1</span> gpgcheck=1 gpgkey=file:<span class="token operator">/</span><span class="token operator">/</span><span class="token operator">/</span>etc/pki/rpm-gpg/RPM-GPG-KEY-ZABBIX-A14FE591 <span class="token comment"># 安装web相关的软件</span> <span class="token namespace">[root@zabbix yum.repos.d]</span><span class="token comment"># yum install zabbix-web-mysql-scl zabbix-nginx-conf-scl -y</span> <span class="token comment"># 4.安装mariadb数据库</span> <span class="token namespace">[root@zabbix yum.repos.d]</span><span class="token comment"># yum install mariadb mariadb-server -y </span> mariadb-server 服务器端的软件包 mariadb 提供客户端命令的软件包 <span class="token comment"># 注意：如果已经安装过mysql的centos系统，就不需要安装mariadb</span> <span class="token namespace">[root@zabbix yum.repos.d]</span><span class="token comment"># service mariadb start # 启动mariadb</span> Redirecting to <span class="token operator">/</span>bin/systemctl <span class="token function">start</span> mariadb<span class="token punctuation">.</span>service <span class="token namespace">[root@zabbix yum.repos.d]</span><span class="token comment"># systemctl enable mariadb # 设置开机启动mariadb数据库</span> Created symlink <span class="token keyword">from</span> <span class="token operator">/</span>etc/systemd/system/multi-user<span class="token punctuation">.</span>target<span class="token punctuation">.</span>wants/mariadb<span class="token punctuation">.</span>service to <span class="token operator">/</span>usr/lib/systemd/system/mariadb<span class="token punctuation">.</span>service<span class="token punctuation">.</span> <span class="token comment"># 查看mysqld进程运行</span> <span class="token namespace">[root@zabbix yum.repos.d]</span><span class="token comment"># ps aux|grep mysqld</span> mysql 11940 0<span class="token punctuation">.</span>1 0<span class="token punctuation">.</span>0 113412 1596 ? Ss 15:09 0:00 <span class="token operator">/</span>bin/sh <span class="token operator">/</span>usr/bin/mysqld_safe <span class="token operator">--</span>basedir=<span class="token operator">/</span>usr mysql 12105 1<span class="token punctuation">.</span>1 4<span class="token punctuation">.</span>3 968920 80820 ? <span class="token function">Sl</span> 15:09 0:00 <span class="token operator">/</span>usr/libexec/mysqld <span class="token operator">--</span>basedir=<span class="token operator">/</span>usr <span class="token operator">--</span>datadir=<span class="token operator">/</span><span class="token keyword">var</span><span class="token operator">/</span>lib/mysql <span class="token operator">--</span>plugin-<span class="token function">dir</span>=<span class="token operator">/</span>usr/lib64/mysql/plugin <span class="token operator">--</span>log-error=<span class="token operator">/</span><span class="token keyword">var</span><span class="token operator">/</span>log/mariadb/mariadb<span class="token punctuation">.</span>log <span class="token operator">--</span>pid-file=<span class="token operator">/</span><span class="token keyword">var</span><span class="token operator">/</span>run/mariadb/mariadb<span class="token punctuation">.</span>pid <span class="token operator">--</span>socket=<span class="token operator">/</span><span class="token keyword">var</span><span class="token operator">/</span>lib/mysql/mysql<span class="token punctuation">.</span>sock root 12159 0<span class="token punctuation">.</span>0 0<span class="token punctuation">.</span>0 112824 980 pts/0 S+ 15:09 0:00 grep <span class="token operator">--</span>color=auto mysqld <span class="token namespace">[root@zabbix yum.repos.d]</span><span class="token comment"># netstat -anplut|grep 3306</span> tcp 0 0 0<span class="token punctuation">.</span>0<span class="token punctuation">.</span>0<span class="token punctuation">.</span>0:3306 0<span class="token punctuation">.</span>0<span class="token punctuation">.</span>0<span class="token punctuation">.</span>0:<span class="token operator">*</span> LISTEN 12105/mysqld <span class="token comment"># 5.在数据库主机上运行以下命令</span> <span class="token namespace">[root@zabbix yum.repos.d]</span><span class="token comment"># mysql -uroot -p</span> Enter password: Welcome to the MariaDB monitor<span class="token punctuation">.</span> Commands <span class="token keyword">end</span> with <span class="token punctuation">;</span> or \g<span class="token punctuation">.</span> Your MariaDB connection id is 2 Server version: 5<span class="token punctuation">.</span>5<span class="token punctuation">.</span>68-MariaDB MariaDB Server Copyright <span class="token punctuation">(</span>c<span class="token punctuation">)</span> 2000<span class="token punctuation">,</span> 2018<span class="token punctuation">,</span> Oracle<span class="token punctuation">,</span> MariaDB Corporation Ab and others<span class="token punctuation">.</span> <span class="token function">Type</span> <span class="token string">'help;'</span> or <span class="token string">'\h'</span> <span class="token keyword">for</span> help<span class="token punctuation">.</span> <span class="token function">Type</span> <span class="token string">'\c'</span> to clear the current input statement<span class="token punctuation">.</span> MariaDB <span class="token punctuation">[</span><span class="token punctuation">(</span>none<span class="token punctuation">)</span><span class="token punctuation">]</span>> show databases<span class="token punctuation">;</span> <span class="token operator">+</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">+</span> <span class="token punctuation">|</span> Database <span class="token punctuation">|</span> <span class="token operator">+</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">+</span> <span class="token punctuation">|</span> information_schema <span class="token punctuation">|</span> <span class="token punctuation">|</span> mysql <span class="token punctuation">|</span> <span class="token punctuation">|</span> performance_schema <span class="token punctuation">|</span> <span class="token punctuation">|</span> test <span class="token punctuation">|</span> <span class="token operator">+</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">+</span> 4 rows in <span class="token function">set</span> <span class="token punctuation">(</span>0<span class="token punctuation">.</span>01 sec<span class="token punctuation">)</span> MariaDB <span class="token punctuation">[</span><span class="token punctuation">(</span>none<span class="token punctuation">)</span><span class="token punctuation">]</span>> create database zabbix character <span class="token function">set</span> utf8 collate utf8_bin<span class="token punctuation">;</span> Query OK<span class="token punctuation">,</span> 1 row affected <span class="token punctuation">(</span>0<span class="token punctuation">.</span>00 sec<span class="token punctuation">)</span> MariaDB <span class="token punctuation">[</span><span class="token punctuation">(</span>none<span class="token punctuation">)</span><span class="token punctuation">]</span>> create user zabbix@localhost identified by <span class="token string">'sc123456'</span><span class="token punctuation">;</span> <span class="token comment"># 创建用户zabbix@localhost 密码是sc123456</span> Query OK<span class="token punctuation">,</span> 0 rows affected <span class="token punctuation">(</span>0<span class="token punctuation">.</span>00 sec<span class="token punctuation">)</span> MariaDB <span class="token punctuation">[</span><span class="token punctuation">(</span>none<span class="token punctuation">)</span><span class="token punctuation">]</span>> grant all privileges on zabbix<span class="token punctuation">.</span><span class="token operator">*</span> to zabbix@localhost<span class="token punctuation">;</span> <span class="token comment">#授权zabbix@localhost用户对zabbix.*库里的表有所有的权限（insert，delete，update，select等）</span> Query OK<span class="token punctuation">,</span> 0 rows affected <span class="token punctuation">(</span>0<span class="token punctuation">.</span>00 sec<span class="token punctuation">)</span> MariaDB <span class="token punctuation">[</span><span class="token punctuation">(</span>none<span class="token punctuation">)</span><span class="token punctuation">]</span>> <span class="token function">set</span> global log_bin_trust_function_creators = 1<span class="token punctuation">;</span> Query OK<span class="token punctuation">,</span> 0 rows affected <span class="token punctuation">(</span>0<span class="token punctuation">.</span>00 sec<span class="token punctuation">)</span> MariaDB <span class="token punctuation">[</span><span class="token punctuation">(</span>none<span class="token punctuation">)</span><span class="token punctuation">]</span>> <span class="token keyword">exit</span> Bye <span class="token comment"># 导入初始化数据，会在zabbix库里新建很多的表</span> <span class="token namespace">[root@zabbix yum.repos.d]</span><span class="token comment"># cd /usr/share/doc/zabbix-server-mysql-5.0.35/</span> <span class="token namespace">[root@zabbix zabbix-server-mysql-5.0.35]</span><span class="token comment"># ls</span> AUTHORS ChangeLog COPYING create<span class="token punctuation">.</span>sql<span class="token punctuation">.</span>gz double<span class="token punctuation">.</span>sql NEWS README <span class="token namespace">[root@zabbix zabbix-server-mysql-5.0.33]</span><span class="token comment"># zcat create.sql.gz |mysql -uzabbix -p'sc123456' zabbix</span> <span class="token namespace">[root@zabbix zabbix-server-mysql-5.0.33]</span><span class="token comment"># mysql -uzabbix -psc123456</span> Welcome to the MariaDB monitor<span class="token punctuation">.</span> Commands <span class="token keyword">end</span> with <span class="token punctuation">;</span> or \g<span class="token punctuation">.</span> Your MariaDB connection id is 4 Server version: 5<span class="token punctuation">.</span>5<span class="token punctuation">.</span>68-MariaDB MariaDB Server Copyright <span class="token punctuation">(</span>c<span class="token punctuation">)</span> 2000<span class="token punctuation">,</span> 2018<span class="token punctuation">,</span> Oracle<span class="token punctuation">,</span> MariaDB Corporation Ab and others<span class="token punctuation">.</span> <span class="token function">Type</span> <span class="token string">'help;'</span> or <span class="token string">'\h'</span> <span class="token keyword">for</span> help<span class="token punctuation">.</span> <span class="token function">Type</span> <span class="token string">'\c'</span> to clear the current input statement<span class="token punctuation">.</span> MariaDB <span class="token punctuation">[</span><span class="token punctuation">(</span>none<span class="token punctuation">)</span><span class="token punctuation">]</span>> show databases<span class="token punctuation">;</span> <span class="token operator">+</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">+</span> <span class="token punctuation">|</span> Database <span class="token punctuation">|</span> <span class="token operator">+</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">+</span> <span class="token punctuation">|</span> information_schema <span class="token punctuation">|</span> <span class="token punctuation">|</span> test <span class="token punctuation">|</span> <span class="token punctuation">|</span> zabbix <span class="token punctuation">|</span> <span class="token operator">+</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">+</span> 3 rows in <span class="token function">set</span> <span class="token punctuation">(</span>0<span class="token punctuation">.</span>00 sec<span class="token punctuation">)</span> MariaDB <span class="token punctuation">[</span><span class="token punctuation">(</span>none<span class="token punctuation">)</span><span class="token punctuation">]</span>> use zabbix<span class="token punctuation">;</span> Reading table information <span class="token keyword">for</span> completion of table and column names You can turn off this feature to get a quicker startup with <span class="token operator">-</span>A Database changed MariaDB <span class="token namespace">[zabbix]</span>> show tables<span class="token punctuation">;</span> <span class="token operator">+</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">+</span> <span class="token punctuation">|</span> Tables_in_zabbix <span class="token punctuation">|</span> <span class="token operator">+</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">--</span><span class="token operator">+</span> <span class="token punctuation">|</span> acknowledges <span class="token punctuation">|</span> <span class="token punctuation">|</span> actions <span class="token punctuation">|</span> <span class="token punctuation">|</span> alerts <span class="token punctuation">|</span> <span class="token punctuation">|</span> application_discovery <span class="token punctuation">|</span> <span class="token punctuation">|</span> application_prototype <span class="token punctuation">|</span> <span class="token comment"># 导入数据库架构后禁用log_bin_trust_function_creators选项</span> <span class="token namespace">[root@zabbix zabbix-server-mysql-5.0.33]</span><span class="token comment"># mysql -uroot -p</span> Enter password: Welcome to the MariaDB monitor<span class="token punctuation">.</span> Commands <span class="token keyword">end</span> with <span class="token punctuation">;</span> or \g<span class="token punctuation">.</span> Your MariaDB connection id is 5 Server version: 5<span class="token punctuation">.</span>5<span class="token punctuation">.</span>68-MariaDB MariaDB Server Copyright <span class="token punctuation">(</span>c<span class="token punctuation">)</span> 2000<span class="token punctuation">,</span> 2018<span class="token punctuation">,</span> Oracle<span class="token punctuation">,</span> MariaDB Corporation Ab and others<span class="token punctuation">.</span> <span class="token function">Type</span> <span class="token string">'help;'</span> or <span class="token string">'\h'</span> <span class="token keyword">for</span> help<span class="token punctuation">.</span> <span class="token function">Type</span> <span class="token string">'\c'</span> to clear the current input statement<span class="token punctuation">.</span> MariaDB <span class="token punctuation">[</span><span class="token punctuation">(</span>none<span class="token punctuation">)</span><span class="token punctuation">]</span>> <span class="token function">set</span> global log_bin_trust_function_creators = 0<span class="token punctuation">;</span> Query OK<span class="token punctuation">,</span> 0 rows affected <span class="token punctuation">(</span>0<span class="token punctuation">.</span>00 sec<span class="token punctuation">)</span> MariaDB <span class="token punctuation">[</span><span class="token punctuation">(</span>none<span class="token punctuation">)</span><span class="token punctuation">]</span>> <span class="token keyword">exit</span> Bye <span class="token comment"># 6.为 Zabbix 服务器配置数据库</span> <span class="token comment"># 编辑文件 /etc/zabbix/zabbix_server.conf</span> <span class="token namespace">[root@zabbix zabbix-server-mysql-5.0.33]</span><span class="token comment"># cd /etc/zabbix/</span> <span class="token namespace">[root@zabbix zabbix]</span><span class="token comment"># vim zabbix_server.conf </span> <span class="token comment"># DBPassword=</span> DBPassword=sc123456 <span class="token comment"># 7.为 Zabbix 前端配置 PHP</span> <span class="token comment"># 编辑文件 /etc/opt/rh/rh-nginx116/nginx/conf.d/zabbix.conf 取消注释</span> <span class="token namespace">[root@zabbix conf.d]</span><span class="token comment"># cd /etc/opt/rh/rh-nginx116/nginx/conf.d/</span> <span class="token namespace">[root@zabbix conf.d]</span><span class="token comment"># ls</span> zabbix<span class="token punctuation">.</span>conf <span class="token namespace">[root@zabbix conf.d]</span><span class="token comment"># vim zabbix.conf </span> server <span class="token punctuation">{<!-- --></span> listen 8080<span class="token punctuation">;</span> server_name zabbix<span class="token punctuation">.</span>com<span class="token punctuation">;</span> <span class="token comment"># 编辑/etc/opt/rh/rh-nginx116/nginx/nginx.conf</span> <span class="token namespace">[root@zabbix conf.d]</span><span class="token comment"># cd /etc/opt/rh/rh-nginx116/nginx/ </span> <span class="token namespace">[root@zabbix nginx]</span><span class="token comment"># vim nginx.conf </span> server <span class="token punctuation">{<!-- --></span> listen 80 default_server<span class="token punctuation">;</span> <span class="token comment">#修改80为8080</span> listen <span class="token punctuation">[</span>::<span class="token punctuation">]</span>:80 default_server<span class="token punctuation">;</span> <span class="token comment"># 避免zabbix和nginx监听同一个端口，导致zabbix启动不起来。</span> <span class="token comment"># 编辑文件 /etc/opt/rh/rh-php72/php-fpm.d/zabbix.conf</span> <span class="token namespace">[root@zabbix nginx]</span><span class="token comment"># cd /etc/opt/rh/rh-php72/php-fpm.d</span> <span class="token namespace">[root@zabbix php-fpm.d]</span><span class="token comment"># ls</span> www<span class="token punctuation">.</span>conf zabbix<span class="token punctuation">.</span>conf <span class="token namespace">[root@zabbix php-fpm.d]</span><span class="token comment"># vim zabbix.conf </span> listen<span class="token punctuation">.</span>acl_users = apache<span class="token punctuation">,</span>nginx php_value<span class="token namespace">[date.timezone]</span> = Asia/Shanghai <span class="token comment"># 建议一定要关闭selinux，不然会导致zabbix_server启动不了</span> <span class="token comment"># 8.启动Zabbix服务器和代理进程并且设置开机启动</span> <span class="token namespace">[root@zabbix php-fpm.d]</span><span class="token comment"># systemctl restart zabbix-server zabbix-agent rh-nginx116-nginx rh-php72-php-fpm</span> <span class="token namespace">[root@zabbix php-fpm.d]</span><span class="token comment"># systemctl enable zabbix-server zabbix-agent rh-nginx116-nginx rh-php72-php-fpm</span> Created symlink <span class="token keyword">from</span> <span class="token operator">/</span>etc/systemd/system/multi-user<span class="token punctuation">.</span>target<span class="token punctuation">.</span>wants/zabbix-server<span class="token punctuation">.</span>service to <span class="token operator">/</span>usr/lib/systemd/system/zabbix-server<span class="token punctuation">.</span>service<span class="token punctuation">.</span> Created symlink <span class="token keyword">from</span> <span class="token operator">/</span>etc/systemd/system/multi-user<span class="token punctuation">.</span>target<span class="token punctuation">.</span>wants/zabbix-agent<span class="token punctuation">.</span>service to <span class="token operator">/</span>usr/lib/systemd/system/zabbix-agent<span class="token punctuation">.</span>service<span class="token punctuation">.</span> Created symlink <span class="token keyword">from</span> <span class="token operator">/</span>etc/systemd/system/multi-user<span class="token punctuation">.</span>target<span class="token punctuation">.</span>wants/rh-nginx116-nginx<span class="token punctuation">.</span>service to <span class="token operator">/</span>usr/lib/systemd/system/rh-nginx116-nginx<span class="token punctuation">.</span>service<span class="token punctuation">.</span> Created symlink <span class="token keyword">from</span> <span class="token operator">/</span>etc/systemd/system/multi-user<span class="token punctuation">.</span>target<span class="token punctuation">.</span>wants/rh-php72-php-fpm<span class="token punctuation">.</span>service to <span class="token operator">/</span>usr/lib/systemd/system/rh-php72-php-fpm<span class="token punctuation">.</span>service<span class="token punctuation">.</span> <span class="token comment"># 9.浏览器里访问</span> http:<span class="token operator">/</span><span class="token operator">/</span>192<span class="token punctuation">.</span>168<span class="token punctuation">.</span>2<span class="token punctuation">.</span>117:8080 <span class="token comment"># 默认登录的账号和密码</span> username: Admin password: zabbix <span class="token comment"># 使用Prometheus监控Kubernetes</span> <span class="token comment"># 1.在所有节点提前下载镜像</span> docker pull prom/node-exporter docker pull prom/prometheus:v2<span class="token punctuation">.</span>0<span class="token punctuation">.</span>0 docker pull grafana/grafana:6<span class="token punctuation">.</span>1<span class="token punctuation">.</span>4 <span class="token namespace">[root@k8smaster ~]</span><span class="token comment"># docker images</span> REPOSITORY TAG IMAGE ID CREATED SIZE prom/node-exporter latest 1dbe0e931976 18 months ago 20<span class="token punctuation">.</span>9MB grafana/grafana 6<span class="token punctuation">.</span>1<span class="token punctuation">.</span>4 d9bdb6044027 4 years ago 245MB prom/prometheus v2<span class="token punctuation">.</span>0<span class="token punctuation">.</span>0 67141fa03496 5 years ago 80<span class="token punctuation">.</span>2MB <span class="token namespace">[root@k8snode1 ~]</span><span class="token comment"># docker images</span> REPOSITORY TAG IMAGE ID CREATED SIZE prom/node-exporter latest 1dbe0e931976 18 months ago 20<span class="token punctuation">.</span>9MB grafana/grafana 6<span class="token punctuation">.</span>1<span class="token punctuation">.</span>4 d9bdb6044027 4 years ago 245MB prom/prometheus <span class="token namespace">[root@k8snode2 ~]</span><span class="token comment"># docker images</span> REPOSITORY TAG IMAGE ID CREATED SIZE prom/node-exporter latest 1dbe0e931976 18 months ago 20<span class="token punctuation">.</span>9MB grafana/grafana 6<span class="token punctuation">.</span>1<span class="token punctuation">.</span>4 d9bdb6044027 4 years ago 245MB prom/prometheus v2<span class="token punctuation">.</span>0<span class="token punctuation">.</span>0 67141fa03496 5 years ago 80<span class="token punctuation">.</span>2MB <span class="token comment"># 2.采用daemonset方式部署node-exporter</span> <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># ll</span> 总用量 36 <span class="token operator">-</span>rw-r-<span class="token operator">-</span>r-<span class="token operator">-</span> 1 root root 5632 6月 25 16:23 configmap<span class="token punctuation">.</span>yaml <span class="token operator">-</span>rw-r-<span class="token operator">-</span>r-<span class="token operator">-</span> 1 root root 1515 6月 25 16:26 grafana-deploy<span class="token punctuation">.</span>yaml <span class="token operator">-</span>rw-r-<span class="token operator">-</span>r-<span class="token operator">-</span> 1 root root 256 6月 25 16:27 grafana-ing<span class="token punctuation">.</span>yaml <span class="token operator">-</span>rw-r-<span class="token operator">-</span>r-<span class="token operator">-</span> 1 root root 225 6月 25 16:27 grafana-svc<span class="token punctuation">.</span>yaml <span class="token operator">-</span>rw-r-<span class="token operator">-</span>r-<span class="token operator">-</span> 1 root root 716 6月 25 16:22 node-exporter<span class="token punctuation">.</span>yaml <span class="token operator">-</span>rw-r-<span class="token operator">-</span>r-<span class="token operator">-</span> 1 root root 1104 6月 25 16:25 prometheus<span class="token punctuation">.</span>deploy<span class="token punctuation">.</span>yml <span class="token operator">-</span>rw-r-<span class="token operator">-</span>r-<span class="token operator">-</span> 1 root root 233 6月 25 16:25 prometheus<span class="token punctuation">.</span>svc<span class="token punctuation">.</span>yml <span class="token operator">-</span>rw-r-<span class="token operator">-</span>r-<span class="token operator">-</span> 1 root root 716 6月 25 16:23 rbac-setukp<span class="token punctuation">.</span>yaml <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># cat node-exporter.yaml </span> <span class="token operator">--</span><span class="token operator">-</span> apiVersion: apps/v1 kind: DaemonSet metadata: name: node-exporter namespace: kube-system labels: k8s-app: node-exporter spec: selector: matchLabels: k8s-app: node-exporter template: metadata: labels: k8s-app: node-exporter spec: containers: <span class="token operator">-</span> image: prom/node-exporter name: node-exporter ports: <span class="token operator">-</span> containerPort: 9100 protocol: TCP name: http <span class="token operator">--</span><span class="token operator">-</span> apiVersion: v1 kind: Service metadata: labels: k8s-app: node-exporter name: node-exporter namespace: kube-system spec: ports: <span class="token operator">-</span> name: http port: 9100 nodePort: 31672 protocol: TCP <span class="token function">type</span>: NodePort selector: k8s-app: node-exporter <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># kubectl apply -f node-exporter.yaml</span> daemonset<span class="token punctuation">.</span>apps/node-exporter created service/node-exporter created <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># kubectl get pods -A</span> NAMESPACE NAME READY STATUS RESTARTS AGE kube-system node-exporter-fcmx5 1/1 Running 0 47s kube-system node-exporter-qccwb 1/1 Running 0 47s <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># kubectl get daemonset -A</span> NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE kube-system calico-node 3 3 3 3 3 kubernetes<span class="token punctuation">.</span>io/os=linux 7d kube-system kube-proxy 3 3 3 3 3 kubernetes<span class="token punctuation">.</span>io/os=linux 7d kube-system node-exporter 2 2 2 2 2 <none> 2m29s <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># kubectl get service -A</span> NAMESPACE NAME <span class="token function">TYPE</span> CLUSTER-IP EXTERNAL-IP PORT<span class="token punctuation">(</span>S<span class="token punctuation">)</span> AGE kube-system node-exporter NodePort 10<span class="token punctuation">.</span>111<span class="token punctuation">.</span>247<span class="token punctuation">.</span>142 <none> 9100:31672/TCP 3m24s <span class="token comment"># 3.部署Prometheus</span> <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># cat rbac-setup.yaml </span> apiVersion: rbac<span class="token punctuation">.</span>authorization<span class="token punctuation">.</span>k8s<span class="token punctuation">.</span>io/v1 kind: ClusterRole metadata: name: prometheus rules: <span class="token operator">-</span> apiGroups: <span class="token punctuation">[</span><span class="token string">""</span><span class="token punctuation">]</span> resources: <span class="token operator">-</span> nodes <span class="token operator">-</span> nodes/proxy <span class="token operator">-</span> services <span class="token operator">-</span> endpoints <span class="token operator">-</span> pods verbs: <span class="token punctuation">[</span><span class="token string">"get"</span><span class="token punctuation">,</span> <span class="token string">"list"</span><span class="token punctuation">,</span> <span class="token string">"watch"</span><span class="token punctuation">]</span> <span class="token operator">-</span> apiGroups: <span class="token operator">-</span> extensions resources: <span class="token operator">-</span> ingresses verbs: <span class="token punctuation">[</span><span class="token string">"get"</span><span class="token punctuation">,</span> <span class="token string">"list"</span><span class="token punctuation">,</span> <span class="token string">"watch"</span><span class="token punctuation">]</span> <span class="token operator">-</span> nonResourceURLs: <span class="token punctuation">[</span><span class="token string">"/metrics"</span><span class="token punctuation">]</span> verbs: <span class="token punctuation">[</span><span class="token string">"get"</span><span class="token punctuation">]</span> <span class="token operator">--</span><span class="token operator">-</span> apiVersion: v1 kind: ServiceAccount metadata: name: prometheus namespace: kube-system <span class="token operator">--</span><span class="token operator">-</span> apiVersion: rbac<span class="token punctuation">.</span>authorization<span class="token punctuation">.</span>k8s<span class="token punctuation">.</span>io/v1 kind: ClusterRoleBinding metadata: name: prometheus roleRef: apiGroup: rbac<span class="token punctuation">.</span>authorization<span class="token punctuation">.</span>k8s<span class="token punctuation">.</span>io kind: ClusterRole name: prometheus subjects: <span class="token operator">-</span> kind: ServiceAccount name: prometheus namespace: kube-system <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># kubectl apply -f rbac-setup.yaml</span> clusterrole<span class="token punctuation">.</span>rbac<span class="token punctuation">.</span>authorization<span class="token punctuation">.</span>k8s<span class="token punctuation">.</span>io/prometheus created serviceaccount/prometheus created clusterrolebinding<span class="token punctuation">.</span>rbac<span class="token punctuation">.</span>authorization<span class="token punctuation">.</span>k8s<span class="token punctuation">.</span>io/prometheus created <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># cat configmap.yaml </span> apiVersion: v1 kind: ConfigMap metadata: name: prometheus-config namespace: kube-system <span class="token keyword">data</span>: prometheus<span class="token punctuation">.</span>yml: <span class="token punctuation">|</span> global: scrape_interval: 15s evaluation_interval: 15s scrape_configs: <span class="token operator">-</span> job_name: <span class="token string">'kubernetes-apiservers'</span> kubernetes_sd_configs: <span class="token operator">-</span> role: endpoints scheme: https tls_config: ca_file: <span class="token operator">/</span><span class="token keyword">var</span><span class="token operator">/</span>run/secrets/kubernetes<span class="token punctuation">.</span>io/serviceaccount/ca<span class="token punctuation">.</span>crt bearer_token_file: <span class="token operator">/</span><span class="token keyword">var</span><span class="token operator">/</span>run/secrets/kubernetes<span class="token punctuation">.</span>io/serviceaccount/token relabel_configs: <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__meta_kubernetes_namespace<span class="token punctuation">,</span> __meta_kubernetes_service_name<span class="token punctuation">,</span> __meta_kubernetes_endpoint_port_name<span class="token punctuation">]</span> action: keep regex: default<span class="token punctuation">;</span>kubernetes<span class="token punctuation">;</span>https <span class="token operator">-</span> job_name: <span class="token string">'kubernetes-nodes'</span> kubernetes_sd_configs: <span class="token operator">-</span> role: node scheme: https tls_config: ca_file: <span class="token operator">/</span><span class="token keyword">var</span><span class="token operator">/</span>run/secrets/kubernetes<span class="token punctuation">.</span>io/serviceaccount/ca<span class="token punctuation">.</span>crt bearer_token_file: <span class="token operator">/</span><span class="token keyword">var</span><span class="token operator">/</span>run/secrets/kubernetes<span class="token punctuation">.</span>io/serviceaccount/token relabel_configs: <span class="token operator">-</span> action: labelmap regex: __meta_kubernetes_node_label_<span class="token punctuation">(</span><span class="token punctuation">.</span><span class="token operator">+</span><span class="token punctuation">)</span> <span class="token operator">-</span> target_label: __address__ replacement: kubernetes<span class="token punctuation">.</span>default<span class="token punctuation">.</span>svc:443 <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__meta_kubernetes_node_name<span class="token punctuation">]</span> regex: <span class="token punctuation">(</span><span class="token punctuation">.</span><span class="token operator">+</span><span class="token punctuation">)</span> target_label: __metrics_path__ replacement: <span class="token operator">/</span>api/v1/nodes/$<span class="token punctuation">{<!-- --></span>1<span class="token punctuation">}</span><span class="token operator">/</span>proxy/metrics <span class="token operator">-</span> job_name: <span class="token string">'kubernetes-cadvisor'</span> kubernetes_sd_configs: <span class="token operator">-</span> role: node scheme: https tls_config: ca_file: <span class="token operator">/</span><span class="token keyword">var</span><span class="token operator">/</span>run/secrets/kubernetes<span class="token punctuation">.</span>io/serviceaccount/ca<span class="token punctuation">.</span>crt bearer_token_file: <span class="token operator">/</span><span class="token keyword">var</span><span class="token operator">/</span>run/secrets/kubernetes<span class="token punctuation">.</span>io/serviceaccount/token relabel_configs: <span class="token operator">-</span> action: labelmap regex: __meta_kubernetes_node_label_<span class="token punctuation">(</span><span class="token punctuation">.</span><span class="token operator">+</span><span class="token punctuation">)</span> <span class="token operator">-</span> target_label: __address__ replacement: kubernetes<span class="token punctuation">.</span>default<span class="token punctuation">.</span>svc:443 <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__meta_kubernetes_node_name<span class="token punctuation">]</span> regex: <span class="token punctuation">(</span><span class="token punctuation">.</span><span class="token operator">+</span><span class="token punctuation">)</span> target_label: __metrics_path__ replacement: <span class="token operator">/</span>api/v1/nodes/$<span class="token punctuation">{<!-- --></span>1<span class="token punctuation">}</span><span class="token operator">/</span>proxy/metrics/cadvisor <span class="token operator">-</span> job_name: <span class="token string">'kubernetes-service-endpoints'</span> kubernetes_sd_configs: <span class="token operator">-</span> role: endpoints relabel_configs: <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__meta_kubernetes_service_annotation_prometheus_io_scrape<span class="token punctuation">]</span> action: keep regex: true <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__meta_kubernetes_service_annotation_prometheus_io_scheme<span class="token punctuation">]</span> action: replace target_label: __scheme__ regex: <span class="token punctuation">(</span>https?<span class="token punctuation">)</span> <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__meta_kubernetes_service_annotation_prometheus_io_path<span class="token punctuation">]</span> action: replace target_label: __metrics_path__ regex: <span class="token punctuation">(</span><span class="token punctuation">.</span><span class="token operator">+</span><span class="token punctuation">)</span> <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__address__<span class="token punctuation">,</span> __meta_kubernetes_service_annotation_prometheus_io_port<span class="token punctuation">]</span> action: replace target_label: __address__ regex: <span class="token punctuation">(</span><span class="token punctuation">[</span>^:<span class="token punctuation">]</span><span class="token operator">+</span><span class="token punctuation">)</span><span class="token punctuation">(</span>?::\d+<span class="token punctuation">)</span>?<span class="token punctuation">;</span><span class="token punctuation">(</span>\d+<span class="token punctuation">)</span> replacement: <span class="token variable">$1</span>:<span class="token variable">$2</span> <span class="token operator">-</span> action: labelmap regex: __meta_kubernetes_service_label_<span class="token punctuation">(</span><span class="token punctuation">.</span><span class="token operator">+</span><span class="token punctuation">)</span> <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__meta_kubernetes_namespace<span class="token punctuation">]</span> action: replace target_label: kubernetes_namespace <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__meta_kubernetes_service_name<span class="token punctuation">]</span> action: replace target_label: kubernetes_name <span class="token operator">-</span> job_name: <span class="token string">'kubernetes-services'</span> kubernetes_sd_configs: <span class="token operator">-</span> role: service metrics_path: <span class="token operator">/</span>probe params: module: <span class="token namespace">[http_2xx]</span> relabel_configs: <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__meta_kubernetes_service_annotation_prometheus_io_probe<span class="token punctuation">]</span> action: keep regex: true <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__address__<span class="token punctuation">]</span> target_label: __param_target <span class="token operator">-</span> target_label: __address__ replacement: blackbox-exporter<span class="token punctuation">.</span>example<span class="token punctuation">.</span>com:9115 <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__param_target<span class="token punctuation">]</span> target_label: instance <span class="token operator">-</span> action: labelmap regex: __meta_kubernetes_service_label_<span class="token punctuation">(</span><span class="token punctuation">.</span><span class="token operator">+</span><span class="token punctuation">)</span> <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__meta_kubernetes_namespace<span class="token punctuation">]</span> target_label: kubernetes_namespace <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__meta_kubernetes_service_name<span class="token punctuation">]</span> target_label: kubernetes_name <span class="token operator">-</span> job_name: <span class="token string">'kubernetes-ingresses'</span> kubernetes_sd_configs: <span class="token operator">-</span> role: ingress relabel_configs: <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__meta_kubernetes_ingress_annotation_prometheus_io_probe<span class="token punctuation">]</span> action: keep regex: true <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__meta_kubernetes_ingress_scheme<span class="token punctuation">,</span>__address__<span class="token punctuation">,</span>__meta_kubernetes_ingress_path<span class="token punctuation">]</span> regex: <span class="token punctuation">(</span><span class="token punctuation">.</span><span class="token operator">+</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token punctuation">(</span><span class="token punctuation">.</span><span class="token operator">+</span><span class="token punctuation">)</span><span class="token punctuation">;</span><span class="token punctuation">(</span><span class="token punctuation">.</span><span class="token operator">+</span><span class="token punctuation">)</span> replacement: $<span class="token punctuation">{<!-- --></span>1<span class="token punctuation">}</span>:<span class="token operator">/</span><span class="token operator">/</span>$<span class="token punctuation">{<!-- --></span>2<span class="token punctuation">}</span>$<span class="token punctuation">{<!-- --></span>3<span class="token punctuation">}</span> target_label: __param_target <span class="token operator">-</span> target_label: __address__ replacement: blackbox-exporter<span class="token punctuation">.</span>example<span class="token punctuation">.</span>com:9115 <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__param_target<span class="token punctuation">]</span> target_label: instance <span class="token operator">-</span> action: labelmap regex: __meta_kubernetes_ingress_label_<span class="token punctuation">(</span><span class="token punctuation">.</span><span class="token operator">+</span><span class="token punctuation">)</span> <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__meta_kubernetes_namespace<span class="token punctuation">]</span> target_label: kubernetes_namespace <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__meta_kubernetes_ingress_name<span class="token punctuation">]</span> target_label: kubernetes_name <span class="token operator">-</span> job_name: <span class="token string">'kubernetes-pods'</span> kubernetes_sd_configs: <span class="token operator">-</span> role: pod relabel_configs: <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__meta_kubernetes_pod_annotation_prometheus_io_scrape<span class="token punctuation">]</span> action: keep regex: true <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__meta_kubernetes_pod_annotation_prometheus_io_path<span class="token punctuation">]</span> action: replace target_label: __metrics_path__ regex: <span class="token punctuation">(</span><span class="token punctuation">.</span><span class="token operator">+</span><span class="token punctuation">)</span> <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__address__<span class="token punctuation">,</span> __meta_kubernetes_pod_annotation_prometheus_io_port<span class="token punctuation">]</span> action: replace regex: <span class="token punctuation">(</span><span class="token punctuation">[</span>^:<span class="token punctuation">]</span><span class="token operator">+</span><span class="token punctuation">)</span><span class="token punctuation">(</span>?::\d+<span class="token punctuation">)</span>?<span class="token punctuation">;</span><span class="token punctuation">(</span>\d+<span class="token punctuation">)</span> replacement: <span class="token variable">$1</span>:<span class="token variable">$2</span> target_label: __address__ <span class="token operator">-</span> action: labelmap regex: __meta_kubernetes_pod_label_<span class="token punctuation">(</span><span class="token punctuation">.</span><span class="token operator">+</span><span class="token punctuation">)</span> <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__meta_kubernetes_namespace<span class="token punctuation">]</span> action: replace target_label: kubernetes_namespace <span class="token operator">-</span> source_labels: <span class="token punctuation">[</span>__meta_kubernetes_pod_name<span class="token punctuation">]</span> action: replace target_label: kubernetes_pod_name <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># kubectl apply -f configmap.yaml</span> configmap/prometheus-config created <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># cat prometheus.deploy.yml </span> apiVersion: apps/v1 kind: Deployment metadata: labels: name: prometheus-deployment name: prometheus namespace: kube-system spec: replicas: 1 selector: matchLabels: app: prometheus template: metadata: labels: app: prometheus spec: containers: <span class="token operator">-</span> image: prom/prometheus:v2<span class="token punctuation">.</span>0<span class="token punctuation">.</span>0 name: prometheus command: <span class="token operator">-</span> <span class="token string">"/bin/prometheus"</span> args: <span class="token operator">-</span> <span class="token string">"--config.file=/etc/prometheus/prometheus.yml"</span> <span class="token operator">-</span> <span class="token string">"--storage.tsdb.path=/prometheus"</span> <span class="token operator">-</span> <span class="token string">"--storage.tsdb.retention=24h"</span> ports: <span class="token operator">-</span> containerPort: 9090 protocol: TCP volumeMounts: <span class="token operator">-</span> mountPath: <span class="token string">"/prometheus"</span> name: <span class="token keyword">data</span> <span class="token operator">-</span> mountPath: <span class="token string">"/etc/prometheus"</span> name: config-volume resources: requests: cpu: 100m memory: 100Mi limits: cpu: 500m memory: 2500Mi serviceAccountName: prometheus volumes: <span class="token operator">-</span> name: <span class="token keyword">data</span> emptyDir: <span class="token punctuation">{<!-- --></span><span class="token punctuation">}</span> <span class="token operator">-</span> name: config-volume configMap: name: prometheus-config <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># kubectl apply -f prometheus.deploy.yml</span> deployment<span class="token punctuation">.</span>apps/prometheus created <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># cat prometheus.svc.yml </span> kind: Service apiVersion: v1 metadata: labels: app: prometheus name: prometheus namespace: kube-system spec: <span class="token function">type</span>: NodePort ports: <span class="token operator">-</span> port: 9090 targetPort: 9090 nodePort: 30003 selector: app: prometheus <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># kubectl apply -f prometheus.svc.yml</span> service/prometheus created 4<span class="token punctuation">.</span>部署grafana <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># cat grafana-deploy.yaml </span> apiVersion: apps/v1 kind: Deployment metadata: name: grafana-core namespace: kube-system labels: app: grafana component: core spec: replicas: 1 selector: matchLabels: app: grafana template: metadata: labels: app: grafana component: core spec: containers: <span class="token operator">-</span> image: grafana/grafana:6<span class="token punctuation">.</span>1<span class="token punctuation">.</span>4 name: grafana-core imagePullPolicy: IfNotPresent <span class="token comment"># env:</span> resources: <span class="token comment"># keep request = limit to keep this container in guaranteed class</span> limits: cpu: 100m memory: 100Mi requests: cpu: 100m memory: 100Mi env: <span class="token comment"># The following env variables set up basic auth twith the default admin user and admin password.</span> <span class="token operator">-</span> name: GF_AUTH_BASIC_ENABLED value: <span class="token string">"true"</span> <span class="token operator">-</span> name: GF_AUTH_ANONYMOUS_ENABLED value: <span class="token string">"false"</span> <span class="token comment"># - name: GF_AUTH_ANONYMOUS_ORG_ROLE</span> <span class="token comment"># value: Admin</span> <span class="token comment"># does not really work, because of template variables in exported dashboards:</span> <span class="token comment"># - name: GF_DASHBOARDS_JSON_ENABLED</span> <span class="token comment"># value: "true"</span> readinessProbe: httpGet: path: <span class="token operator">/</span>login port: 3000 <span class="token comment"># initialDelaySeconds: 30</span> <span class="token comment"># timeoutSeconds: 1</span> <span class="token comment">#volumeMounts: #先不进行挂载</span> <span class="token comment">#- name: grafana-persistent-storage</span> <span class="token comment"># mountPath: /var</span> <span class="token comment">#volumes:</span> <span class="token comment">#- name: grafana-persistent-storage</span> <span class="token comment">#emptyDir: {}</span> <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># kubectl apply -f grafana-deploy.yaml</span> deployment<span class="token punctuation">.</span>apps/grafana-core created <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># cat grafana-svc.yaml </span> apiVersion: v1 kind: Service metadata: name: grafana namespace: kube-system labels: app: grafana component: core spec: <span class="token function">type</span>: NodePort ports: <span class="token operator">-</span> port: 3000 selector: app: grafana component: core <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># kubectl apply -f grafana-svc.yaml </span> service/grafana created <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># cat grafana-ing.yaml </span> apiVersion: extensions/v1beta1 kind: Ingress metadata: name: grafana namespace: kube-system spec: rules: <span class="token operator">-</span> host: k8s<span class="token punctuation">.</span>grafana http: paths: <span class="token operator">-</span> path: <span class="token operator">/</span> backend: serviceName: grafana servicePort: 3000 <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># kubectl apply -f grafana-ing.yaml</span> Warning: extensions/v1beta1 Ingress is deprecated in v1<span class="token punctuation">.</span>14+<span class="token punctuation">,</span> unavailable in v1<span class="token punctuation">.</span>22+<span class="token punctuation">;</span> use networking<span class="token punctuation">.</span>k8s<span class="token punctuation">.</span>io/v1 Ingress ingress<span class="token punctuation">.</span>extensions/grafana created <span class="token comment"># 5.检查、测试</span> <span class="token namespace">[root@k8smaster prometheus]</span><span class="token comment"># kubectl get pods -A</span> NAMESPACE NAME READY STATUS RESTARTS AGE kube-system grafana-core-78958d6d67-49c56 1/1 Running 0 31m kube-system node-exporter-fcmx5 1/1 Running 0 9m33s kube-system node-exporter-qccwb 1/1 Running 0 9m33s kube-system prometheus-68546b8d9-qxsm7 1/1 Running 0 2m47s <span class="token namespace">[root@k8smaster mysql]</span><span class="token comment"># kubectl get svc -A</span> NAMESPACE NAME <span class="token function">TYPE</span> CLUSTER-IP EXTERNAL-IP PORT<span class="token punctuation">(</span>S<span class="token punctuation">)</span> AGE kube-system grafana NodePort 10<span class="token punctuation">.</span>110<span class="token punctuation">.</span>87<span class="token punctuation">.</span>158 <none> 3000:31267/TCP 31m kube-system node-exporter NodePort 10<span class="token punctuation">.</span>111<span class="token punctuation">.</span>247<span class="token punctuation">.</span>142 <none> 9100:31672/TCP 39m kube-system prometheus NodePort 10<span class="token punctuation">.</span>102<span class="token punctuation">.</span>0<span class="token punctuation">.</span>186 <none> 9090:30003/TCP 32m <span class="token comment"># 访问</span> <span class="token comment"># node-exporter采集的数据</span> http:<span class="token operator">/</span><span class="token operator">/</span>192<span class="token punctuation">.</span>168<span class="token punctuation">.</span>2<span class="token punctuation">.</span>104:31672/metrics <span class="token comment"># Prometheus的页面</span> http:<span class="token operator">/</span><span class="token operator">/</span>192<span class="token punctuation">.</span>168<span class="token punctuation">.</span>2<span class="token punctuation">.</span>104:30003 <span class="token comment"># grafana的页面，</span> http:<span class="token operator">/</span><span class="token operator">/</span>192<span class="token punctuation">.</span>168<span class="token punctuation">.</span>2<span class="token punctuation">.</span>104:31267 <span class="token comment"># 账户：admin；密码：*******</span> <div class="hljs-button signin active" data-title="登录复制" data-report-click="{"spm":"1001.2101.3001.4334"}"></div></code><div class="hide-preCode-box"><span class="hide-preCode-bt" data-report-view="{"spm":"1001.2101.3001.7365"}"><img class="look-more-preCode contentImg-no-view" src="https://1000bd.com/contentImg/2022/06/27/191644837.png" alt="" title=""></span></div><ul class="pre-numbering" style=""><li style="color: rgb(153, 153, 153);">1</li><li style="color: rgb(153, 153, 153);">2</li><li style="color: rgb(153, 153, 153);">3</li><li style="color: rgb(153, 153, 153);">4</li><li style="color: rgb(153, 153, 153);">5</li><li style="color: rgb(153, 153, 153);">6</li><li style="color: rgb(153, 153, 153);">7</li><li style="color: rgb(153, 153, 153);">8</li><li style="color: rgb(153, 153, 153);">9</li><li style="color: rgb(153, 153, 153);">10</li><li style="color: rgb(153, 153, 153);">11</li><li style="color: rgb(153, 153, 153);">12</li><li style="color: rgb(153, 153, 153);">13</li><li style="color: rgb(153, 153, 153);">14</li><li style="color: rgb(153, 153, 153);">15</li><li style="color: rgb(153, 153, 153);">16</li><li style="color: rgb(153, 153, 153);">17</li><li style="color: rgb(153, 153, 153);">18</li><li style="color: rgb(153, 153, 153);">19</li><li style="color: rgb(153, 153, 153);">20</li><li style="color: rgb(153, 153, 153);">21</li><li style="color: rgb(153, 153, 153);">22</li><li style="color: rgb(153, 153, 153);">23</li><li style="color: rgb(153, 153, 153);">24</li><li style="color: rgb(153, 153, 153);">25</li><li style="color: rgb(153, 153, 153);">26</li><li style="color: rgb(153, 153, 153);">27</li><li style="color: rgb(153, 153, 153);">28</li><li style="color: rgb(153, 153, 153);">29</li><li style="color: rgb(153, 153, 153);">30</li><li style="color: rgb(153, 153, 153);">31</li><li style="color: rgb(153, 153, 153);">32</li><li style="color: rgb(153, 153, 153);">33</li><li style="color: rgb(153, 153, 153);">34</li><li style="color: rgb(153, 153, 153);">35</li><li style="color: rgb(153, 153, 153);">36</li><li style="color: rgb(153, 153, 153);">37</li><li style="color: rgb(153, 153, 153);">38</li><li style="color: rgb(153, 153, 153);">39</li><li style="color: rgb(153, 153, 153);">40</li><li style="color: rgb(153, 153, 153);">41</li><li style="color: rgb(153, 153, 153);">42</li><li style="color: rgb(153, 153, 153);">43</li><li style="color: rgb(153, 153, 153);">44</li><li style="color: rgb(153, 153, 153);">45</li><li style="color: rgb(153, 153, 153);">46</li><li style="color: rgb(153, 153, 153);">47</li><li style="color: rgb(153, 153, 153);">48</li><li style="color: rgb(153, 153, 153);">49</li><li style="color: rgb(153, 153, 153);">50</li><li style="color: rgb(153, 153, 153);">51</li><li style="color: rgb(153, 153, 153);">52</li><li style="color: rgb(153, 153, 153);">53</li><li style="color: rgb(153, 153, 153);">54</li><li style="color: rgb(153, 153, 153);">55</li><li style="color: rgb(153, 153, 153);">56</li><li style="color: rgb(153, 153, 153);">57</li><li style="color: rgb(153, 153, 153);">58</li><li style="color: rgb(153, 153, 153);">59</li><li style="color: rgb(153, 153, 153);">60</li><li style="color: rgb(153, 153, 153);">61</li><li style="color: rgb(153, 153, 153);">62</li><li style="color: rgb(153, 153, 153);">63</li><li style="color: rgb(153, 153, 153);">64</li><li style="color: rgb(153, 153, 153);">65</li><li style="color: rgb(153, 153, 153);">66</li><li style="color: rgb(153, 153, 153);">67</li><li style="color: rgb(153, 153, 153);">68</li><li style="color: rgb(153, 153, 153);">69</li><li style="color: rgb(153, 153, 153);">70</li><li style="color: rgb(153, 153, 153);">71</li><li style="color: rgb(153, 153, 153);">72</li><li style="color: rgb(153, 153, 153);">73</li><li style="color: rgb(153, 153, 153);">74</li><li style="color: rgb(153, 153, 153);">75</li><li style="color: rgb(153, 153, 153);">76</li><li style="color: rgb(153, 153, 153);">77</li><li style="color: rgb(153, 153, 153);">78</li><li style="color: rgb(153, 153, 153);">79</li><li style="color: rgb(153, 153, 153);">80</li><li style="color: rgb(153, 153, 153);">81</li><li style="color: rgb(153, 153, 153);">82</li><li style="color: rgb(153, 153, 153);">83</li><li style="color: rgb(153, 153, 153);">84</li><li style="color: rgb(153, 153, 153);">85</li><li style="color: rgb(153, 153, 153);">86</li><li style="color: rgb(153, 153, 153);">87</li><li style="color: rgb(153, 153, 153);">88</li><li style="color: rgb(153, 153, 153);">89</li><li style="color: rgb(153, 153, 153);">90</li><li style="color: rgb(153, 153, 153);">91</li><li style="color: rgb(153, 153, 153);">92</li><li style="color: rgb(153, 153, 153);">93</li><li style="color: rgb(153, 153, 153);">94</li><li style="color: rgb(153, 153, 153);">95</li><li style="color: rgb(153, 153, 153);">96</li><li style="color: rgb(153, 153, 153);">97</li><li style="color: rgb(153, 153, 153);">98</li><li style="color: rgb(153, 153, 153);">99</li><li style="color: rgb(153, 153, 153);">100</li><li style="color: rgb(153, 153, 153);">101</li><li style="color: rgb(153, 153, 153);">102</li><li style="color: rgb(153, 153, 153);">103</li><li style="color: rgb(153, 153, 153);">104</li><li style="color: rgb(153, 153, 153);">105</li><li style="color: rgb(153, 153, 153);">106</li><li style="color: rgb(153, 153, 153);">107</li><li style="color: rgb(153, 153, 153);">108</li><li style="color: rgb(153, 153, 153);">109</li><li style="color: rgb(153, 153, 153);">110</li><li style="color: rgb(153, 153, 153);">111</li><li style="color: rgb(153, 153, 153);">112</li><li style="color: rgb(153, 153, 153);">113</li><li style="color: rgb(153, 153, 153);">114</li><li style="color: rgb(153, 153, 153);">115</li><li style="color: rgb(153, 153, 153);">116</li><li style="color: rgb(153, 153, 153);">117</li><li style="color: rgb(153, 153, 153);">118</li><li style="color: rgb(153, 153, 153);">119</li><li style="color: rgb(153, 153, 153);">120</li><li style="color: rgb(153, 153, 153);">121</li><li style="color: rgb(153, 153, 153);">122</li><li style="color: rgb(153, 153, 153);">123</li><li style="color: rgb(153, 153, 153);">124</li><li style="color: rgb(153, 153, 153);">125</li><li style="color: rgb(153, 153, 153);">126</li><li style="color: rgb(153, 153, 153);">127</li><li style="color: rgb(153, 153, 153);">128</li><li style="color: rgb(153, 153, 153);">129</li><li style="color: rgb(153, 153, 153);">130</li><li style="color: rgb(153, 153, 153);">131</li><li style="color: rgb(153, 153, 153);">132</li><li style="color: rgb(153, 153, 153);">133</li><li style="color: rgb(153, 153, 153);">134</li><li style="color: rgb(153, 153, 153);">135</li><li style="color: rgb(153, 153, 153);">136</li><li style="color: rgb(153, 153, 153);">137</li><li style="color: rgb(153, 153, 153);">138</li><li style="color: rgb(153, 153, 153);">139</li><li style="color: rgb(153, 153, 153);">140</li><li style="color: rgb(153, 153, 153);">141</li><li style="color: rgb(153, 153, 153);">142</li><li style="color: rgb(153, 153, 153);">143</li><li style="color: rgb(153, 153, 153);">144</li><li style="color: rgb(153, 153, 153);">145</li><li style="color: rgb(153, 153, 153);">146</li><li style="color: rgb(153, 153, 153);">147</li><li style="color: rgb(153, 153, 153);">148</li><li style="color: rgb(153, 153, 153);">149</li><li style="color: rgb(153, 153, 153);">150</li><li style="color: rgb(153, 153, 153);">151</li><li style="color: rgb(153, 153, 153);">152</li><li style="color: rgb(153, 153, 153);">153</li><li style="color: rgb(153, 153, 153);">154</li><li style="color: rgb(153, 153, 153);">155</li><li style="color: rgb(153, 153, 153);">156</li><li style="color: rgb(153, 153, 153);">157</li><li style="color: rgb(153, 153, 153);">158</li><li style="color: rgb(153, 153, 153);">159</li><li style="color: rgb(153, 153, 153);">160</li><li style="color: rgb(153, 153, 153);">161</li><li style="color: rgb(153, 153, 153);">162</li><li style="color: rgb(153, 153, 153);">163</li><li style="color: rgb(153, 153, 153);">164</li><li style="color: rgb(153, 153, 153);">165</li><li style="color: rgb(153, 153, 153);">166</li><li style="color: rgb(153, 153, 153);">167</li><li style="color: rgb(153, 153, 153);">168</li><li style="color: rgb(153, 153, 153);">169</li><li style="color: rgb(153, 153, 153);">170</li><li style="color: rgb(153, 153, 153);">171</li><li style="color: rgb(153, 153, 153);">172</li><li style="color: rgb(153, 153, 153);">173</li><li style="color: rgb(153, 153, 153);">174</li><li style="color: rgb(153, 153, 153);">175</li><li style="color: rgb(153, 153, 153);">176</li><li style="color: rgb(153, 153, 153);">177</li><li style="color: rgb(153, 153, 153);">178</li><li style="color: rgb(153, 153, 153);">179</li><li style="color: rgb(153, 153, 153);">180</li><li style="color: rgb(153, 153, 153);">181</li><li style="color: rgb(153, 153, 153);">182</li><li style="color: rgb(153, 153, 153);">183</li><li style="color: rgb(153, 153, 153);">184</li><li style="color: rgb(153, 153, 153);">185</li><li style="color: rgb(153, 153, 153);">186</li><li style="color: rgb(153, 153, 153);">187</li><li style="color: rgb(153, 153, 153);">188</li><li style="color: rgb(153, 153, 153);">189</li><li style="color: rgb(153, 153, 153);">190</li><li style="color: rgb(153, 153, 153);">191</li><li style="color: rgb(153, 153, 153);">192</li><li style="color: rgb(153, 153, 153);">193</li><li style="color: rgb(153, 153, 153);">194</li><li style="color: rgb(153, 153, 153);">195</li><li style="color: rgb(153, 153, 153);">196</li><li style="color: rgb(153, 153, 153);">197</li><li style="color: rgb(153, 153, 153);">198</li><li style="color: rgb(153, 153, 153);">199</li><li style="color: rgb(153, 153, 153);">200</li><li style="color: rgb(153, 153, 153);">201</li><li style="color: rgb(153, 153, 153);">202</li><li style="color: rgb(153, 153, 153);">203</li><li style="color: rgb(153, 153, 153);">204</li><li style="color: rgb(153, 153, 153);">205</li><li style="color: rgb(153, 153, 153);">206</li><li style="color: rgb(153, 153, 153);">207</li><li style="color: rgb(153, 153, 153);">208</li><li style="color: rgb(153, 153, 153);">209</li><li style="color: rgb(153, 153, 153);">210</li><li style="color: rgb(153, 153, 153);">211</li><li style="color: rgb(153, 153, 153);">212</li><li style="color: rgb(153, 153, 153);">213</li><li style="color: rgb(153, 153, 153);">214</li><li style="color: rgb(153, 153, 153);">215</li><li style="color: rgb(153, 153, 153);">216</li><li style="color: rgb(153, 153, 153);">217</li><li style="color: rgb(153, 153, 153);">218</li><li style="color: rgb(153, 153, 153);">219</li><li style="color: rgb(153, 153, 153);">220</li><li style="color: rgb(153, 153, 153);">221</li><li style="color: rgb(153, 153, 153);">222</li><li style="color: rgb(153, 153, 153);">223</li><li style="color: rgb(153, 153, 153);">224</li><li style="color: rgb(153, 153, 153);">225</li><li style="color: rgb(153, 153, 153);">226</li><li style="color: rgb(153, 153, 153);">227</li><li style="color: rgb(153, 153, 153);">228</li><li style="color: rgb(153, 153, 153);">229</li><li style="color: rgb(153, 153, 153);">230</li><li style="color: rgb(153, 153, 153);">231</li><li style="color: rgb(153, 153, 153);">232</li><li style="color: rgb(153, 153, 153);">233</li><li style="color: rgb(153, 153, 153);">234</li><li style="color: rgb(153, 153, 153);">235</li><li style="color: rgb(153, 153, 153);">236</li><li style="color: rgb(153, 153, 153);">237</li><li style="color: rgb(153, 153, 153);">238</li><li style="color: rgb(153, 153, 153);">239</li><li style="color: rgb(153, 153, 153);">240</li><li style="color: rgb(153, 153, 153);">241</li><li style="color: rgb(153, 153, 153);">242</li><li style="color: rgb(153, 153, 153);">243</li><li style="color: rgb(153, 153, 153);">244</li><li style="color: rgb(153, 153, 153);">245</li><li style="color: rgb(153, 153, 153);">246</li><li style="color: rgb(153, 153, 153);">247</li><li style="color: rgb(153, 153, 153);">248</li><li style="color: rgb(153, 153, 153);">249</li><li style="color: rgb(153, 153, 153);">250</li><li style="color: rgb(153, 153, 153);">251</li><li style="color: rgb(153, 153, 153);">252</li><li style="color: rgb(153, 153, 153);">253</li><li style="color: rgb(153, 153, 153);">254</li><li style="color: rgb(153, 153, 153);">255</li><li style="color: rgb(153, 153, 153);">256</li><li style="color: rgb(153, 153, 153);">257</li><li style="color: rgb(153, 153, 153);">258</li><li style="color: rgb(153, 153, 153);">259</li><li style="color: rgb(153, 153, 153);">260</li><li style="color: rgb(153, 153, 153);">261</li><li style="color: rgb(153, 153, 153);">262</li><li style="color: rgb(153, 153, 153);">263</li><li style="color: rgb(153, 153, 153);">264</li><li style="color: rgb(153, 153, 153);">265</li><li style="color: rgb(153, 153, 153);">266</li><li style="color: rgb(153, 153, 153);">267</li><li style="color: rgb(153, 153, 153);">268</li><li style="color: rgb(153, 153, 153);">269</li><li style="color: rgb(153, 153, 153);">270</li><li style="color: rgb(153, 153, 153);">271</li><li style="color: rgb(153, 153, 153);">272</li><li style="color: rgb(153, 153, 153);">273</li><li style="color: rgb(153, 153, 153);">274</li><li style="color: rgb(153, 153, 153);">275</li><li style="color: rgb(153, 153, 153);">276</li><li style="color: rgb(153, 153, 153);">277</li><li style="color: rgb(153, 153, 153);">278</li><li style="color: rgb(153, 153, 153);">279</li><li style="color: rgb(153, 153, 153);">280</li><li style="color: rgb(153, 153, 153);">281</li><li style="color: rgb(153, 153, 153);">282</li><li style="color: rgb(153, 153, 153);">283</li><li style="color: rgb(153, 153, 153);">284</li><li style="color: rgb(153, 153, 153);">285</li><li style="color: rgb(153, 153, 153);">286</li><li style="color: rgb(153, 153, 153);">287</li><li style="color: rgb(153, 153, 153);">288</li><li style="color: rgb(153, 153, 153);">289</li><li style="color: rgb(153, 153, 153);">290</li><li style="color: rgb(153, 153, 153);">291</li><li style="color: rgb(153, 153, 153);">292</li><li style="color: rgb(153, 153, 153);">293</li><li style="color: rgb(153, 153, 153);">294</li><li style="color: rgb(153, 153, 153);">295</li><li style="color: rgb(153, 153, 153);">296</li><li style="color: rgb(153, 153, 153);">297</li><li style="color: rgb(153, 153, 153);">298</li><li style="color: rgb(153, 153, 153);">299</li><li style="color: rgb(153, 153, 153);">300</li><li style="color: rgb(153, 153, 153);">301</li><li style="color: rgb(153, 153, 153);">302</li><li style="color: rgb(153, 153, 153);">303</li><li style="color: rgb(153, 153, 153);">304</li><li style="color: rgb(153, 153, 153);">305</li><li style="color: rgb(153, 153, 153);">306</li><li style="color: rgb(153, 153, 153);">307</li><li style="color: rgb(153, 153, 153);">308</li><li style="color: rgb(153, 153, 153);">309</li><li style="color: rgb(153, 153, 153);">310</li><li style="color: rgb(153, 153, 153);">311</li><li style="color: rgb(153, 153, 153);">312</li><li style="color: rgb(153, 153, 153);">313</li><li style="color: rgb(153, 153, 153);">314</li><li style="color: rgb(153, 153, 153);">315</li><li style="color: rgb(153, 153, 153);">316</li><li style="color: rgb(153, 153, 153);">317</li><li style="color: rgb(153, 153, 153);">318</li><li style="color: rgb(153, 153, 153);">319</li><li style="color: rgb(153, 153, 153);">320</li><li style="color: rgb(153, 153, 153);">321</li><li style="color: rgb(153, 153, 153);">322</li><li style="color: rgb(153, 153, 153);">323</li><li style="color: rgb(153, 153, 153);">324</li><li style="color: rgb(153, 153, 153);">325</li><li style="color: rgb(153, 153, 153);">326</li><li style="color: rgb(153, 153, 153);">327</li><li style="color: rgb(153, 153, 153);">328</li><li style="color: rgb(153, 153, 153);">329</li><li style="color: rgb(153, 153, 153);">330</li><li style="color: rgb(153, 153, 153);">331</li><li style="color: rgb(153, 153, 153);">332</li><li style="color: rgb(153, 153, 153);">333</li><li style="color: rgb(153, 153, 153);">334</li><li style="color: rgb(153, 153, 153);">335</li><li style="color: rgb(153, 153, 153);">336</li><li style="color: rgb(153, 153, 153);">337</li><li style="color: rgb(153, 153, 153);">338</li><li style="color: rgb(153, 153, 153);">339</li><li style="color: rgb(153, 153, 153);">340</li><li style="color: rgb(153, 153, 153);">341</li><li style="color: rgb(153, 153, 153);">342</li><li style="color: rgb(153, 153, 153);">343</li><li style="color: rgb(153, 153, 153);">344</li><li style="color: rgb(153, 153, 153);">345</li><li style="color: rgb(153, 153, 153);">346</li><li style="color: rgb(153, 153, 153);">347</li><li style="color: rgb(153, 153, 153);">348</li><li style="color: rgb(153, 153, 153);">349</li><li style="color: rgb(153, 153, 153);">350</li><li style="color: rgb(153, 153, 153);">351</li><li style="color: rgb(153, 153, 153);">352</li><li style="color: rgb(153, 153, 153);">353</li><li style="color: rgb(153, 153, 153);">354</li><li style="color: rgb(153, 153, 153);">355</li><li style="color: rgb(153, 153, 153);">356</li><li style="color: rgb(153, 153, 153);">357</li><li style="color: rgb(153, 153, 153);">358</li><li style="color: rgb(153, 153, 153);">359</li><li style="color: rgb(153, 153, 153);">360</li><li style="color: rgb(153, 153, 153);">361</li><li style="color: rgb(153, 153, 153);">362</li><li style="color: rgb(153, 153, 153);">363</li><li style="color: rgb(153, 153, 153);">364</li><li style="color: rgb(153, 153, 153);">365</li><li style="color: rgb(153, 153, 153);">366</li><li style="color: rgb(153, 153, 153);">367</li><li style="color: rgb(153, 153, 153);">368</li><li style="color: rgb(153, 153, 153);">369</li><li style="color: rgb(153, 153, 153);">370</li><li style="color: rgb(153, 153, 153);">371</li><li style="color: rgb(153, 153, 153);">372</li><li style="color: rgb(153, 153, 153);">373</li><li style="color: rgb(153, 153, 153);">374</li><li style="color: rgb(153, 153, 153);">375</li><li style="color: rgb(153, 153, 153);">376</li><li style="color: rgb(153, 153, 153);">377</li><li style="color: rgb(153, 153, 153);">378</li><li style="color: rgb(153, 153, 153);">379</li><li style="color: rgb(153, 153, 153);">380</li><li style="color: rgb(153, 153, 153);">381</li><li style="color: rgb(153, 153, 153);">382</li><li style="color: rgb(153, 153, 153);">383</li><li style="color: rgb(153, 153, 153);">384</li><li style="color: rgb(153, 153, 153);">385</li><li style="color: rgb(153, 153, 153);">386</li><li style="color: rgb(153, 153, 153);">387</li><li style="color: rgb(153, 153, 153);">388</li><li style="color: rgb(153, 153, 153);">389</li><li style="color: rgb(153, 153, 153);">390</li><li style="color: rgb(153, 153, 153);">391</li><li style="color: rgb(153, 153, 153);">392</li><li style="color: rgb(153, 153, 153);">393</li><li style="color: rgb(153, 153, 153);">394</li><li style="color: rgb(153, 153, 153);">395</li><li style="color: rgb(153, 153, 153);">396</li><li style="color: rgb(153, 153, 153);">397</li><li style="color: rgb(153, 153, 153);">398</li><li style="color: rgb(153, 153, 153);">399</li><li style="color: rgb(153, 153, 153);">400</li><li style="color: rgb(153, 153, 153);">401</li><li style="color: rgb(153, 153, 153);">402</li><li style="color: rgb(153, 153, 153);">403</li><li style="color: rgb(153, 153, 153);">404</li><li style="color: rgb(153, 153, 153);">405</li><li style="color: rgb(153, 153, 153);">406</li><li style="color: rgb(153, 153, 153);">407</li><li style="color: rgb(153, 153, 153);">408</li><li style="color: rgb(153, 153, 153);">409</li><li style="color: rgb(153, 153, 153);">410</li><li style="color: rgb(153, 153, 153);">411</li><li style="color: rgb(153, 153, 153);">412</li><li style="color: rgb(153, 153, 153);">413</li><li style="color: rgb(153, 153, 153);">414</li><li style="color: rgb(153, 153, 153);">415</li><li style="color: rgb(153, 153, 153);">416</li><li style="color: rgb(153, 153, 153);">417</li><li style="color: rgb(153, 153, 153);">418</li><li style="color: rgb(153, 153, 153);">419</li><li style="color: rgb(153, 153, 153);">420</li><li style="color: rgb(153, 153, 153);">421</li><li style="color: rgb(153, 153, 153);">422</li><li style="color: rgb(153, 153, 153);">423</li><li style="color: rgb(153, 153, 153);">424</li><li style="color: rgb(153, 153, 153);">425</li><li style="color: rgb(153, 153, 153);">426</li><li style="color: rgb(153, 153, 153);">427</li><li style="color: rgb(153, 153, 153);">428</li><li style="color: rgb(153, 153, 153);">429</li><li style="color: rgb(153, 153, 153);">430</li><li style="color: rgb(153, 153, 153);">431</li><li style="color: rgb(153, 153, 153);">432</li><li style="color: rgb(153, 153, 153);">433</li><li style="color: rgb(153, 153, 153);">434</li><li style="color: rgb(153, 153, 153);">435</li><li style="color: rgb(153, 153, 153);">436</li><li style="color: rgb(153, 153, 153);">437</li><li style="color: rgb(153, 153, 153);">438</li><li style="color: rgb(153, 153, 153);">439</li><li style="color: rgb(153, 153, 153);">440</li><li style="color: rgb(153, 153, 153);">441</li><li style="color: rgb(153, 153, 153);">442</li><li style="color: rgb(153, 153, 153);">443</li><li style="color: rgb(153, 153, 153);">444</li><li style="color: rgb(153, 153, 153);">445</li><li style="color: rgb(153, 153, 153);">446</li><li style="color: rgb(153, 153, 153);">447</li><li style="color: rgb(153, 153, 153);">448</li><li style="color: rgb(153, 153, 153);">449</li><li style="color: rgb(153, 153, 153);">450</li><li style="color: rgb(153, 153, 153);">451</li><li style="color: rgb(153, 153, 153);">452</li><li style="color: rgb(153, 153, 153);">453</li><li style="color: rgb(153, 153, 153);">454</li><li style="color: rgb(153, 153, 153);">455</li><li style="color: rgb(153, 153, 153);">456</li><li style="color: rgb(153, 153, 153);">457</li><li style="color: rgb(153, 153, 153);">458</li><li style="color: rgb(153, 153, 153);">459</li><li style="color: rgb(153, 153, 153);">460</li><li style="color: rgb(153, 153, 153);">461</li><li style="color: rgb(153, 153, 153);">462</li><li style="color: rgb(153, 153, 153);">463</li><li style="color: rgb(153, 153, 153);">464</li><li style="color: rgb(153, 153, 153);">465</li><li style="color: rgb(153, 153, 153);">466</li><li style="color: rgb(153, 153, 153);">467</li><li style="color: rgb(153, 153, 153);">468</li><li style="color: rgb(153, 153, 153);">469</li><li style="color: rgb(153, 153, 153);">470</li><li style="color: rgb(153, 153, 153);">471</li><li style="color: rgb(153, 153, 153);">472</li><li style="color: rgb(153, 153, 153);">473</li><li style="color: rgb(153, 153, 153);">474</li><li style="color: rgb(153, 153, 153);">475</li><li style="color: rgb(153, 153, 153);">476</li><li style="color: rgb(153, 153, 153);">477</li><li style="color: rgb(153, 153, 153);">478</li><li style="color: rgb(153, 153, 153);">479</li><li style="color: rgb(153, 153, 153);">480</li><li style="color: rgb(153, 153, 153);">481</li><li style="color: rgb(153, 153, 153);">482</li><li style="color: rgb(153, 153, 153);">483</li><li style="color: rgb(153, 153, 153);">484</li><li style="color: rgb(153, 153, 153);">485</li><li style="color: rgb(153, 153, 153);">486</li><li style="color: rgb(153, 153, 153);">487</li><li style="color: rgb(153, 153, 153);">488</li><li style="color: rgb(153, 153, 153);">489</li><li style="color: rgb(153, 153, 153);">490</li><li style="color: rgb(153, 153, 153);">491</li><li style="color: rgb(153, 153, 153);">492</li><li style="color: rgb(153, 153, 153);">493</li><li style="color: rgb(153, 153, 153);">494</li><li style="color: rgb(153, 153, 153);">495</li><li style="color: rgb(153, 153, 153);">496</li><li style="color: rgb(153, 153, 153);">497</li><li style="color: rgb(153, 153, 153);">498</li><li style="color: rgb(153, 153, 153);">499</li><li style="color: rgb(153, 153, 153);">500</li><li style="color: rgb(153, 153, 153);">501</li><li style="color: rgb(153, 153, 153);">502</li><li style="color: rgb(153, 153, 153);">503</li><li style="color: rgb(153, 153, 153);">504</li><li style="color: rgb(153, 153, 153);">505</li><li style="color: rgb(153, 153, 153);">506</li><li style="color: rgb(153, 153, 153);">507</li><li style="color: rgb(153, 153, 153);">508</li><li style="color: rgb(153, 153, 153);">509</li><li style="color: rgb(153, 153, 153);">510</li><li style="color: rgb(153, 153, 153);">511</li><li style="color: rgb(153, 153, 153);">512</li><li style="color: rgb(153, 153, 153);">513</li><li style="color: rgb(153, 153, 153);">514</li><li style="color: rgb(153, 153, 153);">515</li><li style="color: rgb(153, 153, 153);">516</li><li style="color: rgb(153, 153, 153);">517</li><li style="color: rgb(153, 153, 153);">518</li><li style="color: rgb(153, 153, 153);">519</li><li style="color: rgb(153, 153, 153);">520</li><li style="color: rgb(153, 153, 153);">521</li><li style="color: rgb(153, 153, 153);">522</li><li style="color: rgb(153, 153, 153);">523</li><li style="color: rgb(153, 153, 153);">524</li><li style="color: rgb(153, 153, 153);">525</li><li style="color: rgb(153, 153, 153);">526</li><li style="color: rgb(153, 153, 153);">527</li><li style="color: rgb(153, 153, 153);">528</li><li style="color: rgb(153, 153, 153);">529</li><li style="color: rgb(153, 153, 153);">530</li><li style="color: rgb(153, 153, 153);">531</li><li style="color: rgb(153, 153, 153);">532</li><li style="color: rgb(153, 153, 153);">533</li><li style="color: rgb(153, 153, 153);">534</li><li style="color: rgb(153, 153, 153);">535</li><li style="color: rgb(153, 153, 153);">536</li><li style="color: rgb(153, 153, 153);">537</li><li style="color: rgb(153, 153, 153);">538</li><li style="color: rgb(153, 153, 153);">539</li><li style="color: rgb(153, 153, 153);">540</li><li style="color: rgb(153, 153, 153);">541</li><li style="color: rgb(153, 153, 153);">542</li><li style="color: rgb(153, 153, 153);">543</li><li style="color: rgb(153, 153, 153);">544</li><li style="color: rgb(153, 153, 153);">545</li><li style="color: rgb(153, 153, 153);">546</li><li style="color: rgb(153, 153, 153);">547</li><li style="color: rgb(153, 153, 153);">548</li><li style="color: rgb(153, 153, 153);">549</li><li style="color: rgb(153, 153, 153);">550</li><li style="color: rgb(153, 153, 153);">551</li><li style="color: rgb(153, 153, 153);">552</li><li style="color: rgb(153, 153, 153);">553</li><li style="color: rgb(153, 153, 153);">554</li><li style="color: rgb(153, 153, 153);">555</li><li style="color: rgb(153, 153, 153);">556</li><li style="color: rgb(153, 153, 153);">557</li><li style="color: rgb(153, 153, 153);">558</li><li style="color: rgb(153, 153, 153);">559</li><li style="color: rgb(153, 153, 153);">560</li><li style="color: rgb(153, 153, 153);">561</li><li style="color: rgb(153, 153, 153);">562</li><li style="color: rgb(153, 153, 153);">563</li><li style="color: rgb(153, 153, 153);">564</li><li style="color: rgb(153, 153, 153);">565</li><li style="color: rgb(153, 153, 153);">566</li><li style="color: rgb(153, 153, 153);">567</li><li style="color: rgb(153, 153, 153);">568</li><li style="color: rgb(153, 153, 153);">569</li><li style="color: rgb(153, 153, 153);">570</li><li style="color: rgb(153, 153, 153);">571</li><li style="color: rgb(153, 153, 153);">572</li><li style="color: rgb(153, 153, 153);">573</li><li style="color: rgb(153, 153, 153);">574</li><li style="color: rgb(153, 153, 153);">575</li><li style="color: rgb(153, 153, 153);">576</li><li style="color: rgb(153, 153, 153);">577</li><li style="color: rgb(153, 153, 153);">578</li><li style="color: rgb(153, 153, 153);">579</li><li style="color: rgb(153, 153, 153);">580</li><li style="color: rgb(153, 153, 153);">581</li><li style="color: rgb(153, 153, 153);">582</li><li style="color: rgb(153, 153, 153);">583</li><li style="color: rgb(153, 153, 153);">584</li><li style="color: rgb(153, 153, 153);">585</li><li style="color: rgb(153, 153, 153);">586</li><li style="color: rgb(153, 153, 153);">587</li><li style="color: rgb(153, 153, 153);">588</li><li style="color: rgb(153, 153, 153);">589</li><li style="color: rgb(153, 153, 153);">590</li><li style="color: rgb(153, 153, 153);">591</li><li style="color: rgb(153, 153, 153);">592</li><li style="color: rgb(153, 153, 153);">593</li><li style="color: rgb(153, 153, 153);">594</li><li style="color: rgb(153, 153, 153);">595</li><li style="color: rgb(153, 153, 153);">596</li><li style="color: rgb(153, 153, 153);">597</li><li style="color: rgb(153, 153, 153);">598</li><li style="color: rgb(153, 153, 153);">599</li><li style="color: rgb(153, 153, 153);">600</li><li style="color: rgb(153, 153, 153);">601</li><li style="color: rgb(153, 153, 153);">602</li><li style="color: rgb(153, 153, 153);">603</li><li style="color: rgb(153, 153, 153);">604</li><li style="color: rgb(153, 153, 153);">605</li><li style="color: rgb(153, 153, 153);">606</li><li style="color: rgb(153, 153, 153);">607</li><li style="color: rgb(153, 153, 153);">608</li><li style="color: rgb(153, 153, 153);">609</li><li style="color: rgb(153, 153, 153);">610</li><li style="color: rgb(153, 153, 153);">611</li><li style="color: rgb(153, 153, 153);">612</li><li style="color: rgb(153, 153, 153);">613</li><li style="color: rgb(153, 153, 153);">614</li><li style="color: rgb(153, 153, 153);">615</li><li style="color: rgb(153, 153, 153);">616</li><li style="color: rgb(153, 153, 153);">617</li><li style="color: rgb(153, 153, 153);">618</li><li style="color: rgb(153, 153, 153);">619</li><li style="color: rgb(153, 153, 153);">620</li><li style="color: rgb(153, 153, 153);">621</li><li style="color: rgb(153, 153, 153);">622</li><li style="color: rgb(153, 153, 153);">623</li><li style="color: rgb(153, 153, 153);">624</li><li style="color: rgb(153, 153, 153);">625</li><li style="color: rgb(153, 153, 153);">626</li><li style="color: rgb(153, 153, 153);">627</li><li style="color: rgb(153, 153, 153);">628</li><li style="color: rgb(153, 153, 153);">629</li><li style="color: rgb(153, 153, 153);">630</li><li style="color: rgb(153, 153, 153);">631</li><li style="color: rgb(153, 153, 153);">632</li><li style="color: rgb(153, 153, 153);">633</li><li style="color: rgb(153, 153, 153);">634</li><li style="color: rgb(153, 153, 153);">635</li><li style="color: rgb(153, 153, 153);">636</li><li style="color: rgb(153, 153, 153);">637</li><li style="color: rgb(153, 153, 153);">638</li><li style="color: rgb(153, 153, 153);">639</li><li style="color: rgb(153, 153, 153);">640</li><li style="color: rgb(153, 153, 153);">641</li><li style="color: rgb(153, 153, 153);">642</li><li style="color: rgb(153, 153, 153);">643</li><li style="color: rgb(153, 153, 153);">644</li><li style="color: rgb(153, 153, 153);">645</li><li style="color: rgb(153, 153, 153);">646</li><li style="color: rgb(153, 153, 153);">647</li><li style="color: rgb(153, 153, 153);">648</li><li style="color: rgb(153, 153, 153);">649</li><li style="color: rgb(153, 153, 153);">650</li><li style="color: rgb(153, 153, 153);">651</li><li style="color: rgb(153, 153, 153);">652</li><li style="color: rgb(153, 153, 153);">653</li><li style="color: rgb(153, 153, 153);">654</li><li style="color: rgb(153, 153, 153);">655</li><li style="color: rgb(153, 153, 153);">656</li><li style="color: rgb(153, 153, 153);">657</li><li style="color: rgb(153, 153, 153);">658</li><li style="color: rgb(153, 153, 153);">659</li><li style="color: rgb(153, 153, 153);">660</li><li style="color: rgb(153, 153, 153);">661</li><li style="color: rgb(153, 153, 153);">662</li><li style="color: rgb(153, 153, 153);">663</li><li style="color: rgb(153, 153, 153);">664</li><li style="color: rgb(153, 153, 153);">665</li><li style="color: rgb(153, 153, 153);">666</li><li style="color: rgb(153, 153, 153);">667</li><li style="color: rgb(153, 153, 153);">668</li><li style="color: rgb(153, 153, 153);">669</li><li style="color: rgb(153, 153, 153);">670</li><li style="color: rgb(153, 153, 153);">671</li><li style="color: rgb(153, 153, 153);">672</li><li style="color: rgb(153, 153, 153);">673</li><li style="color: rgb(153, 153, 153);">674</li><li style="color: rgb(153, 153, 153);">675</li><li style="color: rgb(153, 153, 153);">676</li><li style="color: rgb(153, 153, 153);">677</li><li style="color: rgb(153, 153, 153);">678</li><li style="color: rgb(153, 153, 153);">679</li><li style="color: rgb(153, 153, 153);">680</li><li style="color: rgb(153, 153, 153);">681</li><li style="color: rgb(153, 153, 153);">682</li><li style="color: rgb(153, 153, 153);">683</li><li style="color: rgb(153, 153, 153);">684</li><li style="color: rgb(153, 153, 153);">685</li><li style="color: rgb(153, 153, 153);">686</li><li style="color: rgb(153, 153, 153);">687</li><li style="color: rgb(153, 153, 153);">688</li><li style="color: rgb(153, 153, 153);">689</li><li style="color: rgb(153, 153, 153);">690</li><li style="color: rgb(153, 153, 153);">691</li><li style="color: rgb(153, 153, 153);">692</li><li style="color: rgb(153, 153, 153);">693</li><li style="color: rgb(153, 153, 153);">694</li><li style="color: rgb(153, 153, 153);">695</li><li style="color: rgb(153, 153, 153);">696</li><li style="color: rgb(153, 153, 153);">697</li><li style="color: rgb(153, 153, 153);">698</li><li style="color: rgb(153, 153, 153);">699</li><li style="color: rgb(153, 153, 153);">700</li><li style="color: rgb(153, 153, 153);">701</li><li style="color: rgb(153, 153, 153);">702</li><li style="color: rgb(153, 153, 153);">703</li><li style="color: rgb(153, 153, 153);">704</li><li style="color: rgb(153, 153, 153);">705</li><li style="color: rgb(153, 153, 153);">706</li><li style="color: rgb(153, 153, 153);">707</li><li style="color: rgb(153, 153, 153);">708</li><li style="color: rgb(153, 153, 153);">709</li><li style="color: rgb(153, 153, 153);">710</li><li style="color: rgb(153, 153, 153);">711</li><li style="color: rgb(153, 153, 153);">712</li><li style="color: rgb(153, 153, 153);">713</li><li style="color: rgb(153, 153, 153);">714</li></ul></pre> <h4><a name="t22"></a><a id="11abk8s_3646"></a>11、使用测试软件ab对整个k8s集群和相关的服务器进行压力测试</h4> <pre data-index="21" class="set-code-hide prettyprint"><code class="prism language-powershell has-numbering" onclick="mdcp.signin(event)" style="position: unset;"><span class="token comment"># 1.运行php-apache服务器并暴露服务</span> <span class="token namespace">[root@k8smaster hpa]</span><span class="token comment"># ls</span> php-apache<span class="token punctuation">.</span>yaml <span class="token namespace">[root@k8smaster hpa]</span><span class="token comment"># cat php-apache.yaml </span> apiVersion: apps/v1 kind: Deployment metadata: name: php-apache spec: selector: matchLabels: run: php-apache template: metadata: labels: run: php-apache spec: containers: <span class="token operator">-</span> name: php-apache image: k8s<span class="token punctuation">.</span>gcr<span class="token punctuation">.</span>io/hpa-example imagePullPolicy: IfNotPresent ports: <span class="token operator">-</span> containerPort: 80 resources: limits: cpu: 500m requests: cpu: 200m <span class="token operator">--</span><span class="token operator">-</span> apiVersion: v1 kind: Service metadata: name: php-apache labels: run: php-apache spec: ports: <span class="token operator">-</span> port: 80 selector: run: php-apache <span class="token namespace">[root@k8smaster hpa]</span><span class="token comment"># kubectl apply -f php-apache.yaml </span> deployment<span class="token punctuation">.</span>apps/php-apache created service/php-apache created <span class="token namespace">[root@k8smaster hpa]</span><span class="token comment"># kubectl get deploy</span> NAME READY UP-TO-DATE AVAILABLE AGE php-apache 1/1 1 1 93s <span class="token namespace">[root@k8smaster hpa]</span><span class="token comment"># kubectl get pod</span> NAME READY STATUS RESTARTS AGE php-apache-567d9f79d-mhfsp 1/1 Running 0 44s <span class="token comment"># 创建HPA功能</span> <span class="token namespace">[root@k8smaster hpa]</span><span class="token comment"># kubectl autoscale deployment php-apache --cpu-percent=10 --min=1 --max=10</span> horizontalpodautoscaler<span class="token punctuation">.</span>autoscaling/php-apache autoscaled <span class="token namespace">[root@k8smaster hpa]</span><span class="token comment"># kubectl get hpa</span> NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE php-apache Deployment/php-apache <unknown><span class="token operator">/</span>10% 1 10 0 7s <span class="token comment"># 测试，增加负载</span> <span class="token namespace">[root@k8smaster hpa]</span><span class="token comment"># kubectl run -i --tty load-generator --rm --image=busybox:1.28 --restart=Never -- /bin/sh -c "while sleep 0.01; do wget -q -O- http://php-apache; done"</span> <span class="token keyword">If</span> you don't see a command prompt<span class="token punctuation">,</span> <span class="token keyword">try</span> pressing enter<span class="token punctuation">.</span> OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK!OK <span class="token namespace">[root@k8smaster hpa]</span><span class="token comment"># kubectl get hpa</span> NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE php-apache Deployment/php-apache 0%<span class="token operator">/</span>10% 1 10 1 3m24s <span class="token namespace">[root@k8smaster hpa]</span><span class="token comment"># kubectl get hpa</span> NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE php-apache Deployment/php-apache 238%<span class="token operator">/</span>10% 1 10 1 3m41s <span class="token namespace">[root@k8smaster hpa]</span><span class="token comment"># kubectl get hpa</span> NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE php-apache Deployment/php-apache 250%<span class="token operator">/</span>10% 1 10 4 3m57s <span class="token comment"># 一旦CPU利用率降至0，HPA会自动将副本数缩减为 1。自动扩缩完成副本数量的改变可能需要几分钟的时间</span> <span class="token comment"># 2.对web服务进行压力测试，观察promethues和dashboard</span> <span class="token comment"># ab命令访问web：192.168.2.112:30001 同时进入prometheus和dashboard观察pod</span> <span class="token comment"># 四种方式观察</span> kubectl top pod http:<span class="token operator">/</span><span class="token operator">/</span>192<span class="token punctuation">.</span>168<span class="token punctuation">.</span>2<span class="token punctuation">.</span>117:3000/ http:<span class="token operator">/</span><span class="token operator">/</span>192<span class="token punctuation">.</span>168<span class="token punctuation">.</span>2<span class="token punctuation">.</span>117:9090/targets https:<span class="token operator">/</span><span class="token operator">/</span>192<span class="token punctuation">.</span>168<span class="token punctuation">.</span>2<span class="token punctuation">.</span>104:32571/ <span class="token namespace">[root@nfs ~]</span><span class="token comment"># yum install httpd-tools -y</span> <span class="token namespace">[root@nfs data]</span><span class="token comment"># ab -n 1000000 -c 10000 -g output.dat http://192.168.2.112:30001/</span> This is ApacheBench<span class="token punctuation">,</span> Version 2<span class="token punctuation">.</span>3 <<span class="token variable">$Revision</span>: 1430300 $> Copyright 1996 Adam Twiss<span class="token punctuation">,</span> Zeus Technology Ltd<span class="token punctuation">,</span> http:<span class="token operator">/</span><span class="token operator">/</span>www<span class="token punctuation">.</span>zeustech<span class="token punctuation">.</span>net/ Licensed to The Apache Software Foundation<span class="token punctuation">,</span> http:<span class="token operator">/</span><span class="token operator">/</span>www<span class="token punctuation">.</span>apache<span class="token punctuation">.</span>org/ Benchmarking 192<span class="token punctuation">.</span>168<span class="token punctuation">.</span>2<span class="token punctuation">.</span>112 <span class="token punctuation">(</span>be patient<span class="token punctuation">)</span> apr_socket_recv: Connection reset by peer <span class="token punctuation">(</span>104<span class="token punctuation">)</span> Total of 3694 requests completed <span class="token comment"># 1000个请求，10并发数 ab -n 1000 -c 10 -g output.dat http://192.168.2.112:30001/</span> <span class="token operator">-</span>t 60 在60秒内发送尽可能多的请求 <div class="hljs-button signin active" data-title="登录复制" data-report-click="{"spm":"1001.2101.3001.4334"}"></div></code><div class="hide-preCode-box"><span class="hide-preCode-bt" data-report-view="{"spm":"1001.2101.3001.7365"}"><img class="look-more-preCode contentImg-no-view" src="https://1000bd.com/contentImg/2022/06/27/191644837.png" alt="" title=""></span></div><ul class="pre-numbering" style=""><li style="color: rgb(153, 153, 153);">1</li><li style="color: rgb(153, 153, 153);">2</li><li style="color: rgb(153, 153, 153);">3</li><li style="color: rgb(153, 153, 153);">4</li><li style="color: rgb(153, 153, 153);">5</li><li style="color: rgb(153, 153, 153);">6</li><li style="color: rgb(153, 153, 153);">7</li><li style="color: rgb(153, 153, 153);">8</li><li style="color: rgb(153, 153, 153);">9</li><li style="color: rgb(153, 153, 153);">10</li><li style="color: rgb(153, 153, 153);">11</li><li style="color: rgb(153, 153, 153);">12</li><li style="color: rgb(153, 153, 153);">13</li><li style="color: rgb(153, 153, 153);">14</li><li style="color: rgb(153, 153, 153);">15</li><li style="color: rgb(153, 153, 153);">16</li><li style="color: rgb(153, 153, 153);">17</li><li style="color: rgb(153, 153, 153);">18</li><li style="color: rgb(153, 153, 153);">19</li><li style="color: rgb(153, 153, 153);">20</li><li style="color: rgb(153, 153, 153);">21</li><li style="color: rgb(153, 153, 153);">22</li><li style="color: rgb(153, 153, 153);">23</li><li style="color: rgb(153, 153, 153);">24</li><li style="color: rgb(153, 153, 153);">25</li><li style="color: rgb(153, 153, 153);">26</li><li style="color: rgb(153, 153, 153);">27</li><li style="color: rgb(153, 153, 153);">28</li><li style="color: rgb(153, 153, 153);">29</li><li style="color: rgb(153, 153, 153);">30</li><li style="color: rgb(153, 153, 153);">31</li><li style="color: rgb(153, 153, 153);">32</li><li style="color: rgb(153, 153, 153);">33</li><li style="color: rgb(153, 153, 153);">34</li><li style="color: rgb(153, 153, 153);">35</li><li style="color: rgb(153, 153, 153);">36</li><li style="color: rgb(153, 153, 153);">37</li><li style="color: rgb(153, 153, 153);">38</li><li style="color: rgb(153, 153, 153);">39</li><li style="color: rgb(153, 153, 153);">40</li><li style="color: rgb(153, 153, 153);">41</li><li style="color: rgb(153, 153, 153);">42</li><li style="color: rgb(153, 153, 153);">43</li><li style="color: rgb(153, 153, 153);">44</li><li style="color: rgb(153, 153, 153);">45</li><li style="color: rgb(153, 153, 153);">46</li><li style="color: rgb(153, 153, 153);">47</li><li style="color: rgb(153, 153, 153);">48</li><li style="color: rgb(153, 153, 153);">49</li><li style="color: rgb(153, 153, 153);">50</li><li style="color: rgb(153, 153, 153);">51</li><li style="color: rgb(153, 153, 153);">52</li><li style="color: rgb(153, 153, 153);">53</li><li style="color: rgb(153, 153, 153);">54</li><li style="color: rgb(153, 153, 153);">55</li><li style="color: rgb(153, 153, 153);">56</li><li style="color: rgb(153, 153, 153);">57</li><li style="color: rgb(153, 153, 153);">58</li><li style="color: rgb(153, 153, 153);">59</li><li style="color: rgb(153, 153, 153);">60</li><li style="color: rgb(153, 153, 153);">61</li><li style="color: rgb(153, 153, 153);">62</li><li style="color: rgb(153, 153, 153);">63</li><li style="color: rgb(153, 153, 153);">64</li><li style="color: rgb(153, 153, 153);">65</li><li style="color: rgb(153, 153, 153);">66</li><li style="color: rgb(153, 153, 153);">67</li><li style="color: rgb(153, 153, 153);">68</li><li style="color: rgb(153, 153, 153);">69</li><li style="color: rgb(153, 153, 153);">70</li><li style="color: rgb(153, 153, 153);">71</li><li style="color: rgb(153, 153, 153);">72</li><li style="color: rgb(153, 153, 153);">73</li><li style="color: rgb(153, 153, 153);">74</li><li style="color: rgb(153, 153, 153);">75</li><li style="color: rgb(153, 153, 153);">76</li><li style="color: rgb(153, 153, 153);">77</li><li style="color: rgb(153, 153, 153);">78</li><li style="color: rgb(153, 153, 153);">79</li><li style="color: rgb(153, 153, 153);">80</li><li style="color: rgb(153, 153, 153);">81</li><li style="color: rgb(153, 153, 153);">82</li><li style="color: rgb(153, 153, 153);">83</li><li style="color: rgb(153, 153, 153);">84</li><li style="color: rgb(153, 153, 153);">85</li><li style="color: rgb(153, 153, 153);">86</li><li style="color: rgb(153, 153, 153);">87</li><li style="color: rgb(153, 153, 153);">88</li><li style="color: rgb(153, 153, 153);">89</li></ul></pre> </div> </div> </li> <li class="list-group-item ul-li"> <b>相关阅读:</b><br> <nobr> <a href="/Article/Index/745194">Vim插件合集 （打造你的专属炫酷IDE）</a> <br /> <a href="/Article/Index/799184">配置windows限时登录</a> <br /> <a href="/Article/Index/1407744">QTableWidget如何在标题行的其他列添加控件</a> <br /> <a href="/Article/Index/612374">KestrelServer详解[1]：注册监听终结点（Endpoint）</a> <br /> <a href="/Article/Index/1074542">HackTheBox Ambassador 枚举获得用户shell，git consul API提权</a> <br /> <a href="/Article/Index/1275028">6.网络编程套接字(下)</a> <br /> <a href="/Article/Index/1353526">es6.x和es7.x如何创建索引？</a> <br /> <a href="/Article/Index/991697">JavaScript：DOM</a> <br /> <a href="/Article/Index/1516732">[2023年]-hadoop面试真题(三)</a> <br /> <a href="/Article/Index/705995">C# 同步 异步 回调 状态机 async await Demo</a> <br /> </nobr> </li> <li class="list-group-item from-a mb-2"> 原文地址：https://blog.csdn.net/m0_54232496/article/details/132922690 </li> </ul> </div> <div class="col-lg-4 col-sm-12"> <ul class="list-group" style="word-break:break-all;"> <li class="list-group-item ul-li-bg" aria-current="true"> 最新文章 </li> <li class="list-group-item ul-li"> <nobr> <a href="/Article/Index/1484446">攻防演习之三天拿下官网站群</a> <br /> <a href="/Article/Index/1515268">数据安全治理学习——前期安全规划和安全管理体系建设</a> <br /> <a href="/Article/Index/1759065">企业安全 | 企业内一次钓鱼演练准备过程</a> <br /> <a href="/Article/Index/1485036">内网渗透测试 | Kerberos协议及其部分攻击手法</a> <br /> <a href="/Article/Index/1877332">0day的产生 | 不懂代码的"代码审计"</a> <br /> <a href="/Article/Index/1887576">安装scrcpy-client模块av模块异常，环境问题解决方案</a> <br /> <a href="/Article/Index/1887578">leetcode hot100【LeetCode 279. 完全平方数】java实现</a> <br /> <a href="/Article/Index/1887512">OpenWrt下安装Mosquitto</a> <br /> <a href="/Article/Index/1887520">AnatoMask论文汇总</a> <br /> <a href="/Article/Index/1887496">【AI日记】24.11.01 LangChain、openai api和github copilot</a> <br /> </nobr> </li> </ul> <ul class="list-group pt-2" style="word-break:break-all;"> <li class="list-group-item ul-li-bg" aria-current="true"> 热门文章 </li> <li class="list-group-item ul-li"> <nobr> <a href="/Article/Index/888177">十款代码表白小特效 一个比一个浪漫 赶紧收藏起来吧！！！</a> <br /> <a href="/Article/Index/797680">奉劝各位学弟学妹们，该打造你的技术影响力了！</a> <br /> <a href="/Article/Index/888183">五年了，我在 CSDN 的两个一百万。</a> <br /> <a href="/Article/Index/888179">Java俄罗斯方块，老程序员花了一个周末，连接中学年代！</a> <br /> <a href="/Article/Index/797730">面试官都震惊，你这网络基础可以啊！</a> <br /> <a href="/Article/Index/797725">你真的会用百度吗？我不信 — 那些不为人知的搜索引擎语法</a> <br /> <a href="/Article/Index/797702">心情不好的时候，用 Python 画棵樱花树送给自己吧</a> <br /> <a href="/Article/Index/797709">通宵一晚做出来的一款类似CS的第一人称射击游戏Demo！原来做游戏也不是很难，连憨憨学妹都学会了！</a> <br /> <a href="/Article/Index/797716">13 万字 C 语言从入门到精通保姆级教程2021 年版</a> <br /> <a href="/Article/Index/888192">10行代码集2000张美女图，Python爬虫120例，再上征途</a> <br /> </nobr> </li> </ul> </div> </div> </div> <!-- 主体 --> <!--body结束--> <!--这里是footer模板--> <!--footer--> <nav class="navbar navbar-inverse navbar-fixed-bottom"> <div class="container"> <div class="row"> <div class="col-md-12"> <div class="text-muted center foot-height"> Copyright © 2022 侵权请联系<a href="mailto:2656653265@qq.com">2656653265@qq.com</a> <a href="https://beian.miit.gov.cn/" target="_blank">京ICP备2022015340号-1</a> </div> <div style="width:300px;margin:0 auto; padding:0px 5px;"> <a href="/regex.html">正则表达式工具</a> <a href="/cron.html">cron表达式工具</a> <a href="/pwdcreator.html">密码生成工具</a> </div> <div style="width:300px;margin:0 auto; padding:5px 0;"> <a target="_blank" href="http://www.beian.gov.cn/portal/registerSystemInfo?recordcode=11010502049817" style="display:inline-block;text-decoration:none;height:20px;line-height:20px;"> <img src="" style="float:left;" /><p style="float:left;height:20px;line-height:20px;margin: 0px 0px 0px 5px; color:#939393;">京公网安备 11010502049817号</p></a> </div> </div> </div> </div> </nav> <!--footer--> <!--footer模板结束--> <script src="/js/plugins/jquery/jquery.js"></script> <script src="/js/bootstrap.min.js"></script> <!--这里是scripts模板--> <!--scripts模板结束--> </body> </html>
