企业网站的发展历史,网站创意策划案,企业服务网站制作,html网页设计模板和源代码本节重点介绍 :
打镜像#xff0c;导出镜像#xff0c;传输到各个节点并导入运行该项目配置prometheus和grafana
打镜像
本地build docker build -t ink8s-pod-metrics:v1 .build过程 导出镜像 docker save ink8s-pod-metrics ink8s-pod-metrics.tar 传输到各个node…本节重点介绍 :
打镜像导出镜像传输到各个节点并导入运行该项目配置prometheus和grafana
打镜像
本地build docker build -t ink8s-pod-metrics:v1 .build过程 导出镜像 docker save ink8s-pod-metrics ink8s-pod-metrics.tar
传输到各个node节点上
scp ink8s-pod-metrics.tar k8s-node01:~各个node节点上导入镜像
使用docker
docker load ink8s-pod-metrics.tar使用containerd
ctr --namespace k8s.io images import ink8s-pod-metrics.tar
运行该项目 kubectl apply -f rbac.yamlkubectl apply -f deployment.yaml
检查
[rootk8s-master01 ink8s-pod-metrics]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
grafana-d5d85bcd6-f74ch 1/1 Running 0 3d9h 10.100.85.199 k8s-node01 none none
grafana-d5d85bcd6-l44mx 1/1 Running 0 3d9h 10.100.85.198 k8s-node01 none none
ink8s-pod-metrics-deployment-85d9795d6-95lsp 1/1 Running 0 13m 10.100.85.207 k8s-node01 none none日志
[rootk8s-master01 ink8s-pod-metrics]# kubectl logs -l appink8s-pod-metrics -f
2021-08-23 20:34:35.377256 INFO app/get_k8s_objs.go:128 [pod.label:map[component:etcd tier:control-plane]]
2021-08-23 20:34:35.377266 INFO app/get_k8s_objs.go:128 [pod.label:map[component:kube-apiserver tier:control-plane]]
2021-08-23 20:34:35.377274 INFO app/get_k8s_objs.go:128 [pod.label:map[component:kube-controller-manager tier:control-plane]]
2021-08-23 20:34:35.377292 INFO app/get_k8s_objs.go:128 [pod.label:map[controller-revision-hash:85c698c6d4 k8s-app:kube-proxy pod-template-generation:1]]
2021-08-23 20:34:35.377299 INFO app/get_k8s_objs.go:128 [pod.label:map[controller-revision-hash:85c698c6d4 k8s-app:kube-proxy pod-template-generation:1]]
2021-08-23 20:34:35.377317 INFO app/get_k8s_objs.go:128 [pod.label:map[component:kube-scheduler tier:control-plane]]
2021-08-23 20:34:35.377324 INFO app/get_k8s_objs.go:128 [pod.label:map[app.kubernetes.io/name:kube-state-metrics app.kubernetes.io/version:v1.9.7 pod-template-hash:564668c858]]
2021-08-23 20:34:35.377331 INFO app/get_k8s_objs.go:128 [pod.label:map[k8s-app:metrics-server pod-template-hash:7dbf6c4558]]
2021-08-23 20:34:35.377336 INFO app/get_k8s_objs.go:128 [pod.label:map[controller-revision-hash:prometheus-5b9cdcfd6c k8s-app:prometheus statefulset.kubernetes.io/pod-name:prometheus-0]]
2021-08-23 20:34:35.377358 INFO app/get_k8s_objs.go:143 server_pod_ips_result][num_pod:11][time_took_seconds:6.189551999]
2021-08-23 20:34:39.197614 INFO app/get_k8s_objs.go:107 server_node_ips_result][num_node:2][time_took_seconds:0.009575987]
2021-08-23 20:34:39.200824 INFO app/get_k8s_objs.go:128 [pod.label:map[k8s-app:kube-dns pod-template-hash:68b9d7b887]]
2021-08-23 20:34:39.200857 INFO app/get_k8s_objs.go:128 [pod.label:map[k8s-app:kube-dns pod-template-hash:68b9d7b887]]
2021-08-23 20:34:39.200871 INFO app/get_k8s_objs.go:128 [pod.label:map[component:etcd tier:control-plane]]
2021-08-23 20:34:39.200889 INFO app/get_k8s_objs.go:128 [pod.label:map[component:kube-apiserver tier:control-plane]]
2021-08-23 20:34:39.200903 INFO app/get_k8s_objs.go:128 [pod.label:map[component:kube-controller-manager tier:control-plane]]
2021-08-23 20:34:39.200920 INFO app/get_k8s_objs.go:128 [pod.label:map[controller-revision-hash:85c698c6d4 k8s-app:kube-proxy pod-template-generation:1]]
2021-08-23 20:34:39.200934 INFO app/get_k8s_objs.go:128 [pod.label:map[controller-revision-hash:85c698c6d4 k8s-app:kube-proxy pod-template-generation:1]]
2021-08-23 20:34:39.200947 INFO app/get_k8s_objs.go:128 [pod.label:map[component:kube-scheduler tier:control-plane]]
2021-08-23 20:34:39.200961 INFO app/get_k8s_objs.go:128 [pod.label:map[app.kubernetes.io/name:kube-state-metrics app.kubernetes.io/version:v1.9.7 pod-template-hash:564668c858]]
2021-08-23 20:34:39.200981 INFO app/get_k8s_objs.go:128 [pod.label:map[k8s-app:metrics-server pod-template-hash:7dbf6c4558]]
2021-08-23 20:34:39.200992 INFO app/get_k8s_objs.go:128 [pod.label:map[controller-revision-hash:prometheus-5b9cdcfd6c k8s-app:prometheus statefulset.kubernetes.io/pod-name:prometheus-0]]
2021-08-23 20:34:39.201022 INFO app/get_k8s_objs.go:143 server_pod_ips_result][num_pod:11][time_took_seconds:0.013052527]node上请求 pod 的metrics
curl pod的ip:8080/metrics
[rootk8s-master01 ink8s-pod-metrics]# curl -s 10.100.85.207:8080/metrics |grep ink8s
# HELP ink8s_pod_metrics_get_node_detail k8s node detail each
# TYPE ink8s_pod_metrics_get_node_detail gauge
ink8s_pod_metrics_get_node_detail{containerRuntimeVersioncontainerd://1.4.4,hostnamek8s-master01,ip172.20.70.205,kubeletVersionv1.20.1} 1
ink8s_pod_metrics_get_node_detail{containerRuntimeVersioncontainerd://1.4.4,hostnamek8s-node01,ip172.20.70.215,kubeletVersionv1.20.1} 1
# HELP ink8s_pod_metrics_get_node_last_duration_seconds get node last duration seconds
# TYPE ink8s_pod_metrics_get_node_last_duration_seconds gauge
ink8s_pod_metrics_get_node_last_duration_seconds 0.008066143
# HELP ink8s_pod_metrics_get_pod_control_plane_pod_detail k8s pod detail of control plane
# TYPE ink8s_pod_metrics_get_pod_control_plane_pod_detail gauge
ink8s_pod_metrics_get_pod_control_plane_pod_detail{componentetcd,ip172.20.70.205,pod_nameetcd-k8s-master01} 1
ink8s_pod_metrics_get_pod_control_plane_pod_detail{componentkube-apiserver,ip172.20.70.205,pod_namekube-apiserver-k8s-master01} 1
ink8s_pod_metrics_get_pod_control_plane_pod_detail{componentkube-controller-manager,ip172.20.70.205,pod_namekube-controller-manager-k8s-master01} 1
ink8s_pod_metrics_get_pod_control_plane_pod_detail{componentkube-scheduler,ip172.20.70.205,pod_namekube-scheduler-k8s-master01} 1
# HELP ink8s_pod_metrics_get_pod_last_duration_seconds get pod last duration seconds
# TYPE ink8s_pod_metrics_get_pod_last_duration_seconds gauge
ink8s_pod_metrics_get_pod_last_duration_seconds 0.01159838prometheus target页面检查pod 发现pod已经发现到了但是 报错向http的server发送https的请求
prometheus和grafana配置
检查 kubernetes-pods的job
如果之前配置的https需要改为http的
- job_name: kubernetes-podshonor_timestamps: truescrape_interval: 30sscrape_timeout: 10smetrics_path: /metricsscheme: httpfollow_redirects: truerelabel_configs:- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]separator: ;regex: truereplacement: $1action: keep- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]separator: ;regex: (.)target_label: __metrics_path__replacement: $1action: replace- source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]separator: ;regex: ([^:])(?::\d)?;(\d)target_label: __address__replacement: $1:$2action: replace- separator: ;regex: __meta_kubernetes_pod_label_(.)replacement: $1action: labelmap- source_labels: [__meta_kubernetes_namespace]separator: ;regex: (.*)target_label: kubernetes_namespacereplacement: $1action: replace- source_labels: [__meta_kubernetes_pod_name]separator: ;regex: (.*)target_label: kubernetes_pod_namereplacement: $1action: replacekubernetes_sd_configs:- role: podkubeconfig_file: follow_redirects: true在prometheus中检查指标
查询 ink8s_pod_metrics_get_node_detail
ink8s_pod_metrics_get_node_detail{appink8s-pod-metrics, containerRuntimeVersioncontainerd://1.4.4, hostnamek8s-master01, instance10.100.85.207:8080, ip172.20.70.205, jobkubernetes-pods, kubeletVersionv1.20.1, kubernetes_namespacedefault, kubernetes_pod_nameink8s-pod-metrics-deployment-85d9795d6-95lsp, pod_template_hash85d9795d6}
1
ink8s_pod_metrics_get_node_detail{appink8s-pod-metrics, containerRuntimeVersioncontainerd://1.4.4, hostnamek8s-node01, instance10.100.85.207:8080, ip172.20.70.215, jobkubernetes-pods, kubeletVersionv1.20.1, kubernetes_namespacedefault, kubernetes_pod_nameink8s-pod-metrics-deployment-85d9795d6-95lsp, pod_template_hash85d9795d6}配置grafana
举例图片
本节重点总结 :
打镜像导出镜像传输到各个节点并导入运行该项目配置prometheus和grafana 文章转载自： http://www.morning.xckqs.cn.gov.cn.xckqs.cn http://www.morning.nrftd.cn.gov.cn.nrftd.cn http://www.morning.nlqgb.cn.gov.cn.nlqgb.cn http://www.morning.myxps.cn.gov.cn.myxps.cn http://www.morning.czrcf.cn.gov.cn.czrcf.cn http://www.morning.qrpx.cn.gov.cn.qrpx.cn http://www.morning.krjrb.cn.gov.cn.krjrb.cn http://www.morning.rmyt.cn.gov.cn.rmyt.cn http://www.morning.c7630.cn.gov.cn.c7630.cn http://www.morning.ubpsa.cn.gov.cn.ubpsa.cn http://www.morning.fxpyt.cn.gov.cn.fxpyt.cn http://www.morning.wnywk.cn.gov.cn.wnywk.cn http://www.morning.bswnf.cn.gov.cn.bswnf.cn http://www.morning.gqjqf.cn.gov.cn.gqjqf.cn http://www.morning.bbxbh.cn.gov.cn.bbxbh.cn http://www.morning.rcyrm.cn.gov.cn.rcyrm.cn http://www.morning.ymtbr.cn.gov.cn.ymtbr.cn http://www.morning.plgbh.cn.gov.cn.plgbh.cn http://www.morning.httzf.cn.gov.cn.httzf.cn http://www.morning.mjxgs.cn.gov.cn.mjxgs.cn http://www.morning.jcxzq.cn.gov.cn.jcxzq.cn http://www.morning.kbkcl.cn.gov.cn.kbkcl.cn http://www.morning.kongpie.com.gov.cn.kongpie.com http://www.morning.zdxss.cn.gov.cn.zdxss.cn http://www.morning.pmdlk.cn.gov.cn.pmdlk.cn http://www.morning.cbvlus.cn.gov.cn.cbvlus.cn http://www.morning.qwzpd.cn.gov.cn.qwzpd.cn http://www.morning.ghxzd.cn.gov.cn.ghxzd.cn http://www.morning.fqyqm.cn.gov.cn.fqyqm.cn http://www.morning.rbrhj.cn.gov.cn.rbrhj.cn http://www.morning.qtzk.cn.gov.cn.qtzk.cn http://www.morning.qcymf.cn.gov.cn.qcymf.cn http://www.morning.ywgrr.cn.gov.cn.ywgrr.cn http://www.morning.ctfh.cn.gov.cn.ctfh.cn http://www.morning.bpmtr.cn.gov.cn.bpmtr.cn http://www.morning.qcrhb.cn.gov.cn.qcrhb.cn http://www.morning.yqsr.cn.gov.cn.yqsr.cn http://www.morning.snjpj.cn.gov.cn.snjpj.cn http://www.morning.ksggl.cn.gov.cn.ksggl.cn http://www.morning.qhjkz.cn.gov.cn.qhjkz.cn http://www.morning.qmtzq.cn.gov.cn.qmtzq.cn http://www.morning.jzlfq.cn.gov.cn.jzlfq.cn http://www.morning.jqcrf.cn.gov.cn.jqcrf.cn http://www.morning.jkrrg.cn.gov.cn.jkrrg.cn http://www.morning.ngcbd.cn.gov.cn.ngcbd.cn http://www.morning.bhmnp.cn.gov.cn.bhmnp.cn http://www.morning.dphmj.cn.gov.cn.dphmj.cn http://www.morning.dgwrz.cn.gov.cn.dgwrz.cn http://www.morning.ghxtk.cn.gov.cn.ghxtk.cn http://www.morning.ghlyy.cn.gov.cn.ghlyy.cn http://www.morning.rkdnm.cn.gov.cn.rkdnm.cn http://www.morning.owenzhi.com.gov.cn.owenzhi.com http://www.morning.pwmpn.cn.gov.cn.pwmpn.cn http://www.morning.qxlhj.cn.gov.cn.qxlhj.cn http://www.morning.zbnkt.cn.gov.cn.zbnkt.cn http://www.morning.hxwhyjh.com.gov.cn.hxwhyjh.com http://www.morning.lrjtx.cn.gov.cn.lrjtx.cn http://www.morning.qkrz.cn.gov.cn.qkrz.cn http://www.morning.bmlcy.cn.gov.cn.bmlcy.cn http://www.morning.kongpie.com.gov.cn.kongpie.com http://www.morning.wjlbb.cn.gov.cn.wjlbb.cn http://www.morning.yzxhk.cn.gov.cn.yzxhk.cn http://www.morning.pmwhj.cn.gov.cn.pmwhj.cn http://www.morning.nwynx.cn.gov.cn.nwynx.cn http://www.morning.bnxfj.cn.gov.cn.bnxfj.cn http://www.morning.dcpbk.cn.gov.cn.dcpbk.cn http://www.morning.mxhcf.cn.gov.cn.mxhcf.cn http://www.morning.zylzk.cn.gov.cn.zylzk.cn http://www.morning.bsghk.cn.gov.cn.bsghk.cn http://www.morning.qmwzz.cn.gov.cn.qmwzz.cn http://www.morning.qmbgb.cn.gov.cn.qmbgb.cn http://www.morning.rfrx.cn.gov.cn.rfrx.cn http://www.morning.ntzfj.cn.gov.cn.ntzfj.cn http://www.morning.jqkjr.cn.gov.cn.jqkjr.cn http://www.morning.krrjb.cn.gov.cn.krrjb.cn http://www.morning.jrqbr.cn.gov.cn.jrqbr.cn http://www.morning.rzdpd.cn.gov.cn.rzdpd.cn http://www.morning.mcjyair.com.gov.cn.mcjyair.com http://www.morning.mnygn.cn.gov.cn.mnygn.cn http://www.morning.rqgjr.cn.gov.cn.rqgjr.cn
