网站源码如何优化,只做网站的,茂名平安建设网站,常州网站建设常州k8s笔记29--使用kyverno提高运维效率 介绍原理安装应用场景自动修正测试环境pod资源强制 Pod 标签限制容器镜像来源禁止特权容器其它潜在场景 注意事项说明 介绍
Kyverno是一个云原生的策略引擎#xff0c;它最初是为k8s构建的#xff0c;现在也可以在k8s集群之外用作统一的… k8s笔记29--使用kyverno提高运维效率 介绍原理安装应用场景自动修正测试环境pod资源强制 Pod 标签限制容器镜像来源禁止特权容器其它潜在场景 注意事项说明 介绍
Kyverno是一个云原生的策略引擎它最初是为k8s构建的现在也可以在k8s集群之外用作统一的策略语言。其允许平台工程师自动化安全、合规和关于验证的最佳实践并为相关团队提供安全的自助服务。 在 K8S的复杂生态系统中确保集群资源的合规性和安全性是一项极具挑战的任务。Kyverno 作为一个强大的k8s策略引擎为我们提供了一种有效的解决方案。本文将深入探讨 Kyverno 的核心原理、经典案例以及使用过程中的注意事项。
原理
Kyverno 通过 Kubernetes 的准入控制器Admission Controller机制来实现策略的执行。当用户向 Kubernetes API Server 提交资源创建、更新或删除请求时准入控制器会拦截这些请求并将其发送给 Kyverno 进行策略检查。 Kyverno 的策略以自定义资源定义CRD的形式存在用户可以通过编写 YAML 文件来定义各种策略规则。这些规则可以基于资源的类型、名称、标签选择器等属性进行匹配并执行相应的操作如拒绝请求、修改资源或添加注释等。 例如我们可以定义一个策略要求所有新建的 Pod 必须包含特定的标签。当用户尝试创建一个不包含该标签的 Pod 时Kyverno 会根据策略规则拒绝这个请求从而确保集群中所有 Pod 都符合我们的标签规范。
kyverno逻辑架构图 https://kyverno.io/docs/introduction/how-kyverno-works/ 策略和规则的运行方式 https://kyverno.io/docs/kyverno-policies/ 如下图所示一个Policy可以包含多个Rule每个规则必须包含一个Match(Exclude可选)和validate,mutate,generate,verifyImages等选项之一。 策略可以定义为集群层面的ClusterPolicy也可以定义为命名空间层面的Policy。
安装
安装文档https://kyverno.io/docs/installation/ 可以直接通过yaml的方式安装也可以通过helm的方式安装 yaml安装方法:
kubectl create -f https://github.com/kyverno/kyverno/releases/download/v1.13.0/install.yamlhelm安装方法:
单机版本
helm repo add kyverno https://kyverno.github.io/kyverno/
helm repo update
helm install kyverno kyverno/kyverno -n kyverno --create-namespace高可用版本
helm install kyverno kyverno/kyverno -n kyverno --create-namespace \
--set admissionController.replicas3 \
--set backgroundController.replicas3 \
--set cleanupController.replicas3 \
--set reportsController.replicas3安装成功会有如下4个running的deploy
$ kubectl -n kyverno get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
kyverno-admission-controller 1/1 1 1 24d
kyverno-background-controller 1/1 1 1 24d
kyverno-cleanup-controller 1/1 1 1 24d
kyverno-reports-controller 1/1 1 1 24d注意不同版本的kyverno支持的k8s版本有限安装前可以在官方文档查看版本支持信息
应用场景
自动添加DR的outlierDetection 火山迁移期间新建DR的时候会默认添加outlierDetection确保优先网格内服务的稳定性 使用一下策略后新建DR如果没有trafficPolicy属性则会自动补充策略中的内容。 策略内容
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:name: general-namespace-add-workload-dr-outlier-detection
spec:admission: truebackground: truerules:- exclude:all:- resources:namespaces:- quicksilver- appsvrmatch:all:- resources:kinds:- networking.istio.io/v1beta1/DestinationRulemutate:patchStrategicMerge:spec:(trafficPolicy):outlierDetection:baseEjectionTime: 60sconsecutive5xxErrors: 0consecutiveLocalOriginFailures: 10interval: 10smaxEjectionPercent: 80splitExternalLocalOriginErrors: truename: general-namespace-add-workload-dr-outlier-detection自动修正测试环境pod资源
测试环境主动限制指定命名空间的服务资源request值可以显著降低业务request过多空资源
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:name: ebc-default-cpu-resources
spec:background: truerules:- name: default-cpu-resourcesmatch:any:- resources:kinds:- Podnamespaces:- ebc-aladdinsite- ebc-bpmexclude:any:- resources:kinds:- Podselector:matchLabels:app: nacos- resources:kinds:- Podselector:matchLabels:app.kubernetes.io/name: ebc-ldapmutate:patchStrategicMerge:spec:containers:- (name): *resources:requests:cpu: 100mmemory: 500Milimits:cpu: 2000mmemory: 4000Mi- name: set-resources-init-containersmatch:any:- resources:kinds:- Podnamespaces:- ebc-aladdinsite- ebc-bpmpreconditions:all:- key: {{ request.object.spec.initContainers || }}operator: NotEqualsvalue: mutate:patchStrategicMerge:spec:initContainers:- (name): *resources:requests:cpu: 100mmemory: 200Milimits:cpu: 1000mmemory: 2000Mi也可以通过如下方式设置资源请求和限制
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:name: test-kyverno-set-resource-limits
spec:validationFailureAction: enforcerules:- name: set-cpu-memory-limitsmatch:resources:kinds:- Deploymentnamespaces:- test-kyverno mutate:patchesJson6902:- op: addpath: /spec/template/spec/containers/0/resourcesvalue:requests:cpu: 100mmemory: 128Milimits:cpu: 500mmemory: 512Mi强制 Pod 标签
确保所有部署到集群中的 Pod 都带有特定的标签用于资源管理和监控。
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:name: test-kyverno-enforce-pod-labels
spec:validationFailureAction: enforcerules:- name: check-pod-labelsmatch:resources:kinds:- Podnamespaces:- test-kyvernovalidate:message: Pod must have environment and app labels.pattern:metadata:labels:environment: ?*app: ?*此时创建一个缺少label的deploy就会报错, 加上 environment 标签后就可以正常访创建了
$ vim test-nginx-with-label.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: test2-deploymentnamespace: test-kyvernolabels:app: test2# environment: prod
spec:replicas: 1selector:matchLabels:app: test2# environment: prodtemplate:metadata:creationTimestamp: nulllabels:app: test2# environment: prodspec:containers:- name: test2image: nginx:1.23ports:- containerPort: 80protocol: TCPresources: {}$ kubectl apply -f test-nginx-with-label.yaml
The Deployment test2-deployment is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{app:test2}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable限制容器镜像来源
为了保障安全只允许从特定的镜像仓库拉取容器镜像
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:name: test-kyverno-restrict-image-registry
spec:validationFailureAction: enforcerules:- name: check-image-registrymatch:resources:kinds:- Podnamespaces:- test-kyverno validate:message: Only images from my-allowed-registry.com are allowed.pattern:spec:containers:- image: my-allowed-registry.com/*此时创建非my-allowed-registry.com 镜像的pod会报错
$ vim test-nginx-with-image-registry.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: test2-deploymentnamespace: test-kyvernolabels:app: test2
spec:replicas: 1selector:matchLabels:app: test2template:metadata:labels:app: test2spec:containers:- name: test2image: nginx:1.23# image: my-allowed-registry.com/nginx:1.23ports:- containerPort: 80protocol: TCPresources: {}$ kubectl apply -f test-nginx-with-image-registry.yaml
Error from server: error when creating test-nginx-with-image-registry.yaml: admission webhook validate.kyverno.svc-fail denied the request: resource Deployment/test-kyverno/test2-deployment was blocked due to the following policies test-kyverno-restrict-image-registry:autogen-check-image-registry: validation error: Only images from my-allowed-registry.comare allowed. rule autogen-check-image-registry failed at path /spec/template/spec/containers/0/image/禁止特权容器
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:name: test-kyverno-prohibit-privileged-containers
spec:validationFailureAction: enforcerules:- name: check-privilegedmatch:resources:kinds:- Podnamespaces:- test-kyverno validate:message: Privileged containers are not allowed.pattern:spec:containers:- privileged: false开启特权就会报错
$ vim test-nginx-with-prohibit-privileged-containers.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: test2-deploymentnamespace: test-kyvernolabels:app: test2
spec:replicas: 1selector:matchLabels:app: test2template:metadata:labels:app: test2spec:containers:- name: test2image: nginx:1.23ports:- containerPort: 80protocol: TCPresources: {}securityContext:privileged: true$ kubectl apply -f test-nginx-with-prohibit-privileged-containers.yaml
Error from server: error when creating test-nginx-with-prohibit-privileged-containers.yaml: admission webhook validate.kyverno.svc-fail denied the request: resource Deployment/test-kyverno/test2-deployment was blocked due to the following policies test-kyverno-prohibit-privileged-containers:autogen-check-privileged: validation error: Privileged containers are not allowed.rule autogen-check-privileged failed at path /spec/template/spec/containers/0/privileged/其它潜在场景
新建命名空间的时候创建对应的configmap新建命名空间的时候创建对应的resource quote给deploy添加默认的环境变量限制pod默认安全策略
注意事项
策略优先级 当存在多个策略时需要注意策略的优先级设置。Kyverno 按照策略的名称顺序进行匹配和执行因此需要合理命名策略以确保正确的执行顺序。资源匹配规则 在编写策略时要仔细定义资源的匹配规则避免误判或漏判。例如使用通配符时要确保其范围符合预期。测试环境 在将策略应用到生产环境之前务必在测试环境中进行全面且充分的测试确保策略的正确性与有效性防止因策略错误导致业务中断。性能影响 尽管Kyverno的设计旨在实现高效运行但大量复杂的策略仍可能对Kubernetes API Server的性能产生一定影响。因此需要定期评估和优化策略以保障系统的稳定运行。
说明
环境 k8s: v1.30.0 kyverno: v1.11.1 参考文档 https://kyverno.io/docs/introduction/#quick-start-guides https://github.com/kyverno/kyverno/
