nginx-ingress-controller日志持久化方案的解决

nginx-ingress-controller日志持久化方案的解决

本文主要介绍nginx-ingress-controller的日志持久化方案的解决方案，通过示例代码进行了详细介绍，对大家的学习或工作有一定的参考价值。有需要的朋友下面和边肖一起学习。

最近看了一篇微信官方账号的文章，关于nginx-ingress-controller的应用。下面有人评论如何坚持日志，正好工作中遇到这个问题，所以我整理了一个方案，仅供参考。

nginx入口控制器的日志

NGX入口控制器的日志由三部分组成:

控制器日志:输出到stdout，可以通过启动参数中的–-log_dir配置为输出到文件。重定向到文件后，会自动旋转，但不会自动清理。

Access:输出到stdout，nginx-configuration中的字段可以配置输出到哪个文件。输出到文件后，它不会自动旋转或清理。

Errorlog:输出到stderr，配置方式类似于accesslog。

下载控制器日志

挂一个hostpath到nginx-ingress-controller:/data/log/nginx_ingress_controller/并映射到容器中的/var/log/nginx_ingress_controller/，

为nginx-ingress-controller配置log-dir和logtostderr参数，并将日志重定向到/var/log/nginx_ingress_controller/。

控制器的日志需要定期清理。由于控制器的日志是通过klog(k8s.io/klog)输出的，会对日志进行滚动，所以我们可以通过脚本定期清理某个时间之前的日志文件。

下载nginx日志

修改配置映射:nginx-configuration。配置accesslog和errorlog的输出路径，替换默认的stdout和stderr。输出路径可以和控制器一致，很容易找到。

accesslog和errorlog都只有一个日志文件。我们可以使用logrotate进行日志旋转，并旋转和清理输出到主机的日志。配置，例如:

$cat/etc/logrotate.d/nginx.log /data/log/nginx_ingress_controller/access.log{ surootlist rotate7 daily maxsize50M copytruncate missingok create0644www-dataroot }

在官方模板中，nginx-ingress-controller默认以用户登录33启动容器，所以挂载hostpath时存在权限问题。我们需要在机器上手动执行chown-r33:33/data/log/nginx_ingress_controller。

自动化 *** 作

nginx日志在磁盘中，第2点和第3点需要手动维护。有什么解决办法吗？

问题的关键是:有没有办法在nginx-ingress-controller容器启动之前添加一个钩子，在主机的指定目录上执行chown？

可以使用initContainer。Init必须在容器initcontainer运行并成功退出之前运行。利用这个k8s特性，我们开发了一个docker映像，它只执行以下脚本:

#!/bin/bash logdir=$LOG_DIR userID=$USER_ID echo"trytosetdir:$logdir'sgroupas$userID" chown-R$userID:$userID$logdir

该脚本读取一些环境变量，确认需要修改哪个目录，以及需要更改到哪种用户组。

将脚本打包到dockerimage中，并作为initcontainers放入nginx-ingress-controller的deployyaml中。注意这个initcontainer的环境变量和volumeMount的配置。

先说第二点。我们注意到nginx-ingress-controller的基本映像附带了logrotate，所以问题很简单。我们可以将写好的logrotate配置文件以configmap的形式挂载到容器中。

部署yaml如下所示:

--- apiVersion:v1 kind:Service metadata: name:ingress-nginx namespace:kube-system spec: type:ClusterIP ports: -name:http port:80 targetPort:80 protocol:TCP -name:https port:443 targetPort:443 protocol:TCP selector: app:ingress-nginx --- apiVersion:v1 kind:Service metadata: name:default-http-backend namespace:kube-system labels: app:default-http-backend spec: ports: -port:80 targetPort:8080 selector: app:default-http-backend --- apiVersion:extensions/v1beta1 kind:Ingress metadata: name:default namespace:kube-system spec: backend: serviceName:default-http-backend servicePort:80 --- kind:ConfigMap apiVersion:v1 metadata: name:nginx-configuration namespace:kube-system labels: app:ingress-nginx data: use-forwarded-headers:"true" #此处配置nginx日志的重定向目标 access-log-path:/var/log/nginx_ingress_controller/access.log error-log-path:/var/log/nginx_ingress_controller/error.log --- #创建一个configmap，配置nginx日志的轮转策略，对应的是nginx日志在容器内的日志文件 apiVersion:v1 data: nginx.log:| {{user_nginx_log.host_path}}/access.log{ rotate{{user_nginx_log.rotate_count}} daily maxsize{{user_nginx_log.rotate_size}} minsize10M copytruncate missingok create0644rootroot } {{user_nginx_log.host_path}}/error.log{ rotate{{user_nginx_log.rotate_count}} daily maxsize{{user_nginx_log.rotate_size}} minsize10M copytruncate missingok create0644rootroot } kind:ConfigMap metadata: name:nginx-ingress-logrotate namespace:kube-system --- kind:ConfigMap apiVersion:v1 metadata: name:tcp-services namespace:kube-system --- kind:ConfigMap apiVersion:v1 metadata: name:udp-services namespace:kube-system --- apiVersion:v1 kind:ServiceAccount metadata: name:nginx-ingress-serviceaccount namespace:kube-system --- apiVersion:rbac.authorization.k8s.io/v1beta1 kind:ClusterRole metadata: name:nginx-ingress-clusterrole rules: -apiGroups: -"" resources: -configmaps -endpoints -nodes -pods -secrets verbs: -list -watch -apiGroups: -"" resources: -nodes verbs: -get -apiGroups: -"" resources: -services verbs: -get -list -watch -apiGroups: -"extensions" resources: -ingresses verbs: -get -list -watch -apiGroups: -"" resources: -events verbs: -create -patch -apiGroups: -"extensions" resources: -ingresses/status verbs: -update --- apiVersion:rbac.authorization.k8s.io/v1beta1 kind:Role metadata: name:nginx-ingress-role namespace:kube-system rules: -apiGroups: -"" resources: -configmaps -pods -secrets -namespaces verbs: -get -apiGroups: -"" resources: -configmaps resourceNames: #Defaultsto"<election-id>-<ingress-class>" #Here:"<ingress-controller-leader>-<nginx>" #Thishastobeadaptedifyouchangeeitherparameter #whenlaunchingthenginx-ingress-controller. -"ingress-controller-leader-nginx" verbs: -get -update -apiGroups: -"" resources: -configmaps verbs: -create -apiGroups: -"" resources: -endpoints verbs: -get --- apiVersion:rbac.authorization.k8s.io/v1beta1 kind:RoleBinding metadata: name:nginx-ingress-role-nisa-binding namespace:kube-system roleRef: apiGroup:rbac.authorization.k8s.io kind:Role name:nginx-ingress-role subjects: -kind:ServiceAccount name:nginx-ingress-serviceaccount namespace:kube-system --- apiVersion:rbac.authorization.k8s.io/v1beta1 kind:ClusterRoleBinding metadata: name:nginx-ingress-clusterrole-nisa-binding roleRef: apiGroup:rbac.authorization.k8s.io kind:ClusterRole name:nginx-ingress-clusterrole subjects: -kind:ServiceAccount name:nginx-ingress-serviceaccount namespace:kube-system --- apiVersion:apps/v1 kind:DaemonSet metadata: name:ingress-nginx namespace:kube-system spec: selector: matchLabels: app:ingress-nginx template: metadata: labels: app:ingress-nginx annotations: prometheus.io/port:'10254' prometheus.io/scrape:'true' spec: serviceAccountName:nginx-ingress-serviceaccount tolerations: -key:dedicated value:ingress-nginx effect:NoSchedule affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: -matchExpressions: -key:"system/ingress" operator:In values: -"true" dnsPolicy:ClusterFirstWithHostNet hostNetwork:true #配置initcontainer，确保在nginx-ingress-controller容器启动前将日志目录的权限配置好 initContainers: -name:adddirperm image:"{{image_registry.addr}}/{{image.adddirperm}}" env: -name:LOG_DIR value:/var/log/nginx_ingress_controller -name:USER_ID value:"33" volumeMounts: -name:logdir mountPath:/var/log/nginx_ingress_controller containers: -name:nginx-ingress-controller image:"{{image_registry.addr}}/{{image.ingress}}" imagePullPolicy:IfNotPresent args: -/nginx-ingress-controller ---default-backend-service=$(POD_NAMESPACE)/default-http-backend ---configmap=$(POD_NAMESPACE)/nginx-configuration ---tcp-services-configmap=$(POD_NAMESPACE)/tcp-services ---udp-services-configmap=$(POD_NAMESPACE)/udp-services ---publish-service=$(POD_NAMESPACE)/ingress-nginx ---annotations-prefix=nginx.ingress.kubernetes.io #设置controller日志的输出路径和方式 ---log_dir=/var/log/nginx_ingress_controller ---logtostderr=false securityContext: capabilities: drop: -ALL add: -NET_BIND_SERVICE #www-data->33 runAsUser:33 env: -name:POD_NAME valueFrom: fieldRef: fieldPath:metadata.name -name:POD_NAMESPACE valueFrom: fieldRef: fieldPath:metadata.namespace ports: -name:http containerPort:80 -name:https containerPort:443 resources: requests: cpu:100m memory:256Mi livenessProbe: failureThreshold:3 httpGet: path:/healthz port:10254 scheme:HTTP initialDelaySeconds:10 periodSeconds:10 successThreshold:1 timeoutSeconds:1 readinessProbe: failureThreshold:3 httpGet: path:/healthz port:10254 scheme:HTTP periodSeconds:10 successThreshold:1 timeoutSeconds:1 volumeMounts: #配置挂载容器中控制器组件和nginx的日志输出路径 -name:logdir mountPath:/var/log/nginx_ingress_controller #配置nginx日志的logrotate配置挂载路径 -name:logrotateconf mountPath:/etc/logrotate.d/nginx.log subPath:nginx.log volumes: #控制器组件和nginx的日志输出路径为宿主机的hostpath -name:logdir hostPath: path:{{user_nginx_log.host_path}} type:"" #nginx日志的轮转配置文件来自于configmap -name:logrotateconf configMap: name:nginx-ingress-logrotate items: -key:nginx.log path:nginx.log --- apiVersion:apps/v1 kind:DaemonSet metadata: name:default-http-backend namespace:kube-system labels: app:default-http-backend spec: selector: matchLabels: app:default-http-backend template: metadata: labels: app:default-http-backend spec: terminationGracePeriodSeconds:60 tolerations: -key:dedicated value:ingress-nginx effect:NoSchedule affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: -matchExpressions: -key:"system/ingress" operator:In values: -"true" containers: -name:default-http-backend #Anyimageispermissibleaslongas: #1.Itservesa404pageat/ #2.Itserves200ona/healthzendpoint image:"{{image_registry.addr}}/{{image.http_backend}}" imagePullPolicy:IfNotPresent livenessProbe: httpGet: path:/healthz port:8080 scheme:HTTP initialDelaySeconds:30 timeoutSeconds:5 ports: -containerPort:8080 resources: limits: cpu:10m memory:20Mi requests: cpu:10m memory:20Mi ---

最后有人建议去掉initcontainer，改为在原有nginx-ingress-controller镜像的基础上增加一层，将配置路径权限的脚本放在这一层执行。个人认为这种方法既不美观也不方便。唯一的优点是部署yaml仍然很简单(但是没有volumeMount这样的配置)。不过还是要看个人经验~

就是这样。本文介绍了nginx-ingress控制器日志持久化方案的解决方案。有关nginxingress控制器日志持久性的更多信息，请搜索我们之前的文章或继续浏览下面的相关文章。希望大家以后能多多支持我们！