Ingress-Nginx 概念解析与实践部署指南

Ingress Nginx 是 Kubernetes 中最常用的 Ingress 控制器之一，由 NGINX 官方提供支持（备注：nginx 和 k8s 官方各自维护了一套 nginx ingress controller）。

NGINX Ingress Controller 是 NGINX 和 NGINX Plus 的 Ingress Controller 实现，可以对 Websocket、gRPC、TCP 和 UDP 应用程序进行负载平衡。它支持标准 Ingress 功能，例如基于内容的路由和 TLS/SSL 终止。

1.4 ingress-nginx 优点和限制

1.5 版本兼容性矩阵

ingress-nginx 项目支持的版本意味着官方已经完成了 E2E 测试，并且它们正在通过列出的版本。 Ingress-Nginx 版本可能适用于旧版本，但该项目不做出这种保证。

2 实践：Ingress Nginx 部署

本文 k8s 环境是 v1.29.7, ingress-nginx 是 v1.11.1

⚠️ 注意：提前将 ingress-nginx 的镜像下载并上传到私有仓库

[root@k8s-master1 ingress-nginx]# grep -n -r image: ingress-nginx.yaml
447: image: registry.k8s.io/ingress-nginx/controller:v1.11.1 @sha256:e6439a12b52076965928e83b7b56aae6731231677b01e81818bce7fa5c60161a
548: image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.1 @sha256:36d05b4077fb8e3d13663702fa337f124675ba8667cbd949c03a8e8ea6fa4366 

2.1 使用 helm 部署 ingress-nginx

官方推荐 helm 方式部署，如果环境没有 helm 需要先下载：helm 下载地址

2.1.1 安装和配置 Helm

安装 helm

tar -zxvf helm-v3.15.3-linux-amd64.tar.gz --strip-components 1 -C /usr/local/bin linux-amd64/helm 

添加 ingress-nginx 的 repo

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx 

更新 repo 信息

helm repo update 

搜索 ingress-nginx 的 Chart

[root@k8s-master1 ingress-nginx]# helm search repo ingress-nginx --version 4.11.1
NAME CHART VERSION APP VERSION DESCRIPTION
ingress-nginx/ingress-nginx 4.11.1 1.11.1 Ingress controller for Kubernetes using NGINX a...

❔ 说明：--version 4.11.1：指定 Chart 版本搜索

下载 ingress-nginx 的 Chart

mkdir helm && cd helm
helm pull ingress-nginx/ingress-nginx --version 4.11.1 

解压 Charts

# 解压 Chart
tar zxvf ingress-nginx-4.11.1.tgz
# 进入 Chart 目录
cd ingress-nginx 

2.1.2 配置和创建 Ingress-Nginx

调整 Values.yaml 的参数

• 镜像仓库和镜像

21 image:
22 ## Keep false as default for now!
23 chroot: false
24 # 注释原仓库地址
25 #registry: registry.k8s.io
26 # 替换成阿里云仓库地址
27 registry: registry.cn-hangzhou.aliyuncs.com
28 image: ingress-nginx/controller
32 tag: "v1.11.1"
33 # 注释 sha256 校验码
34 #digest: sha256:e6439a12b52076965928e83b7b56aae6731231677b01e81818bce7fa5c60161a
---
805 image:
806 # 注释原仓库地址
807 #registry: registry.k8s.io
808 registry: registry.cn-hangzhou.aliyuncs.com
809 image: ingress-nginx/kube-webhook-certgen
813 tag: v1.4.1
814 # 注释 sha256 校验码
815 #digest: sha256:36d05b4077fb8e3d13663702fa337f124675ba8667cbd949c03a8e8ea6fa4366

• 使用本地网络和 dns

# 启用主机网络
103 hostNetwork: true
... # 启用主机 dns 服务
78 dnsPolicy: ClusterFirstWithHostNet 

• 修改 Ingress 模式和指定 port

# 默认是 LoadBalancer
484 #type: LoadBalancer
485 type: NodePort
...
540 appProtocol: true
541 nodePorts:
542 # -- Node port allocated for the external HTTP listener. If left empty, the service controller allocates one from the configured node port range.
543 http: "30080"
544 # -- Node port allocated for the external HTTPS listener. If left empty, the service controller allocates one from the configured node port range.
545 https: "30443"

创建前，执行以下命令检查配置是否修改成功：

helm install ingress-nginx . --dry-run=client --namespace=ingress-nginx 

创建 ingress-nignx

helm install ingress-nginx . --namespace=ingress-nginx --create-namespace 

查看是否创建成功

[root@k8s-master1 ingress-nginx]# helm list --namespace ingress-nginx
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
ingress-nginx ingress-nginx 1 2024-08-27 21:44:34.137344711 +0800 CST deployed ingress-nginx-4.11.1 1.11.1 

卸载 ingress-nginx

[root@k8s-master1 ingress-nginx]# helm uninstall ingress-nginx -n ingress-nginx
release "ingress-nginx" uninstalled 

2.2 使用 yaml 文件部署 ingress-nginx

1. 参考官方 yaml 文件

2. 也可以参考以下配置

apiVersion: v1
kind: Namespace
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx
  namespace: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - namespaces
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - configmaps
  - pods
  - secrets
  - endpoints
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses/status
  verbs:
  - update
- apiGroups:
  - networking.k8s.io
  resources:
  - ingressclasses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - coordination.k8s.io
  resourceNames:
  - ingress-nginx-leader
  resources:
  - leases
  verbs:
  - get
  - update
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - create
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - discovery.k8s.io
  resources:
  - endpointslices
  verbs:
  - list
  - watch
  - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-admission
  namespace: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
  - create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  - endpoints
  - nodes
  - pods
  - secrets
  - namespaces
  verbs:
  - list
  - watch
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses/status
  verbs:
  - update
- apiGroups:
  - networking.k8s.io
  resources:
  - ingressclasses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - discovery.k8s.io
  resources:
  - endpointslices
  verbs:
  - list
  - watch
  - get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-admission
rules:
- apiGroups:
  - admissionregistration.k8s.io
  resources:
  - validatingwebhookconfigurations
  verbs:
  - get
  - update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx
subjects:
- kind: ServiceAccount
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-admission
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx
subjects:
- kind: ServiceAccount
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-admission
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: v1
data:
  allow-snippet-annotations: "false"
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-controller
  namespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  externalTrafficPolicy: Local
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - appProtocol: http
    name: http
    port: 80
    protocol: TCP
    targetPort: http
  - appProtocol: https
    name: https
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-controller-admission
  namespace: ingress-nginx
spec:
  ports:
  - appProtocol: https
    name: https-webhook
    port: 443
    targetPort: webhook
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  minReadySeconds: 0
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/component: controller
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/name: ingress-nginx
  strategy:
    rollingUpdate:
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      labels:
        app.kubernetes.io/component: controller
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.11.1
    spec:
      containers:
      - args:
        - /nginx-ingress-controller
        - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
        - --election-id=ingress-nginx-leader
        - --controller-class=k8s.io/ingress-nginx
        - --ingress-class=nginx
        - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
        - --validating-webhook=:8443
        - --validating-webhook-certificate=/usr/local/certificates/cert
        - --validating-webhook-key=/usr/local/certificates/key
        - --enable-metrics=false
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: LD_PRELOAD
          value: /usr/local/lib/libmimalloc.so
        image: registry.cn-hangzhou.aliyuncs.com/ingress-nginx/controller:v1.11.1
        imagePullPolicy: IfNotPresent
        lifecycle:
          preStop:
            exec:
              command:
              - /wait-shutdown
        livenessProbe:
          failureThreshold: 5
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: controller
        ports:
        - containerPort: 80
          name: http
          protocol: TCP
        - containerPort: 443
          name: https
          protocol: TCP
        - containerPort: 8443
          name: webhook
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources:
          requests:
            cpu: 100m
            memory: 90Mi
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            add:
            - NET_BIND_SERVICE
            drop:
            - ALL
          readOnlyRootFilesystem: false
          runAsNonRoot: true
          runAsUser: 101
          seccompProfile:
            type: RuntimeDefault
        volumeMounts:
        - mountPath: /usr/local/certificates/
          name: webhook-cert
          readOnly: true
      dnsPolicy: ClusterFirst
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: ingress-nginx
      terminationGracePeriodSeconds: 300
      volumes:
      - name: webhook-cert
        secret:
          secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-admission-create
  namespace: ingress-nginx
spec:
  template:
    metadata:
      labels:
        app.kubernetes.io/component: admission-webhook
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.11.1
      name: ingress-nginx-admission-create
    spec:
      containers:
      - args:
        - create
        - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
        - --namespace=$(POD_NAMESPACE)
        - --secret-name=ingress-nginx-admission
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        image: registry.cn-hangzhou.aliyuncs.com/ingress-nginx/kube-webhook-certgen:v1.4.1
        imagePullPolicy: IfNotPresent
        name: create
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          runAsUser: 65532
          seccompProfile:
            type: RuntimeDefault
        nodeSelector:
          kubernetes.io/os: linux
        restartPolicy: OnFailure
        serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-admission-patch
  namespace: ingress-nginx
spec:
  template:
    metadata:
      labels:
        app.kubernetes.io/component: admission-webhook
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.11.1
      name: ingress-nginx-admission-patch
    spec:
      containers:
      - args:
        - patch
        - --webhook-name=ingress-nginx-admission
        - --namespace=$(POD_NAMESPACE)
        - --patch-mutating=false
        - --secret-name=ingress-nginx-admission
        - --patch-failure-policy=Fail
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        image: registry.cn-hangzhou.aliyuncs.com/ingress-nginx/kube-webhook-certgen:v1.4.1
        imagePullPolicy: IfNotPresent
        name: patch
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          runAsUser: 65532
          seccompProfile:
            type: RuntimeDefault
        nodeSelector:
          kubernetes.io/os: linux
        restartPolicy: OnFailure
        serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: nginx
spec:
  controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.11.1
  name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:
  - v1
  clientConfig:
    service:
      name: ingress-nginx-controller-admission
      namespace: ingress-nginx
      path: /networking/v1/ingresses
  failurePolicy: Fail
  matchPolicy: Equivalent
  name: validate.nginx.ingress.kubernetes.io
  rules:
  - apiGroups:
    - networking.k8s.io
    apiVersions:
    - v1
    operations:
    - CREATE
    - UPDATE
    resources:
    - ingresses
  sideEffects: None

• 根据官方 yaml，修改镜像地址：

[root@k8s-master1 ingress-nginx]# grep -n image: ingress-nginx.yaml
448: image: harbor.zx/hcie/controller:v1.11.1
549: image: harbor.zx/hcie/kube-webhook-certgen:v1.4.1
602: image: harbor.zx/hcie/kube-webhook-certgen:v1.4.1 

• 修改 ingress 的端口：

352 ports:
353 - appProtocol: http
354 name: http
355 port: 80
356 protocol: TCP
357 targetPort: http
358 nodePort: 30080
359 - appProtocol: https
360 name: https
361 port: 443
362 protocol: TCP
363 targetPort: https
364 nodePort: 30443

• 修改 ingress 的代理方式：

369 #type: LoadBalancer # 默认是负载均衡，如果是云环境建议使用默认参数
370 type: NodePort 

部署 ingress-nginx：

kubectl apply -f ingress-nginx.yaml 

部署后输出结果：

namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created 

2.3 部署后查看 ingress 状态

部署后，执行以下命令：

kubectl get svc,pod -n ingress-nginx -owide 

查看 Pod 和 SVC 的状态：

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/ingress-nginx-controller NodePort 10.245.187.253 <none>80:30080/TCP,443:30443/TCP 18m app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
service/ingress-nginx-controller-admission ClusterIP 10.245.245.14 <none>443/TCP 18m app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/ingress-nginx-admission-create-h6s42 0/1 Completed 0 18m 172.16.126.2 k8s-worker2 <none><none>
pod/ingress-nginx-admission-patch-79kv5 0/1 Completed 0 18m 172.16.126.3 k8s-worker2 <none><none>
pod/ingress-nginx-controller-746fcdfcdd-6gt4n 1/1 Running 0 18m 172.16.194.85 k8s-worker1 <none><none>

❔ 说明：

• NodePort: Service 的类型是在 ingress-nginx.yaml 中自定义。
• 80:30080/TCP, 443:30443/TCP: 这两个端口也是在 ingress-nginx.yaml 中自定义，后面访问使用这两个端口。
• ingress-nginx-controller 的 Pod 为 ingress-nginx-controller-746fcdfcdd-6gt4n 运行在 k8s-worker1 节点上。后续访问 ingress 代理的服务，都要用 k8s-worker1 节点的 IP 做 DNS 解析。

执行以下命令，查看 IngressClass：

[root@k8s-master1 ingress-nginx]# kubectl get IngressClass
NAME CONTROLLER PARAMETERS AGE
nginx k8s.io/ingress-nginx <none> 25m 

❔ 说明:

• Ingress 类型：nginx 是 ingress-nginx 默认创建的；
• 当创建 Ingress 对象时，需要指定 ingressClass 字段 (.spec.ingressClassName)
• 可以对 ingressClass 添加 ingressclass.kubernetes.io/is-default-class 注解，这样 IngressClass 会被视为默认的 Ingress 类。当某个 IngressClass 资源将此注解设置为 true 时，没有指定类的新 Ingress 资源将被分配到此默认类。

2.4 创建实例测试 Ingress

2.4.1 部署 ingress 对象

最后，我们打算使用 1 个 3 副本的 deployment 组成 nginx-service，然后创建一个 ingress，用于代理 nginx 的服务。yaml 配置如下：

# 部署 3 个 nginx 实例
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-test
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: registry.cn-hangzhou.aliyuncs.com/hcie/nginx:1.26.1-alpine
        ports:
        - containerPort: 80
---
# 部署 nginx 的 Service
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app: nginx-pod
  type: ClusterIP
  ports:
  - port: 80
    targetPort: 80
---
# 创建 ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-http
spec:
  # 上面的 IngressClass 名称
  ingressClassName: "nginx"
  rules:
  # 域名，稍后要通过这个域名访问服务
  - host: nginx.steven.com
    http:
      paths:
      # 路径，访问域名时后面添加的路径
      - path: /
        pathType: Prefix
        backend:
          # 要代理的服务和服务的端口
          service:
            name: nginx-service
            port:
              number: 80

部署 deployment、service、ingress

kubectl apply -f ingress-test.yaml 

执行以下命令，查看部署结果：

[root@k8s-master1 ingress-nginx]# kubectl get deployment,svc,ingress
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx-test 3/3 33 19s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/nginx-service ClusterIP 10.245.127.127 <none>80/TCP 19s
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress.networking.k8s.io/ingress-http nginx nginx.test.com 10.245.187.253 80 19s 

查看下 ingress-http 详细信息：

[root@k8s-master1 ingress-nginx]# kubectl describe ingress ingress-http
Name: ingress-http
Labels: <none>
Namespace: default
Address: 10.245.187.253
Ingress Class: nginx
Default backend: <default>
Rules:
 Host Path Backends
 ---- ---- --------
 nginx.test.com / nginx-service:80 (<none>)
Annotations: <none>
Events:
 Type Reason Age From Message
 ---- ------ ---- ---- -------
 Normal Sync 2m49s (x2 over 2m56s) nginx-ingress-controller Scheduled for sync

查看 ingress-controller 的日志信息：

[root@k8s-master1 ingress-nginx]# kubectl -n ingress-nginx logs -f ingress-nginx-controller-746fcdfcdd-6gt4n
-------------------------------------------------------------------------------
NGINX Ingress controller Release: v1.11.1
Build: 7c44f992012555ff7f4e47c08d7c542ca9b4b1f7
Repository: https://github.com/kubernetes/ingress-nginx
nginx version: nginx/1.25.5
-------------------------------------------------------------------------------
W0826 03:58:30.736251 7 client_config.go:659] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
I0826 03:58:30.736660 7 main.go:205]"Creating API client"host="https://10.245.0.1:443"
I0826 03:58:37.887271 7 main.go:248]"Running in Kubernetes cluster"major="1"minor="29"git="v1.29.7"state="clean"commit="4e4a18878ce330fefda1dc46acca88ba355e9ce7"platform="linux/amd64"
I0826 03:58:38.432286 7 main.go:101]"SSL fake certificate created"file="/etc/ingress-controller/ssl/default-fake-certificate.pem"
I0826 03:58:38.490727 7 ssl.go:535]"loading tls certificate"path="/usr/local/certificates/cert"key="/usr/local/certificates/key"
I0826 03:58:38.516493 7 nginx.go:271]"Starting NGINX Ingress controller"
I0826 03:58:38.538981 7 event.go:377] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress-nginx", Name:"ingress-nginx-controller", UID:"0dba2870-a96b-4b5e-a908-b985139e5d63", APIVersion:"v1", ResourceVersion:"624593", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/ingress-nginx-controller
I0826 03:58:39.719275 7 nginx.go:317]"Starting NGINX process"
I0826 03:58:39.720802 7 leaderelection.go:250] attempting to acquire leader lease ingress-nginx/ingress-nginx-leader...
I0826 03:58:39.723329 7 nginx.go:337]"Starting validation webhook"address=":8443"certPath="/usr/local/certificates/cert"keyPath="/usr/local/certificates/key"
I0826 03:58:39.724636 7 controller.go:193]"Configuration changes detected, backend reload required"
I0826 03:58:39.748482 7 leaderelection.go:260] successfully acquired lease ingress-nginx/ingress-nginx-leader
I0826 03:58:39.748953 7 status.go:85]"New leader elected"identity="ingress-nginx-controller-746fcdfcdd-6gt4n"
I0826 03:58:39.826280 7 controller.go:213]"Backend successfully reloaded"
I0826 03:58:39.826390 7 controller.go:224]"Initial sync, sleeping for 1 second"
I0826 03:58:39.826978 7 event.go:377] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-746fcdfcdd-6gt4n", UID:"68964da5-0da2-4ca7-84f3-742e3d1646ed", APIVersion:"v1", ResourceVersion:"624720", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration 

从最后几条日志来看，我们创建的 ingress 已经与 ingress-nginx 成功关联上了，下面可以进行访问测试了。

2.4.2 访问测试

2.4.2.1 在 HOSTS 文件里添加解析记录：

可使用系统文本编辑器修改 hosts 文件。

添加记录：

192.168.3.44 nginx.steven.com 

访问测试：

2.4.3 pod 负载均衡测试

修改 nginx 的 index.html

[root@k8s-master1 ingress-nginx]# kubectl exec -it nginx-test-6fc95f9f89-dsqkg -- sh
# echo "nginx1"> /usr/share/nginx/html/index.html
# exit
[root@k8s-master1 ingress-nginx]# kubectl exec -it nginx-test-6fc95f9f89-h7khh -- sh
# echo "nginx2"> /usr/share/nginx/html/index.html
# exit
[root@k8s-master1 ingress-nginx]# kubectl exec -it nginx-test-6fc95f9f89-z2v6l -- sh
# echo "nginx3"> /usr/share/nginx/html/index.html
# exit

再次访问测试，默认是轮询方式，负载均衡主要是 service 的作用。

3 DaemonSet 方式部署 Ingress-nginx

DaemonSet 确保 Ingress Controller 的副本在集群中的每个节点上运行。这么做的原因如下：

1. 它确保入口控制器保持服务流量高可用，即使某个节点发生故障。
2. 它允许 Ingress Controller 在集群中的所有节点之间均匀分配流量，从而提高性能。
3. 它允许 Ingress Controller 绑定到主机的网络命名空间，这对于某些功能（例如外部 DNS）是必需的。

3.1 配置 DaemonSet

3.1.1 helm 部署方式

# 切换到 Chart 目录
cd helm/ingress-nginx
vim values.yaml 

• 修改 ingress-nginx-controller 的 kind 字段

222 #kind: Deployment
223 kind: DaemonSet 

更新 helm 配置

helm upgrade ingress-nginx . -n ingress-nginx 

查看部署结果

[root@k8s-master1 ingress-nginx]# kubectl get pod,svc -owide -n ingress-nginx
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/ingress-nginx-controller-2bljr 1/1 Running 0 61s 10.10.0.1 k8s-master1 <none><none>
pod/ingress-nginx-controller-2s59q 1/1 Running 0 3m1s 10.10.0.4 k8s-worker1 <none><none>
pod/ingress-nginx-controller-dppwt 1/1 Running 0 61s 10.10.0.5 k8s-worker2 <none><none>
pod/ingress-nginx-controller-gzshs 1/1 Running 0 61s 10.10.0.3 k8s-master3 <none><none>
pod/ingress-nginx-controller-hbj9d 0/1 Running 0 61s 10.10.0.2 k8s-master2 <none><none>
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
service/ingress-nginx-controller NodePort 10.245.133.125 <none>80:30080/TCP,443:30443/TCP 3m2s app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
service/ingress-nginx-controller-admission ClusterIP 10.245.236.128 <none>443/TCP 3m2s app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx 

• 看到每个节点都部署了一个 ingress-controller 的 pod

3.1.2 yaml 文件部署方式

修改这行即可

396 #kind: Deployment
397 kind: DaemonSet 

• 验证方法如 4.2 节，这里就不重复叙述了。

4 结论

Ingress-Nginx 是 Kubernetes 中强大而灵活的流量管理工具，能够有效地路由外部请求到内部服务。通过本文的讲解与实践示例，您应该能够快速上手并在自己的集群中实现 Ingress-Nginx 的配置与管理。继续探索 Ingress-Nginx 的更多高级功能，将使您的应用架构更加高效和安全。

5 参考

【1】Kubernetes 官方文档-Ingress

【2】Kubernetes 官方文档-IngressController

【3】github-ingress-nginx

【4】what-to-deploy-nginx-ingress-controller-as-replicas-set-or-daemon-set