Kubernetes 边缘 AI 部署实战与最佳实践

# 在边缘节点上执行（替换实际参数）
kubeadm join <主节点 IP>:6443 --token <token> --discovery-token-ca-cert-hash <hash>

3. 边缘 AI 应用部署

3.1 模型准备与存储

模型文件不能随便丢在容器里，最好用 PVC 挂载。这样扩容或迁移时，模型数据不会丢。

# 下载并优化模型
mkdir -p models/yolo/1
wget -O models/yolo/1/model.onnx https://github.com/onnx/models/raw/main/vision/object_detection_segmentation/yolov4/model/yolov4.onnx

# 创建持久化存储卷
kubectl create -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: model-pvc
  namespace: default
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
EOF

3.2 部署服务

Deployment 配置里要注意资源限制，边缘设备内存小，别给太猛。另外加上 nodeSelector，确保 Pod 调度到有标签的边缘节点上。

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: edge-ai-service
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: edge-ai-service
  template:
    metadata:
      labels:
        app: edge-ai-service
    spec:
      nodeSelector:
        node-role.kubernetes.io/edge: "true"
      containers:
      - name: edge-ai-service
        image: edge-ai-service:latest
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: 1
            memory: 1Gi
          requests:
            cpu: 500m
            memory: 512Mi
        volumeMounts:
        - name: model-volume
          mountPath: /models
      volumes:
      - name: model-volume
        persistentVolumeClaim:
          claimName: model-pvc

service.yaml

apiVersion: v1
kind: Service
metadata:
  name: edge-ai-service
  namespace: default
spec:
  selector:
    app: edge-ai-service
  ports:
  - port: 8080
    targetPort: 8080
  type: NodePort

部署完记得测试一下，拿到 NodePort 和节点 IP 就能调用了。

# 部署服务
kubectl apply -f deployment.yaml
kubectl apply -f service.yaml

# 获取端口和 IP
NODE_PORT=$(kubectl get svc edge-ai-service -o jsonpath='{.spec.ports[0].nodePort}')
EDGE_NODE_IP=$(kubectl get nodes -l node-role.kubernetes.io/edge=true -o jsonpath='{.items[0].status.addresses[0].address}')

# 发送请求测试
curl -X POST http://$EDGE_NODE_IP:$NODE_PORT/predict \
  -H "Content-Type: application/json" \
  -d '{"image": "base64_encoded_image"}'

4. 边缘节点管理

4.1 节点调度控制

通过标签和污点，我们可以精准控制哪些应用跑在哪些节点上。比如给边缘节点打上 edge=true 标签，然后设置污点防止普通任务挤占资源。

# 添加标签
kubectl label nodes <edge-node> node-role.kubernetes.io/edge=true

# 添加污点（NoSchedule 表示不允许非容忍的 Pod 调度）
kubectl taint nodes <edge-node> node-role.kubernetes.io/edge:NoSchedule

# 应用添加容忍度
kubectl patch deployment edge-ai-service -p '{"spec":{"template":{"spec":{"tolerations":[{"key":"node-role.kubernetes.io/edge","operator":"Exists","effect":"NoSchedule"}]}}}}'

4.2 资源配额

为了防止某个 Namespace 把节点资源吃光，可以设置 ResourceQuota。

apiVersion: v1
kind: ResourceQuota
metadata:
  name: edge-node-quota
  namespace: default
spec:
  hard:
    requests.cpu: "2"
    requests.memory: "4Gi"
    limits.cpu: "4"
    limits.memory: "8Gi"
    pods: "10"

5. 网络配置

5.1 边缘网络优化

CNI 插件选 Calico 比较稳妥，性能也不错。配合 NetworkPolicy 可以做细粒度的流量控制，只允许特定 Pod 通信。

# 安装 Calico CNI 插件
kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml

# 配置网络策略
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: edge-ai-network-policy
  namespace: default
spec:
  podSelector:
    matchLabels:
      app: edge-ai-service
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: edge-gateway
    ports:
    - protocol: TCP
      port: 8080
  egress:
  - to:
    - podSelector:
        matchLabels:
          app: edge-storage
    ports:
    - protocol: TCP
      port: 9000

5.2 边缘与云端通信

网关层可以用 Nginx 做反向代理，方便统一入口和限流。

configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: edge-gateway-config
  namespace: default
data:
  nginx.conf: |
    events {}
    http {
      server {
        listen 80;
        location / {
          proxy_pass http://edge-ai-service:8080;
          proxy_set_header Host $host;
          proxy_set_header X-Real-IP $remote_addr;
        }
      }
    }

6. 存储配置

6.1 本地存储管理

边缘节点通常用本地磁盘，配置 Local PV 可以避免跨网络读写，提升 IO 性能。

apiVersion: v1
kind: PersistentVolume
metadata:
  name: edge-local-storage
  namespace: default
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  local:
    path: /mnt/edge-storage
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: node-role.kubernetes.io/edge
          operator: In
          values:
          - "true"

7. 监控与可观测性

7.1 节点监控

Prometheus Operator 是标配，配合 ServiceMonitor 自动发现指标。边缘节点资源紧张，采样间隔别设太短。

# 安装 Prometheus Operator
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install prometheus prometheus-community/kube-prometheus-stack -n monitoring --create-namespace

# 配置边缘节点监控
kubectl apply -f - <<EOF
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: edge-ai-service-monitor
  namespace: monitoring
spec:
  selector:
    matchLabels:
      app: edge-ai-service
  endpoints:
  - port: 8080
    path: /metrics
    interval: 15s
EOF

7.2 日志管理

Fluentd 负责收集容器日志，推送到后端存储。注意挂载 /var/log 和 Docker socket 目录。

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd
  namespace: kube-system
  labels:
    k8s-app: fluentd-logging
spec:
  selector:
    matchLabels:
      k8s-app: fluentd-logging
  template:
    metadata:
      labels:
        k8s-app: fluentd-logging
    spec:
      containers:
      - name: fluentd
        image: fluent/fluentd-kubernetes-daemonset:v1.14.6
        env:
        - name: FLUENTD_ARGS
          value: --no-supervisor -q
        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers

8. 安全最佳实践

8.1 边缘节点安全

1. 最小权限原则：不要给所有权限，按需分配。
2. 网络隔离：用 NetworkPolicy 锁死不必要的流量。
3. 加密通信：TLS 必须开，尤其是边缘到云端的链路。
4. 定期更新：固件和软件漏洞要及时修补。

RBAC 配置示例

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: edge-ai-role
  namespace: default
rules:
- apiGroups: [""]
  resources: ["pods", "services"]
  verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: edge-ai-rolebinding
  namespace: default
subjects:
- kind: ServiceAccount
  name: edge-ai-service-account
  namespace: default
roleRef:
  kind: Role
  name: edge-ai-role
  apiGroup: rbac.authorization.k8s.io

8.2 模型安全

模型文件本身也是资产，建议加密存储，限制访问权限，并做好版本审计。

9. 实际应用场景

9.1 智能视频分析

典型的场景就是摄像头接入，本地推理后只回传报警信息。

apiVersion: apps/v1
kind: Deployment
metadata:
  name: video-analytics
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: video-analytics
  template:
    metadata:
      labels:
        app: video-analytics
    spec:
      nodeSelector:
        node-role.kubernetes.io/edge: "true"
      containers:
      - name: video-analytics
        image: video-analytics:latest
        ports:
        - containerPort: 8080
        env:
        - name: MODEL_PATH
          value: /models/yolo
        - name: CAMERA_URL
          value: rtsp://camera:554/stream
        volumeMounts:
        - name: model-volume
          mountPath: /models
      volumes:
      - name: model-volume
        persistentVolumeClaim:
          claimName: model-pvc

9.2 智能传感器数据处理

工业场景下，传感器数据量大，边缘预处理能极大减轻云端压力。

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sensor-processing
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sensor-processing
  template:
    metadata:
      labels:
        app: sensor-processing
    spec:
      nodeSelector:
        node-role.kubernetes.io/edge: "true"
      containers:
      - name: sensor-processing
        image: sensor-processing:latest
        ports:
        - containerPort: 8080
        env:
        - name: SENSOR_ENDPOINT
          value: http://sensor:8000
        - name: MODEL_PATH
          value: /models/anomaly
        volumeMounts:
        - name: model-volume
          mountPath: /models
      volumes:
      - name: model-volume
        persistentVolumeClaim:
          claimName: model-pvc

10. 故障排查

10.1 常见问题解决

遇到 Pod 起不来或者网络不通，先查状态，再查日志。

# 查看节点状态
kubectl get nodes

# 查看应用 Pod
kubectl get pods -l app=edge-ai-service

# 查看应用日志
kubectl logs -l app=edge-ai-service

# 检查资源使用情况
kubectl top node <edge-node>

# 检查网络连接
kubectl exec -it <pod-name> -- ping <target-host>

10.2 调试技巧

1. 开启详细日志：应用层多打 log，定位问题快。
2. 使用 debug 容器：kubectl debug 能帮你进容器修 bug。
3. 核对资源限制：OOMKilled 多半是内存给少了。
4. 验证网络连通性：DNS 解析和防火墙规则最容易出问题。

11. 总结

Kubernetes 为边缘 AI 提供了标准化的编排能力。从节点配置、网络优化到安全加固，每一步都直接影响最终系统的稳定性。重点在于合理分配资源、确保通信安全以及建立完善的监控体系。只要按这些最佳实践走，构建高性能的边缘计算系统就没那么难了。