Docker+K8s 微服务部署与运维实战指南

# 安装依赖
yum install -y yum-utils device-mapper-persistent-data lvm2
# 配置 Docker 镜像源
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
# 安装指定版本 Docker
yum install -y docker-ce-20.10.24 docker-ce-cli-20.10.24 containerd.io
# 启动 Docker 并设置开机自启
systemctl start docker
systemctl enable docker
# 配置 Docker 镜像加速
cat > /etc/docker/daemon.json << EOF
{
  "registry-mirrors": ["https://xxxx.mirror.aliyuncs.com"],
  "exec-opts": ["native.cgroupdriver=systemd"]
}
EOF
systemctl daemon-reload
systemctl restart docker

避坑提醒：Docker 的 cgroup 驱动必须设置为 systemd，否则 K8s 初始化会报错。

三、Docker 实战：微服务容器化

3.1 编写 Dockerfile

以一个标准的 Spring Boot 微服务为例，Dockerfile 放在项目根目录：

# 第一阶段：构建应用
FROM maven:3.8.8-openjdk-11 AS builder
WORKDIR /app
COPY pom.xml .
COPY src ./src
RUN mvn clean package -DskipTests

# 第二阶段：运行应用
FROM openjdk:11-jre-slim
WORKDIR /app
COPY --from=builder /app/target/*.jar app.jar
ENV JAVA_OPTS="-Xms512m -Xmx512m -XX:+UseG1GC -XX:MaxGCPauseMillis=200"
EXPOSE 8080
ENTRYPOINT ["sh", "-c", "java $JAVA_OPTS -jar app.jar"]

关键优化点：分层构建减小镜像体积；JVM 参数优化避免 OOM；入口命令使用 exec 格式。

3.2 构建 Docker 镜像

docker build -t harbor.example.com/microservice/user-service:1.0.0 .

3.3 搭建 Harbor 镜像仓库

1. 安装 Docker Compose

curl -L "https://github.com/docker/compose/releases/download/v2.20.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
docker-compose --version

2. 部署 Harbor

wget https://github.com/goharbor/harbor/releases/download/v2.5.3/harbor-offline-installer-v2.5.3.tgz
tar -zxvf harbor-offline-installer-v2.5.3.tgz -C /usr/local/
cd /usr/local/harbor
cp harbor.yml.tmpl harbor.yml
vi harbor.yml

关键配置修改：

hostname: harbor.example.com
http:
  port: 80
harbor_admin_password: Harbor12345
data_volume: /data/harbor

3. 启动 Harbor

./install.sh
docker-compose -f /usr/local/harbor/docker-compose.yml up -d

4. 推送镜像到 Harbor

docker login harbor.example.com
docker push harbor.example.com/microservice/user-service:1.0.0
docker logout harbor.example.com

避坑提醒：如果 Harbor 用 HTTP 协议，需要在所有 K8s 节点的 Docker 配置中添加 "insecure-registries": ["harbor.example.com"]。

四、K8s 集群搭建（kubeadm 实战版）

4.1 安装 K8s 组件

cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-$basearch/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
exclude=kubelet kubeadm kubectl
EOF
yum install -y kubelet-1.24.10 kubeadm-1.24.10 kubectl-1.24.10 --disableexcludes=kubernetes
systemctl start kubelet
systemctl enable kubelet

4.2 初始化 Master 节点

kubeadm init \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.24.10 \
--pod-network-cidr=10.244.0.0/16 \
--service-cidr=10.96.0.0/12
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl get nodes

4.3 安装网络插件（Calico）

kubectl apply -f https://docs.projectcalico.org/v3.24/manifests/calico.yaml
kubectl get pods -n kube-system

4.4 Worker 节点加入集群

在所有 Worker 节点执行 Master 初始化时输出的 kubeadm join 命令。

4.5 验证集群状态

kubectl get nodes
kubectl get pods -n kube-system

五、微服务部署到 K8s（实战 yaml 配置）

5.1 编写 Deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: user-service
  namespace: microservice
spec:
  replicas: 2
  selector:
    matchLabels:
      app: user-service
  template:
    metadata:
      labels:
        app: user-service
    spec:
      containers:
      - name: user-service
        image: harbor.example.com/microservice/user-service:1.0.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8080
        resources:
          requests:
            cpu: "500m"
            memory: "512Mi"
          limits:
            cpu: "1000m"
            memory: "1024Mi"
        livenessProbe:
          httpGet:
            path: /actuator/health/liveness
            port: 8080
          initialDelaySeconds: 60
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /actuator/health/readiness
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 5
        env:
        - name: SPRING_PROFILES_ACTIVE
          value: "prod"
        - name: DB_HOST
          value: "mysql-service"

5.2 编写 Service.yaml

apiVersion: v1
kind: Service
metadata:
  name: user-service
  namespace: microservice
spec:
  selector:
    app: user-service
  ports:
  - port: 80
    targetPort: 8080
  type: ClusterIP

5.3 编写 Ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: microservice-ingress
  namespace: microservice
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: user.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: user-service
            port:
              number: 80

5.4 执行部署命令

kubectl create namespace microservice
kubectl apply -f Deployment.yaml
kubectl apply -f Service.yaml
kubectl apply -f Ingress.yaml
kubectl get pods -n microservice
kubectl get svc -n microservice
kubectl get ingress -n microservice

5.5 验证访问

curl http://user-service.microservice.svc.cluster.local:80/health
curl http://user.example.com/health

六、运维实战：监控、日志、更新

6.1 监控：Prometheus+Grafana

1. 安装 Prometheus

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install prometheus prometheus-community/prometheus -n monitoring --create-namespace

2. 安装 Grafana

helm repo add grafana https://grafana.github.io/helm-charts
helm install grafana grafana/grafana -n monitoring

3. 配置 Spring Boot 微服务监控

在 pom.xml 中添加依赖：

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
  <groupId>io.micrometer</groupId>
  <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>

在 application-prod.yml 中配置：

management:
  endpoints:
    web:
      exposure:
        include: health,info,prometheus
  metrics:
    tags:
      application: user-service
    endpoint:
      health:
        probes:
          enabled: true
        show-details: always

4. 配置 Prometheus 抓取微服务指标

scrape_configs:
- job_name: 'microservice'
  kubernetes_sd_configs:
  - role: pod
  relabel_configs:
  - source_labels: [__meta_kubernetes_pod_label_app]
    action: keep
    regex: .*service
  - source_labels: [__meta_kubernetes_pod_container_port_number]
    action: keep
    regex: 8080

5. 导入 Grafana 面板

kubectl get secret grafana -n monitoring -o jsonpath="{.data.admin-password}" | base64 --decode

6.2 日志收集：ELK Stack

1. 安装 Elasticsearch

helm repo add elastic https://helm.elastic.co
helm install elasticsearch elastic/elasticsearch -n logging --create-namespace

2. 安装 Logstash

helm install logstash elastic/logstash -n logging

3. 安装 Kibana

helm install kibana elastic/kibana -n logging

4. 配置微服务日志输出到文件

在 logback-spring.xml 中配置：

<appender name="FILE">
  <file>/var/log/user-service.log</file>
  <rollingPolicy>
    <fileNamePattern>/var/log/user-service.%d{yyyy-MM-dd}.log</fileNamePattern>
    <maxHistory>7</maxHistory>
  </rollingPolicy>
  <encoder>
    <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
  </encoder>
</appender>
<root level="INFO">
  <appender-ref ref="FILE" />
</root>

5. 配置 Filebeat 收集日志

filebeat.inputs:
- type: filestream
  paths:
  - /var/log/*.log
processors:
- add_kubernetes_metadata:
    host: ${NODE_NAME}
output.elasticsearch:
  hosts: ["elasticsearch-master:9200"]

6.3 滚动更新与回滚

1. 滚动更新

kubectl set image deployment/user-service user-service=harbor.example.com/microservice/user-service:1.0.1 -n microservice
kubectl rollout status deployment/user-service -n microservice

2. 回滚

kubectl rollout history deployment/user-service -n microservice
kubectl rollout undo deployment/user-service -n microservice
kubectl rollout undo deployment/user-service --to-revision=1 -n microservice

七、八年开发避坑指南

1. Docker 镜像不要用 latest 标签：生产环境一定要指定具体版本。
2. K8s 资源配置要合理：JVM 的 -Xmx 不能超过 Pod 的 memory limits。
3. 探针配置不能少：livenessProbe 和 readinessProbe 一定要配置。
4. 镜像仓库一定要搭建：私有镜像用 Harbor，安全性更高。
5. 网络插件选择 Calico：生产环境更推荐。
6. 监控和日志必须到位：出现问题无法快速定位。
7. 避免单节点故障：Master 节点建议搭建高可用，Worker 节点至少 2 个。
8. 不要直接操作 Pod：要通过 Deployment 管理。

八、总结与展望

Docker+K8s 已经成为微服务部署的标准方案。后续可以进一步优化的方向包括集成 CI/CD、引入 Istio 服务网格、以及使用 PV/PVC 机制处理持久化存储。