Python AI 大模型部署实战：本地运行、API 服务与 Docker 封装

3. 使用 llama.cpp 进行 CPU/GPU 推理

如果显存紧张或需要跨平台支持，llama.cpp 是不错的选择。记得开启 CUDA 编译选项。

# 安装 llama-cpp-python（带 CUDA 支持）
CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python

# 下载 GGUF 格式模型
huggingface-cli download \
  Qwen/Qwen2.5-7B-Instruct-GGUF \
  qwen2.5-7b-instruct-q4_k_m.gguf \
  --localdir ./models

from llama_cpp import Llama

llm = Llama(
    model_path="./models/qwen2.5-7b-instruct-q4_k_m.gguf",
    n_ctx=4096,
    n_gpu_layers=-1,  # 全部卸载到 GPU
    verbose=False,
)
response = llm.create_chat_completion(
    messages=[{"role": "user", "content": "解释 Transformer 的自注意力机制"}],
    temperature=0.7,
    max_tokens=1024,
)
print(response["choices"][0]["message"]["content"])

三、API 服务化

1. 架构流程

客户端请求经过 Nginx 反向代理到达 FastAPI 服务，底层由 vLLM 推理引擎处理，GPU 负责计算，Redis 用于队列管理。

2. 使用 vLLM 启动高性能推理服务

vLLM 原生支持 OpenAI 兼容接口，启动时需注意显存利用率设置，避免 OOM。

# 直接以 OpenAI 兼容模式启动
python -m vllm.entrypoints.openai.api_server \
--model Qwen/Qwen2.5-72B-Instruct-GPTQ-Int4 \
 --served-model-name qwen-72b \
--host 0.0.0.0 \
--port 8000 \
 --max-model-len 4096 \
 --gpu-memory-utilization 0.90 \
 --tensor-parallel-size 2

客户端调用示例：

from openai import OpenAI
client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="not-needed",  # 本地部署无需密钥
)
response = client.chat.completions.create(
    model="qwen-72b",
    messages=[{"role": "system", "content": "你是一位资深 Python 工程师。"}, {"role": "user", "content": "如何优化 asyncio 的并发性能？"}],
    temperature=0.7,
    max_tokens=2048,
)
print(response.choices[0].message.content)

3. 使用 FastAPI 自建 API 服务

如果需要更细粒度的控制，可以用 FastAPI 包装推理逻辑。注意模型加载放在生命周期钩子中，避免每次请求都重新加载。

# api_server.py
import uuid
import time
from contextlib import asynccontextmanager
import torch
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field
from transformers import AutoModelForCausalLM, AutoTokenizer

# ---------- 全局模型 ----------
model = None
tokenizer = None

@asynccontextmanager
async def lifespan(app: FastAPI):
    """应用生命周期：启动时加载模型，关闭时释放资源。"""
    global model, tokenizer
    model_id = "Qwen/Qwen2.5-14B-Instruct-GPTQ-Int4"
    tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
    model = AutoModelForCausalLM.from_pretrained(
        model_id, device_map="auto", torch_dtype=torch.float16, trust_remote_code=True,
    )
    yield
    del model, tokenizer
    torch.cuda.empty_cache()

app = FastAPI(title="LLM API Service", lifespan=lifespan)

# ---------- 请求/响应模型 ----------
class ChatRequest(BaseModel):
    prompt: str = Field(..., min_length=1, max_length=8192)
    max_tokens: int = Field(default=1024, ge=1, le=4096)
    temperature: float = Field(default=0.7, ge=0.0, le=2.0)

class ChatResponse(BaseModel):
    id: str
    response: str
    usage_tokens: int
    latency_ms: float

# ---------- 推理接口 ----------
@app.post("/v1/chat", response_model=ChatResponse)
async def chat_completion(req: ChatRequest):
    if model is None:
        raise HTTPException(status_code=503, detail="模型尚未加载完成")
    start = time.perf_counter()
    input_ids = tokenizer.apply_chat_template([{"role": "user", "content": req.prompt}], return_tensors="pt",).to(model.device)
    with torch.no_grad():
        outputs = model.generate(input_ids, max_new_tokens=req.max_tokens, temperature=req.temperature, top_p=0.9, do_sample=True)
    generated = outputs[0][input_ids.shape[-1]:]
    text = tokenizer.decode(generated, skip_special_tokens=True)
    latency = (time.perf_counter() - start) * 1000
    return ChatResponse(id=str(uuid.uuid4()), response=text, usage_tokens=len(generated), latency_ms=round(latency, 2))

# ---------- 健康检查 ----------
@app.get("/health")
async def health():
    return {"status": "ok", "model_loaded": model is not None, "gpu_available": torch.cuda.is_available()}

启动服务：

uvicorn api_server:app --host 0.0.0.0 --port 8000 --workers 1

四、Docker 容器封装

1. Dockerfile

采用多阶段构建减少镜像体积，确保生产环境只包含运行所需文件。

# ---------- 构建阶段 ----------
FROM nvidia/cuda:12.4.1-devel-ubuntu22.04 AS builder
ENV DEBIAN_FRONTEND=noninteractive \
    PYTHONUNBUFFERED=1
RUN apt-get update && apt-get install -y --no-install-recommends \
    python3.11 python3.11-venv python3-pip \
    && rm -rf /var/lib/apt/lists/*
RUN python3.11 -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
COPY requirements.txt /tmp/requirements.txt
RUN pip install --no-cache-dir -r /tmp/requirements.txt

# ---------- 运行阶段 ----------
FROM nvidia/cuda:12.4.1-runtime-ubuntu22.04
RUN apt-get update && apt-get install -y --no-install-recommends \
    python3.11 \
    && rm -rf /var/lib/apt/lists/*
COPY --from=builder /opt/venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
WORKDIR /app
COPY api_server.py .
EXPOSE 8000
CMD ["uvicorn", "api_server:app", "--host", "0.0.0.0", "--port", "8000"]

2. docker-compose.yml

配置 GPU 设备映射与缓存持久化，方便调试与重启。

version: "3.9"
services:
  llm-api:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: llm-api-server
    ports:
      - "8000:8000"
    volumes:
      - ~/.cache/huggingface:/root/.cache/huggingface  # 模型缓存持久化
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - MODEL_ID=Qwen/Qwen2.5-14B-Instruct-GPTQ-Int4
      - MAX_MODEL_LEN=4096
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
  redis:
    image: redis:7-alpine
    container_name: llm-redis
    ports:
      - "6379:6379"
    restart: unless-stopped

3. 构建与运行

# 构建镜像
docker-compose build
# 启动服务（后台运行）
docker-compose up -d
# 查看日志
docker-compose logs -f llm-api
# 测试接口
curl -X POST http://localhost:8000/v1/chat \
-H "Content-Type: application/json" \
-d '{"prompt": "解释 Docker 的多阶段构建", "max_tokens": 512}'

五、性能调优要点

生产环境中，显存管理至关重要。典型 72B Int4 模型下，权重、KV Cache 与激活值占比需平衡。

关键调优参数如下：

可编写基准测试脚本监控 P95 延迟与吞吐量：

import time
import statistics
import requests

API_URL = "http://localhost:8000/v1/chat"
PROMPT = "请用 200 字介绍 Python 的 GIL 机制。"
NUM_REQUESTS = 50
latencies = []

for i in range(NUM_REQUESTS):
    start = time.perf_counter()
    resp = requests.post(API_URL, json={"prompt": PROMPT, "max_tokens": 256})
    latencies.append((time.perf_counter() - start) * 1000)

print(f"请求次数：{NUM_REQUESTS}")
print(f"平均延迟：{statistics.mean(latencies):.1f} ms")
print(f"P50 延迟：{statistics.median(latencies):.1f} ms")
print(f"P95 延迟：{sorted(latencies)[int(len(latencies)*0.95)]:.1f} ms")
print(f"吞吐量：{NUM_REQUESTS /(sum(latencies)/1000):.1f} req/s")

六、生产部署检查清单

上线前务必核对以下项，避免运行时故障。

七、总结

本文覆盖了 Python AI 大模型部署的三大核心路径：

1. 本地运行 — 适合开发调试，使用 transformers 或 llama.cpp 快速启动
2. API 服务化 — 使用 vLLM 或 FastAPI 提供 OpenAI 兼容接口，支持高并发推理
3. Docker 封装 — 标准化交付，配合 docker-compose 实现一键部署

生产环境中建议以 vLLM + Docker + Nginx + Prometheus 为基础技术栈，并根据实际 QPS 和模型规模水平扩展 GPU 节点。

参考资源： vLLM 官方文档：https://docs.vllm.ai llama.cpp 仓库：https://github.com/ggerganov/llama.cpp Hugging Face Transformers：https://huggingface.co/docs/transformers