Python AI 大模型部署实战：本地运行、API 服务与 Docker 封装

3.3 使用 llama.cpp 进行 CPU/GPU 推理

对于资源受限或需要纯 CPU 的场景，llama-cpp-python 是不错的选择。记得开启 CUDA 支持编译。

# 安装 llama-cpp-python（带 CUDA 支持）
CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python

# 下载 GGUF 格式模型
huggingface-cli download \
  Qwen/Qwen2.5-7B-Instruct-GGUF \
  qwen2.5-7b-instruct-q4_k_m.gguf \
  --localdir ./models

from llama_cpp import Llama

llm = Llama(
    model_path="./models/qwen2.5-7b-instruct-q4_k_m.gguf",
    n_ctx=4096,
    n_gpu_layers=-1,  # 全部卸载到 GPU
    verbose=False,
)
response = llm.create_chat_completion(
    messages=[{"role": "user", "content": "解释 Transformer 的自注意力机制"}],
    temperature=0.7,
    max_tokens=1024,
)
print(response["choices"][0]["message"]["content"])

四、方案二：API 服务化

4.1 架构流程

客户端请求经过 Nginx 反向代理进入 FastAPI 服务，底层由 vLLM 推理引擎处理，GPU 负责计算，Redis 用于队列管理。

4.2 使用 vLLM 启动高性能推理服务

vLLM 原生支持 OpenAI 兼容接口，适合快速接入现有生态。

# 直接以 OpenAI 兼容模式启动
python -m vllm.entrypoints.openai.api_server \
--model Qwen/Qwen2.5-72B-Instruct-GPTQ-Int4 \
 --served-model-name qwen-72b \
--host 0.0.0.0 \
--port 8000 \
 --max-model-len 4096 \
 --gpu-memory-utilization 0.90 \
 --tensor-parallel-size 2

客户端调用示例：

from openai import OpenAI
client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="not-needed",  # 本地部署无需密钥
)
response = client.chat.completions.create(
    model="qwen-72b",
    messages=[
        {"role": "system", "content": "你是一位资深 Python 工程师。"},
        {"role": "user", "content": "如何优化 asyncio 的并发性能？"},
    ],
    temperature=0.7,
    max_tokens=2048,
)
print(response.choices[0].message.content)

4.3 使用 FastAPI 自建 API 服务

如果需要更细粒度的控制，可以用 FastAPI 封装。注意生命周期管理，模型应在应用启动时加载，关闭时释放。

# api_server.py
import uuid
import time
from contextlib import asynccontextmanager
import torch
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field
from transformers import AutoModelForCausalLM, AutoTokenizer

# ---------- 全局模型 ----------
model = None
tokenizer = None

@asynccontextmanager
async def lifespan(app: FastAPI):
    """应用生命周期：启动时加载模型，关闭时释放资源。"""
    global model, tokenizer
    model_id = "Qwen/Qwen2.5-14B-Instruct-GPTQ-Int4"
    tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
    model = AutoModelForCausalLM.from_pretrained(
        model_id, device_map="auto", torch_dtype=torch.float16, trust_remote_code=True
    )
    yield
    del model, tokenizer
    torch.cuda.empty_cache()

app = FastAPI(title="LLM API Service", lifespan=lifespan)

# ---------- 请求/响应模型 ----------
class ChatRequest(BaseModel):
    prompt: str = Field(..., min_length=1, max_length=8192)
    max_tokens: int = Field(default=1024, ge=1, le=4096)
    temperature: float = Field(default=0.7, ge=0.0, le=2.0)

class ChatResponse(BaseModel):
    id: str
    response: str
    usage_tokens: int
    latency_ms: float

# ---------- 推理接口 ----------
@app.post("/v1/chat", response_model=ChatResponse)
async def chat_completion(req: ChatRequest):
    if model is None:
        raise HTTPException(status_code=503, detail="模型尚未加载完成")
    start = time.perf_counter()
    input_ids = tokenizer.apply_chat_template(
        [{"role": "user", "content": req.prompt}], return_tensors="pt",
    ).to(model.device)
    with torch.no_grad():
        outputs = model.generate(
            input_ids, max_new_tokens=req.max_tokens, temperature=req.temperature, top_p=0.9, do_sample=True,
        )
    generated = outputs[0][input_ids.shape[-1]:]
    text = tokenizer.decode(generated, skip_special_tokens=True)
    latency = (time.perf_counter() - start) * 1000
    return ChatResponse(
        id=str(uuid.uuid4()), response=text, usage_tokens=len(generated), latency_ms=round(latency, 2)
    )

# ---------- 健康检查 ----------
@app.get("/health")
async def health():
    return {
        "status": "ok",
        "model_loaded": model is not None,
        "gpu_available": torch.cuda.is_available(),
    }

启动服务：

uvicorn api_server:app --host 0.0.0.0 --port 8000 --workers 1

五、方案三：Docker 容器封装

5.1 Dockerfile

采用多阶段构建可以显著减小镜像体积。构建阶段安装依赖，运行阶段仅保留必要文件。

# ---------- 构建阶段 ----------
FROM nvidia/cuda:12.4.1-devel-ubuntu22.04 AS builder
ENV DEBIAN_FRONTEND=noninteractive \
    PYTHONUNBUFFERED=1
RUN apt-get update && apt-get install -y --no-install-recommends \
    python3.11 python3.11-venv python3-pip \
    && rm -rf /var/lib/apt/lists/*

RUN python3.11 -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
COPY requirements.txt /tmp/requirements.txt
RUN pip install --no-cache-dir -r /tmp/requirements.txt

# ---------- 运行阶段 ----------
FROM nvidia/cuda:12.4.1-runtime-ubuntu22.04
RUN apt-get update && apt-get install -y --no-install-recommends \
    python3.11 \
    && rm -rf /var/lib/apt/lists/*

COPY --from=builder /opt/venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
WORKDIR /app
COPY api_server.py .
EXPOSE 8000
CMD ["uvicorn", "api_server:app", "--host", "0.0.0.0", "--port", "8000"]

5.2 docker-compose.yml

配置环境变量和 GPU 资源限制，确保服务稳定运行。

version: "3.9"
services:
  llm-api:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: llm-api-server
    ports:
      - "8000:8000"
    volumes:
      - ~/.cache/huggingface:/root/.cache/huggingface  # 模型缓存持久化
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - MODEL_ID=Qwen/Qwen2.5-14B-Instruct-GPTQ-Int4
      - MAX_MODEL_LEN=4096
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
  redis:
    image: redis:7-alpine
    container_name: llm-redis
    ports:
      - "6379:6379"
    restart: unless-stopped

5.3 构建与运行

# 构建镜像
docker-compose build
# 启动服务（后台运行）
docker-compose up -d
# 查看日志
docker-compose logs -f llm-api
# 测试接口
curl -X POST http://localhost:8000/v1/chat \
-H"Content-Type: application/json" \
-d'{"prompt": "解释 Docker 的多阶段构建", "max_tokens": 512}'

六、性能调优要点

显存占用通常分布在模型权重、KV Cache 和激活值上。合理设置参数能避免 OOM。

关键调优参数

# 性能基准测试脚本
import time
import statistics
import requests

API_URL = "http://localhost:8000/v1/chat"
PROMPT = "请用 200 字介绍 Python 的 GIL 机制。"
NUM_REQUESTS = 50
latencies = []

for i in range(NUM_REQUESTS):
    start = time.perf_counter()
    resp = requests.post(API_URL, json={"prompt": PROMPT, "max_tokens": 256})
    latencies.append((time.perf_counter() - start) * 1000)

print(f"请求次数：{NUM_REQUESTS}")
print(f"平均延迟：{statistics.mean(latencies):.1f} ms")
print(f"P50 延迟：{statistics.median(latencies):.1f} ms")
print(f"P95 延迟：{sorted(latencies)[int(len(latencies)*0.95)]:.1f} ms")
print(f"吞吐量：{NUM_REQUESTS /(sum(latencies)/1000):.1f} req/s")

七、生产部署检查清单

部署前务必确认 GPU 驱动与 CUDA 版本匹配，模型权重完整性校验通过，显存容量满足需求。

八、总结

本文覆盖了 Python AI 大模型部署的三大核心路径：

1. 本地运行 — 适合开发调试，使用 transformers 或 llama.cpp 快速启动
2. API 服务化 — 使用 vLLM 或 FastAPI 提供 OpenAI 兼容接口，支持高并发推理
3. Docker 封装 — 标准化交付，配合 docker-compose 实现一键部署

生产环境中建议以 vLLM + Docker + Nginx + Prometheus 为基础技术栈，并根据实际 QPS 和模型规模水平扩展 GPU 节点。

参考资源： vLLM 官方文档：https://docs.vllm.ai llama.cpp 仓库：https://github.com/ggerganov/llama.cpp Hugging Face Transformers：https://huggingface.co/docs/transformers