Python AI 大模型部署实战：本地运行、API 服务与 Docker 封装

3.3 使用 llama.cpp 进行 CPU/GPU 推理

如果资源受限或需要跨平台支持，llama.cpp 是不错的选择。注意编译时需开启 CUDA 支持。

# 安装带 CUDA 支持的 llama-cpp-python
CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python

# 下载 GGUF 格式模型
huggingface-cli download \
  Qwen/Qwen2.5-7B-Instruct-GGUF \
  qwen2.5-7b-instruct-q4_k_m.gguf \
  --localdir ./models

from llama_cpp import Llama

llm = Llama(
    model_path="./models/qwen2.5-7b-instruct-q4_k_m.gguf",
    n_ctx=4096,
    n_gpu_layers=-1,  # 全部卸载到 GPU
    verbose=False,
)

response = llm.create_chat_completion(
    messages=[{"role": "user", "content": "解释 Transformer 的自注意力机制"}],
    temperature=0.7,
    max_tokens=1024,
)
print(response["choices"][0]["message"]["content"])

四、方案二：API 服务化

4.1 架构流程

客户端通过 HTTP POST 发送请求，经 Nginx 反向代理转发至 FastAPI 服务，底层由 vLLM 引擎处理推理，GPU 负责计算，Redis 用于队列管理。

4.2 使用 vLLM 启动高性能推理服务

vLLM 原生支持 OpenAI 兼容接口，启动命令如下。注意调整 gpu-memory-utilization 以避免 OOM。

python -m vllm.entrypoints.openai.api_server \
--model Qwen/Qwen2.5-72B-Instruct-GPTQ-Int4 \
--served-model-name qwen-72b \
--host 0.0.0.0 \
--port 8000 \
--max-model-len 4096 \
--gpu-memory-utilization 0.90 \
--tensor-parallel-size 2

客户端调用示例：

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="not-needed",  # 本地部署无需密钥
)

response = client.chat.completions.create(
    model="qwen-72b",
    messages=[
        {"role": "system", "content": "你是一位资深 Python 工程师。"},
        {"role": "user", "content": "如何优化 asyncio 的并发性能？"},
    ],
    temperature=0.7,
    max_tokens=2048,
)
print(response.choices[0].message.content)

4.3 使用 FastAPI 自建 API 服务

若需更细粒度的控制，可基于 FastAPI 封装。关键在于生命周期管理，确保模型在应用启动时加载，关闭时释放显存。

# api_server.py
import uuid
import time
from contextlib import asynccontextmanager
import torch
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field
from transformers import AutoModelForCausalLM, AutoTokenizer

model = None
tokenizer = None

@asynccontextmanager
async def lifespan(app: FastAPI):
    global model, tokenizer
    model_id = "Qwen/Qwen2.5-14B-Instruct-GPTQ-Int4"
    tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
    model = AutoModelForCausalLM.from_pretrained(
        model_id,
        device_map="auto",
        torch_dtype=torch.float16,
        trust_remote_code=True,
    )
    yield
    del model, tokenizer
    torch.cuda.empty_cache()

app = FastAPI(title="LLM API Service", lifespan=lifespan)

class ChatRequest(BaseModel):
    prompt: str = Field(..., min_length=1, max_length=8192)
    max_tokens: int = Field(default=1024, ge=1, le=4096)
    temperature: float = Field(default=0.7, ge=0.0, le=2.0)

class ChatResponse(BaseModel):
    id: str
    response: str
    usage_tokens: int
    latency_ms: float

@app.post("/v1/chat", response_model=ChatResponse)
async def chat_completion(req: ChatRequest):
    if model is None:
        raise HTTPException(status_code=503, detail="模型尚未加载完成")
    start = time.perf_counter()
    input_ids = tokenizer.apply_chat_template(
        [{"role": "user", "content": req.prompt}],
        return_tensors="pt",
    ).to(model.device)
    
    with torch.no_grad():
        outputs = model.generate(
            input_ids,
            max_new_tokens=req.max_tokens,
            temperature=req.temperature,
            top_p=0.9,
            do_sample=True,
        )
    generated = outputs[0][input_ids.shape[-1]:]
    text = tokenizer.decode(generated, skip_special_tokens=True)
    latency = (time.perf_counter() - start) * 1000
    
    return ChatResponse(
        id=str(uuid.uuid4()),
        response=text,
        usage_tokens=len(generated),
        latency_ms=round(latency, 2),
    )

@app.get("/health")
async def health():
    return {
        "status": "ok",
        "model_loaded": model is not None,
        "gpu_available": torch.cuda.is_available(),
    }

启动服务：

uvicorn api_server:app --host 0.0.0.0 --port 8000 --workers 1

五、方案三：Docker 容器封装

5.1 Dockerfile

采用多阶段构建可以显著减小镜像体积。构建阶段安装依赖，运行阶段仅保留运行时环境。

# ---------- 构建阶段 ----------
FROM nvidia/cuda:12.4.1-devel-ubuntu22.04 AS builder
ENV DEBIAN_FRONTEND=noninteractive \
    PYTHONUNBUFFERED=1
RUN apt-get update && apt-get install -y --no-install-recommends \
    python3.11 python3.11-venv python3-pip \
    && rm -rf /var/lib/apt/lists/*

RUN python3.11 -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
COPY requirements.txt /tmp/requirements.txt
RUN pip install --no-cache-dir -r /tmp/requirements.txt

# ---------- 运行阶段 ----------
FROM nvidia/cuda:12.4.1-runtime-ubuntu22.04
RUN apt-get update && apt-get install -y --no-install-recommends \
    python3.11 \
    && rm -rf /var/lib/apt/lists/*

COPY --from=builder /opt/venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
WORKDIR /app
COPY api_server.py .
EXPOSE 8000
CMD ["uvicorn", "api_server:app", "--host", "0.0.0.0", "--port", "8000"]

5.2 docker-compose.yml

配置中挂载了 Hugging Face 缓存目录，避免重复下载模型。同时声明了 GPU 资源限制。

version: "3.9"
services:
  llm-api:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: llm-api-server
    ports:
      - "8000:8000"
    volumes:
      - ~/.cache/huggingface:/root/.cache/huggingface
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - MODEL_ID=Qwen/Qwen2.5-14B-Instruct-GPTQ-Int4
      - MAX_MODEL_LEN=4096
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3

  redis:
    image: redis:7-alpine
    container_name: llm-redis
    ports:
      - "6379:6379"
    restart: unless-stopped

5.3 构建与运行

# 构建镜像
docker-compose build
# 启动服务（后台运行）
docker-compose up -d
# 查看日志
docker-compose logs -f llm-api
# 测试接口
curl -X POST http://localhost:8000/v1/chat \
-H "Content-Type: application/json" \
-d '{"prompt": "解释 Docker 的多阶段构建", "max_tokens": 512}'

六、性能调优要点

在生产环境中，显存管理和延迟优化至关重要。以下是关键参数的建议范围：

为了量化评估，可以使用以下基准测试脚本：

import time
import statistics
import requests

API_URL = "http://localhost:8000/v1/chat"
PROMPT = "请用 200 字介绍 Python 的 GIL 机制。"
NUM_REQUESTS = 50
latencies = []

for i in range(NUM_REQUESTS):
    start = time.perf_counter()
    resp = requests.post(API_URL, json={"prompt": PROMPT, "max_tokens": 256})
    latencies.append((time.perf_counter() - start) * 1000)

print(f"请求次数：{NUM_REQUESTS}")
print(f"平均延迟：{statistics.mean(latencies):.1f} ms")
print(f"P50 延迟：{statistics.median(latencies):.1f} ms")
print(f"P95 延迟：{sorted(latencies)[int(len(latencies)*0.95)]:.1f} ms")
print(f"吞吐量：{NUM_REQUESTS /(sum(latencies)/1000):.1f} req/s")

七、生产部署检查清单

上线前请核对以下项目，确保系统稳定性：

八、总结

本文涵盖了 Python AI 大模型部署的三大核心路径：本地运行适合快速调试，API 服务化支持高并发，Docker 封装实现标准化交付。生产环境建议以 vLLM + Docker + Nginx + Prometheus 为基础技术栈，并根据实际 QPS 和模型规模水平扩展 GPU 节点。

参考资料

• vLLM 官方文档：https://docs.vllm.ai
• llama.cpp 仓库：https://github.com/ggerganov/llama.cpp
• Hugging Face Transformers：https://huggingface.co/docs/transformers