5.1 部署架构设计

5.1 部署架构设计

导读：将大模型应用从开发环境推向生产环境，需要一套完整的部署架构。本节将全面介绍大模型应用的部署架构设计，包括容器化部署、服务编排、负载均衡和高可用方案。

5.1.1 大模型应用部署概述

部署挑战

大模型应用的部署面临以下独特挑战：

1. 高延迟：LLM推理本身需要较长响应时间（通常1-30秒）
2. 高成本：API调用费用或GPU推理资源成本较高
3. 长连接：流式输出需要保持HTTP长连接
4. 弹性需求：流量波动大，需要灵活扩缩容
5. 安全要求：涉及用户数据和模型密钥的安全保护

# 部署配置管理
from pydantic_settings import BaseSettings

class DeploymentSettings(BaseSettings):
    """部署配置"""
    host: str = "0.0.0.0"
    port: int = 8000
    workers: int = 4
    debug: bool = False
    llm_provider: str = "openai"
    llm_model: str = "gpt-4-turbo-preview"
    llm_api_key: str = ""
    llm_base_url: str = "https://api.openai.com/v1"
    llm_timeout: int = 60
    vector_db_host: str = "localhost"
    vector_db_port: int = 19530
    rate_limit_rpm: int = 60
    rate_limit_concurrent: int = 10
    enable_metrics: bool = True
    log_level: str = "INFO"
    
    class Config:
        env_prefix = "APP_"
        env_file = ".env"

settings = DeploymentSettings()
​

5.1.2 应用架构设计

FastAPI应用主入口

# FastAPI应用主入口
from fastapi import FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from contextlib import asynccontextmanager
import logging

logger = logging.getLogger(__name__)

@asynccontextmanager
async def lifespan(app: FastAPI):
    """应用生命周期管理"""
    logger.info("应用启动中...")
    await initialize_resources()
    logger.info("应用启动完成")
    yield
    await cleanup_resources()
    logger.info("应用已关闭")

app = FastAPI(title="大模型应用服务", version="1.0.0", lifespan=lifespan)

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_methods=["*"],
    allow_headers=["*"],
)

@app.get("/health")
async def health_check():
    """健康检查端点"""
    return {"status": "healthy", "version": "1.0.0"}

@app.post("/api/chat")
async def chat(request: ChatRequest):
    """对话接口"""
    try:
        result = await chat_service.process(request)
        return result
    except RateLimitExceeded:
        raise HTTPException(status_code=429, detail="请求过于频繁")
    except LLMAPIError as e:
        raise HTTPException(status_code=502, detail="AI服务暂时不可用")
​

限流与认证中间件

# 限流中间件
from slowapi import Limiter, _rate_limit_exceeded_handler
from slowapi.util import get_remote_address

limiter = Limiter(key_func=get_remote_address)
app.state.limiter = limiter
app.add_exception_handler(429, _rate_limit_exceeded_handler)

@app.post("/api/chat")
@limiter.limit("60/minute")
async def chat(request: Request, body: ChatRequest):
    """带限流的对话接口"""
    return await chat_service.process(body)
​

5.1.3 容器化部署

Dockerfile编写

# Dockerfile - 多阶段构建
FROM python:3.11-slim AS builder

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt

FROM python:3.11-slim

WORKDIR /app
COPY --from=builder /install /usr/local
COPY . .

RUN groupadd -r appuser && useradd -r -g appuser appuser
RUN chown -R appuser:appuser /app
USER appuser

EXPOSE 8000

HEALTHCHECK --interval=30s --timeout=10s --retries=3 \
    CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4"]
​

Docker Compose编排

# docker-compose.yml
version: '3.8'

services:
  app:
    build: .
    ports:
      - "8000:8000"
    environment:
      - APP_LLM_API_KEY=${LLM_API_KEY}
      - APP_LLM_MODEL=gpt-4-turbo-preview
    depends_on:
      - milvus
    restart: unless-stopped
    deploy:
      resources:
        limits:
          memory: 2G
          cpus: "2"

  milvus:
    image: milvusdb/milvus:v2.3.0
    ports:
      - "19530:19530"
    volumes:
      - milvus_data:/var/lib/milvus
    restart: unless-stopped

  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    volumes:
      - redis_data:/data
    restart: unless-stopped

  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
    depends_on:
      - app
    restart: unless-stopped

volumes:
  milvus_data:
  redis_data:
​

5.1.4 服务编排与负载均衡

Nginx反向代理配置

# nginx.conf
upstream llm_backend {
    least_conn;
    server app_1:8000 max_fails=3 fail_timeout=30s;
    server app_2:8000 max_fails=3 fail_timeout=30s;
    server app_3:8000 max_fails=3 fail_timeout=30s;
    keepalive 32;
}

server {
    listen 80;
    server_name api.example.com;

    # 请求体大小限制
    client_max_body_size 10M;

    # 超时配置（适配LLM长响应）
    proxy_read_timeout 120s;
    proxy_send_timeout 120s;

    # 健康检查不记录日志
    location /health {
        proxy_pass http://llm_backend;
        access_log off;
    }

    # API路由
    location /api/ {
        proxy_pass http://llm_backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        
        # SSE/流式输出支持
        proxy_buffering off;
        proxy_cache off;
        chunked_transfer_encoding on;
    }
}
​

Kubernetes部署

# k8s-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: llm-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: llm-app
  template:
    metadata:
      labels:
        app: llm-app
    spec:
      containers:
      - name: llm-app
        image: llm-app:1.0.0
        ports:
        - containerPort: 8000
        env:
        - name: APP_LLM_API_KEY
          valueFrom:
            secretKeyRef:
              name: llm-secrets
              key: api-key
        resources:
          requests:
            memory: "512Mi"
            cpu: "500m"
          limits:
            memory: "2Gi"
            cpu: "2"
        livenessProbe:
          httpGet:
            path: /health
            port: 8000
          initialDelaySeconds: 10
          periodSeconds: 30
        readinessProbe:
          httpGet:
            path: /health
            port: 8000
          initialDelaySeconds: 5
          periodSeconds: 10
---
apiVersion: v1
kind: Service
metadata:
  name: llm-app-service
spec:
  selector:
    app: llm-app
  ports:
  - port: 80
    targetPort: 8000
  type: LoadBalancer
​

5.1.5 高可用与弹性伸缩

HPA自动伸缩

# hpa.yaml - Kubernetes HPA配置
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: llm-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: llm-app
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80
​

缓存策略

# 响应缓存
import hashlib
import json
from typing import Optional
from datetime import datetime, timedelta

class ResponseCache:
    """LLM响应缓存，减少重复API调用"""
    
    def __init__(self, redis_client, ttl_seconds: int = 3600):
        self.redis = redis_client
        self.ttl = ttl_seconds
        self.hit_count = 0
        self.miss_count = 0
    
    def _make_key(self, model: str, messages: list, temperature: float) -> str:
        """生成缓存键"""
        content = json.dumps({"model": model, "messages": messages, "temp": temperature}, sort_keys=True)
        hash_val = hashlib.md5(content.encode()).hexdigest()
        return f"llm_cache:{hash_val}"
    
    def get(self, model: str, messages: list, temperature: float) -> Optional[str]:
        """获取缓存"""
        key = self._make_key(model, messages, temperature)
        cached = self.redis.get(key)
        if cached:
            self.hit_count += 1
            return json.loads(cached)
        self.miss_count += 1
        return None
    
    def set(self, model: str, messages: list, temperature: float, response: str):
        """设置缓存"""
        key = self._make_key(model, messages, temperature)
        self.redis.setex(key, self.ttl, json.dumps(response))
    
    @property
    def hit_rate(self) -> float:
        total = self.hit_count + self.miss_count
        return self.hit_count / total if total > 0 else 0.0
​

本章小结

通过本节的学习，你将：

✅ 了解大模型应用部署的独特挑战
✅ 掌握基于FastAPI的应用架构设计
✅ 学会Docker容器化和Docker Compose编排
✅ 了解Nginx反向代理和负载均衡配置
✅ 掌握Kubernetes部署和HPA弹性伸缩
✅ 学会响应缓存减少API调用成本

在5.2节中，我们将深入性能监控与优化，在5.3节中讨论安全与合规问题。

[FAQ] 常见问题

Q: 大模型应用的部署成本如何控制？
A: 主要从几个方面控制：响应缓存减少API调用、合理设置模型参数（temperature、max_tokens）、使用更便宜的模型处理简单任务、异步处理非实时请求、监控和优化Token使用量。

Q: 如何处理LLM API的超时问题？
A: 设置合理的超时时间（建议60-120秒）、实现重试机制（带指数退避）、为用户提供流式输出减少等待感、实现请求队列防止资源耗尽。

Q: 本地部署开源模型和调用云端API如何选择？
A: 云端API部署简单、扩展容易但成本较高。本地部署需要GPU资源但数据隐私好、长期成本可控。建议初期用API快速验证，稳定后评估本地部署。

Q: 如何保证多副本间的一致性？
A: 无状态设计是关键。将所有状态（会话历史、缓存）外置到Redis等共享存储中。每个请求都应能被任意副本处理，不依赖特定实例的本地状态。

Q: Docker和Kubernetes哪个更适合大模型应用部署？
A: 小规模或初期项目用Docker Compose足够，成本低、运维简单。中大规模或需要自动扩缩容的项目用Kubernetes。建议从Docker开始，根据需求逐步迁移到K8s。