AI知识库搭建全攻略

本文集文档发布于灏天文库

4.4 部署环境配置

文档摘要

4.4 部署环境配置 — AI知识库搭建全攻略 本节导读：全面掌握AI知识库的部署环境配置，包括开发环境、测试环境、生产环境的搭建与管理，确保系统的稳定运行和高可用性。 学习目标 掌握开发环境配置和工具链搭建 学会测试环境的构建和管理 了解生产环境的部署策略 能够实现高可用和容灾配置 掌握监控和运维的最佳实践 环境架构概述 AI知识库的部署环境需要根据不同阶段的需求进行分层配置，确保每个环境都有合适的资源配置和管理策略： 开发环境配置 本地开发环境 基础软件安装 Docker开发环境 开发环境自动化脚本 测试环境配置 测试环境架构 单元测试配置 集成测试配置 生产环境配置 生产环境架构 Nginx配置 Kubernetes生产部署 监控与告警 Prometheus配置 告警规则

4.4 部署环境配置 — AI知识库搭建全攻略

本节导读：全面掌握AI知识库的部署环境配置，包括开发环境、测试环境、生产环境的搭建与管理，确保系统的稳定运行和高可用性。

学习目标

  • 掌握开发环境配置和工具链搭建
  • 学会测试环境的构建和管理
  • 了解生产环境的部署策略
  • 能够实现高可用和容灾配置
  • 掌握监控和运维的最佳实践

环境架构概述

AI知识库的部署环境需要根据不同阶段的需求进行分层配置，确保每个环境都有合适的资源配置和管理策略：

开发环境配置

本地开发环境

基础软件安装

# 系统依赖
sudo apt-get update
sudo apt-get install -y \
    python3-pip \
    python3-venv \
    git \
    docker \
    docker-compose \
    nginx \
    postgresql \
    redis-server

# Python环境
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# 数据库启动
sudo systemctl start postgresql
sudo systemctl start redis-server
​

Docker开发环境

# docker-compose.dev.yml
version: '3.8'

services:
  app:
    build: 
      context: .
      dockerfile: Dockerfile.dev
    volumes:
      - .:/app
      - /app/venv
    ports:
      - "5000:5000"
    environment:
      - FLASK_ENV=development
      - DEBUG=True
      - DATABASE_URL=postgresql://postgres:***@postgres:5432/knowledge_dev
    depends_on:
      - postgres
      - redis

  postgres:
    image: postgres:13
    environment:
      POSTGRES_DB: knowledge_dev
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: ***
    volumes:
      - postgres_dev_data:/var/lib/postgresql/data

  redis:
    image: redis:6-alpine
    volumes:
      - redis_dev_data:/data

volumes:
  postgres_dev_data:
  redis_dev_data:
​

开发环境自动化脚本

#!/bin/bash
# scripts/dev-setup.sh

set -e

echo "🚀 开始设置开发环境..."

# 检查必要工具
check_tools() {
    local tools=("python3" "pip" "git" "docker")
    for tool in "${tools[@]}"; do
        if ! command -v "$tool" &> /dev/null; then
            echo "❌ 缺少工具: $tool"
            exit 1
        fi
    done
    echo "✅ 所有必要工具已安装"
}

# 创建虚拟环境
setup_venv() {
    if [ ! -d "venv" ]; then
        echo "📦 创建Python虚拟环境..."
        python3 -m venv venv
    fi
    echo "✅ 虚拟环境已就绪"
}

# 安装依赖
install_deps() {
    echo "📥 安装Python依赖..."
    source venv/bin/activate
    pip install --upgrade pip
    pip install -r requirements.dev.txt
    echo "✅ 依赖安装完成"
}

# 主函数
main() {
    check_tools
    setup_venv
    install_deps
    
    echo "🎉 开发环境设置完成!"
    echo "📍 应用运行在: http://localhost:5000"
}

main
​

测试环境配置

测试环境架构

# docker-compose.test.yml
version: '3.8'

services:
  app:
    build: 
      context: .
      dockerfile: Dockerfile.test
    volumes:
      - ./tests:/app/tests
    environment:
      - DATABASE_URL=postgresql://postgres:***@postgres-test:5432/knowledge_test
      - TEST_MODE=True
    depends_on:
      - postgres-test
      - redis-test
    command: python -m pytest tests/ -v --cov=app

  postgres-test:
    image: postgres:13
    environment:
      POSTGRES_DB: knowledge_test
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: ***
    volumes:
      - postgres_test_data:/var/lib/postgresql/data

  redis-test:
    image: redis:6-alpine
    volumes:
      - redis_test_data:/data

volumes:
  postgres_test_data:
  redis_test_data:
​

单元测试配置

# tests/conftest.py
import pytest
import tempfile
import os
from knowledge_base import create_app, db
from knowledge_base.models import Document, User

@pytest.fixture
def app():
    """创建测试应用"""
    db_fd, db_path = tempfile.mkstemp()
    app = create_app()
    app.config['SQLALCHEMY_DATABASE_URI'] = f'sqlite:///{db_path}'
    app.config['TESTING'] = True
    
    with app.app_context():
        db.create_all()
    
    yield app
    
    os.close(db_fd)
    os.unlink(db_path)

@pytest.fixture
def client(app):
    """创建测试客户端"""
    return app.test_client()

@pytest.fixture
def init_database(app):
    """初始化测试数据库"""
    with app.app_context():
        db.create_all()
        
        # 创建测试用户
        user = User(
            username='testuser',
            email='test@example.com'
        )
        user.set_password('password123')
        db.session.add(user)
        
        # 创建测试文档
        doc = Document(
            title='测试文档',
            content='这是一个测试文档的内容',
            user_id=user.id
        )
        db.session.add(doc)
        
        db.session.commit()
​

集成测试配置

# tests/test_integration.py
import pytest
from knowledge_base.app import app

class TestKnowledgeBaseIntegration:
    def setup_method(self):
        """测试前置设置"""
        self.app = app
        self.app.config['TESTING'] = True
        self.client = self.app.test_client()
        
    def test_search_endpoint(self, init_database):
        """测试搜索端点"""
        # 准备测试数据
        doc = Document.query.first()
        
        # 执行搜索
        response = self.client.post('/api/search', 
                                  json={'query': '测试文档'})
        
        # 验证结果
        assert response.status_code == 200
        data = response.get_json()
        assert 'results' in data
        assert len(data['results']) > 0
        
    def test_document_crud(self, auth_client, init_database):
        """测试文档CRUD操作"""
        # 创建文档
        response = auth_client.post('/api/documents', 
                                   json={
                                       'title': '新文档',
                                       'content': '新文档内容'
                                   })
        assert response.status_code == 201
        doc_id = response.get_json()['id']
        
        # 获取文档
        response = auth_client.get(f'/api/documents/{doc_id}')
        assert response.status_code == 200
        
        # 更新文档
        response = auth_client.put(f'/api/documents/{doc_id}', 
                                  json={
                                      'title': '更新后的标题',
                                      'content': '更新后的内容'
                                  })
        assert response.status_code == 200
        
        # 删除文档
        response = auth_client.delete(f'/api/documents/{doc_id}')
        assert response.status_code == 204
​

生产环境配置

生产环境架构

# docker-compose.prod.yml
version: '3.8'

services:
  app:
    image: knowledge-base:latest
    deploy:
      replicas: 3
    environment:
      - DATABASE_URL=postgresql://postgres:***@postgres:5432/knowledge_prod
      - SECRET_KEY=${SECRET_KEY}
      - DEBUG=False
      - ENVIRONMENT=production
    depends_on:
      - postgres
      - redis
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:5000/health"]
      interval: 30s
      timeout: 10s
      retries: 3

  postgres:
    image: postgres:13
    environment:
      POSTGRES_DB: knowledge_prod
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: ***
    volumes:
      - postgres_prod_data:/var/lib/postgresql/data
    restart: unless-stopped

  redis:
    image: redis:6-alpine
    volumes:
      - redis_prod_data:/data
    restart: unless-stopped

  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
      - ./ssl:/etc/nginx/ssl
    depends_on:
      - app
    restart: unless-stopped

volumes:
  postgres_prod_data:
  redis_prod_data:
​

Nginx配置

# nginx.conf
upstream app {
    least_conn;
    server app1:5000 max_fails=3 fail_timeout=30s;
    server app2:5000 max_fails=3 fail_timeout=30s;
    server app3:5000 max_fails=3 fail_timeout=30s;
    keepalive 32;
}

server {
    listen 80;
    server_name api.yourdomain.com;

    # HTTPS重定向
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl http2;
    server_name api.yourdomain.com;

    # SSL配置
    ssl_certificate /etc/nginx/ssl/cert.pem;
    ssl_certificate_key /etc/nginx/ssl/key.pem;
    ssl_session_cache shared:SSL:1m;
    ssl_session_timeout 5m;

    # API端点
    location /api {
        proxy_pass http://app;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # 超时设置
        proxy_connect_timeout 30s;
        proxy_read_timeout 30s;
        proxy_send_timeout 30s;
        
        # 限流
        limit_req zone=api burst=20 nodelay;
    }

    # 静态文件
    location /static {
        alias /app/static;
        expires 1y;
        add_header Cache-Control "public, immutable";
    }

    # 健康检查
    location /health {
        access_log off;
        proxy_pass http://app;
    }
}
​

Kubernetes生产部署

# k8s-prod.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: knowledge-base-api
  namespace: knowledge-base
spec:
  replicas: 3
  selector:
    matchLabels:
      app: knowledge-base
  template:
    metadata:
      labels:
        app: knowledge-base
    spec:
      containers:
      - name: knowledge-base
        image: knowledge-base:latest
        ports:
        - containerPort: 5000
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: db-secret
              key: url
        resources:
          requests:
            memory: "1Gi"
            cpu: "500m"
          limits:
            memory: "2Gi"
            cpu: "1000m"
        livenessProbe:
          httpGet:
            path: /health
            port: 5000
          initialDelaySeconds: 30
          periodSeconds: 10

---
apiVersion: v1
kind: Service
metadata:
  name: knowledge-base-service
  namespace: knowledge-base
spec:
  selector:
    app: knowledge-base
  ports:
  - protocol: TCP
    port: 80
    targetPort: 5000
  type: ClusterIP

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: knowledge-base-ingress
  namespace: knowledge-base
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  tls:
  - hosts:
    - api.yourdomain.com
    secretName: knowledge-base-tls
  rules:
  - host: api.yourdomain.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: knowledge-base-service
            port:
              number: 80

---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: knowledge-base-hpa
  namespace: knowledge-base
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: knowledge-base-api
  minReplicas: 3
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
​

监控与告警

Prometheus配置

# prometheus.yml
global:
  scrape_interval: 15s
  evaluation_interval: 15s

rule_files:
  - "alert_rules.yml"

scrape_configs:
  - job_name: 'knowledge-base'
    static_configs:
      - targets: ['knowledge-base:5000']
    metrics_path: '/metrics'

  - job_name: 'postgres'
    static_configs:
      - targets: ['postgres-exporter:9187']

  - job_name: 'redis'
    static_configs:
      - targets: ['redis-exporter:9121']
​

告警规则

# alert_rules.yml
groups:
- name: knowledge-base
  rules:
  - alert: HighErrorRate
    expr: rate(http_requests_total{status=~"5.."}[5m]) > 0.1
    for: 5m
    labels:
      severity: critical
    annotations:
      summary: "高错误率检测到"
      description: "API错误率超过10%，持续5分钟"

  - alert: HighLatency
    expr: histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m])) > 2
    for: 5m
    labels:
      severity: warning
    annotations:
      summary: "高延迟检测到"
      description: "95%请求响应时间超过2秒"
​

Grafana仪表板

{
  "dashboard": {
    "title": "AI知识库监控",
    "panels": [
      {
        "title": "请求总数",
        "type": "stat",
        "targets": [
          {
            "expr": "sum(rate(http_requests_total[5m]))",
            "legendFormat": "总请求数"
          }
        ]
      },
      {
        "title": "错误率",
        "type": "graph",
        "targets": [
          {
            "expr": "sum(rate(http_requests_total{status=~\"5..\"}[5m])) / sum(rate(http_requests_total[5m])) * 100",
            "legendFormat": "5xx错误率"
          }
        ]
      }
    ]
  }
}
​

高可用与容灾

数据库主从复制

# postgres-master-slave.yml
version: '3.8'

services:
  postgres-master:
    image: postgres:13
    environment:
      POSTGRES_DB: knowledge_prod
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: ***
      POSTGRES_REPLICATION_USER: replicator
      POSTGRES_REPLICATION_PASSWORD: ***
    volumes:
      - postgres_master_data:/var/lib/postgresql/data
    command: postgres -c wal_level=replica -c max_wal_senders=3

  postgres-slave:
    image: postgres:13
    environment:
      POSTGRES_DB: knowledge_prod
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: ***
    volumes:
      - postgres_slave_data:/var/lib/postgresql/data
    depends_on:
      - postgres-master
    command: |
      until pg_basebackup --pgdata=/var/lib/postgresql/data --host=postgres-master --port=5432 --username=replicator; do
        echo "等待主数据库连接..."
        sleep 1s
      done
      echo "数据库备份完成，启动从数据库..."
      postgres -c hot_standby=on

volumes:
  postgres_master_data:
  postgres_slave_data:
​

负载均衡配置

# nginx-ha.conf
upstream knowledge_backend {
    least_conn;
    server app1:5000 max_fails=3 fail_timeout=30s;
    server app2:5000 max_fails=3 fail_timeout=30s;
    server app3:5000 max_fails=3 fail_timeout=30s;
    keepalive 32;
}

upstream knowledge_backend_backup {
    least_conn;
    server app4:5000 max_fails=3 fail_timeout=30s;
    server app5:5000 max_fails=3 fail_timeout=30s;
    keepalive 32;
}

server {
    listen 80;
    server_name api.yourdomain.com;

    # 主负载均衡
    location / {
        proxy_pass http://knowledge_backend;
        proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
        proxy_next_upstream_tries 2;
        
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # 超时设置
        proxy_connect_timeout 30s;
        proxy_read_timeout 30s;
        proxy_send_timeout 30s;
        
        # 健康检查
        proxy_intercept_errors on;
        error_page 502 503 504 @fallback;
    }

    # 备份负载均衡（主集群故障时启用）
    location @fallback {
        proxy_pass http://knowledge_backend_backup;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
​

本节小结

本节全面介绍了AI知识库的部署环境配置，涵盖了开发环境、测试环境、生产环境的搭建和管理：

  1. 开发环境配置：提供了本地开发环境的软件安装、Docker容器化和自动化脚本的完整方案，确保开发效率和环境一致性。

  2. 测试环境配置：建立了测试环境的架构，包括单元测试、集成测试的配置方案，确保代码质量和系统稳定性。

  3. 生产环境配置：详细介绍了生产环境的部署策略，包括Docker容器化、Kubernetes集群部署和负载均衡配置，确保系统的高可用性和扩展性。

  4. 监控与告警：实现了完整的监控体系，包括Prometheus配置、告警规则和Grafana仪表板，实时监控系统运行状态。

  5. 高可用与容灾：设计了数据库主从复制和负载均衡的高可用架构，确保系统的稳定运行和故障快速恢复。

通过本节的学习，读者应该能够从零开始搭建完整的AI知识库部署环境，确保系统的稳定运行和高可用性。

延伸阅读

  • 官方文档：Docker、Kubernetes官方文档
  • 相关章节：本教程第5章实战案例与优化

关键词：AI知识库搭建全攻略, 部署环境配置, Docker, Kubernetes, 监控, 高可用
难度：进阶
预计阅读：35分钟

发布者: 作者: 转发
评论区 (0)
U
0/500