LEANN配置指南

LEANN 配置指南

本指南可帮助您针对不同用例优化 LEANN，并了解各种配置选项之间的权衡。

开始使用：简单更好

初次尝试 LEANN 时，从一个小数据集入手，快速验证您的方法：

对于文档 RAG：默认的 data/ 目录非常合适——包含 2 篇 AI 研究论文、《傲慢与偏见》文学作品以及一份技术报告

python -m apps.document_rag --query "What techniques does LEANN use?"
​

对于其他数据源：为快速测试，限制数据集大小

# WeChat: Test with recent messages only
python -m apps.wechat_rag --max-items 100 --query "What did we discuss about the project timeline?"

# Browser history: Last few days
python -m apps.browser_rag --max-items 500 --query "Find documentation about vector databases"

# Email: Recent inbox
python -m apps.email_rag --max-items 200 --query "Who sent updates about the deployment status?"
​

验证无误后，逐步扩大规模：

• 100 个文档 → 1,000 → 10,000 → 完整数据集 (--max-items -1)
• This helps identify issues early before committing to long processing times

Embedding Model Selection: Understanding the Trade-offs

Based on our experience developing LEANN, embedding models fall into three categories:

Small Models (< 100M parameters)

Example: sentence-transformers/all-MiniLM-L6-v2 (22M params)

• Pros: Lightweight, fast for both indexing and inference
• Cons: Lower semantic understanding, may miss nuanced relationships
• Use when: Speed is critical, handling simple queries, interactive mode, or just experimenting with LEANN. If time is not a constraint, consider using a larger/better embedding model

Medium Models (100M-500M parameters)

Example: facebook/contriever (110M params), BAAI/bge-base-en-v1.5 (110M params)

• Pros: Balanced performance, good multilingual support, reasonable speed
• Cons: Requires more compute than small models
• Use when: Need quality results without extreme compute requirements, general-purpose RAG applications

Large Models (500M+ parameters)

Example: Qwen/Qwen3-Embedding-0.6B (600M params), intfloat/multilingual-e5-large (5.6 亿参数)

• 优点：语义理解最佳，能捕捉复杂关系，多语言支持出色。Qwen3-Embedding-0.6B 的性能几乎媲美 OpenAI API！
• 缺点：推理速度较慢，索引构建时间较长
• 适用场景：对质量要求极高且计算资源充足的场合。强烈推荐用于生产环境

快速入门：云端与本地嵌入选项

OpenAI 嵌入（最快设置）
无需下载本地模型即可立即测试（如果您没有 GPU且不太在意文档泄露问题，建议使用此方式，我们将计算嵌入并向 OpenAI API 重新计算）：

# Set OpenAI embeddings (requires OPENAI_API_KEY)
--embedding-mode openai --embedding-model text-embedding-3-small
​

Ollama 嵌入（注重隐私）
实现完全私密的本地嵌入：

# First, pull an embedding model
ollama pull nomic-embed-text

# Use Ollama embeddings
--embedding-mode ollama --embedding-model nomic-embed-text
​

Local & Remote Inference Endpoints

Applies to both LLMs (leann ask) and embeddings (leann build)。

LEANN 现在将 Ollama、LM Studio 和其他兼容 OpenAI 的运行时视为一流提供商。您只需通过几个标志或环境变量，就能让 LEANN 指向任何兼容的端点——无论是在同一台机器上还是跨网络。

一次性环境设置

# Works for OpenAI-compatible runtimes such as LM Studio, vLLM, SGLang, llamafile, etc.
export OPENAI_API_KEY="your-key"            # or leave unset for local servers that do not check keys
export OPENAI_BASE_URL="http://localhost:1234/v1"

# Ollama-compatible runtimes (Ollama, Ollama on another host, llamacpp-server, etc.)
export LEANN_OLLAMA_HOST="http://localhost:11434"   # falls back to OLLAMA_HOST or LOCAL_LLM_ENDPOINT
​

LEANN 还识别 LEANN_LOCAL_LLM_HOST (highest priority), LEANN_OPENAI_BASE_URL, and LOCAL_OPENAI_BASE_URL，因此现有脚本仍可正常工作。

每次命令传递主机

# Build an index with a remote embedding server
leann build my-notes \
  --docs ./notes \
  --embedding-mode openai \
  --embedding-model text-embedding-qwen3-embedding-0.6b \
  --embedding-api-base http://192.168.1.50:1234/v1 \
  --embedding-api-key local-dev-key

# Query using a local LM Studio instance via OpenAI-compatible API
leann ask my-notes \
  --llm openai \
  --llm-model qwen3-8b \
  --api-base http://localhost:1234/v1 \
  --api-key local-dev-key

# Query an Ollama instance running on another box
leann ask my-notes \
  --llm ollama \
  --llm-model qwen3:14b \
  --host http://192.168.1.101:11434
​

⚠️ 确保端点可访问：当您的推理服务器运行在家庭或工作站，而索引/搜索任务运行在云端时，服务器必须能够访问您配置的主机。常见方案包括：

• 在托管 LM Studio/Ollama 的机器上暴露公网 IP（并开放相应端口）。
• 配置路由器或云服务商的端口转发。
• 通过工具如 tailscale, cloudflared, or ssh -R.

When you set these options while building an index, LEANN stores them in meta.json. Any subsequent leann ask or searcher process automatically reuses the same provider settings – even when we spawn background embedding servers. This makes the “server without GPU talking to my local workstation” workflow from issue #80 work out-of-the-box.

Tip: If your runtime does not require an API key (many local stacks don’t), leave --api-key 将流量隧道化。LEANN 将跳过注入凭证。

Python API 使用

您也可以从 Python 传递相同的配置：

from leann.api import LeannBuilder

builder = LeannBuilder(
    backend_name="hnsw",
    embedding_mode="openai",
    embedding_model="text-embedding-qwen3-embedding-0.6b",
    embedding_options={
        "base_url": "http://192.168.1.50:1234/v1",
        "api_key": "local-dev-key",
    },
)
builder.build_index("./indexes/my-notes", chunks)
​

embedding_options is persisted to the index meta.json, so subsequent LeannSearcher or LeannChat 会话会自动复用相同的提供商设置（嵌入服务器管理器会为您将这些设置转发给提供商）。

索引选择：匹配您的规模

HNSW（分层可导航小世界）

适用场景：小型到中型数据集（< 1,000 万向量）——默认且推荐用于极低存储

• 需要完整重新计算
• 构建阶段内存占用高
• 出色的召回率（95%+）

# Optimal for most use cases
--backend-name hnsw --graph-degree 32 --build-complexity 64
​

DiskANN

适用场景：大型数据集，尤其是当您希望 recompute=True.

Key advantages:

• Faster search on large datasets (3x+ speedup vs HNSW in many cases)
• Smart storage: recompute=True enables automatic graph partitioning for smaller indexes
• Better scaling: Designed for 100k+ documents

Recompute behavior:

• recompute=True (recommended): Pure PQ traversal + final reranking - faster and enables partitioning
• recompute=False 时：PQ + 遍历过程中部分真实距离——速度较慢但精度更高

# Recommended for most use cases
--backend-name diskann --graph-degree 32 --build-complexity 64
​

性能基准测试：运行 uv run benchmarks/diskann_vs_hnsw_speed_comparison.py to compare DiskANN and HNSW on your system.

LLM Selection: Engine and Model Comparison

LLM Engines

OpenAI (--llm openai)

• Pros: Best quality, consistent performance, no local resources needed
• Cons: Costs money ($0.15-2.5 per million tokens), requires internet, data privacy concerns
• Models: gpt-4o-mini (fast, cheap), gpt-4o (best quality), o3 (reasoning), o3-mini (reasoning, cheaper)
• Thinking Budget: Use --thinking-budget low/medium/high for o-series reasoning models (o3, o3-mini, o4-mini)
• Note: Our current default, but we recommend switching to Ollama for most use cases

Ollama (--llm ollama)

• Pros: Fully local, free, privacy-preserving, good model variety
• Cons: Requires local GPU/CPU resources, slower than cloud APIs, need to install extra ollama app and pre-download models by ollama pull
• Models: qwen3:0.6b (ultra-fast), qwen3:1.7b (balanced), qwen3:4b (good quality), qwen3:7b (high quality), deepseek-r1:1.5b (reasoning)
• Thinking Budget: Use --thinking-budget low/medium/high for reasoning models like GPT-Oss:20b

HuggingFace (--llm hf)

• Pros: Free tier available, huge model selection, direct model loading (vs Ollama's server-based approach)
• Cons: More complex initial setup
• Models: Qwen/Qwen3-1.7B-FP8

Parameter Tuning Guide

Search Complexity Parameters

--build-complexity (index building)

• Controls thoroughness during index construction
• Higher = better recall but slower build
• Recommendations:
  • 32: Quick prototyping
  • 64: Balanced (default)
  • 128: Production systems
  • 256: Maximum quality

--search-complexity (query time)

• Controls search thoroughness
• Higher = better results but slower
• Recommendations:
  • 16: Fast/Interactive search
  • 32: High quality with diversity
  • 64+: Maximum accuracy

Top-K Selection

--top-k (number of retrieved chunks)

• More chunks = better context but slower LLM processing
• Should be always smaller than --search-complexity
• Guidelines:
  • 10-20: General questions (default: 20)
  • 30+: Complex multi-hop reasoning requiring comprehensive context

Trade-off formula:

• Retrieval time ∝ log(n) × search_complexity
• LLM processing time ∝ top_k × chunk_size
• Total context = top_k × chunk_size tokens

Thinking Budget for Reasoning Models

--thinking-budget (reasoning effort level)

• Controls the computational effort for reasoning models
• Options: low, medium, high
• Guidelines:
  • low: Fast responses, basic reasoning (default for simple queries)
  • medium: Balanced speed and reasoning depth
  • high: Maximum reasoning effort, best for complex analytical questions
• Supported Models:
  • Ollama: gpt-oss:20b, gpt-oss:120b
  • OpenAI: o3, o3-mini, o4-mini, o1 (o-series reasoning models)
• Note: Models without reasoning support will show a warning and proceed without reasoning parameters
• Example: --thinking-budget high 可用于复杂分析问题

** 如需详细用法示例和实现细节，请查看 思考预算文档**

** 快速示例：**

# OpenAI o-series reasoning model
python apps/document_rag.py --query "What are the main techniques LEANN explores?" \
  --index-dir hnswbuild --backend hnsw \
  --llm openai --llm-model o3 --thinking-budget medium

# Ollama reasoning model
python apps/document_rag.py --query "What are the main techniques LEANN explores?" \
  --index-dir hnswbuild --backend hnsw \
  --llm ollama --llm-model gpt-oss:20b --thinking-budget high
​

图度数（HNSW/DiskANN）

--graph-degree

• 图中每个节点的连接数
• 数值越高，召回率越好但内存占用越大
• HNSW：16-32（默认：32）
• DiskANN：32-128（默认：64）

性能优化检查表

如果嵌入速度太慢

1. 切换到更小的模型：

   # From large model
   --embedding-model Qwen/Qwen3-Embedding-0.6B
   # To small model
   --embedding-model sentence-transformers/all-MiniLM-L6-v2
   ​

2. 限制数据集大小进行测试：

   --max-items 1000  # Process first 1k items only
   ​

3. 可选优化：在 Apple Silicon 上使用 MLX：

   --embedding-mode mlx --embedding-model mlx-community/Qwen3-Embedding-0.6B-8bit
   ​

MLX 可能不是最佳选择，因为我们测试发现它仅比 HF 提供 1.3 倍加速，因此可能使用 Ollama 更适合生成嵌入

4. 使用 Ollama

   --embedding-mode ollama --embedding-model nomic-embed-text
   ​

要发现 Ollama 中的更多嵌入模型，请访问 https://ollama.com/search?c=embedding 或阅读更多关于嵌入模型的信息：https://ollama.com/blog/embedding-models，请务必确认最适合您的模型大小

如果搜索质量不佳

1. 增加检索数量：

   --top-k 30  # Retrieve more candidates
   ​

2. 升级嵌入模型：

   # For English
   --embedding-model BAAI/bge-base-en-v1.5
   # For multilingual
   --embedding-model intfloat/multilingual-e5-large
   ​

理解权衡

每种配置选择都涉及权衡：

关键在于为您的特定用例找到合适的平衡。从小处着手，简单开始，测量性能后再根据需要逐步扩展。

低资源设置

如果您没有本地 GPU 或构建/搜索速度太慢，可以使用以下一种或多种方案。

1）使用 OpenAI 嵌入（无需本地计算）

最快路径，无需本地 GPU 要求。设置您的 API 密钥，在构建和搜索时使用 OpenAI 嵌入：

export OPENAI_API_KEY=sk-...

# Build with OpenAI embeddings
leann build my-index \
  --embedding-mode openai \
  --embedding-model text-embedding-3-small

# Search with OpenAI embeddings (recompute at query time)
leann search my-index "your query" \
  --recompute
​

2）使用 SkyPilot 进行远程构建（云端 GPU）

借助 SkyPilot 将嵌入生成和索引构建卸载到 GPU VM 上。模板位于 sky/leann-build.yaml。

# One-time: install and configure SkyPilot
pip install skypilot

# Launch with defaults (L4:1) and mount ./data to ~/leann-data; the build runs automatically
sky launch -c leann-gpu sky/leann-build.yaml

# Override parameters via -e key=value (optional)
sky launch -c leann-gpu sky/leann-build.yaml \
  -e index_name=my-index \
  -e backend=hnsw \
  -e embedding_mode=sentence-transformers \
  -e embedding_model=Qwen/Qwen3-Embedding-0.6B

# Copy the built index back to your local .leann (use rsync)
rsync -Pavz leann-gpu:~/.leann/indexes/my-index ./.leann/indexes/
​

3）禁用重新计算以用存储换速度

如果您需要更低延迟且有更多存储/内存，可以禁用重新计算。这会存储完整嵌入，避免在搜索时重新计算。

# Build without recomputation (HNSW requires non-compact in this mode)
leann build my-index --no-recompute --no-compact

# Search without recomputation
leann search my-index "your query" --no-recompute
​

适用场景：

• 极低延迟需求（高 QPS，交互式助手）
• 读取密集型工作负载，存储成本低于延迟
• 没有始终可用的 GPU

约束条件：

• HNSW：当 --no-recompute is set, LEANN automatically disables compact mode during build
• DiskANN: supported; --no-recompute 在搜索时跳过选择性重新计算

存储影响：

• 存储 N 个维度为 D 的浮点数嵌入，大约需要 N × D × 4 字节
• 示例：1,000,000 个 chunk × 768 维 × 4 字节 ≈ 2.86 GB（加上图/元数据）

转换现有索引（需要重建）：

# Rebuild in-place (ensure you still have original docs or can regenerate chunks)
leann build my-index --force --no-recompute --no-compact
​

Python API 使用：

from leann import LeannSearcher

searcher = LeannSearcher("/path/to/my-index.leann")
results = searcher.search("your query", top_k=10, recompute_embeddings=False)
​

权衡：

• 查询时延迟更低且网络跳数更少
• 存储显著增加（比选择性重新计算高 10–100 倍）
• 构建和搜索期间内存占用稍大

快速基准测试结果（benchmarks/benchmark_no_recompute.py，5k 文本，复杂度=32）：

• HNSW

  recompute=True:  search_time=0.818s, size=1.1MB
  recompute=False: search_time=0.012s, size=16.6MB
  ​

• DiskANN

  recompute=True:  search_time=0.041s, size=5.9MB
  recompute=False: search_time=0.013s, size=24.6MB
  ​

结论：

• HNSW：no-recompute is significantly faster (no embedding recomputation) but requires much more storage (stores all embeddings)
• DiskANN: no-recompute uses PQ + partial real distances during traversal (slower but higher accuracy), while recompute=True 使用纯 PQ 遍历 + 最终重排序（遍历更快，支持构建时分区以减少存储）

更多阅读

• 开发 LEANN 的经验教训
• LEANN 技术论文
• DiskANN 原始论文
• 基于 SSD 的图划分

免责声明：
本文档采用基于机器的 AI 翻译服务进行翻译。尽管我们力求准确，但请注意，自动翻译可能存在错误或不准确之处。应以原文语言版本的文档作为权威依据。如需获取关键信息，建议使用专业的人工翻译。对于因使用本翻译而产生的任何误解或误读，我们概不负责。