LEANN-RAG

git clone https://github.com/yichuan-w/LEANN.git leann
cd leann
git submodule update --init --recursive
​

macOS：

注意：DiskANN要求MacOS 13.3或更高版本。

brew install libomp boost protobuf zeromq pkgconf
uv sync --extra diskann
​

Linux（Ubuntu/Debian）：

注意：在Ubuntu 20.04上，你可能需要编译更新的Abseil并固定Protobuf版本（例如v3.20.x）来构建DiskANN。请参阅问题#30获取逐步说明。

你可以手动安装Intel oneAPI MKL代替下面命令中的libmkl-full-dev for DiskANN. You can also use libopenblas-dev for building HNSW only, by removing --extra diskann。

sudo apt-get update && sudo apt-get install -y \
  libomp-dev libboost-all-dev protobuf-compiler libzmq3-dev \
  pkg-config libabsl-dev libaio-dev libprotobuf-dev \
  libmkl-full-dev

uv sync --extra diskann
​

Linux（Arch Linux）：

sudo pacman -Syu && sudo pacman -S --needed base-devel cmake pkgconf git gcc \
  boost boost-libs protobuf abseil-cpp libaio zeromq

# For MKL in DiskANN
sudo pacman -S --needed base-devel git
git clone https://aur.archlinux.org/paru-bin.git
cd paru-bin && makepkg -si
paru -S intel-oneapi-mkl intel-oneapi-compiler
source /opt/intel/oneapi/setvars.sh

uv sync --extra diskann
​

Linux（RHEL / CentOS Stream / Oracle / Rocky / AlmaLinux）：

请参阅问题#50了解更多信息。

sudo dnf groupinstall -y "Development Tools"
sudo dnf install -y libomp-devel boost-devel protobuf-compiler protobuf-devel \
  abseil-cpp-devel libaio-devel zeromq-devel pkgconf-pkg-config

# For MKL in DiskANN
sudo dnf install -y intel-oneapi-mkl intel-oneapi-mkl-devel \
  intel-oneapi-openmp || sudo dnf install -y intel-oneapi-compiler
source /opt/intel/oneapi/setvars.sh

uv sync --extra diskann
​

将你的OpenAI API密钥设为环境变量：

export OPENAI_API_KEY="your-api-key-here"
​

确保使用--llm openai flag when using the CLI.
You can also specify the model name with --llm-model <model-name> flag.

Thanks to the widespread adoption of the OpenAI API format, LEANN is compatible out-of-the-box with a vast array of LLM and embedding providers. Simply set the OPENAI_BASE_URL and OPENAI_API_KEY环境变量连接到你首选的服务。

export OPENAI_API_KEY="xxx"
export OPENAI_BASE_URL="http://localhost:1234/v1" # base url of the provider
​

要使用CLI界面的OpenAI兼容端点：

如果你用于文本生成，务必使用--llm openai flag and specify the model name with --llm-model <model-name> flag.

If you are using it for embedding, set the --embedding-mode openai flag and specify the model name with --embedding-model <MODEL>.

Below is a list of base URLs for common providers to get you started.

️ Local Inference Engines (Recommended for full privacy)

☁️ Cloud Providers

** A Note on Privacy:** Before choosing a cloud provider, carefully review their privacy and data retention policies. Depending on their terms, your data may be used for their own purposes, including but not limited to human reviews and model training, which can lead to serious consequences if not handled properly.

如果你的提供商不在这个列表中，别担心！查看他们的文档寻找兼容OpenAI的端点——很可能它也兼容OpenAI！

macOS：

首先，下载Ollama for macOS。

# Pull a lightweight model (recommended for consumer hardware)
ollama pull llama3.2:1b
​

Linux：

# Install Ollama
curl -fsSL https://ollama.ai/install.sh | sh

# Start Ollama service manually
ollama serve &

# Pull a lightweight model (recommended for consumer hardware)
ollama pull llama3.2:1b
​

所有RAG示例都共享这些通用参数。交互模式在所有示例中可用——只需不带--query运行，即可启动连续问答会话，你可以提出多个问题。输入'quit'退出。

# Core Parameters (General preprocessing for all examples)
--index-dir DIR              # Directory to store the index (default: current directory)
--query "YOUR QUESTION"      # Single query mode. Omit for interactive chat (type 'quit' to exit), and now you can play with your index interactively
--max-items N                # Limit data preprocessing (default: -1, process all data)
--force-rebuild              # Force rebuild index even if it exists

# Embedding Parameters
--embedding-model MODEL      # e.g., facebook/contriever, text-embedding-3-small, mlx-community/Qwen3-Embedding-0.6B-8bit or nomic-embed-text
--embedding-mode MODE        # sentence-transformers, openai, mlx, or ollama

# LLM Parameters (Text generation models)
--llm TYPE                   # LLM backend: openai, ollama, or hf (default: openai)
--llm-model MODEL            # Model name (default: gpt-4o) e.g., gpt-4o-mini, llama3.2:1b, Qwen/Qwen2.5-1.5B-Instruct
--thinking-budget LEVEL      # Thinking budget for reasoning models: low/medium/high (supported by o3, o3-mini, GPT-Oss:20b, and other reasoning models)

# Search Parameters
--top-k N                    # Number of results to retrieve (default: 20)
--search-complexity N        # Search complexity for graph traversal (default: 32)

# Chunking Parameters
--chunk-size N               # Size of text chunks (default varies by source: 256 for most, 192 for WeChat)
--chunk-overlap N            # Overlap between chunks (default varies: 25-128 depending on source)

# Index Building Parameters
--backend-name NAME          # Backend to use: hnsw or diskann (default: hnsw)
--graph-degree N             # Graph degree for index construction (default: 32)
--build-complexity N         # Build complexity for index construction (default: 64)
--compact / --no-compact     # Use compact storage (default: true). Must be `no-compact` for `no-recompute` build.
--recompute / --no-recompute # Enable/disable embedding recomputation (default: enabled). Should not do a `no-recompute` search in a `recompute` build.
​

参数

--data-dir DIR           # Directory containing documents to process (default: data)
--file-types .ext .ext   # Filter by specific file types (optional - all LlamaIndex supported types if omitted)
​

示例命令

# Process all documents with larger chunks for academic papers
python -m apps.document_rag --data-dir "~/Documents/Papers" --chunk-size 1024

# Filter only markdown and Python files with smaller chunks
python -m apps.document_rag --data-dir "./docs" --chunk-size 256 --file-types .md .py

# Enable AST-aware chunking for code files
python -m apps.document_rag --enable-code-chunking --data-dir "./my_project"

# Or use the specialized code RAG for better code understanding
python -m apps.code_rag --repo-dir "./my_codebase" --query "How does authentication work?"
​

参数

--mail-path PATH         # Path to specific mail directory (auto-detects if omitted)
--include-html          # Include HTML content in processing (useful for newsletters)
​

示例命令

# Search work emails from a specific account
python -m apps.email_rag --mail-path "~/Library/Mail/V10/WORK_ACCOUNT"

# Find all receipts and order confirmations (includes HTML)
python -m apps.email_rag --query "receipt order confirmation invoice" --include-html
​

索引建立完成后，你可以提问：

• “找到我老板关于截止日期的邮件”
• “约翰对项目时间表说了什么？”
• “给我看看关于差旅费用的邮件”

参数

--chrome-profile PATH    # Path to Chrome profile directory (auto-detects if omitted)
​

示例命令

# Search academic research from your browsing history
python -m apps.browser_rag --query "arxiv papers machine learning transformer architecture"

# Track competitor analysis across work profile
python -m apps.browser_rag --chrome-profile "~/Library/Application Support/Google/Chrome/Work Profile" --max-items 5000
​

默认的Chrome配置文件路径针对典型的macOS设置进行了配置。如果你需要找到你的具体Chrome配置文件：

1. 打开终端
2. 运行：ls ~/Library/Application\ Support/Google/Chrome/
3. Look for folders like "Default", "Profile 1", "Profile 2", etc.
4. Use the full path as your --chrome-profile argument

Common Chrome profile locations:

• macOS: ~/Library/Application Support/Google/Chrome/Default
• Linux: ~/.config/google-chrome/Default

索引建立完成后，你可以提问：

• “我访问过哪些关于机器学习的网站？”
• “找到我关于编程的搜索历史”
• “我最近看了哪些YouTube视频？”
• “给我看看我访问过的关于旅行规划的网站”

首先，你需要安装微信导出工具，

brew install sunnyyoung/repo/wechattweak-cli
​

或者手动安装（如果你在Homebrew上有问题）：

sudo packages/wechat-exporter/wechattweak-cli install
​

故障排除：

• 安装问题：查看WeChatTweak-CLI的问题页面
• 导出错误：如果遇到以下错误，尝试重启微信

  Failed to export WeChat data. Please ensure WeChat is running and WeChatTweak is installed.
  Failed to find or export WeChat data. Exiting.
  ​

参数

--export-dir DIR         # Directory to store exported WeChat data (default: wechat_export_direct)
--force-export          # Force re-export even if data exists
​

示例命令

# Search for travel plans discussed in group chats
python -m apps.wechat_rag --query "travel plans" --max-items 10000

# Re-export and search recent chats (useful after new messages)
python -m apps.wechat_rag --force-export --query "work schedule"
​

索引建立完成后，你可以提问：

• “我想买魔术师约翰逊的球衣，给我一些对应聊天记录？”（中文：Show me chat records about buying Magic Johnson's jersey）

分步导出过程：

1. 登录ChatGPT
2. 点击右上角的个人头像
3. 进入设置→数据控制
4. **点击“导出”**下的“导出数据”
5. 确认导出请求
6. 从邮件链接下载ZIP文件（有效期24小时）
7. 解压或直接使用LEANN

支持格式：

• .html files from ChatGPT exports
• .zip归档文件来自ChatGPT
• 包含多个导出文件的目录

参数

--export-path PATH           # Path to ChatGPT export file (.html/.zip) or directory (default: ./chatgpt_export)
--separate-messages         # Process each message separately instead of concatenated conversations
--chunk-size N              # Text chunk size (default: 512)
--chunk-overlap N           # Overlap between chunks (default: 128)
​

示例命令

# Basic usage with HTML export
python -m apps.chatgpt_rag --export-path conversations.html

# Process ZIP archive from ChatGPT
python -m apps.chatgpt_rag --export-path chatgpt_export.zip

# Search with specific query
python -m apps.chatgpt_rag --export-path chatgpt_data.html --query "Python programming help"

# Process individual messages for fine-grained search
python -m apps.chatgpt_rag --separate-messages --export-path chatgpt_export.html

# Process directory containing multiple exports
python -m apps.chatgpt_rag --export-path ./chatgpt_exports/ --max-items 1000
​

ChatGPT对话索引完成后，你可以用以下查询搜索：

• “我问ChatGPT关于Python编程的什么？”
• “给我看看关于机器学习算法的对话”
• “找到关于Web开发框架的讨论”
• “ChatGPT给了我哪些编程建议？”
• “搜索关于调试技巧的对话”
• “找到ChatGPT的学习资源推荐”

分步导出过程：

1. 在浏览器中打开Claude
2. 进入设置（查找齿轮图标或设置菜单）
3. **在

• 在 developer.twitter.com 申请一个 Twitter 开发者账号
  • 在 Twitter 开发者门户中创建一个新应用
  • 生成具有“读取”权限的 API 密钥和访问令牌
  • 如果需要访问书签，可能需要使用具有适当范围的 Twitter API v2

  export TWITTER_API_KEY="your-api-key"
  export TWITTER_API_SECRET="your-api-secret"
  export TWITTER_ACCESS_TOKEN="your-access-token"
  export TWITTER_ACCESS_TOKEN_SECRET="your-access-token-secret"
  ​

3. 使用 --test-connection flag

Arguments:

• --mcp-server: Command to start the Twitter MCP server
• --username: Filter bookmarks by username (optional)
• --max-bookmarks: Maximum bookmarks to fetch (default: 1000)
• --no-tweet-content: Exclude tweet content, only metadata
• --no-metadata 测试连接：排除互动元数据

Slack 查询：

• “团队讨论了项目截止日期的哪些内容？”
• “查找有关新功能发布的消息”
• “给我展示关于预算规划的对话”
• “开发团队频道里做出了哪些决定？”

Twitter 查询：

• “我上个月收藏了哪些人工智能文章？”
• “查找有关机器学习技术的推文”
• “给我展示收藏的关于创业建议的帖子”
• “我保存了哪些 Python 教程？”

想为其他平台添加支持吗？LEANN 的 MCP 集成旨在轻松扩展：

1. 为您的平台找到或创建一个 MCP 服务器
2. 按照 apps/slack_data/slack_mcp_reader.py
3. Create a RAG application following the pattern in apps/slack_rag.py 中的模式创建一个读取器类
4. 测试并回馈社区！

值得探索的热门 MCP 服务器：

• GitHub 仓库和问题
• Discord 消息
• Notion 页面
• Google Drive 文档
• 以及 MCP 生态系统中的更多平台！

LEANN 提供智能代码分块功能，可保留语义边界（函数、类、方法），适用于 Python、Java、C# 和 TypeScript，相比基于文本的分块，能更好地理解代码。

阅读AST 分块指南 →

You can use leann --help, or leann build --help, leann search --help, leann ask --help, leann list --help, leann remove --help 获取完整的 CLI 参考。

构建命令：

leann build INDEX_NAME --docs DIRECTORY|FILE [DIRECTORY|FILE ...] [OPTIONS]

Options:
  --backend {hnsw,diskann}     Backend to use (default: hnsw)
  --embedding-model MODEL      Embedding model (default: facebook/contriever)
  --graph-degree N             Graph degree (default: 32)
  --complexity N               Build complexity (default: 64)
  --force                      Force rebuild existing index
  --compact / --no-compact     Use compact storage (default: true). Must be `no-compact` for `no-recompute` build.
  --recompute / --no-recompute Enable recomputation (default: true)
​

搜索命令：

leann search INDEX_NAME QUERY [OPTIONS]

Options:
  --top-k N                     Number of results (default: 5)
  --complexity N                Search complexity (default: 64)
  --recompute / --no-recompute  Enable/disable embedding recomputation (default: enabled). Should not do a `no-recompute` search in a `recompute` build.
  --pruning-strategy {global,local,proportional}
​

提问命令：

leann ask INDEX_NAME [OPTIONS]

Options:
  --llm {ollama,openai,hf}    LLM provider (default: ollama)
  --model MODEL               Model name (default: qwen3:8b)
  --interactive              Interactive chat mode
  --top-k N                  Retrieval count (default: 20)
​

列表命令：

leann list

# Lists all indexes across all projects with status indicators:
# ✅ - Index is complete and ready to use
# ❌ - Index is incomplete or corrupted
#  - CLI-created index (in .leann/indexes/)
#  - App-created index (*.leann.meta.json files)
​

删除命令：

leann remove INDEX_NAME [OPTIONS]

Options:
  --force, -f    Force removal without confirmation

# Smart removal: automatically finds and safely removes indexes
# - Shows all matching indexes across projects
# - Requires confirmation for cross-project removal
# - Interactive selection when multiple matches found
# - Supports both CLI and app-created indexes
​