AI 原生开放教材

外观

Context Engineering Intro工程手册·英文原文 ↗仓库 ↗

🔍 Semantic Search Agent ​

An intelligent knowledge base search system powered by Pydantic AI and PostgreSQL with PGVector. This agent provides both semantic and hybrid search capabilities with automatic strategy selection and result summarization.

一个由 Pydantic AI 和带 PGVector 的 PostgreSQL 驱动的智能知识库检索系统。该 agent 同时提供语义检索与混合检索能力，并支持自动策略选择与结果摘要。

Features ​

  • Semantic Search: Pure vector similarity search using embeddings
  • Hybrid Search: Combined semantic and keyword matching for precise results
  • Intelligent Strategy Selection: Agent automatically chooses the best search approach
  • Result Summarization: Coherent insights generated from search results
  • Interactive CLI: Rich command-line interface with real-time streaming
  • Multi-Provider Support: Works with any OpenAI-compatible API (OpenAI, Gemini, Ollama, etc.)

语义检索：使用 embedding 的纯向量相似度检索；混合检索：把语义匹配与关键词匹配结合起来以获得精确结果；智能策略选择：agent 自动选择最合适的检索方式；结果摘要：从检索结果中生成连贯的洞见；交互式 CLI：带实时流式输出的丰富命令行界面；多 provider 支持：兼容任何 OpenAI 风格的 API（OpenAI、Gemini、Ollama 等）。

Prerequisites ​

  • Python 3.10+
  • PostgreSQL with PGVector extension
  • LLM API key (OpenAI, Gemini, Ollama, Groq, or any OpenAI-compatible provider)
  • Existing database with documents and chunks (schema provided)

环境要求：Python 3.10+；带 PGVector 扩展的 PostgreSQL；LLM API key（OpenAI、Gemini、Ollama、Groq 或任何 OpenAI 兼容的 provider）；已有包含文档与文本块的数据库（schema 已提供）。

Installation ​

  1. Clone or copy the agent directory:

1）克隆或复制 agent 目录：

bash
cd agents/rag_agent
  1. Install dependencies:

2）安装依赖：

bash
pip install -r requirements.txt
  1. Set up PostgreSQL with PGVector:

3）搭建带 PGVector 的 PostgreSQL：

bash
# SIMPLEST: Run the SQL in your SQL editor if you are using a platform like Supabase/Postgres

# Or run the schema with psql
psql -d your_database -f sql/schema.sql
  1. Configure environment variables:

4）配置环境变量：

bash
cp .env.example .env
# Edit .env with your credentials
  1. Ingest documents into the database:

5）把文档写入数据库：

bash
# This step is required before running the agent
# It will process documents and generate embeddings
python -m ingestion.ingest --documents documents/

Configuration ​

Required Environment Variables ​

  • DATABASE_URL: PostgreSQL connection string with PGVector
  • LLM_PROVIDER: Provider name (openai, anthropic, ollama, etc.)
  • LLM_API_KEY: Your LLM provider API key
  • LLM_MODEL: Model to use (e.g., gpt-4.1-mini, gemini-2.5-flash)
  • LLM_BASE_URL: API base URL (default: https://api.openai.com/v1)
  • EMBEDDING_MODEL: Embedding model to use (e.g., text-embedding-3-small, text-embedding-3-large)

DATABASE_URL：带 PGVector 的 PostgreSQL 连接串；LLM_PROVIDER：provider 名称（openai、anthropic、ollama 等）；LLM_API_KEY：你的 LLM provider API key；LLM_MODEL：要使用的模型（例如 gpt-4.1-mini、gemini-2.5-flash）；LLM_BASE_URL：API base URL（默认 https://api.openai.com/v1 ）；EMBEDDING_MODEL：要使用的 embedding 模型（例如 text-embedding-3-small、text-embedding-3-large）。

Usage ​

Command Line Interface ​

Run the interactive CLI:

运行交互式 CLI：

bash
python -m cli

The CLI provides:

  • Real-time streaming responses
  • Tool execution visibility
  • Session persistence
  • User preference management

该 CLI 提供：实时流式响应；工具执行的可见性；会话持久化；用户偏好管理。

Available Commands ​

  • help - Show available commands
  • info - Display system configuration
  • clear - Clear the screen
  • set <key>=<value> - Set preferences (e.g., set text_weight=0.5)
  • exit/quit - Exit the application

可用命令：help——显示可用命令；info——展示系统配置；clear——清屏；set <key>=<value>——设置偏好（例如 set text_weight=0.5）；exit/quit——退出程序。

Search Strategies ​

The agent intelligently selects between two search strategies:

该 agent 会在两种检索策略之间智能选择：

Best for conceptual queries and finding related content:

  • "concepts similar to machine learning"
  • "ideas about artificial intelligence"
  • "related to neural networks"

Best for specific facts and technical terms:

  • "OpenAI GPT-4 specifications"
  • "NASDAQ:NVDA stock price"
  • "specific quote from Sam Altman"

The agent automatically chooses the appropriate strategy based on your query, or you can explicitly request a specific search type in your prompt.

agent 会根据你的查询自动选择合适的策略，你也可以在提示词中明确要求某种检索类型。

Database Setup ​

Schema Overview ​

  • documents: Stores full documents with metadata
  • chunks: Stores document chunks with embeddings
  • match_chunks(): Function for semantic search
  • hybrid_search(): Function for combined search

数据库结构：documents 存放带元数据的完整文档；chunks 存放带 embedding 的文档块；match_chunks() 是语义检索函数；hybrid_search() 是组合检索函数。

Development ​

Running Tests ​

bash
pytest tests/

Code Formatting ​

bash
black .
ruff check .

Project Structure ​

semantic_search_agent/
├── agent.py           # Main agent implementation
├── cli.py            # Command-line interface
├── dependencies.py   # Agent dependencies
├── providers.py      # Model providers
├── prompts.py        # System prompts
├── settings.py       # Configuration
├── tools.py          # Search tools
├── ingestion/        # Document ingestion pipeline
├── sql/              # Database schema
└── documents/        # Sample documents