docs/en/faq/faq.md
OpenViking is an open-source context database designed specifically for AI Agents. It solves core pain points when building AI Agents:
OpenViking unifies all context management through a filesystem paradigm, enabling tiered delivery and self-iteration.
| Dimension | Traditional Vector DB | OpenViking |
|---|---|---|
| Storage Model | Flat vector storage | Hierarchical filesystem (AGFS) |
| Retrieval Method | Single vector similarity search | Directory recursive retrieval + Intent analysis + Rerank |
| Output Format | Raw chunks | Structured context (L0 Abstract/L1 Overview/L2 Details) |
| Memory Capability | Not supported | Built-in 6 memory categories with auto-extraction and iteration |
| Observability | Black box | Fully traceable retrieval trajectory |
| Context Types | Documents only | Resource + Memory + Skill three types |
L0/L1/L2 is OpenViking's progressive content loading mechanism, solving the problem of "stuffing massive context into prompts all at once":
| Layer | Name | Token Limit | Purpose |
|---|---|---|---|
| L0 | Abstract | ~100 tokens | Vector search recall, quick filtering, list display |
| L1 | Overview | ~2000 tokens | Rerank refinement, content navigation, decision reference |
| L2 | Details | Unlimited | Complete original content, on-demand deep loading |
This design allows Agents to browse abstracts for quick positioning, then load details on demand, significantly saving token consumption.
Viking URI is OpenViking's unified resource identifier, formatted as viking://{scope}/{path}. It enables precise location of any context:
viking://
├── resources/ # Knowledge base: documents, code, web pages, etc.
│ └── my_project/
├── user/ # User context
│ └── memories/ # User memories (preferences, entities, events)
└── agent/ # Agent context
├── skills/ # Callable skills
└── memories/ # Agent memories (cases, patterns)
binding-client and http-client? Which one should I choose?binding-client (Default): Runs AGFS logic directly within the Python process via CGO bindings. Advantages: extremely high performance, zero network latency; Disadvantages: requires a compiled AGFS shared library locally.http-client: Communicates with a standalone agfs-server via HTTP. Advantages: decoupled deployment, no local Go compilation needed; Disadvantages: some network communication overhead.If your environment supports Go compilation or you've installed a Wheel package containing pre-compiled libraries, the default binding-client is recommended.
This usually means the AGFS shared library is not pre-built in your environment. You can:
pip install -e . --force-reinstall in the project root (requires Go environment).storage.agfs.mode = "http-client" in your ov.conf and ensure an agfs-server is running.pip install openviking --upgrade --force-reinstall
Create an ~/.openviking/ov.conf configuration file in your project directory:
{
"embedding": {
"dense": {
"provider": "volcengine",
"api_key": "your-api-key",
"model": "doubao-embedding-vision-251215",
"dimension": 1024,
"input": "multimodal"
}
},
"vlm": {
"provider": "volcengine",
"api_key": "your-api-key",
"model": "doubao-seed-2-0-pro-260215",
"api_base": "https://ark.cn-beijing.volces.com/api/v3"
},
"rerank": {
"provider": "volcengine",
"api_key": "your-api-key",
"model": "doubao-rerank-250615"
},
"storage": {
"workspace": "./data",
"agfs": { "backend": "local" },
"vectordb": { "backend": "local" }
}
}
Config files at the default path ~/.openviking/ov.conf are loaded automatically; you can also specify a different path via the OPENVIKING_CONFIG_FILE environment variable or --config flag. See Configuration Guide for details.
| Provider | Description |
|---|---|
volcengine | Volcengine Embedding API (Recommended) |
openai | OpenAI Embedding API |
vikingdb | VikingDB Embedding API |
jina | Jina AI Embedding API |
ollama | Ollama (local OpenAI-compatible server, no API key required) |
Supports Dense, Sparse, and Hybrid embedding modes.
import openviking as ov
# Async client - embedded mode (recommended)
client = ov.AsyncOpenViking(path="./my_data")
await client.initialize()
# Async client - HTTP client mode
client = ov.AsyncHTTPClient(url="http://localhost:1933", api_key="your-key")
await client.initialize()
The SDK constructor only accepts url, api_key, and path parameters. Other configuration (embedding, vlm, etc.) is managed through the ov.conf config file.
| Type | Supported Formats |
|---|---|
| Text | .txt, .md, .json, .yaml |
| Code | .py, .js, .ts, .go, .java, .cpp, etc. |
| Documents | .pdf, .docx |
| Images | .png, .jpg, .jpeg, .gif, .webp |
| Video | .mp4, .mov, .avi |
| Audio | .mp3, .wav, .m4a |
# Add single file
await client.add_resource(
"./document.pdf",
reason="Project technical documentation", # Describe resource purpose to improve retrieval quality
target="viking://resources/docs/" # Specify storage location
)
# Add web page
await client.add_resource(
"https://example.com/api-docs",
reason="API reference documentation"
)
# Wait for processing to complete
await client.wait_processed()
find() and search()? Which should I use?| Feature | find() | search() |
|---|---|---|
| Session Context | Not required | Required |
| Intent Analysis | Not used | Uses LLM to analyze and generate 0-5 queries |
| Latency | Low | Higher |
| Use Case | Simple semantic search | Complex tasks requiring context understanding |
# find(): Simple direct semantic search
results = await client.find(
"OAuth authentication flow",
target_uri="viking://resources/"
)
# search(): Complex tasks requiring intent analysis
results = await client.search(
"Help me implement user login functionality",
session_info=session
)
Selection Guide:
find()search()Session management is a core capability of OpenViking, supporting conversation tracking and memory extraction:
# Create session
session = client.session()
# Add conversation messages
await session.add_message("user", [{"type": "text", "text": "Help me analyze performance issues in this code"}])
await session.add_message("assistant", [{"type": "text", "text": "Let me analyze..."}])
# Mark used context (for tracking)
await session.used(["viking://resources/code/main.py"])
# Commit session to trigger memory extraction
await session.commit()
OpenViking has 6 built-in memory categories, automatically extracted during session commit:
| Category | Belongs To | Description |
|---|---|---|
| profile | user | User basic info (name, role, etc.) |
| preferences | user | User preferences (code style, tool choices, etc.) |
| entities | user | Entity memories (people, projects, organizations, etc.) |
| events | user | Event records (decisions, milestones, etc.) |
| cases | agent | Cases learned by Agent |
| patterns | agent | Patterns learned by Agent |
# List directory contents
items = await client.ls("viking://resources/")
# Read full content (L2)
content = await client.read("viking://resources/doc.md")
# Get abstract (L0)
abstract = await client.abstract("viking://resources")
# Get overview (L1)
overview = await client.overview("viking://resources")
reason: Describe purpose when adding resources to help system understand resource valuetarget parameter to group related resources togethersearch() leverages session history for intent analysismultimodal input for multimodal contentOpenViking uses a score propagation mechanism:
Final Score = 0.5 × Embedding Similarity + 0.5 × Parent Directory Score
This design gives content under high-scoring directories a boost, reflecting the importance of "contextual environment".
Directory recursive retrieval is OpenViking's innovative retrieval strategy:
This strategy finds semantically matching fragments while understanding the complete context of the information.
Possible causes and solutions:
Didn't wait for processing to complete
await client.add_resource("./doc.pdf")
await client.wait_processed() # Must wait
Embedding model configuration error
api_key in ~/.openviking/ov.conf is correctUnsupported file format
View processing logs
import logging
logging.basicConfig(level=logging.DEBUG)
Troubleshooting steps:
Confirm resources have been processed
# Check if resources exist
items = await client.ls("viking://resources/")
Check target_uri filter condition
Try different query approaches
find() and search()Check L0 abstract quality
abstract = await client.abstract("viking://resources/your-doc")
print(abstract) # Confirm abstract accurately reflects content
Troubleshooting steps:
Ensure commit() was called
await session.commit() # Triggers memory extraction
Check VLM configuration
vlm configuration is correctConfirm conversation content is meaningful
View extracted memories
memories = await client.find("", target_uri="viking://user/memories/")
Optimization suggestions:
batch_size: Adjust batch processing size in Embedding configurationlocal backend during development to reduce network latencyAsyncOpenViking / AsyncHTTPClient's async capabilities| Mode | Use Case | Characteristics |
|---|---|---|
| Embedded | Local development, single-process apps | Auto-starts AGFS subprocess, uses local vector index |
| Service Mode | Production, distributed deployment | Connects to remote services, supports multi-instance concurrency, independently scalable |
# Embedded mode
client = ov.AsyncOpenViking(path="./data")
# HTTP client mode (connects to a remote server)
client = ov.AsyncHTTPClient(url="http://localhost:1933", api_key="your-key")
Yes, OpenViking main project is open source under the AGPL-3.0 license, and examples/ and crates/ov_cli are licensed under the Apache 2.0 license.