docs/en/faq/faq.md
OpenViking is an open-source context database designed specifically for AI Agents. It solves core pain points when building AI Agents:
OpenViking unifies all context management through a filesystem paradigm, enabling tiered delivery and self-iteration.
| Dimension | Traditional Vector DB | OpenViking |
|---|---|---|
| Storage Model | Flat vector storage | Hierarchical filesystem (AGFS) |
| Retrieval Method | Single vector similarity search | Directory recursive retrieval + Intent analysis + Rerank |
| Output Format | Raw chunks | Structured context (L0 Abstract/L1 Overview/L2 Details) |
| Memory Capability | Not supported | Multiple extensible memory types with automatic extraction and continuous iteration |
| Observability | Black box | Fully traceable retrieval trajectory |
| Context Types | Documents only | Resource + Memory + Skill three types |
L0/L1/L2 is OpenViking's progressive content loading mechanism, solving the problem of "stuffing massive context into prompts all at once":
| Layer | Name | Token Limit | Purpose |
|---|---|---|---|
| L0 | Abstract | ~100 tokens | Vector search recall, quick filtering, list display |
| L1 | Overview | ~2000 tokens | Rerank refinement, content navigation, decision reference |
| L2 | Details | Unlimited | Complete original content, on-demand deep loading |
This design allows Agents to browse abstracts for quick positioning, then load details on demand, significantly saving token consumption.
Viking URI is OpenViking's unified resource identifier, formatted as viking://{scope}/{path}. It enables precise location of any context:
viking://
├── resources/ # Knowledge base: documents, code, web pages, etc.
│ └── my_project/
├── user/
│ └── {user_id}/ # Private context for a user
│ ├── memories/ # User memories
│ ├── resources/ # Private user resources
│ ├── skills/ # Private user skills (default)
│ └── peers/{peer_id}/
│ ├── memories/ # Memories scoped to a peer
│ └── resources/ # Resources scoped to a peer
└── agent/ # Optional account-wide Agent capabilities
└── skills/ # Shared skills
OpenViking runs the RAGFS filesystem in-process through the Rust binding
(ragfs_python / RAGFSBindingClient). The binding executes filesystem logic
directly within the Python process, giving extremely high performance and zero
network latency. A compiled RAGFS shared library must be available locally
(shipped in the prebuilt Wheel, or built from source).
[!WARNING] OpenViking no longer supports the AGFS HTTP client mode. AGFS / RAGFS filesystem access now happens only through the in-process Rust binding (
RAGFSBindingClient). This does not affect the OpenViking server HTTP API, theovCLI, orAsyncHTTPClient/SyncHTTPClientwhen they connect to an OpenViking server.
This usually means the RAGFS shared library is not available in your
environment. Re-compile and install it by running
pip install -e . --force-reinstall in the project root (requires a Rust
toolchain).
pip install openviking --upgrade --force-reinstall
Create an ~/.openviking/ov.conf configuration file in your project directory:
{
"embedding": {
"dense": {
"provider": "volcengine",
"api_key": "your-api-key",
"model": "doubao-embedding-vision-251215",
"dimension": 1024,
"input": "multimodal"
}
},
"vlm": {
"provider": "volcengine",
"api_key": "your-api-key",
"model": "doubao-seed-2-0-lite-260428",
"api_base": "https://ark.cn-beijing.volces.com/api/v3"
},
"rerank": {
"provider": "volcengine",
"api_key": "your-api-key",
"model": "doubao-rerank-250615"
},
"storage": {
"workspace": "./data",
"agfs": { "backend": "local" },
"vectordb": { "backend": "local" }
}
}
Config files at the default path ~/.openviking/ov.conf are loaded automatically; you can also specify a different path via the OPENVIKING_CONFIG_FILE environment variable or --config flag. See Configuration Guide for details.
| Provider | Description |
|---|---|
volcengine | Volcengine Embedding API (Recommended) |
openai | OpenAI Embedding API |
vikingdb | VikingDB Embedding API |
jina | Jina AI Embedding API |
ollama | Ollama (local OpenAI-compatible server, no API key required) |
Supports Dense, Sparse, and Hybrid embedding modes.
import openviking as ov
# Async client - embedded mode (recommended)
client = ov.AsyncOpenViking(path="./my_data")
await client.initialize()
# Async client - HTTP client mode
client = ov.AsyncHTTPClient(url="http://localhost:1933", api_key="your-key")
await client.initialize()
The SDK constructor only accepts url, api_key, and path parameters. Other configuration (embedding, vlm, etc.) is managed through the ov.conf config file.
| Type | Supported Formats |
|---|---|
| Text | .txt, .md, .json, .yaml |
| Code | .py, .js, .ts, .go, .java, .cpp, etc. |
| Documents | .pdf, .docx |
| Images | .png, .jpg, .jpeg, .gif, .webp |
| Video | .mp4, .mov, .avi |
| Audio | .mp3, .wav, .m4a |
# Add single file
await client.add_resource(
"./document.pdf",
reason="Project technical documentation", # Describe resource purpose to improve retrieval quality
to="viking://resources/docs/" # Specify storage location
)
# Add web page
await client.add_resource(
"https://example.com/api-docs",
reason="API reference documentation"
)
# Wait for processing to complete
await client.wait_processed()
find() and search()? Which should I use?| Feature | find() | search() |
|---|---|---|
| Session Context | Not required | Required |
| Intent Analysis | Not used | Uses LLM to analyze and generate 0-5 queries |
| Latency | Low | Higher |
| Use Case | Simple semantic search | Complex tasks requiring context understanding |
# find(): Simple direct semantic search
results = await client.find(
"OAuth authentication flow",
target_uri="viking://resources/"
)
# search(): Complex tasks requiring intent analysis
results = await client.search(
"Help me implement user login functionality",
session_info=session
)
Selection Guide:
find()search()Session management is a core capability of OpenViking, supporting conversation tracking and memory extraction:
# Create session
session = client.session()
# Add conversation messages
await session.add_message("user", [{"type": "text", "text": "Help me analyze performance issues in this code"}])
await session.add_message("assistant", [{"type": "text", "text": "Let me analyze..."}])
# Mark used context (for tracking)
await session.used(["viking://resources/code/main.py"])
# Commit session to trigger memory extraction
await session.commit()
OpenViking includes memory types such as profile, preferences, entities, events, identity, soul, cases, trajectories, experiences, tools, and skills. After a session is committed, the active memory policy determines which useful information to extract. Applications can also extend or adjust the memory types for their own needs.
Memories are stored in the current User or Peer namespace; there is no current writable viking://agent/memories directory. See Context Types for the complete type and path mapping.
# List directory contents
items = await client.ls("viking://resources/")
# Read full content (L2)
content = await client.read("viking://resources/doc.md")
# Get abstract (L0)
abstract = await client.abstract("viking://resources")
# Get overview (L1)
overview = await client.overview("viking://resources")
reason: Describe purpose when adding resources to help system understand resource valuetarget parameter to group related resources togethersearch() leverages session history for intent analysismultimodal input for multimodal contentOpenViking uses a score propagation mechanism:
Final Score = 0.5 × Embedding Similarity + 0.5 × Parent Directory Score
This design gives content under high-scoring directories a boost, reflecting the importance of "contextual environment".
Directory recursive retrieval is OpenViking's innovative retrieval strategy:
This strategy finds semantically matching fragments while understanding the complete context of the information.
Possible causes and solutions:
Didn't wait for processing to complete
await client.add_resource("./doc.pdf")
await client.wait_processed() # Must wait
Embedding model configuration error
api_key in ~/.openviking/ov.conf is correctUnsupported file format
View processing logs
import logging
logging.basicConfig(level=logging.DEBUG)
Troubleshooting steps:
Confirm resources have been processed
# Check if resources exist
items = await client.ls("viking://resources/")
Check target_uri filter condition
Try different query approaches
find() and search()Check L0 abstract quality
abstract = await client.abstract("viking://resources/your-doc")
print(abstract) # Confirm abstract accurately reflects content
Troubleshooting steps:
Ensure commit() was called
await session.commit() # Triggers memory extraction
Check VLM configuration
vlm configuration is correctConfirm conversation content is meaningful
View extracted memories
memories = await client.find("", target_uri="viking://user/memories/")
Optimization suggestions:
batch_size: Adjust batch processing size in Embedding configurationlocal backend during development to reduce network latencyAsyncOpenViking / AsyncHTTPClient's async capabilities| Mode | Use Case | Characteristics |
|---|---|---|
| Embedded | Local development, single-process apps | Auto-starts AGFS subprocess, uses local vector index |
| Service Mode | Production, distributed deployment | Connects to remote services, supports multi-instance concurrency, independently scalable |
# Embedded mode
client = ov.AsyncOpenViking(path="./data")
# HTTP client mode (connects to a remote server)
client = ov.AsyncHTTPClient(url="http://localhost:1933", api_key="your-key")
Yes, OpenViking main project is open source under the AGPL-3.0 license, and examples/ and crates/ov_cli are licensed under the Apache 2.0 license.