Resource Management

Resources are external knowledge that agents can reference. This module provides functionality for adding, importing/exporting, and uploading temporary files for resources.

Core Concepts

Resource Types

OpenViking supports various resource types, categorized by functionality:

Documents

Spreadsheets & Presentations

Code

Media

Cloud Documents

Resource Processing Pipeline

Resources go through the following processing stages when added:

Source Input -> Parse -> Resource Tree Build -> Persistence -> Semantic Processing
    ↓           ↓            ↓                 ↓               ↓
  URL/File    Parser    TreeBuilder        AGFS       Summarizer/Vector

Stage 1: Parse

• Uses UnifiedResourceProcessor to parse content based on resource type
• Supports multiple formats: documents (PDF/Markdown/Word), spreadsheets (Excel/PPT), code, media files, etc.
• Parsed results are written to a temporary VikingFS directory
• Media files have descriptions generated via VLM (Vision Language Model)

Stage 2: Resource Tree Build (TreeBuilder)

• TreeBuilder.finalize_from_temp() scans the temporary directory structure
• Builds resource tree nodes, handles URI conflicts (auto-renames)
• Establishes relationships between directories and resources

Stage 3: Persistence

• Checks if target URI already exists
• New resources: moves temporary files to permanent AGFS location
• Existing resources: retains temporary tree for subsequent diff comparison
• Acquires lifecycle lock to prevent concurrent modifications
• Cleans up temporary directory

Stage 4: Semantic Processing

• Summary Generation: Summarizer generates L0 (abstract) and L1 (overview)
• Vector Index: Vectorizes content for semantic search
• Processed asynchronously via SemanticQueue, can wait for completion with wait=True

Incremental Updates for Resources

Resource incremental updates are implemented via the Watch Task mechanism:

Watch Task Creation

• Set watch_interval > 0 (in minutes) when calling add_resource to create a watch task
• Must specify the to parameter to define the target URI
• WatchManager handles task persistence
• Supports multi-tenant permission control (ROOT/ADMIN/USER permission levels)

Task Scheduling & Execution

• WatchScheduler checks for expired tasks every 60 seconds
• Default concurrency control prevents duplicate execution
• Expired tasks automatically re-invoke add_resource
• Updates task's last execution time and next execution time

Task Management Operations

• Create: Creates new task or reactivates disabled task when watch_interval > 0
• Update: Re-sets parameters for the same target URI
• Cancel: Disables task when watch_interval <= 0 for the same target URI
• Query: Queries task status by task ID or target URI

API Reference

add_resource

Add a resource to the knowledge base. Supports local files/directories, URLs, and other sources.

1. API Implementation Overview

This endpoint is the core entry point for resource management, supporting adding resources from various sources with optional waiting for semantic processing completion.

Processing Flow:

1. Identify resource source (URL or uploaded temporary file)
2. Call corresponding Parser to parse content
3. Build directory tree and write to AGFS
4. Set up scheduled update task if watch_interval is specified
5. Wait for semantic processing completion if wait=true

Code Entry Points:

• openviking/client/local.py:LocalClient.add_resource - SDK entry (embedded)
• openviking_cli/client/http.py:AsyncHTTPClient.add_resource - SDK entry (HTTP)
• openviking/server/routers/resources.py:add_resource - HTTP router
• openviking/service/resource_service.py - Core service implementation
• crates/ov_cli/src/handlers.rs:handle_add_resource - CLI handler

2. Interface and Parameter Description

Parameters

Additional Notes:

• to and parent cannot be specified together
• path and temp_file_id cannot be specified together
• Raw HTTP calls for local files require first uploading via temp_upload to obtain temp_file_id
• When to is specified and the target already exists, triggers incremental update
• watch_interval only takes effect when to is provided
• For local directory inputs, scanning respects .gitignore files (root and nested) with standard Git semantics; ignore_dirs, include, and exclude further refine what is ingested.

3. Usage Examples

HTTP API

POST /api/v1/resources
Content-Type: application/json

# Add resource from URL
curl -X POST http://localhost:1933/api/v1/resources \
  -H "Content-Type: application/json" \
  -H "X-API-Key: your-key" \
  -d '{
    "path": "https://example.com/guide.md",
    "reason": "User guide documentation",
    "wait": true
  }'

# Add from local file (requires temp_upload first)
TEMP_FILE_ID=$(
  curl -s -X POST http://localhost:1933/api/v1/resources/temp_upload \
    -H "X-API-Key: your-key" \
    -F "file=@./documents/guide.md" \
  | jq -r '.result.temp_file_id'
)

curl -X POST http://localhost:1933/api/v1/resources \
  -H "Content-Type: application/json" \
  -H "X-API-Key: your-key" \
  -d "{
    \"temp_file_id\": \"$TEMP_FILE_ID\",
    \"to\": \"viking://resources/guide.md\",
    \"reason\": \"User guide\"
  }"

Python SDK

import openviking as ov

# Using embedded mode
client = ov.OpenViking(path="./data")
client.initialize()

# Or using HTTP client
client = ov.SyncHTTPClient(url="http://localhost:1933", api_key="your-key")
client.initialize()

# Add local file
result = client.add_resource(
    "./documents/guide.md",
    reason="User guide documentation"
)
print(f"Added: {result['root_uri']}")

# Add from URL to specific location
result = client.add_resource(
    "https://example.com/api-docs.md",
    to="viking://resources/external/api-docs.md",
    reason="External API documentation"
)

# Wait for processing to complete
client.wait_processed()

# Enable scheduled updates
client.add_resource(
    "./documents/guide.md",
    to="viking://resources/guide.md",
    watch_interval=60  # Update every 60 minutes
)

CLI

# Add local file
ov add-resource ./documents/guide.md --reason "User guide"

# Add from URL
ov add-resource https://example.com/guide.md --to viking://resources/guide.md

# Wait for processing to complete
ov add-resource ./documents/guide.md --wait

# Enable scheduled updates (check every 60 minutes)
ov add-resource https://github.com/example/repo.git --to viking://resources/guide.md --watch-interval 60

# Cancel scheduled updates
ov add-resource https://github.com/example/repo.git --to viking://resources/guide.md --watch-interval 0

Response Example

HTTP API Response (JSON)

{
  "status": "ok",
  "result": {
    "status": "success",
    "root_uri": "viking://resources/guide.md",
    "temp_uri": "viking://temp/username/04291108_b62dc7/guide.md",
    "source_path": "./documents/guide.md",
    "meta": {},
    "errors": [],
    "queue_status": {
      "pending": 5,
      "processing": 2,
      "completed": 10
    }
  },
  "telemetry": {
    "operation_id": "550e8400-e29b-41d4-a716-446655440000"
  }
}

CLI Response (Default Table Format)

Note: Resource is being processed in the background.
Use 'ov wait' to wait for completion, or 'ov observer queue' to check status.
status       success
errors       []
source_path  /Users/bytedance/workspace/github.com/OpenViking/docs/en/api/01-overview.md
meta         {}
root_uri     viking://resources/01-overview
temp_uri     viking://temp/shengmaojia/04291108_b62dc7/01-overview

CLI Response (JSON Format, using -o json)

{
  "status": "success",
  "root_uri": "viking://resources/01-overview",
  "temp_uri": "viking://temp/shengmaojia/04291108_b62dc7/01-overview",
  "source_path": "/Users/bytedance/workspace/github.com/OpenViking/docs/en/api/01-overview.md",
  "meta": {},
  "errors": []
}

Field Description

add_skill

Add a skill to the knowledge base.

1. API Implementation Overview

Skills are special resources used to define operations or tools that agents can execute.

Processing Flow:

1. Receive skill data or uploaded temporary file
2. Parse skill definition
3. Store to skill directory
4. Wait for skill processing completion if wait=true

Code Entry Points:

• openviking/client/local.py:LocalClient.add_skill - SDK entry (embedded)
• openviking_cli/client/http.py:AsyncHTTPClient.add_skill - SDK entry (HTTP)
• openviking/server/routers/resources.py:add_skill - HTTP router
• openviking/service/resource_service.py - Core service implementation
• crates/ov_cli/src/handlers.rs:handle_add_skill - CLI handler

2. Interface and Parameter Description

Parameters

3. Usage Examples

HTTP API

POST /api/v1/skills
Content-Type: application/json

# Using inline data
curl -X POST http://localhost:1933/api/v1/skills \
  -H "Content-Type: application/json" \
  -H "X-API-Key: your-key" \
  -d '{
    "data": {
      "name": "my-skill",
      "description": "My custom skill",
      "steps": []
    }
  }'

# Using local file (requires temp_upload first)
TEMP_FILE_ID=$(
  curl -s -X POST http://localhost:1933/api/v1/resources/temp_upload \
    -H "X-API-Key: your-key" \
    -F "file=@./skills/my-skill.json" \
  | jq -r '.result.temp_file_id'
)

curl -X POST http://localhost:1933/api/v1/skills \
  -H "Content-Type: application/json" \
  -H "X-API-Key: your-key" \
  -d "{
    \"temp_file_id\": \"$TEMP_FILE_ID\"
  }"

Python SDK

import openviking as ov

client = ov.SyncHTTPClient(url="http://localhost:1933", api_key="your-key")
client.initialize()

# Add skill from local file
result = client.add_skill("./skills/my-skill.json")

# Wait for processing to complete
client.wait_processed()

CLI

# Add skill
ov add-skill ./skills/my-skill.json

# Wait for processing to complete
ov add-skill ./skills/my-skill.json --wait

4. Response Example

HTTP API Response (JSON)

{
  "status": "ok",
  "result": {
    "status": "success",
    "root_uri": "viking://agent/skills/my-skill",
    "uri": "viking://agent/skills/my-skill",
    "name": "my-skill",
    "auxiliary_files": 2,
    "queue_status": {
      "pending": 0,
      "processing": 0,
      "completed": 1
    }
  },
  "telemetry": {
    "operation_id": "550e8400-e29b-41d4-a716-446655440000"
  }
}

CLI Response (Default Table Format)

Note: Skill is being processed in the background.
Use 'ov wait' to wait for completion, or 'ov observer queue' to check status.
status          success
root_uri        viking://agent/skills/my-skill
uri             viking://agent/skills/my-skill
name            my-skill
auxiliary_files 2

CLI Response (JSON Format, using -o json)

{
  "status": "success",
  "root_uri": "viking://agent/skills/my-skill",
  "uri": "viking://agent/skills/my-skill",
  "name": "my-skill",
  "auxiliary_files": 2
}

Field Description

temp_upload

Upload a temporary file for subsequent importing of local files via add_resource or add_skill.

1. API Implementation Overview

This endpoint is used to upload local files to server temporary storage, returning a temp_file_id for use with subsequent API calls. This is a helper endpoint typically not called directly but used automatically via the SDK or CLI.

Processing Flow:

1. Receive uploaded file
2. Clean up expired temporary files
3. Save to temporary directory and record original filename
4. Return temporary file ID

Code Entry Points:

• openviking/server/routers/resources.py:temp_upload - HTTP router
• openviking/service/resource_service.py - Service implementation

2. Interface and Parameter Description

Parameters

3. Usage Examples

HTTP API

POST /api/v1/resources/temp_upload
Content-Type: multipart/form-data

curl -X POST http://localhost:1933/api/v1/resources/temp_upload \
  -H "X-API-Key: your-key" \
  -F "file=@./documents/guide.md"

Python SDK

The add_resource, add_skill and other endpoints in the Python SDK automatically handle local file uploads, no need to call this endpoint manually.

CLI

CLI commands also automatically handle local file uploads, no need to call this endpoint manually.

Response Example

{
  "status": "ok",
  "result": {
    "temp_file_id": "upload_abc123def456.md"
  },
  "telemetry": {
    "operation_id": "550e8400-e29b-41d4-a716-446655440000"
  }
}

Related Documentation

• File System - File and directory operations
• Skills - Skill management APIs
• Retrieval - Search and context acquisition
• ovpack Guide - Detailed ovpack import/export documentation