docs/design/memory-extractor-optimization.md
Implement a memory templating system based on OpenViking storage, such that: / 实现基于 OpenViking 存储的记忆模版化系统,使得:
This design references the following core ideas from the ../memory project: / 本设计参考了 ../memory 项目的以下核心思想:
The system provides two memory modes, distinguished by whether content_template exists. / 系统提供两种记忆模式,用是否有 content_template 来区分。
Use Cases: profile, preferences, entities, events, cases, patterns / 适用场景:profile、preferences、entities、events、cases、patterns
Characteristics: / 特点:
name, content two fields (or a few simple fields) / 只有 name、content 两个字段(或简单的几个字段)content is Markdown content, no rendering needed / content 就是 Markdown 内容,不需要渲染MEMORY_FIELDS comment needed / 不需要 MEMORY_FIELDS 注释Config Examples / 配置示例:
profile.yaml
name: profile
description: |
User profile memory - captures "who the user is" as a person.
Extract relatively stable personal attributes that define the user's identity, work style, and preferences.
Include: profession, experience level, technical background, communication style, work habits, etc.
Do NOT include transient conversation content or temporary mood states.
directory: "viking://user/{user_space}/memories"
filename_template: "profile.md"
fields:
- name: content
type: string
description: |
User profile content describing "who the user is".
Includes relatively stable personal attributes: profession, experience, tech stack, communication style, etc.
Example: "User is an AI development engineer with 3 years of LLM application development experience, mainly using Python and LangChain tech stack. Communication style is concise and direct, prefers efficient code implementation."
merge_op: patch
preferences.yaml
name: preferences
description: |
User preference memory - captures "what the user likes/dislikes or is accustomed to".
Extract specific preferences the user has expressed across conversations.
Each preference should be about a specific topic (not generic).
Topics can be: code style, communication style, tools, workflow, food, commute, etc.
Store different topics as separate memory files, do NOT mix unrelated preferences.
directory: "viking://user/{user_space}/memories/preferences"
filename_template: "{topic}.md"
fields:
- name: topic
type: string
description: |
Preference topic used to uniquely identify this preference memory.
Should be a semantic topic description such as "Python code style", "Communication style", "Food preference", "Commute preference", etc.
Different preference topics should be stored as separate memories, do not mix unrelated preferences.
merge_op: immutable
- name: content
type: string
description: |
Specific preference content describing "what the user prefers/is accustomed to".
Example: "User has shown clear preferences for Python code style in multiple conversations: dislikes using type hints, considers them redundant; requires concise function comments, limited to 1-2 lines; prefers direct implementation, avoids excessive fallbacks and over-engineering."
merge_op: patch
entities.yaml
name: entities
description: |
Entity memory - captures "what this named thing is, what properties it has".
Extract information about specific entities mentioned in conversation.
Entity types include: projects, people, organizations, systems, technologies, concepts, products, etc.
Each entity is a named thing that has attributes worth remembering for future conversations.
Store each entity as a separate memory file, keyed by entity name.
directory: "viking://user/{user_space}/memories/entities"
filename_template: "{entity_name}.md"
fields:
- name: entity_name
type: string
description: |
Entity name used to uniquely identify this entity memory.
Should be the specific name of the entity such as "OpenViking project", "Alice (colleague)", "Redis", etc.
merge_op: immutable
- name: entity_type
type: string
description: |
Entity type describing what type of entity this is.
Possible values: project, person, organization, system, technology, concept, etc.
- name: content
type: string
description: |
Detailed entity content describing "what this named thing is, what properties it has".
Includes: basic information, core attributes, status, etc.
Example: "OpenViking is an AI Agent long-term memory management system the user is developing. The project uses Python and AGFS tech stack, core features include memory extraction, deduplication, and retrieval. Currently in active development, goal is to build Claude-like long-term memory capabilities."
merge_op: patch
events.yaml
name: events
description: |
Event memory - captures "what happened, what decision was made, and why".
Extract notable events, decisions, milestones, and turning points from the conversation.
Events should be things worth remembering for future context: decisions made, agreements reached, milestones achieved, problems solved, etc.
Each event should include: what happened, why it happened, what the outcome was, and any relevant context/timeline.
Use absolute dates for event_time, not relative time like "today" or "recently".
directory: "viking://user/{user_space}/memories/events"
filename_template: "{event_time}_{event_name}.md"
fields:
- name: event_name
type: string
description: |
Event name used to uniquely identify this event memory.
Should be a specific event description such as "Decided to refactor memory system", "Started OpenViking project", "Completed Q3 review", etc.
Example: "[Action]: [Description]"
merge_op: immutable
- name: event_time
type: string
description: |
Time when the event occurred, use absolute time format, do not use relative time (such as "today", "recently").
Can be empty if time is unknown.
Example: "2026-03-17"
merge_op: immutable
- name: content
type: string
description: |
Detailed event content describing "what happened".
Includes: decision content, reasons, results, background, timeline, etc.
Example: "During memory system design discussion, found that the original 6 categories had blurry boundaries. Especially states, lessons, insights often overlapped and were hard to distinguish. Decided to refactor to 5 categories, removing these three to make classification boundaries clearer."
merge_op: patch
cases.yaml
name: cases
description: |
Case memory - captures "what problem was encountered and how it was solved".
Extract specific problem-solution pairs from the conversation that are worth remembering for future reference.
Cases should be about specific problems that have clear solutions.
Each case should include: what the problem was (symptoms, error messages, context), what the solution was (steps taken, principles used), and why it worked.
Case names should be in "Problem → Solution" format to make them easily searchable.
directory: "viking://agent/{agent_space}/memories/cases"
filename_template: "{case_name}.md"
fields:
- name: case_name
type: string
description: |
Case name used to uniquely identify this case memory.
Should be in "Problem → Solution" format such as "Band not recognized → Request member/album/style details", "Memory merge timeout → Split into smaller chunks", etc.
Example: "[Problem] → [Solution]"
merge_op: immutable
- name: problem
type: string
description: |
Problem description specifically detailing what problem was encountered.
Includes: error messages, symptoms, context, and other specific details.
Example: "User feedback that a band cannot be recognized by system."
merge_op: patch
- name: solution
type: string
description: |
Solution description detailing how to solve this problem.
Includes: solution method, steps, principles, etc.
Example: "Request user to provide more identification details: band member names, representative album names, music style, etc. This information can improve recognition accuracy."
merge_op: patch
- name: content
type: string
description: |
Complete case content including full narrative of problem and solution.
Example: "User feedback mentioned a band that the system could not recognize. Solution is to request user to provide more identification details: band member names, representative album names, music style, etc. This information can improve recognition accuracy."
merge_op: patch
patterns.yaml
name: patterns
description: |
Pattern memory - captures "under what circumstances to follow what process".
Extract reusable workflows, processes, and methods that the agent should follow in similar future situations.
Patterns should be about: how to approach certain types of tasks, what steps to follow, what considerations to keep in mind.
Each pattern should include: trigger conditions (when to use this pattern), process steps (what to do), and considerations (what to watch out for).
Pattern names should be in "Process name: Step description" format.
directory: "viking://agent/{agent_space}/memories/patterns"
filename_template: "{pattern_name}.md"
fields:
- name: pattern_name
type: string
description: |
Pattern name used to uniquely identify this pattern memory.
Should be in "Process name: Step description" format such as "Teaching topic handling: Outline→Plan→Generate PPT", "Code refactoring: Understand→Test→Refactor→Verify", etc.
Example: "[Pattern name]: [Step sequence]"
merge_op: immutable
- name: pattern_type
type: string
description: |
Pattern type describing what type of pattern this is.
Possible values: workflow, method, process, etc.
- name: content
type: string
description: |
Detailed pattern content describing "under what circumstances to follow what process".
Includes: trigger conditions, process steps, considerations, etc.
Example: "When user requests teaching content for a topic, use a four-step process: first list the topic outline to understand overall structure; then create a detailed learning plan; next generate PPT framework; finally refine specific content for each section. This process ensures content is systematic and complete."
merge_op: patch
Memory File Storage Format Example / 记忆文件存储格式示例:
# User Profile
User is an AI development engineer with 3 years of experience...
Use Cases: tools, skills / 适用场景:tools、skills
Characteristics: / 特点:
content is rendered from fields via content_template / content 是通过 content_template 从字段渲染出来的MEMORY_FIELDS JSON comment is always placed at the end of the file / MEMORY_FIELDS JSON 注释永远放在文件最后<!-- MEMORY_FIELDS --> comment at the end of Markdown file / 从 Markdown 文件最后的 <!-- MEMORY_FIELDS --> 注释中解析 JSON 字段merge_op / 根据字段的 merge_op 做记忆更新content_template after update / 更新后通过 content_template 重新渲染 Markdown 内容MEMORY_FIELDS back to end of file / 把更新后的 MEMORY_FIELDS 写回文件最后Config Examples / 配置示例:
tools.yaml
name: tools
description: |
Tool usage memory - captures "how this tool is used, what works well, and what doesn't".
Extract tool usage patterns, statistics, and learnings from [ToolCall] records and conversation context.
For each tool, track: how many times it's been called, success rate, average time/tokens, what it's best for, optimal parameters, common failure modes, and actionable recommendations.
Also accumulate complete guidelines with "Good Cases" and "Bad Cases" examples.
Tool memories help the agent learn from experience and use tools more effectively over time.
directory: "viking://agent/{agent_space}/memories/tools"
filename_template: "{tool_name}.md"
content_template: |
Tool: {tool_name}
Static Description:
"{static_desc}"
Tool Memory Context:
Based on {total_calls} historical calls:
- Success rate: {success_rate}% ({success_count} successful, {fail_count} failed)
- Avg time: {avg_time}, Avg tokens: {avg_tokens}
- Best for: {best_for}
- Optimal params: {optimal_params}
- Common failures: {common_failures}
- Recommendation: {recommendation}
{guidelines}
fields:
- name: tool_name
type: string
description: |
Tool name, copied exactly from [ToolCall] records without modification.
Used to uniquely identify this tool memory.
Examples: "web_search", "read_file", "execute_code"
merge_op: immutable
- name: static_desc
type: string
description: |
Static description of the tool, basic functionality description.
Examples: "Searches the web for information", "Reads files from the file system"
- name: total_calls
type: int64
description: |
Total number of tool calls, accumulated from historical statistics.
Used to calculate success rate and average duration.
merge_op: sum
- name: success_count
type: int64
description: |
Number of successful tool calls, accumulated from historical statistics.
Counts calls with status "completed".
merge_op: sum
- name: fail_count
type: int64
description: |
Number of failed tool calls, accumulated from historical statistics.
Counts calls with status not "completed".
merge_op: sum
- name: total_time_ms
type: int64
description: |
Total tool call duration in milliseconds, accumulated from historical statistics.
Used to calculate average duration.
merge_op: sum
- name: total_tokens
type: int64
description: |
Total tokens used by tool calls (prompt tokens + completion tokens), accumulated from historical statistics.
Used to calculate average token consumption.
merge_op: sum
- name: best_for
type: string
description: |
Best use cases for the tool, describing in what scenarios this tool works best.
Examples: "Technical documentation, tutorials, API references"
merge_op: patch
- name: optimal_params
type: string
description: |
Optimal parameter range/best practices for the tool, describing general parameter optimization suggestions.
Should describe general best practices (such as "max_results=5-20", "timeout>30s for large files"),
do not describe specific case values (such as "command: 'echo hello'").
Examples: "max_results: 5-20 (larger values may timeout); language: 'en' for better results; query: specific multi-word phrases with qualifiers"
merge_op: patch
- name: common_failures
type: string
description: |
Common failure modes of the tool, describing problems and error patterns this tool frequently encounters.
Examples: "Single-word queries return irrelevant results; max_results>50 causes timeout; non-English queries have lower quality"
merge_op: patch
- name: recommendation
type: string
description: |
Actionable recommendations for tool usage, short actionable recommendations.
Examples: "Use specific multi-word queries like 'Python asyncio tutorial'; add qualifiers like 'guide', 'docs', 'example'"
merge_op: patch
- name: guidelines
type: string
description: |
Tool usage guidelines, complete usage guide content.
Must include exact English headings:
- "## Guidelines" - best practices
- "### Good Cases" - successful usage examples
- "### Bad Cases" - failed usage examples
Headings must be in English, content can be in target language.
merge_op: patch
skills.yaml
name: skills
description: |
Skill execution memory - captures "how this skill is executed, what works well, and what doesn't".
Extract skill execution patterns, statistics, and learnings from skill usage in conversation.
For each skill, track: how many times it's been executed, success rate, what it's best for, recommended execution flow, key dependencies, common failure modes, and actionable recommendations.
Also accumulate complete guidelines with "Good Cases" and "Bad Cases" examples.
Skill memories help the agent learn from experience and execute skills more effectively over time.
directory: "viking://agent/{agent_space}/memories/skills"
filename_template: "{skill_name}.md"
content_template: |
Skill: {skill_name}
Skill Memory Context:
Based on {total_executions} historical executions:
- Success rate: {success_rate}% ({success_count} successful, {fail_count} failed)
- Best for: {best_for}
- Recommended flow: {recommended_flow}
- Key dependencies: {key_dependencies}
- Common failures: {common_failures}
- Recommendation: {recommendation}
{guidelines}
fields:
- name: skill_name
type: string
description: |
Skill name, copied exactly from [ToolCall] if skill_name is present, otherwise inferred from conversation context.
Used to uniquely identify this skill memory.
Examples: "create_presentation", "analyze_code", "write_document"
merge_op: immutable
- name: total_executions
type: int64
description: |
Total number of skill executions, accumulated from historical statistics.
Used to calculate success rate.
merge_op: sum
- name: success_count
type: int64
description: |
Number of successful skill executions, accumulated from historical statistics.
Counts successful executions.
merge_op: sum
- name: fail_count
type: int64
description: |
Number of failed skill executions, accumulated from historical statistics.
Counts failed executions.
merge_op: sum
- name: best_for
type: string
description: |
Best use cases for the skill, describing in what scenarios this skill works best.
Examples: "Slide creation tasks with clear topic and target audience"
merge_op: patch
- name: recommended_flow
type: string
description: |
Recommended execution flow for the skill, describing the best steps to execute this skill.
Examples: "1. Confirm topic and audience → 2. Collect reference materials → 3. Generate outline → 4. Create slides → 5. Refine content"
merge_op: patch
- name: key_dependencies
type: string
description: |
Key dependencies/prerequisites for the skill, describing what prerequisites and inputs are needed to execute this skill.
Examples: "Clear topic (e.g., 'Q3 project update', 'Python tutorial'); Target audience (e.g., 'executives', 'beginners'); Reference materials (optional but recommended)"
merge_op: patch
- name: common_failures
type: string
description: |
Common failure modes of the skill, describing problems and error patterns this skill frequently encounters.
Examples: "Vague topic like 'make a PPT' leads to multiple rework cycles; Missing audience info causes style mismatch; No reference materials results in generic content"
merge_op: patch
- name: recommendation
type: string
description: |
Actionable recommendations for skill usage, short actionable recommendations.
Examples: "Always confirm topic and audience before starting; Collect 2-3 reference materials for better quality"
merge_op: patch
- name: guidelines
type: string
description: |
Skill usage guidelines, complete usage guide content.
Must include exact English headings:
- "## Guidelines" - best practices
- "### Good Cases" - successful usage examples
- "### Bad Cases" - failed usage examples
Headings must be in English, content can be in target language.
merge_op: patch
Memory File Storage Format Example / 记忆文件存储格式示例:
Tool: web_search
Static Description:
"Searches the web for information"
Tool Memory Context:
Based on 100 historical calls:
- Success rate: 92.0% (92 successful, 8 failed)
- Avg time: 1.2s, Avg tokens: 1500
- Best for: Technical documentation, tutorials, API references
- Optimal params: max_results=5-20, timeout>30s
- Common failures: Single-word queries return irrelevant results
- Recommendation: Use specific multi-word queries
## Guidelines
...
### Good Cases
...
### Bad Cases
...
<!-- MEMORY_FIELDS
{
"tool_name": "web_search",
"static_desc": "Searches the web for information",
"total_calls": 100,
"success_count": 92,
"fail_count": 8,
"total_time_ms": 120000,
"total_tokens": 150000,
"best_for": "Technical documentation, tutorials, API references",
"optimal_params": "max_results=5-20, timeout>30s",
"common_failures": "Single-word queries return irrelevant results",
"recommendation": "Use specific multi-word queries",
"guidelines": "## Guidelines\n...\n\n### Good Cases\n...\n\n### Bad Cases\n..."
}
-->
┌─────────────────────────────────────────────────────────────────┐
│ Phase 0: Pre-fetch (System) │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ System automatically executes before LLM reasoning: │ │
│ │ 1. ls: Get all memory directory structures │ │
│ │ 2. read: Read all .abstract.md (L0) and .overview.md (L1)│ │
│ │ 3. search: Perform one semantic search in all directories│ │
│ │ Output: Pre-fetched Context (dir + L0/L1 + search results)│ │
│ └───────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────────┐
│ Phase 1: Reasoning + Action (LLM + Optional Reads) │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ LLM: Analyze conversation + Pre-fetched Context │ │
│ │ Output: Reasoning + Actions (optional additional reads) │ │
│ │ - Reasoning: What memories need to change │ │
│ │ - Actions: Additional read operations (only if needed) │ │
│ └───────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────────┐
│ Phase 2: Generate Operations (Final Output) │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ LLM: Generate final operations based on existing memories │ │
│ │ Output: MemoryOperations │ │
│ │ - WriteOp: New memory data (create or full replace) │ │
│ │ - EditOp: patch (SEARCH/REPLACE) incremental update │ │
│ │ - DeleteOp: URI to delete │ │
│ │ [Note] This is model's final output, directly executed │ │
│ └───────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────────┐
│ Phase 3: System Execute │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ MemoryUpdater: Directly execute MemoryOperations │ │
│ │ - Write to OpenViking Storage (L0/L1/L2) │ │
│ │ - Update VikingDB vector index │ │
│ └───────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
Field properties / 字段属性:
Special fields / 特殊字段:
content: Default Markdown content field, only this field is used in simple mode / 默认的 Markdown 内容字段,简单模式下只用这个字段abstract: L0 summary field (one-sentence summary for indexing) / L0 摘要字段(用于索引的一句话摘要)overview: L1 overview field (structured Markdown summary) / L1 概览字段(结构化 Markdown 摘要)Type properties / 类型属性:
content_template example (for tools type) / content_template 示例(用于 tools 类型):
content_template: |
# Tool: {tool_name}
Tool Memory Context:
Based on {total_calls} historical calls:
- Success rate: {success_rate}% ({success_count} successful, {fail_count} failed)
- Best for: {best_for}
- Common failures: {common_failures}
- Recommendation: {recommendation}
{content}
Data properties / 数据属性:
ReasoningAction:
- reasoning: str # LLM's thinking string (natural language description)
- memory_changes: List[MemoryChange] # Memories that need changes
- actions: List[ReadAction] # Read operations to execute
MemoryChange:
- change_type: write/edit/delete
- memory_type: Memory type
- uri: Memory URI (if known)
- reason: Reason for change
ReadAction:
- action_type: read/find/ls
- params: Dict # Call parameters
Available Tools (consistent with OpenViking VikingFS API) / 可使用的工具(与 OpenViking VikingFS API 保持一致):
read
uri: str, offset: int = 0, limit: int = -1find
query: str, target_uri: str = "", limit: int = 10, score_threshold: Optional[float] = None, filter: Optional[Dict] = Nonels
uri: str, output: str = "agent", abs_limit: int = 256, show_all_hidden: bool = False, node_limit: int = 1000tree
uri: str = "viking://", output: str = "agent", abs_limit: int = 256, show_all_hidden: bool = False, node_limit: int = 1000, level_limit: int = 3Important / 重要: No function calls like add_memory/update_memory/delete_memory, add/edit/delete operations are model's final output as MemoryOperations, directly executed by system / 没有 add_memory/update_memory/delete_memory 这样的 function call,增删改操作通过模型最终输出 MemoryOperations,由系统直接执行
MemoryOperations:
- write_operations: List[WriteOp]
- edit_operations: List[EditOp]
- delete_operations: List[DeleteOp]
WriteOp:
- uri: Target memory URI / 目标记忆的 URI
- memory_data: MemoryData
EditOp:
- uri: Target memory URI / 目标记忆的 URI
- patches: Field-level updates / 字段级更新
DeleteOp:
- uri: Memory URI to delete / 要删除的记忆 URI
Patch format / Patch 格式:
<<<<<<< SEARCH
:start_line:10
-------
Original content / 原始内容
=======
New content / 新内容
>>>>>>> REPLACE
Handle according to field's merge_op / 根据字段的 merge_op 处理:
Features / 功能:
openviking/session/memory/schemas/ / 默认从 openviking/session/memory/schemas/ 加载内置类型Optimization Strategy / 优化策略: To avoid excessive time from multiple ReAct rounds, system automatically performs pre-fetch before LLM reasoning / 为避免多次 ReAct 导致耗时过长,系统在 LLM 推理前自动执行前置读取:
Phase 0: Pre-fetch (system executes, before LLM reasoning) / 阶段 0: Pre-fetch(系统执行,LLM 推理前):
viking://user/{user_space}/memories/ and subdirectories / 及子目录viking://agent/{agent_space}/memories/ and subdirectories / 及子目录.abstract.md (L0) and .overview.md (L1) / 读取所有 .abstract.md (L0) 和 .overview.md (L1)
Phase 1: Reasoning + Action: LLM analyzes conversation + Pre-fetched Context, outputs ReasoningAction / LLM 分析对话 + Pre-fetched Context,输出 ReasoningAction
Phase 2: Generate Operations: LLM generates MemoryOperations based on existing memories (final output) / LLM 基于现有记忆生成 MemoryOperations(最终输出)
Phase 3: System Execute: System directly executes MemoryOperations / 系统直接执行 MemoryOperations
Features / 功能:
Based on the implementation from ../memory project, the following tech stack is used / 基于 ../memory 项目的实现方案,采用以下技术栈:
Pydantic BaseModel: Used to define all data structures / 用于定义所有数据结构
json_repair: Fault-tolerant LLM output parsing / 容错解析 LLM 输出
Pydantic TypeAdapter: Type validation and conversion / 类型验证和转换
BaseModelCompat: Compatibility base class (refer to ../memory) / 兼容性基类(参考 ../memory)
1. format() - Build prompt / 构建提示
- prepare_instructions(): Generate input/output field descriptions / 生成输入输出字段说明
- format_turn(): Format user/assistant messages / 格式化用户/助手消息
- format_fields(): Format field values / 格式化字段值
2. llm_request() - Call LLM / 调用 LLM
- Check if json_schema is supported / 检查是否支持 json_schema
- If supported: generate response_format (type: json_schema) / 如果支持:生成 response_format (type: json_schema)
- If not supported: use json_object or no format / 如果不支持:使用 json_object 或无格式
- Call LM with static_messages + dynamic_messages / 调用 LM,传入 static_messages + dynamic_messages
3. parse() - Parse output / 解析输出
- remove_trailing_content(): Remove content after JSON ends / 去除 JSON 结束后的内容
- json_repair.loads(): Fault-tolerant parsing / 容错解析
- parse_value(): Type conversion + fault tolerance / 类型转换 + 容错
- List type: filter invalid items / 列表类型:过滤无效项目
- Other types: use TypeAdapter validation / 其他类型:使用 TypeAdapter 验证
JSON Parsing Fault Tolerance / JSON 解析容错:
Type Validation Fault Tolerance / 类型验证容错:
Field Fault Tolerance / 字段容错:
File Storage: L0/L1/L2 three-level structure / L0/L1/L2 三层结构
.abstract.md - summary / 摘要.overview.md - overview / 概览URI Structure:
viking://user/{space}/memories/{category}/viking://agent/{space}/memories/{category}/Vector Index: stored in context collection of VikingDB / 存储在 VikingDB 的 context 集合中
| File / 文件 | Purpose / 作用 |
|---|---|
openviking/session/memory_extractor.py | Memory extraction main logic (~1200 lines) / 记忆提取主逻辑 (~1200行) |
openviking/session/memory_deduplicator.py | Deduplication decision (~395 lines) / 去重决策 (~395行) |
openviking/session/compressor.py | Session compressor (~447 lines) / 会话压缩器 (~447行) |
openviking/prompts/templates/compression/memory_extraction.yaml | Extraction prompt template (~400 lines) / 提取提示模板 (~400行) |
Create openviking/session/memory/memory_data.py / 创建 openviking/session/memory/memory_data.py
Create openviking/session/memory/memory_operations.py / 创建 openviking/session/memory/memory_operations.py
Create openviking/session/memory/memory_react.py / 创建 openviking/session/memory/memory_react.py
Create openviking/session/memory/memory_functions.py / 创建 openviking/session/memory/memory_functions.py
openviking/session/memory/memory_patch.py / 创建 openviking/session/memory/memory_patch.py
Create openviking/session/memory/memory_types.py / 创建 openviking/session/memory/memory_types.py
Create YAML config files / 创建 YAML 配置文件
openviking/session/memory/schemas/ directory / 放在 openviking/session/memory/schemas/ 目录openviking/session/memory/memory_updater.py / 创建 openviking/session/memory/memory_updater.py
openviking/session/memory/memory_react.py / 完善 openviking/session/memory/memory_react.py
Create LLM prompt templates / 创建 LLM 提示模板
Create new memory_extractor_v2.py / 创建新的 memory_extractor_v2.py
Unit tests / 单元测试
Integration tests / 集成测试
openviking/session/
├── memory_extractor_v2.py # New memory extractor (replaces existing) / 新的记忆提取器(替换现有)
openviking/session/memory/
├── memory_data.py # Core data structures / 核心数据结构
├── memory_operations.py # Three operation definitions (LLM final output) / 三种操作定义(模型最终输出)
├── memory_react.py # ReAct orchestrator / ReAct 编排器
├── memory_functions.py # Function call definitions (read/find/ls) / Function call 定义(read/find/ls)
├── memory_patch.py # Patch handler / Patch 处理器
├── memory_types.py # Type registry / 类型注册表
├── memory_updater.py # Patch applier (system execution) / Patch 应用器(系统执行)
└── schemas/ # Config directory / 配置目录
├── profile.yaml # directory: viking://user/{user_space}/memories
├── preferences.yaml # directory: viking://user/{user_space}/memories/preferences
├── entities.yaml # directory: viking://user/{user_space}/memories/entities
├── events.yaml # directory: viking://user/{user_space}/memories/events
├── cases.yaml # directory: viking://agent/{agent_space}/memories/cases
├── patterns.yaml # directory: viking://agent/{agent_space}/memories/patterns
├── tools.yaml # directory: viking://agent/{agent_space}/memories/tools
└── skills.yaml # directory: viking://agent/{agent_space}/memories/skills
tests/session/memory/
├── test_memory_data.py
├── test_memory_operations.py
├── test_memory_react.py
├── test_memory_patch.py
├── test_memory_types.py
└── test_memory_updater.py
openviking/session/memory_extractor.py # Preserve existing version, do not modify / 保留现有版本,不修改
Purpose / 目的: Avoid excessive time from multiple ReAct rounds / 避免多次 ReAct 导致耗时过长
System automatically executes / 系统自动执行:
viking://user/{user_space}/memories/ and subdirectories / 及子目录viking://agent/{agent_space}/memories/ and subdirectories / 及子目录.abstract.md (L0) and .overview.md (L1) / 读取所有 .abstract.md (L0) 和 .overview.md (L1)Output / 输出: Pre-fetched Context (directory structure + L0/L1 summaries + search results) / Pre-fetched Context(目录结构 + L0/L1 摘要 + 搜索结果)
Input / 输入: Conversation history + Pre-fetched Context / 对话历史 + Pre-fetched Context
LLM Task / LLM 任务: Analyze conversation + Pre-fetched Context, identify memories that need changes + decide if additional reads are needed / 分析对话 + Pre-fetched Context,识别需要变更的记忆 + 决定是否需要额外读取
Output / 输出: ReasoningAction (reasoning + optional additional actions) / ReasoningAction(reasoning + 可选的额外 actions)
System Execution / 系统执行: Execute additional read operations in actions (if needed) / 执行 actions 中的额外读取操作(如需要)
Input / 输入: Conversation history + Reasoning + read existing memories / 对话历史 + Reasoning + 读取的现有记忆
LLM Task / LLM 任务: Generate specific operations based on existing memories / 基于现有记忆生成具体的操作
Output / 输出: MemoryOperations (model final output) / MemoryOperations(模型最终输出)
Important / 重要: This is the model's final output, directly executed by system, no more function calls / 这是模型的最后输出,直接由系统执行,不再通过 function call
Input / 输入: MemoryOperations
Execution / 执行: MemoryUpdater.apply_operations() directly applies patch, writes to OpenViking Storage / MemoryUpdater.apply_operations() 直接应用 patch,写入 OpenViking Storage
This design is based on practices from ../memory project, uses ReAct pattern: / 本设计基于 ../memory 项目的实践,采用 ReAct 模式: