docs/public/progressive-disclosure.mdx
Show what exists and its retrieval cost first. Let the agent decide what to fetch based on relevance and need.
Progressive disclosure is an information architecture pattern where you reveal complexity gradually rather than all at once. In the context of AI agents, it means:
This mirrors how humans work: We scan headlines before reading articles, review table of contents before diving into chapters, and check file names before opening files.
Traditional RAG (Retrieval-Augmented Generation) systems fetch everything upfront:
❌ Traditional Approach:
┌─────────────────────────────────────┐
│ Session Start │
│ │
│ [15,000 tokens of past sessions] │
│ [8,000 tokens of observations] │
│ [12,000 tokens of file summaries] │
│ │
│ Total: 35,000 tokens │
│ Relevant: ~2,000 tokens (6%) │
└─────────────────────────────────────┘
Problems:
✅ Progressive Disclosure Approach:
┌─────────────────────────────────────┐
│ Session Start │
│ │
│ Index of 50 observations: ~800 tokens│
│ ↓ │
│ Agent sees: "🔴 Hook timeout issue" │
│ Agent decides: "Relevant!" │
│ ↓ │
│ Fetch observation #2543: ~120 tokens│
│ │
│ Total: 920 tokens │
│ Relevant: 920 tokens (100%) │
└─────────────────────────────────────┘
Benefits:
Every SessionStart hook provides a compact index:
### Oct 26, 2025
**General**
| ID | Time | T | Title | Tokens |
|----|------|---|-------|--------|
| #2586 | 12:58 AM | 🔵 | Context hook file exists but is empty | ~51 |
| #2587 | ″ | 🔵 | Context hook script file is empty | ~46 |
| #2589 | ″ | 🟡 | Investigated hook debug output docs | ~105 |
**src/hooks/context-hook.ts**
| ID | Time | T | Title | Tokens |
|----|------|---|-------|--------|
| #2591 | 1:15 AM | ⚖️ | Stderr messaging abandoned | ~155 |
| #2592 | 1:16 AM | ⚖️ | Web UI strategy redesigned | ~193 |
What the agent sees:
🎯 session-request - User's original goal
🔴 gotcha - Critical edge case or pitfall
🟡 problem-solution - Bug fix or workaround
🔵 how-it-works - Technical explanation
🟢 what-changed - Code/architecture change
🟣 discovery - Learning or insight
🟠 why-it-exists - Design rationale
🟤 decision - Architecture decision
⚖️ trade-off - Deliberate compromise
Purpose:
The index includes usage guidance:
💡 **Progressive Disclosure:** This index shows WHAT exists and retrieval COST.
- Use MCP search tools to fetch full observation details on-demand
- Prefer searching observations over re-reading code for past decisions
- Critical types (🔴 gotcha, 🟤 decision, ⚖️ trade-off) often worth fetching immediately
What this does:
Think of context window as a bank account:
| Approach | Metaphor | Outcome |
|---|---|---|
| Dump everything | Spending your entire paycheck on groceries you might need someday | Waste, clutter, can't afford what you actually need |
| Fetch nothing | Refusing to spend any money | Starvation, can't accomplish tasks |
| Progressive disclosure | Check your pantry, make a shopping list, buy only what you need | Efficiency, room for unexpected needs |
LLMs have finite attention:
Claude-Mem's approach:
"As models improve, let them act intelligently"
Progressive disclosure treats the agent as an intelligent information forager, not a passive recipient of pre-selected context.
Traditional RAG:
System → [Decides relevance] → Agent
↑
Hope this helps!
Progressive Disclosure:
System → [Shows index] → Agent → [Decides relevance] → [Fetches details]
↑
You know best!
The agent knows:
We don't.
Every item in the index shows token count:
| #2591 | 1:15 AM | ⚖️ | Stderr messaging abandoned | ~155 |
^^^^
Retrieval cost
Why:
Titles compress full observations into ~10 words:
Bad title:
Observation about a thing
Good title:
🔴 Hook timeout issue: 60s default too short for npm install
What makes a good title:
Observations are grouped by:
**src/hooks/context-hook.ts**
| ID | Time | T | Title | Tokens |
|----|------|---|-------|--------|
| #2591 | 1:15 AM | ⚖️ | Stderr messaging abandoned | ~155 |
| #2594 | 1:17 AM | 🟠 | Removed stderr section from docs | ~93 |
Benefit: If agent is working on src/hooks/context-hook.ts, related observations are already grouped together.
The index is useless without retrieval mechanisms:
*Use claude-mem MCP search to access records with the given ID*
Available MCP tools:
search - Search memory index (Layer 1: Get IDs)timeline - Get chronological context (Layer 2: See narrative arc)get_observations - Fetch full details (Layer 3: Deep dive)The 3-layer workflow ensures progressive disclosure: index → context → details.
Without progressive disclosure:
SessionStart injects 25,000 tokens of past context
Agent reads everything
Agent finds 1 relevant observation (buried in middle)
Total tokens consumed: 25,000
Relevant tokens: ~200
Efficiency: 0.8%
With progressive disclosure:
SessionStart shows index: ~800 tokens
Agent sees title: "🔴 Hook timeout issue: 60s too short"
Agent thinks: "This looks relevant to my bug!"
Agent fetches observation #2543: ~155 tokens
Total tokens consumed: 955
Relevant tokens: 955
Efficiency: 100%
| #2543 | 2:14 PM | 🔴 | Hook timeout: 60s too short for npm install | ~155 |
What the agent learns WITHOUT fetching:
Decision tree:
Is my task related to hooks? → YES
Is my task related to timeouts? → YES
Is my task related to npm? → YES
155 tokens is cheap → FETCH IT
Claude-Mem implements progressive disclosure through a 3-layer workflow pattern:
Start by searching to get a compact index with IDs:
search({
query: "hook timeout",
limit: 10
})
Returns:
Found 3 observations matching "hook timeout":
| ID | Date | Type | Title |
|----|------|------|-------|
| #2543 | Oct 26 | gotcha | Hook timeout: 60s too short |
| #2891 | Oct 25 | how-it-works | Hook timeout configuration |
| #2102 | Oct 20 | problem-solution | Fixed timeout in CI |
Cost: ~50-100 tokens per result Value: Agent can scan and decide which observations are relevant
Get chronological context around interesting observations:
timeline({
anchor: 2543, // Observation ID from search
depth_before: 3,
depth_after: 3
})
Returns: Chronological view showing what happened before/during/after observation #2543
Cost: Variable based on depth Value: Understand narrative arc and context
Fetch full details only for relevant observations:
get_observations({
ids: [2543, 2102] // Selected from search results
})
Returns:
#2543 🔴 Hook timeout: 60s too short for npm install
─────────────────────────────────────────────────
Date: Oct 26, 2025 2:14 PM
Type: gotcha
Project: claude-mem
Narrative:
Discovered that the default 60-second hook timeout is insufficient
for npm install operations, especially with large dependency trees
or slow network conditions. This causes SessionStart hook to fail
silently, preventing context injection.
Facts:
- Default timeout: 60 seconds
- npm install with cold cache: ~90 seconds
- Configured timeout: 120 seconds in plugin/hooks/hooks.json:25
Files Modified:
- plugin/hooks/hooks.json
Concepts: hooks, timeout, npm, configuration
Cost: ~155 tokens for full details Value: Complete understanding of the issue
Progressive disclosure is grounded in Cognitive Load Theory:
The inherent difficulty of the task itself.
Example: "Fix authentication bug"
This load is unavoidable.
The cognitive burden of poorly presented information.
Traditional RAG adds extraneous load:
Progressive disclosure minimizes extraneous load:
The effort of building mental models and schemas.
Progressive disclosure supports germane load:
Bad:
| #2543 | 2:14 PM | 🔴 | Investigation into the issue where hooks time out | ~155 |
Good:
| #2543 | 2:14 PM | 🔴 | Hook timeout: 60s too short for npm install | ~155 |
Bad:
| #2543 | 2:14 PM | 🔴 | Hook timeout issue |
Good:
| #2543 | 2:14 PM | 🔴 | Hook timeout issue | ~155 |
Bad:
Here are 10 observations. [No instructions on how to get full details]
Good:
Here are 10 observations.
*Use MCP search tools to fetch full observation details on-demand*
Bad:
// Fetching full details immediately
get_observations({
ids: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] // Guessing which are relevant
})
Good:
// Follow the 3-layer workflow
// Layer 1: Search for index
search({
query: "hooks",
limit: 20
})
// Layer 2: Review index, identify 2-3 relevant IDs
// Layer 3: Fetch only relevant observations
get_observations({
ids: [2543, 2891] // Just the most relevant
})
Decision: Show approximate token counts (~155, ~203) rather than exact counts.
Rationale:
Decision: Use emoji icons (🔴, 🟡, 🔵) rather than text (GOTCHA, PROBLEM, HOWTO).
Rationale:
Decision: Always show index first, even if we "know" what's relevant.
Rationale:
Decision: Group observations by file path in addition to date.
Rationale:
Progressive disclosure is working when:
Relevant Tokens / Total Context Tokens > 80%
Most of the context consumed is actually useful.
Index Shown: 50 observations
Details Fetched: 2-3 observations
Agent is being selective, not fetching everything.
Session with index: 30 seconds to find relevant context
Session without: 90 seconds scanning all context
Time-to-relevant-information is faster.
Simple task: Only index needed
Medium task: 1-2 observations fetched
Complex task: 5-10 observations + code reads
Depth scales with task complexity.
// Vary index size based on session type
SessionStart({ source: "startup" }):
→ Show last 10 sessions (small index)
SessionStart({ source: "resume" }):
→ Show only current session (micro index)
SessionStart({ source: "compact" }):
→ Show last 20 sessions (larger index)
// Use embeddings to pre-sort index by relevance
search({
query: "authentication bug",
orderBy: "relevance" // Based on semantic similarity (future enhancement)
})
💡 **Budget Estimate:**
- Fetching all 🔴 gotchas: ~450 tokens
- Fetching all file-related: ~1,200 tokens
- Fetching everything: ~8,500 tokens
Layer 1: Index (titles only)
Layer 2: Summaries (2-3 sentences)
Layer 3: Full details (complete observation)
Layer 4: Source files (referenced code)
"The best interface is one that disappears when not needed, and appears exactly when it is."
Progressive disclosure respects the agent's intelligence and autonomy. We provide the map; the agent chooses the path.
This philosophy emerged from real-world usage of Claude-Mem across hundreds of coding sessions. The pattern works because it aligns with both human cognition and LLM attention mechanics.