packages/kilo-vscode/src/services/autocomplete/continuedev/ARCHITECTURE.md
Technical architecture documentation for the ContinueDev library within the Kilocode monorepo.
Note: This library is a streamlined extraction from the Continue project, containing only autocomplete and NextEdit functionality. All GUI, chat, agents, and other features have been removed. It's integrated into Kilocode as a TypeScript service library at
src/services/continuedev/.
The library is architected as a layered system with clear separation of concerns:
┌─────────────────────────────────────────────────────────────┐
│ IDE Integration Layer │
│ (VSCode, JetBrains, Custom IDEs) │
└────────────────────────┬────────────────────────────────────┘
│ IDE Interface
│
┌────────────────────────▼────────────────────────────────────┐
│ Provider Layer │
│ ┌──────────────────────┐ ┌──────────────────────────┐ │
│ │ CompletionProvider │ │ NextEditProvider │ │
│ │ (Autocomplete) │ │ (Edit Prediction) │ │
│ └──────────┬───────────┘ └───────────┬──────────────┘ │
└─────────────┼────────────────────────────┼─────────────────┘
│ │
┌─────────────▼────────────────────────────▼─────────────────┐
│ Core Services Layer │
│ ┌────────────────┐ ┌─────────────┐ ┌─────────────────┐ │
│ │ Context │ │ LLM │ │ Configuration │ │
│ │ Retrieval │ │ Integration │ │ Management │ │
│ └────────────────┘ └─────────────┘ └─────────────────┘ │
└──────────────────────────────────────────────────────────────┘
│ │
┌─────────────▼────────────────────────────▼─────────────────┐
│ Utility & Infrastructure │
│ ┌────────────┐ ┌──────────┐ ┌───────────┐ ┌─────────┐ │
│ │Tree-sitter │ │ Diff │ │ Caching │ │ Logging │ │
│ │ (AST) │ │ Engine │ │ (LRU) │ │ Service │ │
│ └────────────┘ └──────────┘ └───────────┘ └─────────┘ │
└──────────────────────────────────────────────────────────────┘
IDE interfacegraph TB
subgraph "Autocomplete System"
CP[CompletionProvider]
subgraph "Input Processing"
PRE[Prefiltering]
DEB[Debouncer]
HELP[HelperVars]
end
subgraph "Context Gathering"
CTX[ContextRetrievalService]
SNIP[Snippet Retrieval]
LSP[LSP Integration]
end
subgraph "Generation"
TMPL[Template Rendering]
STRM[CompletionStreamer]
LLM[LLM Interface]
end
subgraph "Post-Processing"
FILT[Filtering]
BRKT[Bracket Matching]
POST[Postprocessing]
end
subgraph "Supporting Services"
CACHE[LRU Cache]
LOG[Logging Service]
MULTI[Multiline Detection]
end
end
CP --> PRE
PRE --> HELP
HELP --> CTX
CTX --> SNIP
CTX --> LSP
SNIP --> TMPL
TMPL --> STRM
STRM --> LLM
LLM --> FILT
FILT --> BRKT
BRKT --> POST
POST --> CACHE
POST --> LOG
HELP --> MULTI
DEB --> CP
CompletionProvider (core/autocomplete/CompletionProvider.ts)
ContextRetrievalService (core/autocomplete/context/ContextRetrievalService.ts)
CompletionStreamer (core/autocomplete/generation/CompletionStreamer.ts)
BracketMatchingService (core/autocomplete/filtering/BracketMatchingService.ts)
graph TB
subgraph "NextEdit System"
NEP[NextEditProvider]
subgraph "Model Providers"
BASE[BaseNextEditProvider]
INST[InstinctProvider]
MERC[MercuryCoderProvider]
FACT[NextEditProviderFactory]
end
subgraph "Edit Calculation"
CALC[EditableRegionCalculator]
HIST[DocumentHistoryTracker]
DIFF[Diff Engine]
end
subgraph "Context & Prompts"
PCTX[Prompt Context Builder]
PMETA[Prompt Metadata]
UEDITS[User Edits Capture]
end
subgraph "Output Processing"
EXTRACT[Completion Extraction]
CURSOR[Cursor Positioning]
OUTCOME[Outcome Builder]
end
subgraph "Supporting Services"
PREFETCH[Prefetch Queue]
NLOG[NextEdit Logging]
end
end
NEP --> FACT
FACT --> BASE
BASE --> INST
BASE --> MERC
NEP --> CALC
CALC --> HIST
NEP --> PCTX
PCTX --> PMETA
PMETA --> UEDITS
PCTX --> LLM[LLM Interface]
LLM --> EXTRACT
EXTRACT --> DIFF
DIFF --> CURSOR
CURSOR --> OUTCOME
NEP --> PREFETCH
NEP --> NLOG
NextEditProvider (core/nextEdit/NextEditProvider.ts)
BaseNextEditProvider (core/nextEdit/providers/BaseNextEditProvider.ts)
NextEditEditableRegionCalculator (core/nextEdit/NextEditEditableRegionCalculator.ts)
DocumentHistoryTracker (core/nextEdit/DocumentHistoryTracker.ts)
User Types
│
▼
┌─────────────────────┐
│ IDE Integration │
│ • Captures input │
│ • Gets cursor pos │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Debouncer │
│ • Delays request │
│ • Prevents spam │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Prefiltering │
│ • Early checks │
│ • Security check │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Context Gathering │
│ • File contents │
│ • Recent edits │
│ • LSP definitions │
│ • Snippets │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Prompt Building │
│ • Tokenization │
│ • Template render │
│ • Token limiting │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ LLM Generation │
│ • Stream request │
│ • Token by token │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Real-time Filter │
│ • Stop sequences │
│ • Invalid syntax │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Post-processing │
│ • Format clean-up │
│ • Bracket match │
│ • Cache storage │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Display to User │
│ • Ghost text │
│ • Inline display │
└─────────────────────┘
User Makes Edit
│
▼
┌─────────────────────┐
│ History Tracking │
│ • Capture change │
│ • Store in history │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Trigger Condition │
│ • Selection change │
│ • Edit completion │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Region Calculation │
│ • Editable range │
│ • Window size │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Context Building │
│ • File contents │
│ • Recent edits │
│ • User excerpts │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Prompt Generation │
│ • Model-specific │
│ • Format context │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ LLM Prediction │
│ • Generate edits │
│ • Full response │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Response Parsing │
│ • Extract code │
│ • Parse structure │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Diff Calculation │
│ • Myers algorithm │
│ • Line-by-line │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Cursor Positioning │
│ • Calculate new pos│
│ • Apply offsets │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Display Edits │
│ • Visual feedback │
│ • Jump navigation │
└─────────────────────┘
Autocomplete and NextEdit both rely on gathering relevant context:
Context Types:
Context Retrieval Strategy:
// Prioritized context gathering
1. Current file context (prefix + suffix)
2. Recently edited ranges in current file
3. Recently edited files in workspace
4. LSP definitions at cursor
5. Similar code snippets from workspace
6. Imports and dependencies
Tree-sitter provides accurate syntax-aware code analysis:
Usage:
Query Files (core/tag-qry/):
The library abstracts LLM communication through the ILLM interface:
Supported Operations:
Provider Implementation:
// LLM providers implement ILLM interface
class OpenAI implements ILLM {
async *streamFim(prefix: string, suffix: string, signal: AbortSignal) {
// Call OpenAI API with FIM format
// Stream tokens back
for await (const token of stream) {
yield token
}
}
}
LRU Cache (core/autocomplete/util/AutocompleteLruCache.ts):
Cache Key: Normalized prefix (whitespace-stripped, lowercased)
Cache Hit Flow:
Request → Check Cache → Hit? → Return Cached
↓ Miss
Generate → Store → Return
Myers Diff (core/diff/myers.ts):
Stream Diff (core/diff/streamDiff.ts):
Prefiltering (core/autocomplete/prefiltering/):
Real-time Filtering:
Post-filtering:
Algorithm (core/autocomplete/classification/shouldCompleteMultiline.ts):
Heuristics:
core/
├── autocomplete/ # Autocomplete feature
│ ├── CompletionProvider.ts # Main orchestrator
│ ├── MinimalConfig.ts # Configuration provider
│ ├── types.ts # Type definitions
│ │
│ ├── classification/ # Completion classification
│ │ └── shouldCompleteMultiline.ts
│ │
│ ├── context/ # Context gathering
│ │ ├── ContextRetrievalService.ts
│ │ └── ... (context providers)
│ │
│ ├── filtering/ # Completion filtering
│ │ ├── BracketMatchingService.ts
│ │ └── ... (filters)
│ │
│ ├── generation/ # LLM completion generation
│ │ ├── CompletionStreamer.ts
│ │ └── ... (generation utils)
│ │
│ ├── postprocessing/ # Post-processing
│ │ └── index.ts
│ │
│ ├── prefiltering/ # Pre-filtering
│ │ └── index.ts
│ │
│ ├── snippets/ # Code snippet retrieval
│ │ ├── index.ts
│ │ └── types.ts
│ │
│ ├── templating/ # Prompt templates
│ │ └── index.ts
│ │
│ └── util/ # Autocomplete utilities
│ ├── AutocompleteDebouncer.ts
│ ├── AutocompleteLruCache.ts
│ ├── AutocompleteLoggingService.ts
│ ├── HelperVars.ts
│ └── types.ts
│
├── nextEdit/ # NextEdit feature
│ ├── NextEditProvider.ts # Main provider
│ ├── NextEditProviderFactory.ts
│ ├── NextEditEditableRegionCalculator.ts
│ ├── DocumentHistoryTracker.ts
│ ├── NextEditPrefetchQueue.ts
│ ├── NextEditLoggingService.ts
│ ├── types.ts
│ ├── utils.ts
│ │
│ ├── providers/ # Model-specific providers
│ │ ├── BaseNextEditProvider.ts
│ │ ├── InstinctNextEditProvider.ts
│ │ └── MercuryCoderNextEditProvider.ts
│ │
│ └── diff/ # Diff utilities for NextEdit
│ └── diff.ts
│
├── llm/ # LLM integration
│ ├── index.ts
│ ├── messages.ts # Message formatting
│ ├── countTokens.ts # Token counting
│ │
│ └── llms/ # LLM implementations
│ ├── OpenAI.ts
│ └── ... (other providers)
│
├── diff/ # Diff algorithms
│ ├── myers.ts # Myers diff algorithm
│ ├── streamDiff.ts # Streaming diff
│ └── util.ts # Diff utilities
│
├── util/ # Shared utilities
│ ├── treeSitter.ts # Tree-sitter integration
│ ├── text.ts # Text manipulation
│ ├── history.ts # History management
│ ├── withExponentialBackoff.ts
│ └── ... (other utilities)
│
├── indexing/ # Code indexing
│ ├── ignore.ts # .continueignore handling
│ ├── continueignore.ts # Ignore pattern parsing
│ └── ... (indexing utils)
│
├── fetch/ # HTTP utilities
│ ├── fetch.ts # Fetch wrapper
│ ├── stream.ts # Stream handling
│ └── ... (fetch utilities)
│
├── tag-qry/ # Tree-sitter queries
│ ├── tree-sitter-typescript-tags.scm
│ ├── tree-sitter-python-tags.scm
│ └── ... (language queries)
│
└── vscode-test-harness/ # VSCode integration example
├── src/
│ ├── VsCodeIde.ts # IDE implementation
│ ├── autocomplete/ # Autocomplete integration
│ │ ├── completionProvider.ts
│ │ ├── GhostTextAcceptanceTracker.ts
│ │ └── ...
│ └── activation/ # NextEdit UI managers
│ ├── NextEditWindowManager.ts
│ ├── JumpManager.ts
│ └── SelectionChangeManager.ts
└── test/ # Integration tests
├── ContinueCompletionProvider.vitest.ts
├── GhostTextAcceptanceTracker.vitest.ts
└── ... (test files)
The library is designed to be extensible at multiple levels:
Implement the IDE interface to integrate with any editor:
class MyCustomIDE implements IDE {
async readFile(filepath: string): Promise<string> {
// Your implementation
}
async getWorkspaceDirs(): Promise<string[]> {
// Your implementation
}
// ... implement all required methods
}
Key Methods to Customize:
Implement the ILLM interface for custom LLM backends:
class MyCustomLLM implements ILLM {
async *streamFim(prefix: string, suffix: string, signal: AbortSignal) {
// Stream from your LLM
for await (const token of yourLLMStream(prefix, suffix)) {
yield token
}
}
// ... implement other methods
}
Supported LLM Operations:
Extend context gathering with custom providers:
class CustomContextProvider {
async getContext(filepath: string, position: Position): Promise<string> {
// Fetch custom context (e.g., from a database, API)
return customContext
}
}
Integration Point: Modify ContextRetrievalService to include custom providers
Add custom filters to the completion pipeline:
function customFilter(completion: string, context: HelperVars): boolean {
// Return false to reject completion
return isValid(completion)
}
Integration Point: Add to CompletionStreamer filter chain
Create model-specific NextEdit providers:
class CustomNextEditProvider extends BaseNextEditProvider {
async generatePrompts(context: ModelSpecificContext): Promise<Prompt[]> {
// Build custom prompts for your model
}
extractCompletion(message: string): string {
// Extract completion from model response
}
// ... implement other required methods
}
Registration:
// Add to NextEditProviderFactory
NextEditProviderFactory.registerProvider("my-model", CustomNextEditProvider)
Problem: Original Continue used a complex config system with YAML parsing, control-plane integration, and profile management.
Solution: Created MinimalConfigProvider with hardcoded defaults.
Benefits:
Reasoning:
Implementation: Separate providers that can be used together or independently
Benefits:
Trade-off: More complex implementation vs. simpler batch processing
Benefits:
Trade-off: Additional dependency, but worth it for accuracy
Reasoning:
Configuration: Cache size configurable via TabAutocompleteOptions
Benefits:
Trade-off: Some IDE-specific optimizations harder to implement
Reasoning:
Pattern: Factory pattern with model detection
core/
├── **/*.test.ts # Jest unit tests (legacy)
├── **/*.vitest.ts # Vitest unit tests (current)
└── vscode-test-harness/
└── test/
└── *.vitest.ts # Integration tests
Mock Factories (core/autocomplete/util/completionTestUtils.ts):
Typical Autocomplete Request:
Total: 200-500ms
├── Debounce wait: 150ms
├── Context gathering: 10-30ms
├── Prompt building: 5-10ms
├── LLM call: 50-200ms
└── Post-processing: 5-10ms
Typical NextEdit Request:
Total: 500-2000ms
├── Region calculation: 10-20ms
├── Context building: 20-50ms
├── LLM call: 400-1800ms
└── Diff calculation: 10-50ms
isSecurityConcern).continueignore)Potential areas for extension: