packages/llmz/DOCS.md
LLMz: A Revolutionary TypeScript AI Agent Framework
Stop chaining tools. Start generating real code.
LLMz is a revolutionary TypeScript AI agent framework that fundamentally changes how AI agents work. Like other agent frameworks, LLMz calls LLM models in a loop to achieve desired outcomes with access to tools and memory. However, LLMz is code-first – meaning it generates and runs TypeScript code in a sandbox rather than using traditional JSON tool calling.
Traditional agent frameworks rely on JSON tool calling, which has significant limitations:
LLMz leverages the fact that models have been trained extensively on millions of TypeScript codebases, making them incredibly reliable at generating working code. This enables:
LLMz operates as an LLM-native TypeScript VM built on top of Zui (Botpress's internal schema library), battle-tested in production powering millions of AI agents worldwide.
Traditional tool-calling agents are fundamentally limited by the JSON interface between the LLM and tools. This requires multiple roundtrips for complex tasks and cannot handle conditional logic, loops, or sophisticated data processing.
LLMz solves this by letting LLMs do what they do best: generate code. Since models are trained extensively on code, they can reliably generate TypeScript that:
Traditional Tool Calling:
LLM → JSON: {"tool": "getPrice", "params": {"from": "quebec", "to": "new york"}}
System → Response: {"price": 600}
LLM → JSON: {"tool": "checkBudget", "params": {"amount": 600}}
System → Response: {"canAfford": false}
LLM → JSON: {"tool": "notifyUser", "params": {"message": "Price too high"}}
LLMz Code Generation:
// Check the ticket price and user's budget in one go
const price = await getTicketPrice({ from: 'quebec', to: 'new york' })
const budget = await getUserBudget()
if (price > budget) {
await notifyUser({ message: `Price $${price} exceeds budget $${budget}` })
return { action: 'budget_exceeded', price, budget }
} else {
const ticketId = await buyTicket({ from: 'quebec', to: 'new york' })
return { action: 'done', result: ticketId }
}
npm install llmz @botpress/client
import { execute } from 'llmz'
import { Client } from '@botpress/client'
const client = new Client({
botId: process.env.BOTPRESS_BOT_ID!,
token: process.env.BOTPRESS_TOKEN!,
})
const result = await execute({
instructions: 'Calculate the sum of numbers 1 to 100',
client,
})
if (result.isSuccess()) {
console.log('Result:', result.output)
console.log('Generated code:', result.iteration.code)
}
import { execute } from 'llmz'
import { Client } from '@botpress/client'
import { CLIChat } from './utils/cli-chat'
const client = new Client({
botId: process.env.BOTPRESS_BOT_ID!,
token: process.env.BOTPRESS_TOKEN!,
})
const chat = new CLIChat()
while (await chat.iterate()) {
await execute({
instructions: 'You are a helpful assistant',
chat,
client,
})
}
At its core, LLMz exposes a single method (execute) that runs in a loop until one of these conditions is met:
The loop automatically handles:
Every LLMz code block follows a predictable structure that LLMs can reliably generate:
At minimum, an LLMz response must contain a return statement with an Exit:
// Chat mode - give turn back to user
return { action: 'listen' }
// Worker mode - complete with result
return { action: 'done', result: calculatedValue }
Unlike traditional tool calling, LLMz enables complex logic impossible with JSON:
// Complex conditional logic and error handling
const price = await getTicketPrice({ from: 'quebec', to: 'new york' })
if (price > 500) {
throw new Error('Price too high')
} else {
const ticketId = await buyTicket({ from: 'quebec', to: 'new york' })
return { action: 'done', result: ticketId }
}
Comments help LLMs think step-by-step and plan ahead:
// Check user's budget first before proceeding with purchase
const budget = await getUserBudget()
// Only proceed if we have enough funds
if (budget >= price) {
// Purchase the ticket
const ticket = await buyTicket(ticketDetails)
}
In Chat Mode, agents can yield React components for rich user interaction:
// Multi-line text support
yield <Text>
Hello, world!
This is a second line.
</Text>
// Composed/nested components
yield <Message>
<Text>What do you prefer?</Text>
<Button>Cats</Button>
<Button>Dogs</Button>
</Message>
return { action: 'listen' }
LLMz uses a sophisticated Babel-based compilation system to transform generated code:
Key transformations include:
LLMz supports multiple execution environments:
isolated-vm for security isolationThe VM provides:
LLMz operates in two distinct modes depending on whether a chat interface is provided:
Enabled when: chat parameter is provided to execute()
Chat Mode is designed for interactive conversational agents that need to:
Key characteristics:
ListenExit automatically availableconst result = await execute({
instructions: 'You are a helpful assistant',
chat: myChatInstance,
tools: [searchTool, calculatorTool],
client,
})
if (result.is(ListenExit)) {
// Agent is waiting for user input
}
Enabled when: chat parameter is omitted from execute()
Worker Mode is designed for automated execution environments that need to:
Key characteristics:
DefaultExit if no custom exits providedconst result = await execute({
instructions: 'Process the customer data and generate insights',
tools: [dataProcessorTool, analyticseTool],
exits: [dataProcessedExit],
client,
})
if (result.is(dataProcessedExit)) {
console.log('Analysis complete:', result.output)
}
| Feature | Chat Mode | Worker Mode |
|---|---|---|
| User Interaction | ✅ Interactive | ❌ Automated |
| UI Components | ✅ React components | ❌ No UI |
| Conversation History | ✅ Full transcript | ❌ No history |
| Default Exits | ListenExit | DefaultExit |
| Primary Use Case | Conversational AI | Data processing |
| Execution Pattern | Turn-based | Continuous |
Tools are the primary way to extend LLMz agents with external capabilities. Unlike traditional agent frameworks, LLMz tools are called through generated TypeScript code, enabling complex orchestration and error handling.
Tools are defined using Zui schemas for complete type safety:
import { Tool } from 'llmz'
import { z } from '@bpinternal/zui'
const weatherTool = new Tool({
name: 'getWeather',
description: 'Get current weather for a location',
input: z.object({
location: z.string().describe('City name or coordinates'),
units: z.enum(['celsius', 'fahrenheit']).optional().default('celsius'),
}),
output: z.object({
temperature: z.number(),
conditions: z.string(),
humidity: z.number(),
}),
handler: async ({ location, units }) => {
// Implementation here
return {
temperature: 22,
conditions: 'sunny',
humidity: 65,
}
},
})
The LLM generates TypeScript code that calls tools naturally:
// Simple tool call
const weather = await getWeather({ location: 'New York' })
// Complex logic with multiple tools
const weather = await getWeather({ location: userLocation })
if (weather.temperature < 0) {
const clothing = await getSuggestions({ type: 'winter', temperature: weather.temperature })
yield <Text>It's {weather.temperature}°C! {clothing.suggestion}</Text>
} else {
yield <Text>Nice weather! {weather.conditions} at {weather.temperature}°C</Text>
}
return { action: 'listen' }
Tools can have multiple names for flexible calling:
const tool = new Tool({
name: 'calculatePrice',
aliases: ['getPrice', 'checkCost'],
// ... rest of definition
})
// All of these work in generated code:
// await calculatePrice(params)
// await getPrice(params)
// await checkCost(params)
Force specific inputs to be always included:
const tool = new Tool({
name: 'logEvent',
input: z.object({
event: z.string(),
userId: z.string(),
timestamp: z.number(),
}),
staticInputs: {
userId: 'user-123',
timestamp: () => Date.now(), // Dynamic static input
},
handler: async ({ event, userId, timestamp }) => {
// userId and timestamp are automatically provided
},
})
Clone and modify existing tools:
const originalTool = new Tool({
/* definition */
})
const wrappedTool = originalTool.clone({
name: 'wrappedVersion',
description: 'Enhanced version with logging',
handler: async (input) => {
console.log('Tool called with:', input)
const result = await originalTool.execute(input)
console.log('Tool returned:', result)
return result
},
})
Use tool.getTypings() to see the TypeScript definitions generated for the LLM:
console.log(weatherTool.getTypings())
// Output:
// /**
// * Get current weather for a location
// */
// declare function getWeather(input: {
// location: string; // City name or coordinates
// units?: "celsius" | "fahrenheit";
// }): Promise<{
// temperature: number;
// conditions: string;
// humidity: number;
// }>;
Objects in LLMz provide namespaced containers for related tools and variables, enabling sophisticated state management and data organization.
Objects group related functionality and provide scoped variables:
import { ObjectInstance } from 'llmz'
import { z } from '@bpinternal/zui'
const userObject = new ObjectInstance({
name: 'user',
properties: [
{
name: 'name',
value: 'John Doe',
writable: true,
type: z.string(),
},
{
name: 'age',
value: 30,
writable: false, // Read-only
type: z.number(),
},
{
name: 'preferences',
value: { theme: 'dark', language: 'en' },
writable: true,
type: z.object({
theme: z.enum(['light', 'dark']),
language: z.string(),
}),
},
],
tools: [
new Tool({
name: 'updateProfile',
input: z.object({ name: z.string() }),
handler: async ({ name }) => {
// This tool is scoped to the user object
return { success: true }
},
}),
],
})
The LLM can read and write object properties in generated code:
// Reading variables
const userName = user.name // "John Doe"
const userAge = user.age // 30
// Writing to writable variables
user.name = 'Jane Smith' // ✅ Succeeds
user.preferences = { theme: 'light', language: 'es' } // ✅ Succeeds
// Attempting to write read-only variables
user.age = 25 // ❌ Throws AssignmentError
Variables are validated against their schemas:
// Valid assignment
user.preferences = { theme: 'dark', language: 'fr' } // ✅
// Invalid assignment - wrong type
user.preferences = { theme: 'blue', language: 'fr' } // ❌ Throws validation error
// Invalid assignment - missing required fields
user.preferences = { theme: 'dark' } // ❌ Missing language field
LLMz automatically tracks changes to object properties:
// In generated code
user.name = 'Updated Name'
user.preferences.theme = 'light'
// After execution, mutations are available
console.log(result.iteration.mutations)
// [
// {
// object: 'user',
// property: 'name',
// before: 'John Doe',
// after: 'Updated Name'
// },
// {
// object: 'user',
// property: 'preferences',
// before: { theme: 'dark', language: 'en' },
// after: { theme: 'light', language: 'en' }
// }
// ]
Tools within objects are called with object namespace:
// Tool is scoped to the user object
await user.updateProfile({ name: 'New Name' })
// This automatically updates the user object's properties
// and is tracked as a mutation
Objects are automatically sealed to prevent unauthorized modifications:
// In generated code - these will throw errors
user.newProperty = 'value' // ❌ Cannot add new properties
delete user.name // ❌ Cannot delete properties
// Only predefined properties can be modified (if writable)
user.name = 'New Name' // ✅ Allowed if writable: true
Variables persist across iterations and thinking cycles:
// Iteration 1: Set a variable
user.preferences = { theme: 'dark', language: 'es' }
return { action: 'think' } // Trigger thinking
// Iteration 2: Variable is still available
const currentTheme = user.preferences.theme // 'dark'
Every call to execute() returns an ExecutionResult that provides type-safe access to the execution outcome. LLMz execution can result in three different types of results.
Agent completed successfully with an Exit. Contains the structured data produced by the agent.
const result = await execute({
instructions: 'Calculate the sum',
client,
})
if (result.isSuccess()) {
console.log('Output:', result.output)
console.log('Exit used:', result.exit.name)
console.log('Generated code:', result.iteration.code)
}
Execution failed with an unrecoverable error:
if (result.isError()) {
console.error('Error:', result.error)
console.error('Failed iteration:', result.iteration?.error)
// Analyze failure progression
result.iterations.forEach((iter, i) => {
console.log(`Iteration ${i + 1}: ${iter.status.type}`)
})
}
Execution was interrupted by a SnapshotSignal for pauseable operations:
if (result.isInterrupted()) {
console.log('Interrupted:', result.signal.message)
// Save snapshot for later resumption
const serialized = result.snapshot.toJSON()
await database.saveSnapshot(serialized)
}
Use result.is(exit) for type-safe access to specific exit data:
const successExit = new Exit({
name: 'success',
schema: z.object({
recordsProcessed: z.number(),
processingTime: z.number(),
}),
})
const errorExit = new Exit({
name: 'error',
schema: z.object({
errorCode: z.string(),
details: z.string(),
}),
})
const result = await execute({
instructions: 'Process the data',
exits: [successExit, errorExit],
client,
})
// Type-safe exit handling with automatic output typing
if (result.is(successExit)) {
// TypeScript knows result.output has the success schema
console.log(`Processed ${result.output.recordsProcessed} records`)
console.log(`Processing took ${result.output.processingTime}ms`)
} else if (result.is(errorExit)) {
// TypeScript knows result.output has the error schema
console.error(`Error ${result.output.errorCode}: ${result.output.details}`)
}
import { ListenExit, DefaultExit, ThinkExit } from 'llmz'
// Check for built-in exits
if (result.is(ListenExit)) {
console.log('Agent is waiting for user input')
}
if (result.is(DefaultExit)) {
// DefaultExit has success/failure discriminated union
if (result.output.success) {
console.log('Completed successfully:', result.output.result)
} else {
console.error('Completed with error:', result.output.error)
}
}
if (result.is(ThinkExit)) {
console.log('Agent requested thinking time')
console.log('Current variables:', result.output.variables)
}
// Access the final iteration
const lastIteration = result.iteration
if (lastIteration) {
console.log('Generated code:', lastIteration.code)
console.log('Status:', lastIteration.status.type)
console.log('Duration:', lastIteration.duration)
}
// Access all iterations to see full execution flow
result.iterations.forEach((iteration, index) => {
console.log(`Iteration ${index + 1}:`)
console.log(' Status:', iteration.status.type)
console.log(' Code length:', iteration.code?.length || 0)
console.log(' Variables:', Object.keys(iteration.variables).length)
})
// If agent generates: const hello = '1234'
const lastIteration = result.iteration
if (lastIteration) {
console.log(lastIteration.variables.hello) // '1234'
// Access all variables from the final iteration
Object.entries(lastIteration.variables).forEach(([name, value]) => {
console.log(`Variable ${name}:`, value)
})
}
// Access tool calls from all iterations
const allToolCalls = result.iterations.flatMap((iter) => iter.traces.filter((trace) => trace.type === 'tool_call'))
console.log('Total tool calls:', allToolCalls.length)
// Access other trace types
const lastIteration = result.iteration
if (lastIteration) {
const yields = lastIteration.traces.filter((trace) => trace.type === 'yield')
const comments = lastIteration.traces.filter((trace) => trace.type === 'comment')
const propertyAccess = lastIteration.traces.filter((trace) => trace.type === 'property')
}
if (result.isSuccess()) {
// Access original execution parameters
console.log('Instructions:', result.context.instructions)
console.log('Loop limit:', result.context.loop)
console.log('Temperature:', result.context.temperature)
console.log('Model:', result.context.model)
// Access tools and exits that were available
console.log(
'Available tools:',
result.context.tools?.map((t) => t.name)
)
console.log(
'Available exits:',
result.context.exits?.map((e) => e.name)
)
}
if (result.isError()) {
console.error('Execution failed:', result.error)
// Analyze the failure progression
const failedIteration = result.iteration
if (failedIteration) {
switch (failedIteration.status.type) {
case 'execution_error':
console.error('Code execution failed:', failedIteration.status.execution_error.message)
console.error('Stack trace:', failedIteration.status.execution_error.stack)
console.error('Failed code:', failedIteration.code)
break
case 'generation_error':
console.error('LLM generation failed:', failedIteration.status.generation_error.message)
break
case 'invalid_code_error':
console.error('Invalid code generated:', failedIteration.status.invalid_code_error.message)
console.error('Invalid code:', failedIteration.code)
break
case 'aborted':
console.error('Execution aborted:', failedIteration.status.aborted.reason)
break
}
}
// Review all iterations to understand failure progression
console.log('Iterations before failure:', result.iterations.length)
result.iterations.forEach((iter, i) => {
console.log(`Iteration ${i + 1}: ${iter.status.type}`)
})
}
Handle interrupted executions with snapshot resumption:
const result = await execute({
instructions: 'Process large dataset with pauseable operation',
tools: [snapshotCapableTool],
client,
})
if (result.isInterrupted()) {
console.log('Execution paused:', result.signal.message)
// Serialize snapshot for persistence
const serialized = result.snapshot.toJSON()
await database.saveSnapshot('execution-123', serialized)
// Later, resume from snapshot
const snapshot = Snapshot.fromJSON(serialized)
snapshot.resolve({ resumeData: 'Operation completed' })
const continuation = await execute({
snapshot,
instructions: result.context.instructions,
tools: result.context.tools,
exits: result.context.exits,
client,
})
if (continuation.isSuccess()) {
console.log('Resumed execution completed:', continuation.output)
}
}
LLMz provides a comprehensive hook system that allows you to inject custom logic at various points during execution. Hooks are categorized as either blocking (execution waits) or non-blocking, and either mutation (can modify data) or non-mutation.
| Hook | Blocking | Mutation | Called When |
|---|---|---|---|
onTrace | ❌ | ❌ | Each trace generated |
onIterationEnd | ✅ | ❌ | After iteration completion |
onExit | ✅ | ❌ | When exit is reached |
onBeforeExecution | ✅ | ✅ | Before code execution |
onBeforeTool | ✅ | ✅ | Before tool execution |
onAfterTool | ✅ | ✅ | After tool execution |
Called for each trace generated during iteration. Useful for logging, debugging, or monitoring execution progress.
await execute({
onTrace: ({ trace, iteration }) => {
console.log(`Iteration ${iteration}: ${trace.type}`, trace)
// Log specific trace types
if (trace.type === 'tool_call') {
console.log(`Tool ${trace.tool_name} called with:`, trace.input)
}
},
// ... other props
})
Available Trace Types:
abort_signal: Abort signal receivedcomment: Comment found in generated codellm_call_success: LLM generation completed successfullyproperty: Object property accessed or modifiedthink_signal: ThinkSignal throwntool_call: Tool executedyield: Component yielded in chat modelog: General logging eventCalled after each iteration ends, regardless of status. Useful for logging, cleanup, or controlling iteration timing.
await execute({
onIterationEnd: async (iteration, controller) => {
console.log(`Iteration ${iteration.id} ended with status: ${iteration.status.type}`)
// Add delays, cleanup, or conditional logic
if (iteration.status.type === 'execution_error') {
await logError(iteration.error)
// Add delay before retry
await new Promise((resolve) => setTimeout(resolve, 1000))
}
// Can use controller to abort execution if needed
if (shouldAbort(iteration)) {
controller.abort('Custom abort reason')
}
},
// ... other props
})
Called when an exit is reached. Useful for logging, notifications, or implementing guardrails by throwing errors to prevent exit.
await execute({
onExit: async (result) => {
console.log(`Exiting with: ${result.exit.name}`, result.result)
// Implement guardrails
if (result.exit.name === 'approve_loan' && result.result.amount > 10000) {
throw new Error('Manager approval required for loans over $10,000')
}
// Send notifications
await notifyStakeholders(result)
// Log to audit trail
await auditLog.record({
action: result.exit.name,
data: result.result,
timestamp: Date.now(),
})
},
// ... other props
})
Called after LLM generates code but before execution. Allows code modification and guardrails implementation.
await execute({
onBeforeExecution: async (iteration, controller) => {
console.log('Generated code:', iteration.code)
// Code modification
if (iteration.code?.includes('dangerousOperation')) {
return {
code: iteration.code.replace('dangerousOperation', 'safeOperation'),
}
}
// Guardrails - throw to prevent execution
if (iteration.code?.includes('forbidden')) {
throw new Error('Forbidden operation detected')
}
// Add security checks
const securityIssues = await scanCodeForSecurity(iteration.code)
if (securityIssues.length > 0) {
throw new Error(`Security issues found: ${securityIssues.join(', ')}`)
}
// Log code generation for audit
await auditCodeGeneration(iteration.code)
},
// ... other props
})
Called before any tool execution. Allows input modification and tool execution control.
await execute({
onBeforeTool: async ({ iteration, tool, input, controller }) => {
console.log(`Executing tool: ${tool.name}`, input)
// Input modification
if (tool.name === 'sendEmail') {
return {
input: {
...input,
subject: `[Automated] ${input.subject}`, // Add prefix
from: '[email protected]', // Override sender
},
}
}
// Access control
if (tool.name === 'deleteFile' && !hasPermission(input.path)) {
throw new Error('Insufficient permissions to delete file')
}
// Rate limiting
const rateLimit = await checkRateLimit(tool.name)
if (rateLimit.exceeded) {
throw new Error(`Rate limit exceeded for ${tool.name}`)
}
// Validation
await validateToolUsage(tool, input)
},
// ... other props
})
Called after tool execution. Allows output modification and post-processing.
await execute({
onAfterTool: async ({ iteration, tool, input, output, controller }) => {
console.log(`Tool ${tool.name} completed`, { input, output })
// Output modification
if (tool.name === 'fetchUserData') {
return {
output: {
...output,
// Remove sensitive data before LLM sees it
ssn: undefined,
creditCard: undefined,
// Add metadata
fetchedAt: Date.now(),
},
}
}
// Result enhancement
if (tool.name === 'calculatePrice') {
return {
output: {
...output,
currency: 'USD',
timestamp: Date.now(),
exchangeRate: await getCurrentExchangeRate(),
},
}
}
// Logging and caching
await Promise.all([
cacheResult(tool.name, input, output),
logToolExecution(tool.name, input, output),
updateMetrics(tool.name, Date.now() - tool.startTime),
])
},
// ... other props
})
For each iteration:
await execute({
onBeforeTool: async ({ tool, input }) => {
// Apply different logic based on tool
switch (tool.name) {
case 'payment':
return await handlePaymentValidation(input)
case 'notification':
return await handleNotificationThrottling(input)
default:
return // No modification
}
},
})
await execute({
onExit: async (result) => {
try {
await criticalPostProcessing(result)
} catch (error) {
// Log error but don't fail the entire execution
console.error('Post-processing failed:', error)
// Optionally throw to retry the iteration
if (error.retryable) {
throw new Error('Retrying due to recoverable error')
}
}
},
})
let executionMetrics = { toolCalls: 0, totalTime: 0 }
await execute({
onBeforeTool: async ({ tool }) => {
executionMetrics.toolCalls++
tool.startTime = Date.now()
},
onAfterTool: async ({ tool }) => {
executionMetrics.totalTime += Date.now() - tool.startTime
},
onIterationEnd: async () => {
console.log('Execution metrics:', executionMetrics)
},
})
onTraceonBeforeExecution and onBeforeTool for security validationonTrace for comprehensive execution monitoringonExitonBeforeTool/onAfterTool for input/output processingSnapshots allow you to pause and resume LLMz execution, enabling long-running workflows that can be interrupted and continued later.
Inside a tool, throw a SnapshotSignal to halt execution and create a serializable snapshot:
import { SnapshotSignal, Tool } from 'llmz'
const longRunningTool = new Tool({
name: 'processLargeDataset',
input: z.object({ datasetId: z.string() }),
async handler({ datasetId }) {
// Start processing
const dataset = await loadDataset(datasetId)
// At any point, pause execution for later resumption
if (dataset.size > LARGE_THRESHOLD) {
throw new SnapshotSignal(
'Dataset is large, pausing for background processing',
'Processing will continue once background job completes'
)
}
return { processed: true }
},
})
const result = await execute({
instructions: 'Process the uploaded dataset',
tools: [longRunningTool],
client,
})
if (result.isInterrupted()) {
console.log('Execution paused:', result.signal.message)
// Serialize snapshot for persistence
const serialized = result.snapshot.toJSON()
await database.saveSnapshot('job-123', serialized)
// Start background processing
await backgroundJobQueue.add('process-dataset', {
snapshotId: 'job-123',
datasetId: result.signal.toolCall?.input.datasetId,
})
}
// Later, when background job completes
const serialized = await database.getSnapshot('job-123')
const snapshot = Snapshot.fromJSON(serialized)
// Resolve the snapshot with the result
snapshot.resolve({
processed: true,
recordCount: 1000000,
processingTime: 3600000,
})
// Continue execution from where it left off
const continuation = await execute({
snapshot,
instructions: 'Process the uploaded dataset', // Same as original
tools: [longRunningTool], // Same tools
client,
})
if (continuation.isSuccess()) {
console.log('Processing completed:', continuation.output)
}
// If background processing fails
const snapshot = Snapshot.fromJSON(serialized)
snapshot.reject(new Error('Background processing failed'))
const continuation = await execute({
snapshot,
// ... same parameters
})
// The agent will receive the error and can handle it
The thinking system allows agents to pause and reflect on variables and context before proceeding.
Tools can force thinking by throwing a ThinkSignal:
const analysisTool = new Tool({
name: 'analyzeData',
input: z.object({ data: z.array(z.number()) }),
async handler({ data }) {
const result = performAnalysis(data)
// Force the agent to think about the results before responding
throw new ThinkSignal(
'Analysis complete, consider the implications',
`Found ${result.anomalies.length} anomalies and ${result.patterns.length} patterns`
)
},
})
Agents can request thinking time in generated code:
// In generated code
const analysisResult = await analyzeData({ data: userInputData })
// Think about the results before responding to user
return { action: 'think' }
Pass specific variables for reflection:
// In generated code
const price = await calculatePrice({ items: cartItems })
const budget = await getUserBudget()
// Think about pricing vs budget with specific context
return {
action: 'think',
price,
budget,
recommendation: price > budget ? 'deny' : 'approve',
}
const result = await execute({
instructions: 'Analyze the user data and provide recommendations',
tools: [analysisTool],
client,
})
if (result.is(ThinkExit)) {
console.log('Agent is thinking about:', result.output.variables)
// Continue execution after thinking
const continuation = await execute({
instructions: result.context.instructions,
tools: result.context.tools,
// Variables from thinking are automatically preserved
client,
})
}
CitationsManager provides standardized source tracking and referencing for RAG (Retrieval-Augmented Generation) systems.
Citations use rare Unicode symbols (【】) as markers that are unlikely to appear in natural text. The system supports:
【0】, 【1】【0,1,3】import { CitationsManager } from 'llmz'
const citations = new CitationsManager()
// Register sources and get citation tags
const source1 = citations.registerSource({
file: 'document.pdf',
page: 5,
title: 'Company Policy',
})
const source2 = citations.registerSource({
url: 'https://example.com/article',
title: 'Best Practices',
})
console.log(source1.tag) // "【0】"
console.log(source2.tag) // "【1】"
// Use tags in content
const content = `The policy states employees must arrive on time${source1.tag}. However, best practices suggest flexibility${source2.tag}.`
const ragTool = new Tool({
name: 'search',
description: 'Searches in the knowledge base for relevant information.',
input: z.string().describe('The query to search in the knowledge base.'),
async handler(query) {
// Perform semantic search
const { passages } = await client.searchFiles({
query,
limit: 20,
contextDepth: 3,
})
if (!passages.length) {
throw new ThinkSignal(
'No results found',
'No results were found in the knowledge base. Try rephrasing your question.'
)
}
// Build response with citations
let message: string[] = ['Here are the search results:']
let { tag: example } = chat.citations.registerSource({}) // Example citation
// Register each passage as a source
for (const passage of passages) {
const { tag } = chat.citations.registerSource({
file: passage.file.key,
title: passage.file.tags.title,
})
message.push(`<${tag} file="${passage.file.key}">`)
message.push(`**${passage.file.tags.title}**`)
message.push(passage.content)
message.push(`</${tag}>`)
}
// Provide context with citation instructions
throw new ThinkSignal(
`Got search results. When answering, you MUST add inline citations (eg: "The price is $10${example} ...")`,
message.join('\n').trim()
)
},
})
class CLIChat extends Chat {
public citations: CitationsManager = new CitationsManager()
private async sendMessage(input: RenderedComponent) {
if (input.type === 'Text') {
let sources: string[] = []
// Extract citations and format them for display
const { cleaned } = this.citations.extractCitations(input.text, (citation) => {
let idx = chalk.bgGreenBright.black.bold(` ${sources.length + 1} `)
sources.push(`${idx}: ${JSON.stringify(citation.source)}`)
return `${idx}` // Replace 【0】 with [1]
})
// Display cleaned text and sources
console.log(`🤖 Agent: ${cleaned}`)
if (sources.length) {
console.log(chalk.dim('Citations'))
console.log(chalk.dim('========='))
console.log(chalk.dim(sources.join('\n')))
}
}
}
}
Multiple Citation Support:
// Agent can reference multiple sources in one citation
const content = 'This fact is supported by multiple studies【0,1,3】'
const { cleaned, citations } = manager.extractCitations(content)
// citations array contains entries for sources 0, 1, and 3
Object Citation Processing:
// Remove citations from complex objects
const dataWithCitations = {
summary: 'The report shows positive trends【0】',
details: {
revenue: 'Increased by 15%【1】',
costs: 'Reduced by 8%【2】',
},
}
const [cleanData, extractedCitations] = manager.removeCitationsFromObject(dataWithCitations)
// cleanData has citations removed, extractedCitations contains path + citation info
Citation Stripping:
// Remove all citation tags from content
const textWithCitations = 'This statement【0】 has multiple【1,2】 citations.'
const cleaned = CitationsManager.stripCitationTags(textWithCitations)
// Result: "This statement has multiple citations."
LLMz supports dynamic evaluation of most parameters, allowing context-aware configuration:
await execute({
// Dynamic instructions based on context
instructions: (ctx) => {
const timeOfDay = new Date().getHours()
const greeting = timeOfDay < 12 ? 'Good morning' : 'Good afternoon'
return `${greeting}! You are a helpful assistant with access to ${ctx.tools?.length || 0} tools.`
},
// Dynamic tools based on user permissions
tools: async (ctx) => {
const userPermissions = await getUserPermissions(ctx.userId)
return allTools.filter((tool) => userPermissions.includes(tool.permission))
},
// Dynamic objects with current state
objects: async (ctx) => {
const userPreferences = await loadUserPreferences(ctx.userId)
return [
new ObjectInstance({
name: 'user',
properties: [{ name: 'preferences', value: userPreferences, writable: true }],
}),
]
},
client,
})
Main execution function that runs LLMz agents in either Chat Mode or Worker Mode.
Parameters:
props.client - Botpress Client or Cognitive Client instance for LLM generationprops.instructions - System prompt/instructions for the LLM (static string or dynamic function)props.chat - Optional Chat instance to enable Chat Mode with user interactionprops.tools - Array of Tool instances available to the agent (static or dynamic)props.objects - Array of ObjectInstance for namespaced tools and variables (static or dynamic)props.exits - Array of Exit definitions for structured completion (static or dynamic)props.snapshot - Optional Snapshot to resume paused executionprops.signal - Optional AbortSignal to cancel executionprops.options - Optional execution options (loop limit, temperature, model, timeout)props.onTrace - Optional non-blocking hook for monitoring traces during executionprops.onIterationEnd - Optional blocking hook called after each iterationprops.onExit - Optional blocking hook called when an exit is reachedprops.onBeforeExecution - Optional blocking hook to modify code before VM executionprops.onBeforeTool - Optional blocking hook to modify tool inputs before executionprops.onAfterTool - Optional blocking hook to modify tool outputs after executionReturns: Promise<ExecutionResult> - Result containing success/error/interrupted status with type-safe exit checking
Creates a new tool definition with type-safe schemas.
Properties:
name: string - Tool name used in generated codedescription?: string - Description for LLM understandinginput?: ZuiSchema - Input validation schemaoutput?: ZuiSchema - Output validation schemahandler: (input: any) => Promise<any> | any - Tool implementationaliases?: string[] - Alternative names for the toolstaticInputs?: Record<string, any> - Force specific input valuesMethods:
execute(input: any, context?: ToolContext): Promise<any> - Execute the toolgetTypings(): string - Get TypeScript definitions for LLMclone(overrides: Partial<ToolConfig>): Tool - Create a modified copyDefines a structured exit point for agent execution.
Properties:
name: string - Exit name used in generated codedescription?: string - Description for LLM understandingschema?: ZuiSchema - Output validation schemaaliases?: string[] - Alternative names for the exitCreates a namespaced container for tools and variables.
Properties:
name: string - Object name used in generated codeproperties?: PropertyConfig[] - Object properties/variablestools?: Tool[] - Tools scoped to this objectPropertyConfig:
name: string - Property namevalue: any - Initial valuewritable: boolean - Whether property can be modifiedtype?: ZuiSchema - Validation schemaProperties:
isSuccess(): boolean - Type guard for successoutput: any - The result data from the exitexit: Exit - The exit that was usediteration: Iteration - Final iteration detailsiterations: Iteration[] - All iterationscontext: Context - Execution contextis(exit: Exit): boolean - Type-safe exit checkingProperties:
isError(): boolean - Type guard for errorerror: Error | string - The error that occurrediteration?: Iteration - Failed iteration detailsiterations: Iteration[] - All iterations before failurecontext: Context - Execution contextProperties:
isInterrupted(): boolean - Type guard for interruptionsignal: SnapshotSignal - The signal that caused interruptionsnapshot: Snapshot - Serializable execution stateiterations: Iteration[] - All iterations before interruptioncontext: Context - Execution contextAbstract base class for implementing chat interfaces.
Abstract Methods:
getTranscript(): Promise<Transcript.Message[]> | Transcript.Message[] - Get conversation historygetComponents(): Promise<ComponentDefinition[]> | ComponentDefinition[] - Get available UI componentshandler(component: RenderedComponent): Promise<void> - Handle agent messagesManages source citations for RAG systems.
Methods:
registerSource(source: any): { tag: string, id: number } - Register a source and get citation tagextractCitations(text: string, replacer?: (citation) => string): { cleaned: string, citations: Citation[] } - Extract and process citationsremoveCitationsFromObject(obj: any): [cleanedObj: any, citations: Citation[]] - Remove citations from objectsstatic stripCitationTags(text: string): string - Remove all citation tagsManages pauseable execution state.
Methods:
toJSON(): string - Serialize snapshotstatic fromJSON(json: string): Snapshot - Deserialize snapshotresolve(data: any): void - Resume with successreject(error: Error): void - Resume with errorListenExit - Automatically available in Chat Mode for user interactionDefaultExit - Default exit for Worker Mode with success/failure discriminationThinkExit - Used when agent requests thinking timeSnapshotSignal - Thrown to pause execution for later resumptionThinkSignal - Thrown to request agent reflection timeLoopExceededError - Thrown when maximum iterations reachedVM_DRIVER: 'isolated-vm' | 'node' - Choose VM execution environmentCI: boolean - Automatically detected, affects VM driver selectionThe LLMz repository includes 20 comprehensive examples demonstrating different patterns and capabilities:
# Install dependencies
pnpm install
# Set up environment variables
cp .env.example .env
# Edit .env with your Botpress credentials
# Run a specific example
pnpm start 01_chat_basic
pnpm start chat_basic
pnpm start 01
# List all available examples
pnpm start
Create a .env file in the examples directory:
BOTPRESS_BOT_ID=your_bot_id_here
BOTPRESS_TOKEN=your_token_here
Getting Started:
01_chat_basic and 11_worker_minimalIntermediate Concepts:
09_chat_variables for state management16_worker_tool_chaining for complex workflows14_worker_snapshot for pauseable executionAdvanced Patterns:
08_chat_multi_agent for orchestration20_chat_rag for knowledge integration18_worker_security for production deploymentEach example includes detailed comments explaining the concepts and implementation patterns, making them excellent learning resources for understanding LLMz capabilities.
This documentation covers the complete LLMz framework. For the latest updates and community contributions, visit the LLMz repository.