README.md
Experimental - This project is still in development, and not ready for prime time.
A minimal, secure Python interpreter written in Rust for use by AI.
Monty avoids the cost, latency, complexity and general faff of using a full container based sandbox for running LLM generated code.
Instead, it lets you safely run Python code written by an LLM embedded in your agent, with startup times measured in single digit microseconds not hundreds of milliseconds.
What Monty can do:
sys, os, typing, asyncio, re, datetime, json, dataclasses (soon)What Monty cannot do:
In short, Monty is extremely limited and designed for one use case:
To run code written by agents.
For motivation on why you might want to do this, see:
In very simple terms, the idea of all the above is that LLMs can work faster, cheaper and more reliably if they're asked to write Python (or Javascript) code, instead of relying on traditional tool calling. Monty makes that possible without the complexity of a sandbox or risk of running code directly on the host.
Note: Monty will (soon) be used to implement codemode in Pydantic AI
Monty can be called from Python, JavaScript/TypeScript or Rust.
To install:
uv add pydantic-monty
(Or pip install pydantic-monty for the boomers)
pydantic-monty is a metapackage pairing pydantic-monty-client (the
pydantic_monty module) with pydantic-monty-runtime (the monty worker
binary). Install pydantic-monty-client alone if the binary already comes from
somewhere else.
Usage:
from typing import Any
import pydantic_monty
code = """
async def agent(prompt: str, messages: Messages):
while True:
print(f'messages so far: {messages}')
output = await call_llm(prompt, messages)
if isinstance(output, str):
return output
messages.extend(output)
await agent(prompt, [])
"""
type_definitions = """
from typing import Any
Messages = list[dict[str, Any]]
async def call_llm(prompt: str, messages: Messages) -> str | Messages:
raise NotImplementedError()
prompt: str = ''
"""
Messages = list[dict[str, Any]]
async def call_llm(prompt: str, messages: Messages) -> str | Messages:
if len(messages) < 2:
return [{'role': 'system', 'content': 'example response'}]
else:
return f'example output, message count {len(messages)}'
async def main():
async with pydantic_monty.AsyncMonty() as pool:
async with pool.checkout(
script_name='agent.py',
type_check=True,
type_check_stubs=type_definitions,
) as session:
output = await session.feed_run(
code,
inputs={'prompt': 'testing'},
external_lookup={'call_llm': call_llm},
)
print(output)
#> example output, message count 2
if __name__ == '__main__':
import asyncio
asyncio.run(main())
Execution happens in a pool of monty worker subprocesses, so even a memory
error triggered by adversarial code (stack overflow, allocator abort) can
never crash your process — the worker dies, raises MontyCrashedError, and
is replaced. There is also a fully synchronous API:
import pydantic_monty
with pydantic_monty.Monty() as pool:
with pool.checkout() as session:
# session state persists between feed_run calls
session.feed_run('x = 21')
print(session.feed_run('x * 2'))
#> 42
To install:
npm install @pydantic/monty
The JS package is a native (napi) binding over the same Rust worker pool the
Python package uses — the binding and the monty worker binary ship via
platform-specific npm packages:
import { Monty } from '@pydantic/monty'
await using pool = await Monty.create()
await using session = await pool.checkout()
// session state persists between feedRun calls
await session.feedRun('x = 21')
console.log(await session.feedRun('x * 2')) // 42
// external functions may be async
const result = await session.feedRun('await fetch_data()', {
externalLookup: { fetch_data: async () => 'data' },
})
For browsers (or anywhere subprocesses are impossible) the same package
exposes an in-process WebAssembly build under the @pydantic/monty/wasm
subpath (no crash isolation: a sandbox crash is a host crash there).
For running untrusted code from Rust, we recommend the
monty-pool crate rather than the in-process API below.
monty-pool only runs code in monty worker subprocesses, which affords extra protections:
a crash triggered by adversarial code (stack overflow, allocator abort) kills only the worker —
the pool detects the death and replaces the worker — and a parent-side watchdog can kill workers
that exceed a hard timeout. It is the same engine the Python and JavaScript packages above are
built on. See the monty-pool README
for usage.
The monty crate itself provides the in-process interpreter:
use monty::MontyRun;
use monty_types::{CompileOptions, ResourceTracker, MontyObject, PrintWriter, ResourceLimits};
let code = r#"
def fib(n):
if n <= 1:
return n
return fib(n - 1) + fib(n - 2)
fib(x)
"#;
let runner = MontyRun::new(code.to_owned(), "fib.py", vec!["x".to_owned()], CompileOptions::default()).unwrap();
let result = runner.run(vec![MontyObject::Int(10)], ResourceTracker::default(), PrintWriter::Stdout).unwrap();
assert_eq!(result, MontyObject::Int(55));
A REPL session can be serialized with dump() and restored with Dump::load(). The dump carries the session metadata (script name, type-check stubs) alongside the interpreter state, behind a version the loading build checks:
use monty::{Dump, MontyRepl, Session, SessionRef, dump};
use monty_types::{CompileOptions, MontyObject, PrintWriter, ResourceTracker};
// Snapshot a session between snippets
let mut repl = MontyRepl::new("main.py", ResourceTracker::default(), CompileOptions::default());
repl.feed_run("x = 41", vec![], PrintWriter::Stdout).unwrap();
let bytes = dump("main.py", None, SessionRef::Idle(&repl)).unwrap();
// Later, restore and carry on feeding
let Session::Idle(mut restored) = Dump::load(&bytes).unwrap().state else {
panic!("dumped an idle session")
};
let result = restored.feed_run("x + 1", vec![], PrintWriter::Stdout).unwrap();
assert_eq!(result, MontyObject::Int(42));
MontyRun and RunProgress have no dump format of their own, but both implement serde::Serialize/Deserialize, so a host can serialize parsed code or a paused run with whatever format it already uses.
A session's max_memory is measured by the worker's allocator. The interpreter
reports a graceful MemoryError after crossing the soft limit; a higher hard
limit kills and replaces the worker if one allocation jumps too far between checkpoints.
See limitations/resource_limits.md for how
exceeding a limit surfaces to a host, and monty-alloc for the allocator both
the subprocess and WebAssembly workers run under.
Monty will power code-mode in Pydantic AI. Instead of making sequential tool calls, the LLM writes Python code that calls your tools as functions and Monty executes it safely.
import asyncio
import json
import logfire
from httpx import AsyncClient
from pydantic_ai import Agent, RunContext
from pydantic_ai.toolsets.code_mode import CodeModeToolset
from pydantic_ai.toolsets.function import FunctionToolset
from typing_extensions import TypedDict
logfire.configure()
logfire.instrument_pydantic_ai()
class LatLng(TypedDict):
lat: float
lng: float
weather_toolset: FunctionToolset[AsyncClient] = FunctionToolset()
@weather_toolset.tool
async def get_lat_lng(
ctx: RunContext[AsyncClient], location_description: str
) -> LatLng:
"""Get the latitude and longitude of a location."""
# NOTE: the response here will be random, and is not related to the location description.
r = await ctx.deps.get(
'https://demo-endpoints.pydantic.workers.dev/latlng',
params={'location': location_description},
)
r.raise_for_status()
return json.loads(r.content)
@weather_toolset.tool
async def get_temp(ctx: RunContext[AsyncClient], lat: float, lng: float) -> float:
"""Get the temp at a location."""
# NOTE: the responses here will be random, and are not related to the lat and lng.
r = await ctx.deps.get(
'https://demo-endpoints.pydantic.workers.dev/number',
params={'min': 10, 'max': 30},
)
r.raise_for_status()
return float(r.text)
@weather_toolset.tool
async def get_weather_description(
ctx: RunContext[AsyncClient], lat: float, lng: float
) -> str:
"""Get the weather description at a location."""
# NOTE: the responses here will be random, and are not related to the lat and lng.
r = await ctx.deps.get(
'https://demo-endpoints.pydantic.workers.dev/weather',
params={'lat': lat, 'lng': lng},
)
r.raise_for_status()
return r.text
agent = Agent(
'gateway/anthropic:claude-sonnet-4-5',
# toolsets=[weather_toolset],
toolsets=[CodeModeToolset(weather_toolset)],
deps_type=AsyncClient,
)
async def main():
async with AsyncClient() as client:
await agent.run('Compare the weather of London, Paris, and Tokyo.', deps=client)
if __name__ == '__main__':
asyncio.run(main())
There are generally two responses when you show people Monty:
Where X is some alternative technology. Oddly often these responses are combined, suggesting people have not yet found an alternative that works for them, but are incredulous that there's really no good alternative to creating an entire Python implementation from scratch.
I'll try to run through the most obvious alternatives, and why there aren't right for what we wanted.
NOTE: all these technologies are impressive and have widespread uses, this commentary on their limitations for our use case should not be seen as a criticism. Most of these solutions were not conceived with the goal of providing an LLM sandbox, which is why they're not necessary great at it.
| Tech | Language completeness | Security | Start latency | FOSS | Setup complexity | File mounting | Snapshotting |
|---|---|---|---|---|---|---|---|
| Monty | partial | strict | 0.06ms | free / OSS | easy | easy | easy |
| Docker | full | good | 195ms | free / OSS | intermediate | easy | intermediate |
| Pyodide | full | poor | 2800ms | free / OSS | intermediate | easy | hard |
| starlark-rust | very limited | good | 1.7ms | free / OSS | easy | not available? | impossible? |
| WASI / Wasmer | partial, almost full | strict | 66ms | free * | intermediate | easy | intermediate |
| sandboxing service | full | strict | 1033ms | not free | intermediate | hard | intermediate |
| YOLO Python | full | non-existent | 0.1ms / 30ms | free / OSS | easy | easy / scary | hard |
See ./scripts/startup_performance.py for the script used to calculate the startup performance numbers.
Details on each row below:
pip install pydantic-monty or npm install @pydantic/monty, ~4.5MB downloaddump() and load() makes it trivial to pause, resume and fork executionpython:3.14-alpine is 50MB - docker can't be installed from PyPISee starlark-rust.
Running Python in WebAssembly via Wasmer.
python/python wasmer package package has no readme, no license, no source link and no indication of how it's built, the recently uploaded versions show size as "0B" although the download is ~50MB - the build process for the Python binary is not clear and transparent. (If I'm wrong here, please create an issue to correct correct me)Services like Daytona, E2B, Modal.
There are similar challenges, more setup complexity but lower network latency for setting up your own sandbox setup with k8s.
Running Python directly via exec() (~0.1ms) or subprocess (~30ms).
exec(), ~30ms for subprocessThe Pydantic Stack is everything you need to ship production-grade AI agents: