========= Sandboxes

.. important::

Sandbox is currently a preview feature. If you'd like early access, please reach out to us <https://www.bentoml.com/contact>_.

Modern AI applications like agentic systems often generate and execute code autonomously. However, letting these agents use tools, write scripts, and make decisions without human review is risky. Consider these common scenarios:

• An LLM agent writes Python code during its reasoning loop and runs it on the fly.
• A prompt injection leads the agent to execute harmful or unexpected logic.
• The agent integrates third-party tools or APIs with unpredictable behavior.
• The agent clones a GitHub repository and runs tasks like test suites and static analysis.

In each case, you can't assume the code is safe. Untrusted code can damage your infrastructure or expose sensitive data.

A BentoML Sandbox provides an isolated, secure, and ephemeral environment for running untrusted or dynamically generated code. With Sandboxes, you can:

• Run code generated by LLMs in a contained environment.
• Customize each Sandbox environment with your choice of base image, packages, environment variables, and startup behavior.
• Scale workloads from 1 to thousands of instances with fast cold starts.

Create a Sandbox

You can create a Sandbox and specify its :doc:runtime </build-with-bentoml/runtime-environment>:

.. code-block:: python

Use BentoCloud client

client = bentoml.BentoCloudClient()

sb = client.sandbox.create( image=bentoml.images.Image(python_version="3.12") .run("apt-get install -y curl") .python_packages("fastapi") )

Block until sandbox is ready (show logs if enabled)

sb.wait_until_ready(log_polling=True)

To run commands in a Sandbox:

.. code-block:: python

result = sb.exec("python", "-c", "import sys;print(sys.version)") print(result.stdout.read())

Output streaming

result = sb.exec("python", "-c", "import sys;print(sys.version)") for line in result.stdout: print(line)

On the Monitoring tab of the Sandbox details page, you can view real-time metrics such as replica count and resource usage:

.. image:: ../../_static/img/build-with-bentoml/sandboxes/bento-sandboxes-scaling.png :alt: Sandbox scaling based on real-time traffic

.. image:: ../../_static/img/build-with-bentoml/sandboxes/bento-sandboxes-resources.png :alt: Sandbox resource usage

Reuse an existing Sandbox

Previously created Sandboxes can be reused:

.. code-block:: python

sb = bentoml.sandbox.get(name=sandbox_name)

Using BentoCloud client

client = bentoml.BentoCloudClient() sb = client.sandbox.get(name=sandbox_name)

Parameters

You can customize sandbox behavior via more parameters. For example:

.. code-block:: python

Custom command

sb = bentoml.sandbox.create(cmd=["python", "-m", "http.server"])

Env vars and secrets

sb = bentoml.sandbox.create(secrets=["my-secret"], envs={"HF_TOKEN": "abdef"})

Add labels

sb = bentoml.sandbox.create(labels={"foo": "bar"})

Full signature:

.. list-table:: :header-rows: 1 :widths: 20 20 60

• • Parameter
  • Type
  • Description
• • image
  • Image | None
  • Base image specification (e.g. Python version, packages, OS dependencies).
• • labels
  • Mapping[str, str] | None
  • Optional key-value metadata to tag or group the sandbox.
• • cmd
  • list[str] | None
  • Custom command to run when the Sandbox starts.
• • secrets
  • list[str] | None
  • List of secret keys to mount into the Sandbox environment.
• • envs
  • Mapping[str, str] | None
  • Environment variables to set inside the Sandbox.
• • cluster
  • str | None
  • Optional BentoCloud cluster name to launch the Sandbox on.
• • client
  • BentoCloudClient
  • The BentoCloud client instance to use. It defaults to default_client.

Terminate a Sandbox

.. code-block:: python

Terminate and delete the sandbox

sb.destroy()

Destroy a sandbox by name

bentoml.sandbox.destroy(name=sandbox_name)