Testing Functions - Baml

You can test your BAML functions in the VSCode Playground by adding a test snippet into a BAML file:

baml

enum Category {
    Refund
    CancelOrder
    TechnicalSupport
    AccountIssue
    Question
}

function ClassifyMessage(input: string) -> Category {
  client GPT4Turbo
  prompt #"
    ... truncated ...
  "#
}

test Test1 {
  functions [ClassifyMessage]
  args {
    // input is the first argument of ClassifyMessage
    input "Can't access my account using my usual login credentials, and each attempt results in an error message stating 'Invalid username or password.' I have tried resetting my password using the 'Forgot Password' link, but I haven't received the promised password reset email."
  }
  // 'this' is the output of the function
  @@assert( {{ this == "AccountIssue" }})
}

Try it! Press 'Run Test' below!

{" "}

See more interactive examples

The BAML playground will give you a starting snippet to copy that will match your function signature.

<Warning> BAML doesn't use colons `:` between key-value pairs except in function parameters. </Warning> <hr /> ## Complex object inputs

Objects are injected as dictionaries

rust

class Message {
  user string
  content string
}

function ClassifyMessage(messages: Messages[]) -> Category {
...
}

test Test1 {
  functions [ClassifyMessage]
  args {
    messages [
      {
        user "hey there"
        // multi-line string using the #"..."# syntax
        content #"
          You can also add a multi-line
          string with the hashtags
          Instead of ugly json with \n
        "#
      }
    ]
  }
}

<hr /> ## Test Image Inputs in the Playground

For a function that takes an image as input, like so:

baml

function MyFunction(myImage: image) -> string {
  client GPT4o
  prompt #"
    Describe this image: {{myImage}}
  "#
}

You can define test cases using image files, URLs, or base64 strings.

<Tabs> <Tab title="File" language="baml"> <Warning> Committing a lot of images into your repository can make it slow to clone and pull your repository. If you expect to commit >500MiB of images, please read [GitHub's size limit documentation][github-large-files] and consider setting up [large file storage][github-lfs]. </Warning>

baml

test Test1 {
  functions [MyFunction]
  args {
    myImage {
      file "../path/to/image.png"
    }
  }
}

<ParamField path="file" type="string" required="true"> The path to the image file, relative to the directory containing the current BAML file.

Image files must be somewhere in baml_src/.

</ParamField> <ParamField path="media_type" type="string"> The mime-type of the image. If not set, and the provider expects a mime-type to be provided, BAML will try to infer it based on first, the file extension, and second, the contents of the file. </ParamField> </Tab> <Tab title="URL" language="baml"> ```baml test Test1 { functions [MyFunction] args { myImage { url "https...." } } } ``` <ParamField path="url" type="string" required="true"> The publicly accessible URL from which the image may be downloaded. </ParamField> <ParamField path="media_type" type="string"> The mime-type of the image. If not set, and the provider expects a mime-type to be provided, BAML will try to infer it based on the contents of the file. </ParamField> </Tab> <Tab title="Base64" language="baml"> ```baml test Test1 { args { myImage { base64 "base64string" media_type "image/png" } } } ``` <ParamField path="base64" type="string" required="true"> The base64-encoded image data. </ParamField> <ParamField path="media_type" type="string"> The mime-type of the image. If not set, and the provider expects a mime-type to be provided, BAML will try to infer it based on the contents of the file.

If base64 is a data URL, this field will be ignored.

</ParamField> </Tab> </Tabs>

Test Audio Inputs in the Playground

For a function that takes audio as input, like so:

baml

function MyFunction(myAudio: audio) -> string {
  client GPT4o
  prompt #"
    Describe this audio: {{myAudio}}
  "#
}

You can define test cases using audio files, URLs, or base64 strings.

<Tabs> <Tab title="File" language="baml"> <Warning> Committing a lot of audio files into your repository can make it slow to clone and pull your repository. If you expect to commit >500MiB of audio, please read [GitHub's size limit documentation][github-large-files] and consider setting up [large file storage][github-lfs]. </Warning>

baml

test Test1 {
  functions [MyFunction]
  args {
    myAudio {
      file "../path/to/audio.mp3"
    }
  }
}

<ParamField path="file" type="string" required="true"> The path to the audio file, relative to the directory containing the current BAML file.

audio files must be somewhere in baml_src/.

</ParamField> <ParamField path="media_type" type="string"> The mime-type of the audio. If not set, and the provider expects a mime-type to be provided, BAML will try to infer it based on first, the file extension, and second, the contents of the file. </ParamField> </Tab> <Tab title="URL" language="baml"> ```baml test Test1 { functions [MyFunction] args { myAudio { url "https...." } } } ``` <ParamField path="url" type="string" required="true"> The publicly accessible URL from which the audio may be downloaded. </ParamField> <ParamField path="media_type" type="string"> The mime-type of the audio. If not set, and the provider expects a mime-type to be provided, BAML will try to infer it based on the contents of the file. </ParamField> </Tab> <Tab title="Base64" language="baml"> ```baml test Test1 { args { myAudio { base64 "base64string" media_type "audio/mp3" } } } ``` <ParamField path="base64" type="string" required="true"> The base64-encoded audio data. </ParamField> <ParamField path="media_type" type="string"> The mime-type of the audio. If not set, and the provider expects a mime-type to be provided, BAML will try to infer it based on the contents of the file.

If base64 is a data URL, this field will be ignored.

</ParamField> </Tab> </Tabs>

Test Pdf Inputs in the Playground

For a function that takes a Pdf as input, like so:

baml

function MyFunction(myPdf: pdf) -> string {
  client GPT4o
  prompt #"
    Summarize this Pdf: {{myPdf}}
  "#
}

You can define test cases using Pdf files, URLs, or base64 strings.

<Tabs> <Tab title="File" language="baml"> <Warning> Committing a lot of Pdf files into your repository can make it slow to clone and pull your repository. If you expect to commit >500MiB of Pdfs, please read [GitHub's size limit documentation][github-large-files] and consider setting up [large file storage][github-lfs]. </Warning>

baml

test Test1 {
  functions [MyFunction]
  args {
    myPdf {
      file "../path/to/document.pdf"
    }
  }
}

<ParamField path="file" type="string" required="true"> The path to the Pdf file, relative to the directory containing the current BAML file.

Pdf files must be somewhere in baml_src/.

</ParamField> <ParamField path="media_type" type="string"> The mime-type of the Pdf. If not set, and the provider expects a mime-type to be provided, BAML will try to infer it based on first, the file extension, and second, the contents of the file. </ParamField> </Tab> <Tab title="URL" language="baml"> ```baml test Test1 { functions [MyFunction] args { myPdf { url "https...." } } } ``` <ParamField path="url" type="string" required="true"> The publicly accessible URL from which the Pdf may be downloaded. </ParamField> <ParamField path="media_type" type="string"> The mime-type of the Pdf. If not set, and the provider expects a mime-type to be provided, BAML will try to infer it based on the contents of the file. </ParamField> </Tab> <Tab title="Base64" language="baml"> ```baml test Test1 { args { myPdf { base64 "base64string" media_type "application/pdf" } } } ``` <ParamField path="base64" type="string" required="true"> The base64-encoded Pdf data. </ParamField> <ParamField path="media_type" type="string"> The mime-type of the Pdf. If not set, and the provider expects a mime-type to be provided, BAML will try to infer it based on the contents of the file.

If base64 is a data URL, this field will be ignored.

</ParamField> </Tab> </Tabs>

Test Video Inputs in the Playground

For a function that takes a video as input, like so:

baml

function MyFunction(myVideo: video) -> string {
  client GPT4o
  prompt #"
    Describe this video: {{myVideo}}
  "#
}

You can define test cases using video files, URLs, or base64 strings.

<Tabs> <Tab title="File" language="baml"> <Warning> Committing large video files into your repository can make it slow to clone and pull your repository. If you expect to commit >500MiB of videos, please read [GitHub's size limit documentation][github-large-files] and consider setting up [large file storage][github-lfs]. </Warning>

baml

test Test1 {
  functions [MyFunction]
  args {
    myVideo {
      file "../path/to/video.mp4"
    }
  }
}

<ParamField path="file" type="string" required="true"> The path to the video file, relative to the directory containing the current BAML file.

Video files must be somewhere in baml_src/.

</ParamField> <ParamField path="media_type" type="string"> The mime-type of the video. If not set, and the provider expects a mime-type to be provided, BAML will try to infer it based on first, the file extension, and second, the contents of the file. </ParamField> </Tab> <Tab title="URL" language="baml"> ```baml test Test1 { functions [MyFunction] args { myVideo { url "https...." } } } ``` <ParamField path="url" type="string" required="true"> The publicly accessible URL from which the video may be downloaded. </ParamField> <ParamField path="media_type" type="string"> The mime-type of the video. If not set, and the provider expects a mime-type to be provided, BAML will try to infer it based on the contents of the file. </ParamField> </Tab> <Tab title="Base64" language="baml"> ```baml test Test1 { args { myVideo { base64 "base64string" media_type "video/mp4" } } } ``` <ParamField path="base64" type="string" required="true"> The base64-encoded video data. </ParamField> <ParamField path="media_type" type="string"> The mime-type of the video. If not set, and the provider expects a mime-type to be provided, BAML will try to infer it based on the contents of the file.

If base64 is a data URL, this field will be ignored.

</ParamField> </Tab> </Tabs>

Assertions

Test blocks in BAML code may contain checks and asserts. These attributes behave similarly to value-level Checks and Asserts, with several additional variables available in the context of the jinja expressions you can write in a test:

The _ variable contains fields result, checks and latency_ms.
The this variable refers to the value computed by the test, and is shorthand for _.result.
In a given check or assert, _.checks.$NAME can refer to the NAME of any earlier check that was run in the same test block. By referring to prior checks, you can build compound checks and asserts, for example asserting that all checks of a certain type passed.

The following example illustrates how each of these features can be used to validate a test result.

rust

test MyTest {
  functions [EchoString]
  args {
    input "example input"
  }
  @@check( nonempty, {{ this|length > 0 }} )
  @@check( small_enough, {{ _.result|length < 1000 }} )
  @@assert( {{ _.checks.nonempty and _.checks.small_enough }})
  @@assert( {{ _.latency_ms < 1000 }})
}

@@check and @@assert behave differently:

A @@check represents a property of the test result that should either be manually checked or checked by a subsequent stage in the test. Multiple @@check predicates can fail without causing a hard failure of the test.
An @@assert represents a hard guarantee. The first failing assert will halt the remainder of the checks and asserts in this particular test.

For more information about the syntax used inside @@check and @@assert attributes, see Checks and Asserts

Dynamic Types Tests

Classes and enums marked with the @@dynamic attribute can be modified in tests using the type_builder and dynamic blocks.

The type_builder block can contain new types scoped to the parent test block and also dynamic blocks that act as modifiers for dynamic classes or enums.

Try it! Press 'Run Test' below!

{" "}

Command Line Testing

While the VSCode playground is excellent for interactive development and debugging, you can also run your tests from the command line using the BAML CLI:

bash

# Run all tests
baml-cli test

# Run tests for a specific function
baml-cli test -i "ClassifyMessage::"

# Run tests in parallel with custom concurrency
baml-cli test --parallel 5

# List available tests without running them
baml-cli test --list

See the CLI Test Reference for complete documentation of all available options, filtering capabilities, and output formats.

Production Builds

When deploying to production, you may want to reduce the size of your generated baml_client by excluding test blocks. Use the --no-tests flag with the generate command:

bash

baml-cli generate --no-tests

This strips test blocks from the inlined BAML code in the generated client, reducing bundle size without affecting runtime functionality. Your BAML functions will continue to work normally; only the embedded test definitions are removed.

See the CLI Generate Reference for more details.