docs/observability/query-historical-inferences.mdx
You can query historical inferences to analyze model behavior, debug issues, export data for fine-tuning, and more. The TensorZero UI provides an interface to browse and filter historical inferences. You can also query historical inferences programmatically using the TensorZero Gateway.
<Tip>You can find a complete runnable example of this guide on GitHub.
</Tip><span style={{ display: "block" }}>
<Badge color="blue">HTTP</Badge> POST /v1/inferences/get_inferences
</span>
<span style={{ display: "block" }}>
<Badge color="orange">TensorZero SDK</Badge> client.get_inferences(...)
</span>
Retrieve specific inferences when you know their IDs.
Source of the output to return:
"inference": Returns the original model output"demonstration": Returns human-curated feedback output (ignores inferences without one)"none": Returns the inference without outputYou can retrieve inferences by ID using the TensorZero Python SDK.
from tensorzero import TensorZeroGateway
t0 = TensorZeroGateway.build_http(gateway_url="http://localhost:3000")
t0.get_inferences(ids=["00000000-0000-0000-0000-000000000000"])
You can retrieve inferences by ID using the HTTP API.
curl -X POST http://localhost:3000/v1/inferences/get_inferences \
-H "Content-Type: application/json" \
-d '{"ids": ["00000000-0000-0000-0000-000000000000"]}'
List inferences with filtering, pagination, and sorting.
<span style={{ display: "block" }}>
<Badge color="blue">HTTP</Badge> POST /v1/inferences/list_inferences
</span>
<span style={{ display: "block" }}>
<Badge color="orange">TensorZero SDK</Badge>{" "}
client.list_inferences(request=ListInferencesRequest(...))
</span>
Source of the output to return:
"inference": Returns the original model output"demonstration": Returns human-curated feedback output (ignores inferences without one)"none": Returns the inference without outputYou can list inferences with filters using the TensorZero Python SDK.
from tensorzero import TensorZeroGateway, ListInferencesRequest, InferenceFilterTag
t0 = TensorZeroGateway.build_http(gateway_url="http://localhost:3000")
t0.list_inferences(
request=ListInferencesRequest(
filters=InferenceFilterTag(
key="my_tag",
value="my_value",
comparison_operator="=",
),
limit=10,
)
)
You can list inferences with filters using the HTTP API.
curl -X POST http://localhost:3000/v1/inferences/list_inferences \
-H "Content-Type: application/json" \
-d '{
"filters": {
"type": "tag",
"key": "my_tag",
"value": "my_value",
"comparison_operator": "="
},
"limit": 10
}'