Guardrails server

The guardrails server is a standalone backend that runs the machine learning models behind Opik's built-in guardrails. The Python SDK sends text to this service over HTTP, and the service returns whether each check passed along with detailed scores.

It powers these validation types:

• Topic — a zero-shot classifier (facebook/bart-large-mnli) that checks whether text is on or off topic.
• PII — named entity recognition (Microsoft Presidio with the spaCy en_core_web_lg model) that detects sensitive information such as names, emails, and credit card numbers.
• Prompt injection — a fine-tuned classifier (a Qwen LoRA adapter) that detects prompt injection and jailbreak attempts. Its model lives in a private Hugging Face repository, so the server needs a Hugging Face token (see Configuration).
• Custom models — models fine-tuned on your own labeled examples. Training runs on this server, and it loads the results by name from its adapters directory.

The server is optional and disabled by default. You need it to run the built-in Topic, PII, or Prompt injection guardrails, or any custom model. The LLM judge guardrail does not use this server: it calls the LLM provider configured in your workspace.

Installation and startup

Run alongside Opik (recommended)

If you self-host Opik with Docker Compose, start the guardrails backend by passing the --guardrails flag:

./opik.sh --guardrails

This enables the guardrails profile, starts the guardrails-backend container, and configures the Opik frontend to proxy requests to it. When Opik is started this way, the Python SDK reaches the server automatically with no extra configuration.

Run as a standalone container

You can also run the guardrails backend on its own, which is useful when you want to run it on a dedicated GPU node.

The image is not published to a registry, so build it locally from apps/opik-guardrails-backend (only needed once):

cd apps/opik-guardrails-backend
docker build -t opik-guardrails-backend:latest .

Run it with GPU acceleration:

docker run -p 5000:5000 --gpus all opik-guardrails-backend:latest

Or run it CPU-only:

docker run -p 5000:5000 opik-guardrails-backend:latest

The server listens on port 5000. Confirm it is up with the health check:

curl http://localhost:5000/healthcheck
# OK

Configuration

The server reads the following environment variables:

The model is downloaded lazily the first time the prompt injection guardrail is used, so the server starts without these variables set; only the prompt injection guardrail fails (closed) until they are provided. Pass them to the container with -e HF_TOKEN=<token> -e OPIK_GUARDRAILS_PROMPT_INJECTION_MODEL=<repo> -e OPIK_GUARDRAILS_PROMPT_INJECTION_BASE_MODEL=<base-model>.

For example, to pin inference to the second GPU:

docker run -p 5000:5000 --gpus all -e OPIK_GUARDRAILS_DEVICE=cuda:1 opik-guardrails-backend:latest

The service detects at startup whether CUDA is available. If no GPU is present, or the NVIDIA Container Toolkit is not configured, it automatically falls back to CPU mode without any configuration changes.

If you run the server at a custom address (for example, on a separate node), point the Python SDK at it with the OPIK_GUARDRAILS_URL_OVERRIDE environment variable:

export OPIK_GUARDRAILS_URL_OVERRIDE=http://my-guardrails-host:5000

Calling the server

Most users never call the server directly — the Python SDK handles the request, response parsing, and trace logging for you. Call the API directly only when integrating from a language or environment without the SDK.

Send a POST request to /api/v1/guardrails/validations with the text to check and a list of validations to run:

curl -X POST http://localhost:5000/api/v1/guardrails/validations \
  -H "Content-Type: application/json" \
  -d '{
    "text": "My name is John Doe and my email is [email protected].",
    "validations": [
      {
        "type": "TOPIC",
        "config": {
          "topics": ["politics", "religion", "artificial intelligence"],
          "threshold": 0.5,
          "mode": "restrict"
        }
      },
      {
        "type": "PII",
        "config": {
          "entities": ["PERSON", "EMAIL_ADDRESS"],
          "language": "en",
          "threshold": 0.5
        }
      }
    ]
  }'

The response reports an overall validation_passed flag plus per-validation results with the scores that drove each decision.

- `text` (required) — the text to validate.
- `validations` (required) — a list of validations to run. Each has a `type` (`TOPIC` or `PII`) and a `config`:
  - **`TOPIC` config:**
    - `topics` (required) — the list of topics to check against.
    - `threshold` (default `0.5`) — confidence threshold for topic detection.
    - `mode` (required) — `restrict` passes when none of the topics match (content filtering); `allow` passes when at least one matches (content classification).
  - **`PII` config:**
    - `entities` (optional) — entity types to detect. Defaults to `IP_ADDRESS`, `PHONE_NUMBER`, `PERSON`, `MEDICAL_LICENSE`, `URL`, `EMAIL_ADDRESS`, `IBAN_CODE`. See the [Presidio supported entities](https://microsoft.github.io/presidio/supported_entities/) for the full list.
    - `language` (default `en`) — language of the text.
    - `threshold` (default `0.5`) — confidence threshold for PII detection.

**Example response:**

```json
{
  "validation_passed": false,
  "validations": [
    {
      "validation_passed": false,
      "type": "PII",
      "validation_config": {
        "entities": ["PERSON", "EMAIL_ADDRESS"],
        "language": "en",
        "threshold": 0.5
      },
      "validation_details": {
        "detected_entities": {
          "PERSON": [{ "start": 11, "end": 19, "score": 0.85, "text": "John Doe" }],
          "EMAIL_ADDRESS": [{ "start": 36, "end": 56, "score": 1.0, "text": "[email protected]" }]
        }
      }
    }
  ]
}
```

Next steps