Guardrails server

The guardrails server is a standalone backend that runs the machine learning models behind Opik's built-in guardrails. The Python SDK sends text to this service over HTTP, and the service returns whether each check passed along with detailed scores.

It powers these validation types:

• Topic — a zero-shot classifier (facebook/bart-large-mnli) that checks whether text is on or off topic.
• PII — named entity recognition (Microsoft Presidio with the spaCy en_core_web_lg model) that detects sensitive information such as names, emails, and credit card numbers.
• Prompt injection — a fine-tuned classifier (a Qwen LoRA adapter) that detects prompt injection and jailbreak attempts. Its model lives in a private Hugging Face repository, so the server needs a Hugging Face token (see Configuration).
• Custom guardrails — binary classifiers you train on your own labeled examples. The server loads them by name from its adapters directory.

The server is optional and disabled by default. You need it to run the built-in Topic, PII, or Prompt injection guardrails, or any custom guardrail. The LLM judge guardrail does not use this server: it calls the LLM provider configured in your workspace.

Installation and startup

Run alongside Opik (recommended)

If you self-host Opik with Docker Compose, start the guardrails backend by passing the --guardrails flag:

./opik.sh --guardrails

This enables the guardrails profile, starts the guardrails-backend container, and configures the Opik frontend to proxy requests to it. When Opik is started this way, the Python SDK reaches the server automatically with no extra configuration.

Run as a standalone container

You can also run the guardrails backend on its own, which is useful when you want to run it on a dedicated GPU node.

The image is not published to a registry, so build it locally from apps/opik-guardrails-backend (only needed once):

cd apps/opik-guardrails-backend
docker build -t opik-guardrails-backend:latest .

Run it with GPU acceleration:

docker run -p 5000:5000 --gpus all opik-guardrails-backend:latest

Or run it CPU-only:

docker run -p 5000:5000 opik-guardrails-backend:latest

The server listens on port 5000. Confirm it is up with the health check:

curl http://localhost:5000/healthcheck
# OK

Configuration

The server reads the following environment variables:

The model is downloaded lazily the first time the prompt injection guardrail is used, so the server starts without these variables set; only the prompt injection guardrail fails (closed) until they are provided. Pass them to the container with -e HF_TOKEN=<token> -e OPIK_GUARDRAILS_PROMPT_INJECTION_MODEL=<repo> -e OPIK_GUARDRAILS_PROMPT_INJECTION_BASE_MODEL=<base-model>.

For example, to pin inference to the second GPU:

docker run -p 5000:5000 --gpus all -e OPIK_GUARDRAILS_DEVICE=cuda:1 opik-guardrails-backend:latest

The service detects at startup whether CUDA is available. If no GPU is present, or the NVIDIA Container Toolkit is not configured, it automatically falls back to CPU mode without any configuration changes.

If you run the server at a custom address (for example, on a separate node), point the Python SDK at it with the OPIK_GUARDRAILS_URL_OVERRIDE environment variable:

export OPIK_GUARDRAILS_URL_OVERRIDE=http://my-guardrails-host:5000

Calling the server

Most users never call the server directly — the Python SDK handles the request, response parsing, and trace logging for you. Call the API directly only when integrating from a language or environment without the SDK.

Send a POST request to /api/v1/guardrails/validations with the text to check and a list of validations to run:

curl -X POST http://localhost:5000/api/v1/guardrails/validations \
  -H "Content-Type: application/json" \
  -d '{
    "text": "My name is John Doe and my email is [email protected].",
    "validations": [
      {
        "type": "TOPIC",
        "config": {
          "topics": ["politics", "religion", "artificial intelligence"],
          "threshold": 0.5,
          "mode": "restrict"
        }
      },
      {
        "type": "PII",
        "config": {
          "entities": ["PERSON", "EMAIL_ADDRESS"],
          "language": "en",
          "threshold": 0.5
        }
      }
    ]
  }'

Request fields:

• text (required) — the text to validate.
• validations (required) — a list of validations to run. Each has a type (TOPIC or PII) and a config:
  • TOPIC config:
    • topics (required) — the list of topics to check against.
    • threshold (default 0.5) — confidence threshold for topic detection.
    • mode (required) — restrict passes when none of the topics match (content filtering); allow passes when at least one matches (content classification).
  • PII config:
    • entities (optional) — entity types to detect. Defaults to IP_ADDRESS, PHONE_NUMBER, PERSON, MEDICAL_LICENSE, URL, EMAIL_ADDRESS, IBAN_CODE. See the Presidio supported entities for the full list.
    • language (default en) — language of the text.
    • threshold (default 0.5) — confidence threshold for PII detection.

The response reports an overall validation_passed flag plus per-validation results with the scores that drove each decision:

{
  "validation_passed": false,
  "validations": [
    {
      "validation_passed": false,
      "type": "PII",
      "validation_config": {
        "entities": ["PERSON", "EMAIL_ADDRESS"],
        "language": "en",
        "threshold": 0.5
      },
      "validation_details": {
        "detected_entities": {
          "PERSON": [{ "start": 11, "end": 19, "score": 0.85, "text": "John Doe" }],
          "EMAIL_ADDRESS": [{ "start": 36, "end": 56, "score": 1.0, "text": "[email protected]" }]
        }
      }
    }
  ]
}

Next steps