Local Development Setup - Opik

This guide provides detailed instructions for setting up your local Opik development environment. We offer multiple development modes optimized for different workflows.

Quick Start

Choose the approach that best fits your development needs:

Development Mode	Use Case	Command	Speed
Docker Mode	Testing full stack, closest to production	`./opik.sh --build`	Slow
Local Process Mode	Fast BE + FE development	`scripts/dev-runner.sh`	Fast
BE-Only Mode	Backend development only	`scripts/dev-runner.sh --be-only-restart`	Fast
Infrastructure	Manual with IDE development	`./opik.sh --infra --port-mapping`	Medium

<Tip> **Working with multiple branches?** Opik supports [multi-worktree development](#multi-worktree-support) - run multiple environments simultaneously with automatic port isolation. </Tip>

Prerequisites

Required Tools

Docker and Docker Compose - For running infrastructure services
Java 21 and Maven - For backend development
Node.js 18+ and npm - For frontend development
Python 3.10+ and pip - For SDK development

Verify Installation

bash

# Check Docker
docker --version
docker compose version

# Check Java and Maven
java -version
mvn -version

# Check Node.js and npm
node --version
npm --version

# Check Python
python --version

Development Modes

1. Docker Mode (Full Stack)

Best for: Testing complete system, integration testing, or when you need an environment closest to production.

How It Works

The opik.sh script manages Docker Compose profiles to start different combinations of services:

Infrastructure: MySQL, ClickHouse, Redis, MinIO, ZooKeeper
Backend: Java backend application
Frontend: React application
Optional: Guardrails service

Starting Opik in Docker

bash

# Build and start all services (recommended for first time)
./opik.sh --build

# Start without rebuilding (if no code changes)
./opik.sh

# Enable port mapping (useful for debugging)
./opik.sh --build --port-mapping

# Enable debug logging
./opik.sh --build --debug

Available Profiles

bash

# Infrastructure only (MySQL, Redis, ClickHouse, ZooKeeper, MinIO)
./opik.sh --infra --port-mapping

# Infrastructure + Backend services
./opik.sh --backend --port-mapping

# All services EXCEPT backend (for local backend development)
./opik.sh --local-be --port-mapping

# Add guardrails services
./opik.sh --build --guardrails

Managing Docker Services

bash

# Check service health
./opik.sh --verify

# View system status
./opik.sh --info

# Stop all services and clean up
./opik.sh --stop

# Rebuild specific service
docker compose -f deployment/docker-compose/docker-compose.yaml build backend

Accessing Services

UI: http://localhost:5173
Backend API: http://localhost:8080
MySQL: localhost:3306
ClickHouse: localhost:8123
Redis: localhost:6379

2. Local Process Mode (Fast Development)

Best for: Rapid backend and frontend development with instant code reloading.

How It Works

The dev-runner.sh script:

Starts infrastructure services in Docker (MySQL, Redis, ClickHouse, etc.)
Builds backend and runs it as a local process
Runs frontend with Vite dev server as a local process
Runs database migrations automatically

Starting Development Environment

bash

# Full restart (stop, build, start) - DEFAULT
scripts/dev-runner.sh

# Or explicitly
scripts/dev-runner.sh --restart

# Start without rebuilding (faster if no dependency changes)
scripts/dev-runner.sh --start

# Stop all services
scripts/dev-runner.sh --stop

# Check status
scripts/dev-runner.sh --verify

# View logs
scripts/dev-runner.sh --logs

Debug Mode

bash

# Enable verbose logging
scripts/dev-runner.sh --restart --debug

# Or set environment variable
DEBUG_MODE=true scripts/dev-runner.sh --restart

Service Details

Backend Process:

Port: 8080 (default, may vary with multi-worktree support)
Logs: /tmp/opik-<worktree-id>-backend.log
PID file: /tmp/opik-<worktree-id>-backend.pid
CORS enabled for local frontend
Auto-built from apps/opik-backend

Frontend Process:

Port: 5174 (default, may vary with multi-worktree support)
Logs: /tmp/opik-<worktree-id>-frontend.log
PID file: /tmp/opik-<worktree-id>-frontend.pid
Hot-reload enabled
Proxies API calls to backend

Infrastructure (Docker):

Same services as Docker mode
Ports mapped for local access (may be offset with multi-worktree support)

<Tip> Port assignments are shown when the environment starts. With multi-worktree support, ports may be offset from the defaults. </Tip>

Accessing Services

UI: http://localhost:5174 (local Vite dev server, port may vary)
Backend API: http://localhost:8080 (port may vary)
Infrastructure: Same ports as Docker mode (may be offset)

SDK Configuration

After starting, configure the SDK to use your local instance:

bash

opik configure --use_local

IMPORTANT: You must manually edit ~/.opik.config to remove /api from the URL:

ini

[opik]
# Change from:
url_override = http://localhost:8080/api/

# To:
url_override = http://localhost:8080
workspace = default

Or use environment variables:

bash

export OPIK_URL_OVERRIDE='http://localhost:8080'
export OPIK_WORKSPACE='default'

<Note> If using [multi-worktree support](#multi-worktree-support), replace `8080` with your actual backend port (shown when the environment starts). </Note>

3. BE-Only Mode (Backend Development)

Best for: Backend-focused development when you don't need to modify frontend code.

How It Works

This mode:

Starts infrastructure services in Docker
Starts frontend in Docker (pre-built)
Runs backend as a local process with hot-reload

The frontend in Docker proxies API calls to your local backend process.

Starting BE-Only Mode

bash

# Full restart (stop, build backend, start)
scripts/dev-runner.sh --be-only-restart

# Start without rebuilding
scripts/dev-runner.sh --be-only-start

# Stop services
scripts/dev-runner.sh --be-only-stop

# Check status
scripts/dev-runner.sh --be-only-verify

Service Details

Backend Process (Local):

Port: 8080
Logs: /tmp/opik-backend.log
Auto-built and hot-reloadable

Frontend (Docker):

Port: 5173 (Docker container)
Pre-built image
Proxies to localhost:8080

Infrastructure (Docker):

All infrastructure services

Accessing Services

UI: http://localhost:5173 (Docker frontend)
Backend API: http://localhost:8080 (local process)

SDK Configuration

Configure SDK without the manual edit requirement:

bash

opik configure --use_local
# Use URL: http://localhost:5173

Or with environment variables:

bash

export OPIK_URL_OVERRIDE='http://localhost:5173/api'
export OPIK_WORKSPACE='default'

4. Infrastructure Only Mode

Best for: SDK development, integration testing, or when you need just the databases.

bash

# Start only infrastructure services
./opik.sh --infra --port-mapping

# Verify infrastructure is running
./opik.sh --infra --verify

# Stop infrastructure
./opik.sh --infra --stop

This gives you access to:

MySQL on port 3306
ClickHouse on port 8123
Redis on port 6379
MinIO on port 9000

Multi-Worktree Support

Opik supports running multiple development environments simultaneously from different git worktrees. This is useful when you need to work on multiple features or compare branches side-by-side.

How It Works

Each worktree automatically gets:

Unique port assignments - All services use offset ports to avoid conflicts
Isolated Docker containers - Separate container namespaces per worktree
Separate log and PID files - No interference between worktrees

The port offset (0-99) is deterministically calculated from an MD5 hash of your project path, ensuring consistent port assignments across restarts.

Port Assignments

Service	Base Port	With Offset (e.g., 42)
Backend	8080	8122
Frontend	5174	5216
MySQL	3306	3348
Redis	6379	6421
ClickHouse HTTP	8123	8165
ClickHouse Native	9000	9042
Python Backend	8000	8042
Zookeeper	2181	2223
MinIO API	9001	9043
MinIO Console	9090	9132

Running Multiple Worktrees

bash

# Terminal 1: Main branch
cd ~/opik
scripts/dev-runner.sh --restart
# Access at ports based on hash of ~/opik

# Terminal 2: Feature branch
cd ~/opik-worktrees/feature-xyz
scripts/dev-runner.sh --restart
# Access at different ports based on hash of ~/opik-worktrees/feature-xyz

Manual Port Override

If you need specific ports (e.g., to use standard ports or avoid conflicts):

bash

# Use standard ports (offset 0)
OPIK_PORT_OFFSET=0 scripts/dev-runner.sh --restart

# Use a specific offset
OPIK_PORT_OFFSET=10 scripts/dev-runner.sh --restart

Port Collision Detection

The script automatically checks for port conflicts before starting:

bash

# If ports are in use, you'll see:
[ERROR] Port 8122 (Backend) is already in use

Port collision detected! Another process is using one or more required ports.
This might be caused by:
  - Another Opik instance running from a different worktree
  - Stale containers from a previous run
  - Other services using the same ports

To resolve:
  1. Stop other Opik instances: ./scripts/dev-runner.sh --stop
  2. Use a different port offset: export OPIK_PORT_OFFSET=<0-99>
  3. Check running processes: lsof -i :8122

Docker Container Naming

Containers are prefixed with the worktree project name:

Main repo: opik-opik-mysql-1, opik-opik-backend-1
Worktree: opik-feature-xyz-mysql-1, opik-feature-xyz-backend-1

SDK Configuration for Worktrees

Configure the SDK to use your worktree's backend port (shown when the environment starts):

bash

# Configure SDK (use the backend port shown at startup)
export OPIK_URL_OVERRIDE='http://localhost:8080'  # or your worktree's port
export OPIK_WORKSPACE='default'

Or edit ~/.opik.config:

ini

[opik]
url_override = http://localhost:8122
workspace = default

Windows Development

All scripts have PowerShell equivalents for Windows developers.

Docker Mode (Windows)

powershell

# Build and start all services
.\opik.ps1 --build

# Different profiles
.\opik.ps1 --infra --port-mapping
.\opik.ps1 --backend --port-mapping
.\opik.ps1 --local-be --port-mapping

# Manage services
.\opik.ps1 --verify
.\opik.ps1 --stop

Local Process Mode (Windows)

powershell

# Full restart
scripts\dev-runner.ps1

# Specific commands
scripts\dev-runner.ps1 --restart
scripts\dev-runner.ps1 --start
scripts\dev-runner.ps1 --stop
scripts\dev-runner.ps1 --verify

# BE-only mode
scripts\dev-runner.ps1 --be-only-restart

# Debug mode
scripts\dev-runner.ps1 --restart --debug

Windows-Specific Notes

Logs location: $env:TEMP directory
PID files: $env:TEMP directory
Use Get-Content -Wait instead of tail -f for log following
Configuration file: $env:USERPROFILE\.opik.config

Common Development Tasks

Building Components

bash

# Build backend only
scripts/dev-runner.sh --build-be

# Build frontend only
scripts/dev-runner.sh --build-fe

# Lint backend
scripts/dev-runner.sh --lint-be

# Lint frontend
scripts/dev-runner.sh --lint-fe

Database Migrations

bash

# Run migrations only
scripts/dev-runner.sh --migrate

# This will:
# 1. Start infrastructure if not running
# 2. Build backend if needed
# 3. Run MySQL migrations
# 4. Run ClickHouse migrations

If migrations fail, you may need to clean up:

bash

# Stop all services
scripts/dev-runner.sh --stop  # or ./opik.sh --stop

# Remove Opik Docker volumes (WARNING: DATA LOSS - removes Opik databases)
./opik.sh --clean

# Restart
scripts/dev-runner.sh --restart

Viewing Logs

bash

# Show recent logs (last 20 lines)
scripts/dev-runner.sh --logs

# Follow logs in real-time
tail -f /tmp/opik-backend.log
tail -f /tmp/opik-frontend.log

# On Windows
Get-Content -Wait $env:TEMP\opik-backend.log
Get-Content -Wait $env:TEMP\opik-frontend.log

Working with Docker Services

bash

# View all Opik containers
docker ps --filter "name=opik-"

# View logs from Docker services
docker logs -f opik-backend-1
docker logs -f opik-frontend-1
docker logs -f opik-clickhouse-1

# Execute commands in containers
docker exec -it opik-mysql-1 mysql -u root -p
docker exec -it opik-clickhouse-1 clickhouse-client

# Restart a specific Docker service
docker restart opik-backend-1

Troubleshooting

Services Won't Start

bash

# Check Docker is running
docker info

# Check port conflicts (ports shown when environment starts)
lsof -i :5174  # Frontend (default)
lsof -i :8080  # Backend (default)
lsof -i :3306  # MySQL (default)
lsof -i :8123  # ClickHouse (default)

# On Windows
Get-NetTCPConnection -LocalPort 5174
Get-NetTCPConnection -LocalPort 8080

<Tip> If you have port conflicts from another worktree, you can override the port offset: ```bash OPIK_PORT_OFFSET=0 scripts/dev-runner.sh --restart ``` </Tip>

Build Failures

bash

# Clean backend build
cd apps/opik-backend
mvn clean
mvn spotless:apply  # Fix formatting issues
mvn clean install

# Clean frontend build
cd apps/opik-frontend
rm -rf node_modules
npm install
npm run lint

Database Connection Issues

bash

# Check MySQL is accessible
docker exec -it opik-mysql-1 mysql -u root -p

# Check ClickHouse is accessible
docker exec -it opik-clickhouse-1 clickhouse-client

# Or via HTTP
echo 'SELECT version()' | curl -H 'X-ClickHouse-User: opik' -H 'X-ClickHouse-Key: opik' 'http://localhost:8123/' -d @-

Process Management Issues

bash

# Kill stuck backend process
pkill -f "opik-backend.*jar"

# Kill stuck frontend process
pkill -f "vite.*opik-frontend"

# On Windows
Get-Process | Where-Object {$_.Path -like "*opik-backend*"} | Stop-Process -Force
Get-Process | Where-Object {$_.Path -like "*opik-frontend*"} | Stop-Process -Force

Clean Slate Restart

bash

# Complete cleanup and restart
scripts/dev-runner.sh --stop
./opik.sh --clean  # WARNING: Deletes Opik data
scripts/dev-runner.sh --restart

Development Workflow Examples

Backend Feature Development

bash

# 1. Start BE-only mode (fastest for backend work)
scripts/dev-runner.sh --be-only-restart

# 2. Make changes in apps/opik-backend

# 3. Rebuild and restart backend
scripts/dev-runner.sh --build-be
scripts/dev-runner.sh --be-only-start

# 4. Test changes via UI at http://localhost:5173

Frontend Feature Development

bash

# 1. Start local process mode
scripts/dev-runner.sh --restart

# 2. Make changes in apps/opik-frontend
# Frontend hot-reloads automatically

# 3. View changes at http://localhost:5174

Full Stack Feature Development

bash

# 1. Start local process mode
scripts/dev-runner.sh --restart

# 2. Make changes to backend and frontend
# Frontend changes hot-reload
# Backend changes require rebuild:

scripts/dev-runner.sh --build-be
# Backend automatically restarts

# 3. Test at http://localhost:5174

SDK Development

bash

# 1. Start infrastructure only
./opik.sh --infra --port-mapping

# 2. Start backend separately if needed
cd apps/opik-backend
mvn clean install
java -jar target/opik-backend-*.jar server config.yml

# 3. Configure SDK
opik configure --use_local

# 4. Test SDK changes
cd sdks/python
pip install -e .
pytest tests/e2e

Integration Testing

bash

# 1. Start full Docker stack
./opik.sh --build

# 2. Run tests against full environment
cd tests_end_to_end
pytest tests/

# 3. Clean up
./opik.sh --stop

Performance Tips

Use local process mode for fastest development cycle
Use BE-only mode if you're not changing frontend
Use --start instead of --restart when dependencies haven't changed
Enable debug mode only when needed - it increases log verbosity
Keep Docker images up to date - rebuild periodically with --build

Best Practices

Always run linters before committing:

bash

scripts/dev-runner.sh --lint-be
scripts/dev-runner.sh --lint-fe

Test migrations locally before committing:
bash
```
scripts/dev-runner.sh --migrate
```

Clean up regularly to free disk space:

bash

# Clean up Opik Docker resources (WARNING: DATA LOSS - removes Opik databases)
./opik.sh --clean  # Removes Opik containers and volumes

# Or clean up dangling Docker containers, networks, images (affects all projects)
docker system prune

Use debug mode for troubleshooting:
bash
```
scripts/dev-runner.sh --restart --debug
```
Check service status before reporting issues:
bash
```
scripts/dev-runner.sh --verify
./opik.sh --verify
```

IDE Configuration

Opik provides AI coding rules and configurations for editors like Cursor, Codex, and Claude Code. These are stored in .agents/ and can be synced to your editor of choice using Makefile.

Setup

bash

# For Cursor users - creates .cursor symlink to .agents/
make cursor

# For Codex users - creates .codex symlink and generates Codex-compatible AGENTS files
make codex

# For Claude Code users - syncs rules to .claude/ and generates .mcp.json
make claude

make codex keeps .codex linked to .agents and generates AGENTS.override.md plus .agents/generated/codex/rules/*.md so Codex can consume rule content derived from .mdc files.

Directory Structure

.agents/
├── rules/                    # AI coding rules (.mdc files)
│   ├── *.mdc                 # Root-level rules (git, clean-code, etc.)
│   ├── apps/                 # App-specific rules
│   │   ├── opik-backend/     # Java backend rules
│   │   └── opik-frontend/    # React frontend rules
│   └── sdks/                 # SDK-specific rules
├── commands/                 # Slash commands
└── mcp.json                  # MCP server configuration

Git Hooks

Install pre-commit hooks to automatically run linting before commits:

bash

# Install hooks
make hooks

# Remove hooks
make hooks-remove

Next Steps

Backend Development Guide - Deep dive into backend development
Frontend Development Guide - Deep dive into frontend development
Python SDK Development Guide - SDK contribution guidelines
Testing Guide - Running and writing tests