Beads Performance Benchmarks

This document describes the performance benchmarks available in the beads project and how to use them.

Running Benchmarks

All Dolt Benchmarks

go test -tags=bench -bench=. -benchmem ./internal/storage/dolt/...

Specific Benchmark

go test -tags=bench -bench=BenchmarkGetReadyWork_Large -benchmem ./internal/storage/dolt/...

With CPU Profiling

go test -tags=bench -bench=BenchmarkGetReadyWork_Large -cpuprofile=cpu.prof ./internal/storage/dolt/...
go tool pprof -http=:8080 cpu.prof

Benchmark Categories

Compaction Operations

• BenchmarkGetTier1Candidates - Identify L1 compaction candidates
• BenchmarkGetTier2Candidates - Identify L2 compaction candidates
• BenchmarkCheckEligibility - Check if issue is eligible for compaction

Cycle Detection

Tests on graphs with different topologies (linear chains, trees, dense graphs):

• BenchmarkCycleDetection_Linear_100/1000/5000 - Linear dependency chains
• BenchmarkCycleDetection_Tree_100/1000 - Tree-structured dependencies
• BenchmarkCycleDetection_Dense_100/1000 - Dense graphs

Ready Work / Filtering

• BenchmarkGetReadyWork_Large - Filter unblocked issues (10K dataset)
• BenchmarkGetReadyWork_XLarge - Filter unblocked issues (20K dataset)
• BenchmarkGetReadyWork_FromJSONL - Ready work on imported database

Search Operations

• BenchmarkSearchIssues_Large_NoFilter - Search all open issues (10K dataset)
• BenchmarkSearchIssues_Large_ComplexFilter - Search with priority/status filters (10K dataset)
• BenchmarkPerfSearchTypedLabelFilter_5K - Label/type search over a 5K issue/label catalog
• BenchmarkPerfResolvePartialIDInvalidInput_5K - Invalid partial-ID rejection without a broad fallback scan

CRUD Operations

• BenchmarkCreateIssue_Large - Create new issue in 10K database
• BenchmarkUpdateIssue_Large - Update existing issue in 10K database
• BenchmarkBulkCloseIssues - Close 100 issues sequentially (NEW)

Specialized Operations

• BenchmarkLargeDescription - Handling 100KB+ issue descriptions (NEW)
• BenchmarkSyncMerge - Simulate sync cycle with create/update operations (NEW)

Recent Perf Regression References

These benchmarks cover the May 2026 Dolt hot-path changes so future perf PRs can run before/after checks against the same fixture shapes:

Measured with -benchtime=1x -benchmem -count=1 on the same host, copying this benchmark file onto each before/after ref:

Run the recent perf reference set with:

go test -run=^$ -bench='BenchmarkPerf(SearchTypedLabelFilter|ResolvePartialIDInvalidInput|AddDependencyCycleCheck|ReadyWorkLimited|BlockedIssues|ReadyWorkDeferredParentExclusion|GetIssuePrimaryFirst)' -benchtime=1x -benchmem ./internal/storage/dolt

For production-shaped CLI timeout and index experiments, use:

go run ./scripts/repro-dolt-prod-timeouts --bd ./bd --scenario all
go run ./scripts/bench-ready-indexes --dsn 'root@tcp(127.0.0.1:33307)/mc?timeout=30s&readTimeout=30s&writeTimeout=30s'

When repro-dolt-prod-timeouts targets an existing workspace with --workspace, fixture seeding defaults to --seed-mode=none; pass --seed-mode=full or --seed-mode=dep-only only when intentionally writing and committing synthetic fixture rows into that workspace.

bench-ready-indexes drops its candidate indexes again before exit by default; pass --keep-indexes only when intentionally leaving the final index set installed.

Performance Targets

Typical Results (M2 Pro)

Dataset Caching

Benchmark datasets are cached in /tmp/beads-bench-cache/:

• large.db - 10,000 issues (16.6 MB)
• xlarge.db - 20,000 issues (generated on demand)

Cached databases are reused across runs. To regenerate:

rm /tmp/beads-bench-cache/*.db

Adding New Benchmarks

Follow the pattern in sqlite_bench_test.go:

// BenchmarkMyTest benchmarks a specific operation
func BenchmarkMyTest(b *testing.B) {
	runBenchmark(b, setupLargeBenchDB, func(store *SQLiteStorage, ctx context.Context) error {
		// Your test code here
		return err
	})
}

Or for custom setup:

func BenchmarkMyTest(b *testing.B) {
	store, cleanup := setupLargeBenchDB(b)
	defer cleanup()
	ctx := context.Background()

	b.ResetTimer()
	b.ReportAllocs()

	for i := 0; i < b.N; i++ {
		// Your test code here
	}
}

CPU Profiling

The benchmark suite automatically enables CPU profiling on the first benchmark run:

CPU profiling enabled: bench-cpu-2025-12-07-174417.prof
View flamegraph: go tool pprof -http=:8080 bench-cpu-2025-12-07-174417.prof

This generates a flamegraph showing where time is spent across all benchmarks.

Performance Optimization Strategy

1. Identify bottleneck - Run benchmarks to find slow operations
2. Profile - Use CPU profiling to see which functions consume time
3. Measure - Run baseline benchmark before optimization
4. Optimize - Make targeted changes
5. Verify - Re-run benchmark to measure improvement

Example:

# Baseline
go test -tags=bench -bench=BenchmarkGetReadyWork_Large -benchmem ./internal/storage/dolt/...

# Make changes...

# Measure improvement
go test -tags=bench -bench=BenchmarkGetReadyWork_Large -benchmem ./internal/storage/dolt/...