Task: Update Benchmarking Documentation

Overview

Update docs/benchmarking.md and related documentation to reflect the current state of the benchmark suite, including any changes from tinybench migration and newly added benchmarks.

Current State

The docs/benchmarking.md file was created as part of the initial benchmark implementation. It needs to be reviewed and updated to ensure accuracy after:

• Benchmark.js to tinybench migration (task 001)
• CodSpeed plugin version bump (task 002)
• Query performance benchmark refinements (task 003)
• Interpreter benchmark refinements (task 004)
• Baseline establishment (task 005)

Documentation Areas to Update

1. Library References

2. Code Examples

All code examples need updating for tinybench API:

Creating a benchmark:

// Update from Benchmark.Suite to Bench
import { withCodSpeed } from '@codspeed/tinybench-plugin'
import { Bench } from 'tinybench'

const bench = withCodSpeed(new Bench({ name: 'my-benchmarks' }))

bench.add('benchmark name', async () => {
  await myAsyncOperation()
})

await bench.run()
console.table(bench.table())

Event handling:

// Update event listener pattern
bench.addEventListener('cycle', (evt) => {
  console.log(evt.task?.name, evt.task?.result)
})

3. Sections to Review

4. New Sections to Add

Consider adding:

• Migration Guide - How to migrate existing benchmarks
• Performance Baselines - Documented baseline measurements
• Regression Investigation - How to debug performance issues
• Local vs CI Results - Differences and interpretation
• CodSpeed Dashboard Guide - How to use the dashboard

Specific Updates Required

Update Import Statements

All code examples should use:

import { withCodSpeed } from '@codspeed/tinybench-plugin'
import { Bench } from 'tinybench'

Update Suite Creation

Replace all instances of:

const suite = withCodSpeed(new Benchmark.Suite('name'))

With:

const bench = withCodSpeed(new Bench({ name: 'name' }))

Update Async Pattern

Replace deferred pattern:

suite.add('name', {
  defer: true,
  fn: (deferred) => {
    asyncOp().then(() => deferred.resolve())
  },
})

With native async:

bench.add('name', async () => {
  await asyncOp()
})

Update Output Format

tinybench uses console.table(bench.table()) which outputs a different format:

┌─────────┬───────────────┬─────────────┬───────────────────┬──────────┬─────────┐
│ (index) │   Task Name   │   ops/sec   │ Average Time (ns) │  Margin  │ Samples │
├─────────┼───────────────┼─────────────┼───────────────────┼──────────┼─────────┤
│    0    │ 'benchmark1'  │ '1,234,567' │      810.005      │ '±0.50%' │  617284 │
└─────────┴───────────────┴─────────────┴───────────────────┴──────────┴─────────┘

Update Dependencies List

Document required dependencies:

{
  "devDependencies": {
    "@codspeed/tinybench-plugin": "5.0.1",
    "tinybench": "^4.0.1"
  }
}

Update AGENTS.md

The benchmarking section in AGENTS.md should also be reviewed:

• Verify benchmark file paths
• Update library references
• Ensure commands are accurate

Action Items

High Priority

1. Update all code examples to use tinybench API
2. Update "Adding New Benchmarks" section with correct templates
3. Verify all commands in "Running Benchmarks" section
4. Update troubleshooting for tinybench-specific issues

Medium Priority

5. Add performance baselines section with measurements from task 005
6. Update file listings if new benchmark files added
7. Review profiling commands for compatibility

Low Priority

8. Add CodSpeed dashboard guide with screenshots
9. Add regression investigation guide
10. Improve formatting and organization

Verification Steps

After updates:

# Verify all documented commands work
pnpm bench
pnpm bench query-performance
pnpm bench compilation
pnpm bench interpreter

# Verify profiling commands work
node --cpu-prof -r esbuild-register packages/client/src/__tests__/benchmarks/query-performance/query-performance.bench.ts

Review Checklist

• All code examples use tinybench API
• Import statements are correct
• Async patterns use native async/await
• Output format examples are accurate
• Commands in documentation work
• File paths are correct
• Dependencies are documented correctly
• AGENTS.md benchmarking section updated
• No references to Benchmark.js remain (except historical context)
• Performance targets section added (after task 005)

Dependencies

• Task 001: Migrate to tinybench (required for accurate docs)
• Task 005: Establish baselines (for performance targets section)

Related Tasks

• Task 001: Migration provides the API changes to document
• Task 003: Query benchmark changes may need documentation
• Task 004: Interpreter benchmark changes may need documentation
• Task 005: Baselines provide data for performance targets

Notes

• Keep documentation concise and practical
• Focus on "how to" over "what is"
• Include working code examples that can be copy-pasted
• Link to external resources (tinybench docs, CodSpeed docs) rather than duplicating