crates/editor/PLAYBACK-BENCHMARKS.md
This document tracks performance benchmarks for Cap's playback and decoding system over time. Each benchmark run is timestamped, enabling comparison across different versions, configurations, and hardware.
| Metric | Target | Tolerance |
|---|---|---|
| Decoder Init | <200ms | - |
| Decode Latency (p95) | <50ms | - |
| Effective FPS | ≥30 fps | ±2 fps |
| Decode Jitter | <10ms | - |
| A/V Sync (mic↔video) | <100ms | - |
| A/V Sync (system↔video) | <100ms | - |
| Camera-Display Drift | <100ms | - |
| Format | Extension | Hardware Accel | Notes |
|---|---|---|---|
| MP4 (H.264) | .mp4 | ✅ VideoToolbox (macOS), MediaFoundation (Win) | Primary format |
| Fragmented MP4 | init.mp4 + .m4s | ✅ | Segmented recording |
| MP4 (HEVC/H.265) | .mp4 | ✅ | Higher compression |
| WebM (VP9) | .webm | ⚠️ Software fallback | Web compatibility |
# Run full playback validation on recordings from real-device-test-runner
cargo run -p cap-recording --example playback-test-runner -- full
# Run specific test categories
cargo run -p cap-recording --example playback-test-runner -- decoder
cargo run -p cap-recording --example playback-test-runner -- playback
cargo run -p cap-recording --example playback-test-runner -- audio-sync
cargo run -p cap-recording --example playback-test-runner -- camera-sync
# Test a specific recording
cargo run -p cap-recording --example playback-test-runner -- --recording-path /path/to/recording full
# List available recordings
cargo run -p cap-recording --example playback-test-runner -- list
# Benchmark decode performance on a video file
cargo run -p cap-editor --example decode-benchmark -- --video /path/to/video.mp4
# With custom FPS and iterations
cargo run -p cap-editor --example decode-benchmark -- --video /path/to/video.mp4 --fps 60 --iterations 50
# Step 1: Create test recordings with outputs kept
cargo run -p cap-recording --example real-device-test-runner -- full --keep-outputs
# Step 2: Validate playback of those recordings
cargo run -p cap-recording --example playback-test-runner -- full
| Metric | Description | How Measured |
|---|---|---|
| Init Time | Time to create and initialize decoder | spawn_decoder() latency |
| Decoder Type | Backend used (AVAssetReader, FFmpeg, MediaFoundation) | Reported by decoder |
| Hardware Accel | Whether GPU decoding is active | Decoder capability check |
| Fallback Reason | Why software decoding was used (if applicable) | Decoder error chain |
| Metric | Description | How Measured |
|---|---|---|
| Frames Decoded | Successfully retrieved frames | Count of non-None results |
| Failed Frames | Frames that failed to decode | Count of None results |
| Avg Decode Time | Mean frame retrieval latency | Avg of all frame times |
| Min/Max Decode Time | Latency range | Min/max of frame times |
| P50/P95/P99 | Latency percentiles | Sorted distribution |
| Effective FPS | Actual decode throughput | frames / elapsed_time |
| Jitter | Decode time variance (std dev) | sqrt(variance) |
| Metric | Description | How Measured |
|---|---|---|
| Mic Duration | Total mic audio length | sample_count / sample_rate |
| System Audio Duration | Total system audio length | sample_count / sample_rate |
| Video Duration | Total video length | FFmpeg duration query |
| Mic-Video Diff | Audio/video duration mismatch | |
| System-Video Diff | System audio/video mismatch | |
| Detected Sync Offset | Cross-correlation sync analysis | SyncAnalyzer result |
| Sync Confidence | Reliability of sync detection | 0.0 - 1.0 score |
| Metric | Description | How Measured |
|---|---|---|
| Camera Start Time | Recorded start timestamp | Metadata |
| Display Start Time | Display recording start | Metadata |
| Drift | Camera vs display timing offset | |
| Camera Frame Count | Frames decoded from camera | Test loop count |
| Display Frame Count | Frames decoded from display | Test loop count |
When analyzing benchmark results, focus on:
| Issue | Possible Causes | Investigation Steps |
|---|---|---|
| High Init Time | Codec probe delay, HW init | Check decoder type, try software fallback |
| Low Effective FPS | Slow decode, I/O bound | Profile decode times, check disk speed |
| High Jitter | Thread contention, GC | Check system load, memory pressure |
| A/V Sync Drift | Timestamp mismatch, sample rate | Verify audio metadata, check recording |
| Camera Drift | Start time alignment | Check segment start_time values |
| Decoder Fallback | Missing codec, HW unavailable | Check codec support, permissions |
| Resolution | Target Decode (p95) | Notes |
|---|---|---|
| 1080p (1920x1080) | <30ms | Standard target |
| 1440p (2560x1440) | <40ms | High DPI displays |
| 4K (3840x2160) | <60ms | May need HW accel |
| 5K (5120x2880) | <80ms | Apple displays |