Back to Oh My Openagent

Performance Benchmarks

packages/shared-skills/skills/data-scientist/references/performance-benchmarks.md

4.19.42.5 KB
Original Source

Performance Benchmarks

Routing heuristics distilled from public 2024-2025 results in the db-benchmark suite (rendered at h2oai.github.io/db-benchmark) and from the vendors' own documentation (duckdb.org, pola.rs).

Read these as routing guidance, not guarantees. Multipliers vary with dataset size, cardinality, data types, and hardware. When the choice materially matters, measure on the actual data.

Operation Performance Comparison

Operation TypeWinnerIndicative advantageWhen to Use
FilteringPolarsoften the fastest by a wide margin (SIMD, predicate pushdown)Single-table filters
SortingPolarstypically the fastestORDER BY operations, ranking
JoinsDuckDBtypically faster; richer join typesMulti-table joins, especially complex joins
AggregationsDuckDBtypically faster on large datasetsGROUP BY, complex aggregations
Window FuncsPolarstypically fasterRANK, LAG/LEAD, running totals
TransformationsPolarstypically fasterPivot, melt, string operations
Direct QueryDuckDBavoids loading to memory entirelyAd-hoc exploration without loading to memory
StreamingPolarshandles datasets larger than RAMDatasets larger than RAM

Both tools are decisively faster than pandas on 1M+ row workloads, which is why pandas is banned outright in this skill.

Operation Detection Keywords

Automatically detect operation types from user requests:

  • Filter: "where", "filter", "condition", "select rows", "find"
  • Sort: "sort", "order", "top", "bottom", "rank"
  • Join: "join", "merge", "combine tables"
  • Aggregate: "group", "sum", "avg", "count", "mean", "total", "aggregate"
  • Transform: "pivot", "melt", "reshape", "string operations", "clean", "transform"
  • Window: "running", "cumulative", "lag", "lead", "rank", "partition"

Benchmarking Notes

  • Public suites exercise typical analytical workloads (1M-100M rows).
  • Performance advantages are approximate and vary by dataset characteristics.
  • Real-world performance depends on data types, cardinality, and hardware.