Back to Data Formulator

Chart Engine Test Plan

src/lib/agents-chart/docs/test_plan.md

0.614.6 KB
Original Source

Chart Engine Test Plan

Overview

Test data lives in test-data/ as fixture generators (not executable test suites). Each file exports generator functions that produce TestCase[] arrays. The gallery UI (ChartGallery.tsx) uses TEST_GENERATORS and GALLERY_SECTIONS from test-data/index.ts to render all tests interactively.

20 test-data files, ~11,100 lines, 53 named test generators.

Test categories

CategoryFilesDescription
VL chart matricesscatter-tests, line-tests, bar-tests, area-testsMatrix-driven tests for core VL chart types
Distribution chartsdistribution-testsHistogram, Boxplot, Density, Strip Plot
Specialized chartsspecialized-testsPie, Heatmap, Lollipop, Candlestick, Waterfall, Ranged Dot, Bump, Radar, Pyramid, Rose, Custom
Semantic contextsemantic-tests39 tests validating semantic type → ChannelSemantics resolution
ECharts backendecharts-testsAll ECharts chart types (reuses VL inputs + ECharts-only types)
Chart.js backendchartjs-testsChart.js chart types
GoFish backendgofish-testsGoFish imperative rendering
Facetsfacet-testsColumn, row, col+row, wrap, clip, overflow faceting
Stress/sizingstress-tests, gas-pressure-tests, line-area-stretch-tests, discrete-axis-testsOverflow, elasticity, pressure model, discrete axis sizing
Temporaldate-testsYear, Month, YearMonth, Decade, DateTime, Hours parsing/formatting
Line/area variantsline-area-testsDotted Line, Bump Chart

Scatter Plot

A scatter plot places marks (points/bubbles) in a 2D space. The core axes are continuous (quantitative), but one or both axes can be discrete (nominal), which changes how the engine computes layout, step sizing, and overflow.

Temporal axes are omitted from scatter tests because T behaves identically to Q in scatter layout — no special handling. Temporal is still tested as a color channel (color: 'T').

Default test canvas: 300 × 300 px.

Matrix-driven approach

Tests are generated from a declarative matrix (SCATTER_MATRIX in scatter-tests.ts). Each row describes one test via its axis types, optional third channels, cardinality, and special flags. A generator function converts each matrix entry into a full TestCase.

Matrix dimensions

DimensionValuesNotes
x axis typeQ, NQuantitative, Nominal
y axis typeQ, NSame
color channel—, Q, T, NOptional 3rd encoding
size channel—, Q, NOptional 4th encoding
n (density)10–500Or 0 for N×N grid mode
cardinalityxCard, yCard, colorCard, sizeCardCardinality of nominal dims
flagshugeRangeSpecial data distributions

Full test matrix (25 tests)

Q × Q — 15 tests

#colorsizenflagswhat it tests
120Baseline scatter
2N(3)20Nominal color groups
3Q20Continuous color gradient
4T30Temporal color gradient
5Q50Bubble chart
6N(4)20Ordinal size — 4 ranked levels
7N(3)Q15Gapminder-style
8QQ30Dual continuous (4D)
9N(20)Q20hugeRangeSize 1K–1B, sqrt scale
10100Moderate density
11500High density
12N(20)200Dense, many groups
13N(50)100Legend overflow
14Q10Sparse bubbles
15Q200Dense bubbles

N × Q — 4 tests

#xCardcolorsizenwhat it tests
15Q25Strip + continuous color
25Q25Bubble strip
3230Binary category strip (edge)
4606060 cats — overflow

Q × N — 3 tests (mirrors N×Q with flipped orientation)

#yCardcolorsizenwhat it tests
15Q25Horizontal strip + continuous color
25Q25Horizontal bubble strip
36060Horizontal 60-cat overflow

N × N — 3 tests

#xCardyCardcolorsizewhat it tests
156QBubble grid
254QHeatmap-like grid
31512QLarge grid — overflow

Coverage summary

Axis combos: Q×Q, N×Q, Q×N, N×N. Third-channel variants (color and size, typed Q/T/N) crossed with Q×Q. Density from 10 to 500. Edge cases: binary categories, legend overflow, huge value ranges.

How to add a test

Add one row to SCATTER_MATRIX in scatter-tests.ts:

typescript
{ x: 'N', y: 'Q', n: 40, xCard: 8, color: 'Q', desc: 'Strip + continuous color, moderate density' },

The generator handles field naming, data synthesis, metadata, tags, and title automatically.


Line Chart

A line chart connects data points with lines. Lines imply sequential progression, so axes use T (temporal), O (ordinal), or Q (quantitative) — never purely nominal. N (nominal) is used only for color groups.

Channels: x, y, color, opacity, column, row

Matrix-driven approach

Tests are generated from LINE_MATRIX in line-tests.ts. Each row specifies axis types, optional color channel, point count, and flags like sparse (20% dropout).

Full test matrix (16 tests)

T × Q — 6 tests (core time series)

#colornflagswhat it tests
130Simple time series
2N(4)2004 series × 50 dates
3N(8)8008 series crowded
4N(20)4000stress20 series spaghetti
5N(3)180sparse3 series, ~20% missing
6Q30Continuous color gradient

O × Q — 4 tests (ordinal x)

#xCardcolornwhat it tests
755Ordinal line
812N(4)4812 ordinal × 4 series
93030Label overflow
105Q5Ordinal + gradient

Q × Q — 3 tests

#colornwhat it tests
1130Quantitative x line
12N(3)1503 parametric curves
13200Dense single curve

Q × O — 3 tests (mirror)

#yCardcolornwhat it tests
1455Horizontal ordinal
1512N(4)48Horizontal 12 ordinal × 4
163030Horizontal 30 ordinal overflow

Excluded combos

  • T×T, Q×T — date-pair data (start vs end date) doesn't suit line charts. Each row is an independent event, not a sequential series; lines connect points in data order producing random zig-zags. Better served by scatter or dumbbell charts.
  • O×O — ordinal×ordinal lines are degenerate.
  • N×N, T×N, N×T — purely nominal axes don't suit line charts. Lines imply sequence/progression; connecting unordered categories is misleading.

Coverage summary

Axis combos: T×Q, O×Q, Q×Q, Q×O. Color variants (N, Q) crossed with primary combos. Density from 5 to 4000 (stress). Sparse dropout tests irregular gaps. Total 16 tests.


Bar Chart / Stacked Bar Chart / Grouped Bar Chart

Bar charts encode values as rectangular bars. Three variants share a common matrix format in bar-tests.ts:

  • Bar Chart: x, y, color, opacity — basic bars with optional color
  • Stacked Bar Chart: x, y, color — bars stacked by color dimension
  • Grouped Bar Chart: x, y, group — bars side-by-side by group dimension

Matrix-driven approach

Three matrices (BAR_MATRIX, STACKED_BAR_MATRIX, GROUPED_BAR_MATRIX) share one generator function barMatrixToTestCase. The third channel key is 'color' for bar/stacked and 'group' for grouped.

Bar Chart matrix (19 tests)

N × Q — 6 tests (classic vertical)

#xCardcolornwhat it tests
155Basic 5 bars
22020Label rotation
33030Thin bar handling
4100100Discrete cutoff
55N(3)155 cats × 3 colors
65N(20)100Color saturation

Q × N — 3 tests (horizontal)

#yCardcolornwhat it tests
71010Horizontal 10 bars
8100100Horizontal cutoff
910N(3)30Horizontal + 3 colors

T × Q — 3 tests (temporal)

#colornwhat it tests
1024Temporal bars
11100100 dates — dynamic sizing
12N(3)72Temporal + 3 colors

Q × T — 2 tests (horizontal temporal)

#colornwhat it tests
1318Horizontal temporal
14N(3)54Horizontal temporal + color

Q × Q — 2 tests (continuous banded)

#nwhat it tests
1520Both quant — dynamic resizing
1630Equally spaced 1..30

Edge combos — 3 tests

#xynwhat it tests
17NNgridCat × cat (degenerate)
18TT20Date × date (degenerate)
19TN25Temporal × categorical

Stacked Bar Chart matrix (12 tests)

#xycolornwhat it tests
1NQN(3)12Basic stack 4×3
2NQN(5)75Large 15×5
3NQN(3)240Very large 80×3 (cutoff)
4NQQ(4)24Numeric color (1–4)
5NQQ(30)150Numeric color (1–30)
6TQN(3)30Temporal stack
7TQN(4)8020 dates × 4
8QQN(3)30Both quant stacked
9QNN(3)24Horizontal stack
10QTN(3)45Horizontal temporal stack
11NNN(3)gridCat×cat stacked (edge)
12TTN(3)30Date×date stacked (edge)

Grouped Bar Chart matrix (12 tests)

#xygroupnwhat it tests
1NQN(3)12Basic grouped 4×3
2NQ8No group — fallback
3NQN(3)270Very large 90×3 (cutoff)
4NQQ(5)30Numeric group (1–5)
5TQN(3)36Temporal grouped
6QQN(4)20Both quant + group
7QNN(4)24Horizontal grouped
8QTN(3)30Horizontal temporal grouped
9NQQ(50)400Numeric group (1–50)
10NQQ50Continuous float on group
11NNN(3)gridCat×cat grouped (edge)
12TTN(3)30Date×date grouped (edge)

Coverage summary

All three bar variants cover common xy-type combinations. Bar Chart has 19 tests, Stacked Bar has 12, Grouped Bar has 12 — total 43 bar tests. Covers horizontal/vertical orientation, discrete cutoff, numeric/continuous color, edge combos.

Area Chart & Streamgraph

File: area-tests.ts Approach: Matrix-driven — AREA_MATRIX (17 entries) + STREAMGRAPH_MATRIX (6 entries). Shared generator: areaMatrixToTestCase(entry, chartType, rand) — same infrastructure for both chart types. Data characteristic: Uses genAreaTrend() with upward drift (natural for cumulative / stacked-area metrics).

Area charts use O (ordinal) for categorical axes (like line charts) — area fills imply continuity. N (nominal) is used only for color groups. Purely nominal axis combos are excluded.

Area Chart matrix (17 tests)

T × Q — 7 tests (core stacked / layered area)

#colornflagswhat it tests
130Simple time-series area
2N(4)964 stacked series
3N(8)4808 series large stacked
4N(15)1800stress15 series stress
5N(3)1203 layered/overlapping
6N(3)180sparse3 series, ~20% missing
7Q30Continuous color gradient

O × Q — 4 tests (ordinal x)

#xCardcolornwhat it tests
855Ordinal area 5 cats
912N(4)4812 ordinal × 4 stacked
10303030 ordinal overflow
115Q5Ordinal + continuous color

Q × O — 3 tests (mirror)

#yCardcolornwhat it tests
1255Horizontal ordinal 5 cats
1312N(4)48Horizontal 12 ordinal × 4
143030Horizontal 30 ordinal overflow

Q × Q — 3 tests

#colornwhat it tests
1530Quantitative x area
16N(3)1503 stacked curves
17200Dense single-series

Excluded combos

  • T×T, Q×T — date-pair data doesn't suit area charts. Area fills imply sequential progression; T×T/Q×T lack monotonic relationships.
  • N×N, T×N, N×T — purely nominal axes don't suit area charts. Area fills imply continuity/progression; nominal axes lack this.

Streamgraph matrix (6 tests)

#xycolornwhat it tests
1TQN(5)2005 genres basic streamgraph
2TQN(10)80010 industries large
3TQN(20)300020 series stress
4TQN(5)2005 series ~20% sparse
5OQN(5)60Ordinal streamgraph
6QQN(3)150Quant-x streamgraph

Coverage summary

Area Chart covers T×Q, O×Q, Q×O, Q×Q axis combos (17 tests). Streamgraph adds 6 tests exercising T×Q, O×Q, and Q×Q with multi-series color. Total 23 area/streamgraph tests.


Grand total (matrix-driven chart tests)

Chart typeTests
Scatter25
Line16
Bar19
Stacked Bar12
Grouped Bar12
Area17
Streamgraph6
Matrix subtotal107

Plus additional non-matrix test generators:

  • Distribution charts (Histogram, Boxplot, Density, Strip)
  • Specialized charts (Pie, Heatmap, Lollipop, Candlestick, Waterfall, etc.)
  • Semantic context (39 tests)
  • Facets (9 generators)
  • Stress/sizing (4 generators)
  • Temporal (7 generators)
  • ECharts backend (24 generators)
  • Chart.js backend (11 generators)
  • GoFish backend (10 generators)

53 named test generators total across all categories.