docs/reference/spec.md
The TOON specification is the authoritative reference for implementing encoders, decoders, and validators. It defines the concrete syntax, normative encoding/decoding behavior, and strict-mode validation rules.
You don't need this page to use TOON. It's mainly for implementers and contributors. If you're looking to learn how to use TOON, start with the Getting Started guide instead.
[!TIP] The TOON specification is stable, but also an idea in progress. Nothing's set in stone – help shape where it goes by contributing to it or sharing feedback!
Spec v{{ $spec.version }} (2025-11-24) is the current published Working Draft. It is stable for implementation but not yet finalized; see "Status of This Document" in the spec for details.
The spec defines a provisional media type and file extension in §18.2:
text/toon (provisional, not yet IANA‑registered; UTF‑8 only).toonTOON documents are always UTF‑8 with LF (\n) line endings; the optional charset parameter, when present, MUST be utf-8 per the spec.
§1 Terminology and Conventions: Defines key terms like "indentation level", "active delimiter", "strict mode", and RFC2119 keywords (MUST, SHOULD, MAY).
§2 Data Model: Specifies the JSON data model (objects, arrays, primitives), array/object ordering requirements, and canonical number formatting (no exponent notation, no leading/trailing zeros).
§3 Encoding Normalization: Defines how non-JSON types (Date, BigInt, NaN, Infinity, undefined, etc.) are normalized before encoding. Required reading for encoder implementers.
§4 Decoding Interpretation:
Specifies how decoders map text tokens to host values (quoted strings, unquoted primitives, numeric parsing with leading-zero handling). Decoders default to strict mode (strict = true) in the reference implementation; strict-mode errors are enumerated in §14.
§5 Concrete Syntax and Root Form: Defines TOON's line-oriented, indentation-based notation and how to determine whether the root is an object, array, or primitive.
§6 Header Syntax:
Normative ABNF grammar for array headers: key[N<delim?>]{fields}:. Specifies bracket segments, delimiter symbols, and field lists.
§7 Strings and Keys:
Complete quoting rules (when strings MUST be quoted), escape sequences (only \\, \", \n, \r, \t are valid), and key encoding requirements.
§8 Objects: Object field encoding (key: value), nesting rules, key order preservation, and empty object handling.
§9 Arrays: Covers all array forms: primitive (inline), arrays of objects (tabular), mixed/non-uniform (list), and arrays of arrays. Includes tabular detection requirements.
§10 Objects as List Items: Indentation rules for objects appearing in list items (first field on the hyphen line), including the canonical pattern when the first field is a tabular array (header on the hyphen line, rows at depth +2, sibling fields at depth +1).
§11 Delimiters: Delimiter scoping (document vs active), delimiter-aware quoting, and parsing rules for comma/tab/pipe delimiters.
§12 Indentation and Whitespace: Encoding requirements (consistent spaces, no tabs in indentation, no trailing spaces/newlines) and decoding rules (strict vs non-strict indentation handling).
§13 Conformance and Options: Defines conformance classes (encoder, decoder, validator), standardized options, and conformance checklists.
§13.4 Key Folding and Path Expansion: Optional encoder feature (key folding) and decoder feature (path expansion) for collapsing/expanding dotted paths, with deep-merge semantics and strict/non-strict conflict resolution.
§14 Strict Mode Errors and Diagnostics: Authoritative checklist of all strict-mode errors: array count mismatches, syntax errors, indentation errors, structural errors, and path expansion conflicts.
§15 Security Considerations: Injection risks, quoting rules, and strict-mode checks relevant to security.
§16 Internationalization: Unicode handling and locale-independent number formatting.
§17 Interoperability and Mappings: JSON/CSV/YAML mappings and conversion guidance.
§18 IANA Considerations: Media type registration plans and provisional status.
§19 TOON Core Profile: Normative subset of the most common, memory-friendly rules. Useful for minimal implementations.
Appendix G: Host Type Normalization Examples: Non-normative guidance for Go, JavaScript, Python, and Rust implementations on normalizing language-specific types.
Appendix C: Test Suite and Compliance: Reference test suite at github.com/toon-format/spec/tree/main/tests for validating implementations.
| Section | Topic | When to Read |
|---|---|---|
| §1-4 | Data model, normalization, decoding | Implementing encoders/decoders |
| §5-6 | Syntax, headers, root form | Implementing parsers |
| §7 | Strings, keys, quoting, escaping | Implementing string handling |
| §8-10 | Objects, arrays, list items | Implementing structure encoding |
| §11-12 | Delimiters, indentation, whitespace | Implementing formatting and validation |
| §13 | Conformance, options, key folding/path expansion | Implementing options and features |
| §14 | Strict-mode errors | Implementing validators |
| §15-18 | Security, i18n, interoperability, media type | Operational and ecosystem considerations |
| §19 | Core profile | Minimal implementations |
| §20-21 | Versioning, extensibility, IP | Long-term stability and licensing |
The spec includes three conformance checklists:
Key requirements:
\\, \", \n, \r, \t in quoted strings; any other escape is invalid[N] matching actual count-0 to 0, NaN/±Infinity to nullkeyFolding="safe" is enabled, folding MUST follow §13.4:
flattenDepth is set, folding MUST stop at the configured number of segments (§13.4).Key requirements:
strict=trueexpandPaths="safe" is enabled, expand dotted keys into nested objects per §13.4:
., only expand when all segments are IdentifierSegments,expandPaths="safe" and strict=true (default), MUST error on any expansion conflict (§14.5).expandPaths="safe" and strict=false, MUST apply deterministic last-write-wins (LWW) conflict resolution (§13.4).Validators should verify:
[N]The spec uses semantic versioning (major.minor):
See Appendix D: Document Changelog for detailed version history.
The spec is community-maintained at github.com/toon-format/spec. We welcome contributions of all kinds: reporting ambiguities or errors, proposing clarifications and examples, adding test cases to the reference suite, or discussing edge cases and normative behavior. Your feedback helps shape the format.