Back to Node Newrelic

Agent Config Schema Generator

.fleetControl/schemaGeneration/README.md

14.3.911.2 KB
Original Source

Agent Config Schema Generator

This directory contains the scripts that turn the agent's own config definition into a JSON Schema (../schemas/config.json) and manage version bumps in ../configurationDefinitions.yml for Fleet Control.

Files

FileDescription
generate-schema.jsPost-merge regenerator. Reads lib/config/default.js's definition(), writes config.json. Never touches configurationDefinitions.yml.
bump-schema-version.jsRelease-time version bumper. Compares the schema at a prior git ref to the current schema and writes a new version into configurationDefinitions.yml.
schema-diff.jsShared library (no CLI). Holds the diff classification (classifyChanges), bump arithmetic (recommendBump, applyBump, bumpVersionLine), and schema loading (loadExisting). Required by both scripts above.
tests/generate-schema.test.jsTests for the generator (comment extraction, per-leaf type inference, override precedence, generateSchema).
tests/schema-diff.test.jsTests for the shared library (classifyChanges, recommendBump, applyBump, bumpVersionLine).
tests/bump-schema-version.test.jsTests for the bump script (parsing helpers, decideBump's bootstrap/no-change/bump paths).
../schemas/config.jsonGenerated JSON Schema (Draft 2020-12).
../configurationDefinitions.ymlFleet Control metadata, including the schema's semver version. Bumped only at release time.

How the generator works

generate-schema.js:

  1. Requires lib/config/default.js directly and calls its definition() function. That function is the agent's own source of truth for config defaults, types (via each leaf's formatter), and env var overrides — there's no separate mirror file to keep in sync.
  2. Re-reads lib/config/default.js as plain text to extract the JSDoc comments documenting each key, since require() only gives you the evaluated values, not the comments above them. distributed_tracing.sampler's root/remote_parent_sampled/remote_parent_not_sampled fields are spread in from lib/config/samplers.js rather than written directly in default.js, so that file is read too and its comments reindexed under the right dotted path (mergeSamplerDescriptions).
  3. Applies the TYPE_OVERRIDES, ENUM_OVERRIDES, and EXCLUDE_KEYS configured in the script.
  4. Validates the result against the JSON Schema Draft 2020-12 meta-schema (via ajv).
  5. Writes config.json.

The generator does not touch configurationDefinitions.yml — version bumps live in the next section.

How versioning works

Schema regeneration runs post-merge, via .github/workflows/agent-config-schema.yml: once a change to the config definition lands on main, the workflow regenerates config.json and opens a separate PR with the result for review. It writes config.json and nothing else.

Version bumps run as part of release prep, via the bump-config-schema job in .github/workflows/prepare-release.yml. Once agent-release-notes has pushed the release branch, that job:

  1. Checks out the release branch and finds the latest v* tag on main.
  2. Reads the historical configurationDefinitions.yml from that tag — the version stored there is the starter version for the bump.
  3. Reads the historical schema from .fleetControl/schemas/config.json at that same tag.
  4. Compares the historical schema to the current on-disk config.json, classifies the cumulative diff, and applies the recommended bump kind (major/minor/patch).
  5. Commits the bumped configurationDefinitions.yml straight into the release branch (--write), so it rides along in the same release PR.

If the latest release tag predates the schema (no config.json, or no version in configurationDefinitions.yml, at that tag), bump-schema-version.js exits 0 with a bootstrap message and commits nothing. The first release that includes the schema ships at whatever version is currently in configurationDefinitions.yml.

Because the bump lands in the release PR itself, there's no ordering step to get wrong — reviewing and merging the release PR carries the version bump with it.

Quick start

bash
# Regenerate the schema (from repo root)
node .fleetControl/schemaGeneration/generate-schema.js
# or: npm run generate:config-schema

# Preview a release-time bump against a tag (dry-run)
node .fleetControl/schemaGeneration/bump-schema-version.js --since=v14.2.0

# Apply a release-time bump (writes configurationDefinitions.yml)
node .fleetControl/schemaGeneration/bump-schema-version.js --since=v14.2.0 --write
# or, against the latest v* tag: npm run bump:config-schema

Adding new configuration keys

New keys under defaultConfig.definition() in lib/config/default.js are picked up automatically — the generator walks that structure directly. Most of the time nothing else is needed: each leaf's formatter (boolean, int, float, array, object, objectList, regex, or an allowList.bind(...) call) tells the generator exactly what JSON Schema type — and, for allowList, what enum — to emit, and the JSDoc comment directly above the key becomes its description.

Three cases need manual handling, all configured via override maps in generate-schema.js:

  • A key accepts more than one shape. app_name is parsed with a custom formatter that splits a string on ;/,, so it accepts either a real array or a delimited string — that shape can't be inferred from a single default value, so it's declared explicitly in TYPE_OVERRIDES.
  • A key has a fixed set of values not expressed via allowList. Add it to ENUM_OVERRIDES.
  • A key's default in definition() is computed at require-time, not a stable literal — e.g. logging.filepath defaults to require('path').join(process.cwd(), ...), and serverless_mode.enabled defaults to whether an env var happens to be set. Left alone, the generator bakes in whatever that expression evaluates to on the machine that last ran it (an absolute path specific to that checkout, in the logging.filepath case — this is exactly the kind of bug the generator should never produce silently). Add a corrected schema to TYPE_OVERRIDES.

Excluding keys

Add a key's dotted path to EXCLUDE_KEYS to drop it, and everything nested under it, from the schema entirely. This schema is scoped to public-facing config — settings a user is meant to set:

js
const EXCLUDE_KEYS = new Set([
  'agent_control', // Fleet Control sets this itself.
  'logging.diagnostics',
  'infinite_tracing.trace_observer.insecure',
  'ssl' // no-op: the formatter always forces true regardless of input.
])

Missing descriptions

The generator prints every config path it wrote without a description — usually because the JSDoc comment documents a parent stanza (or, in a few spots, a single child written under its parent's comment) rather than that specific leaf. Check the printout after each run; fixing this means adding or moving a comment in lib/config/default.js, not editing the generated schema.

Checklist for new config keys

  1. Add the key to lib/config/default.js with a JSDoc comment, as usual.
  2. Run the generator. Check the inferred type in config.json.
  3. If the type or enum came out wrong, or the key needs to be hidden, add an entry to the appropriate override map above and re-run.
  4. Run the tests (npm run unit:config-schema).
  5. The version doesn't bump on post-merge regeneration. The next release will pick up your changes when someone runs the bump workflow as part of release prep.

CLI options

generate-schema.js

None — every run regenerates unconditionally and writes config.json.

bump-schema-version.js

OptionDescription
--since=<ref>Compare the current schema to the schema at <ref>. Defaults to the latest v* tag.
--writeWrite the bumped version to configurationDefinitions.yml. Without this, the script just prints the recommendation.

Exit codes

generate-schema.js

CodeMeaning
0Ran successfully. config.json reflects the current config definition, whether or not its contents actually changed.
non-zeroGenerator or meta-schema validation failure — config.json was not written.

bump-schema-version.js

CodeMeaning
0Ran successfully — whether or not a bump was applied or recommended.
non-zeroA real failure (bad ref, malformed YAML, unknown flag, etc).

Both scripts use this same two-way contract deliberately, rather than a three-way "unchanged/changed/failed" one: Node's default exit code for any uncaught exception is 1, which would collide with "changed" if that were also 1, letting a genuine crash slip through as a false success. Callers that need to know whether a file's contents actually changed check that directly — e.g. git status/git diff on config.json or configurationDefinitions.yml — rather than relying on the exit code.

Version bumping rules

bump-schema-version.js classifies each schema change and the bump kind is the highest severity across all changes:

Change typeSeverityBump
Property removedBreakingMajor
Type changedBreakingMajor
Enum value removedBreakingMajor
Enum newly introducedBreakingMajor
Required field addedBreakingMajor
additionalProperties tightened (truefalse)BreakingMajor
Property addedAdditiveMinor
Enum value addedAdditiveMinor
Enum removed entirelyAdditiveMinor
Required field removedAdditiveMinor
Default changedAdditiveMinor
additionalProperties loosened (falsetrue)AdditiveMinor
Description changedCosmeticPatch

additionalProperties is only compared when it's a plain boolean on both sides — labels and instrumentation constrain their dictionary values with an object-shaped additionalProperties instead, which isn't a bump signal. instrumentation also enumerates every currently-instrumented package as a real property (falling back to additionalProperties for anything newer); each addition/removal there is a normal property add/remove, not a bump-signal exemption.

Running the tests

bash
node --test .fleetControl/schemaGeneration/tests/

# Or via npm
npm run unit:config-schema

tests/generate-schema.test.js covers the comment-extraction scanner, per-leaf type inference (including override precedence and allowList enum extraction), the exclusion/recursion logic, and generateSchema itself — once against a small synthetic fixture (fast, isolated from the real config) and once against the actual lib/config/default.js (catches real drift). tests/schema-diff.test.js and tests/bump-schema-version.test.js cover the version-bump classification and driver logic above, entirely with synthetic schema fixtures — no git or real config involved. Every function across all three scripts takes its inputs — definitions, source text, override maps, schemas — as parameters rather than reading module-level constants directly, specifically so tests can supply synthetic ones instead of depending on production data.