Back to Netdata

In-app dashboard contract

.agents/skills/integrations-lifecycle/in-app-contract.md

2.11.08.4 KB
Original Source

In-app dashboard contract

The Netdata cloud-frontend dashboard (the React app that powers app.netdata.cloud) renders the Integrations page from the integrations.js artifact this repo produces. Collector dashboard taxonomy is published separately as integrations/taxonomy.json. This guide documents the contract between the two repositories so maintainers know what is and is NOT in scope when working on integrations-lifecycle changes.

The cloud-frontend repo lives at ${NETDATA_REPOS_DIR}/dashboard/cloud-frontend/ (private, Netdata-org). React component internals are explicitly OUT OF SCOPE for this skill; only the artifact contract matters.

What gets shipped

This repo produces: integrations/integrations.js (and integrations/integrations.json) on every CI run of generate-integrations.yml (or local run of gen_integrations.py). It also produces integrations/taxonomy.json from gen_taxonomy.py. All three files are gitignored in this repo.

The cloud-frontend repo consumes: integrations/integrations.js -- specifically, it copies the file into src/domains/integrations/data/integrations.js in its own source tree.

integrations/taxonomy.json is a new opt-in downstream contract. This repo validates and emits it, but cloud-frontend consumption is owned by the dashboard team and may land independently.

How the consumption works

Confirmed at ${NETDATA_REPOS_DIR}/dashboard/cloud-frontend/.github/workflows/sync-to-s3.yaml:48-66:

  1. Cloud-frontend's CI checks out netdata/netdata (this repo).
  2. Runs python3 integrations/gen_integrations.py against the freshly checked-out master.
  3. cp ./integrations/integrations.js ../src/domains/integrations/data/integrations.js into the dashboard source tree.
  4. The dashboard builds with the just-copied artifact baked in.

A second script in the dashboard repo, scripts/checkIntegrations.js, fetches https://raw.githubusercontent.com/netdata/netdata/master/integrations/integrations.json and validates against the in-tree copy as a drift detector (scripts/checkIntegrations.js:13).

A third script, scripts/checkLinks.js, validates that links in src/domains/integrations/data/integrations.js and src/domains/integrations/utils/integrations.js resolve.

At the time this skill was updated, cloud-frontend had not yet consumed taxonomy.json; its CI needs an explicit follow-up change to run python3 integrations/gen_taxonomy.py and copy the JSON artifact if/when the dashboard switches chart TOC ownership.

The artifact shape

integrations/integrations.js:

js
// DO NOT EDIT THIS FILE DIRECTLY
// It gets generated by integrations/gen_integrations.py in the Netdata repo

export const categories = [
  /* recursive tree of category objects, each:
     { id, name, description, children: [...], collector_default?: boolean }
   */
];

export const integrations = [
  /* flat array of integration objects, each carries
     integration_type, id, meta, keywords, plus the rendered
     section keys per type (e.g. setup, troubleshooting, alerts,
     metrics, functions, overview, related_resources for
     collectors) and their `clean_*` siblings (e.g. clean_setup,
     clean_alerts) */
];

integrations/taxonomy.json:

json
{
  "taxonomy_schema_version": 1,
  "source": {
    "netdata_commit": "...",
    "generated_at": "..."
  },
  "sections": [],
  "placements": [],
  "opted_out_collectors": []
}

Each taxonomy placement keeps the ordered recursive items: tree and snapshot fields generated from current metadata for CI/review diffing. resolved_contexts contains owned contexts; referenced_contexts contains display/widget references, and unresolved_references carries explicit unresolved-reference escape hatches for downstream consumers. The schema lives at integrations/schemas/taxonomy_output.json.

FE adapters must discriminate these v1 taxonomy node kinds:

  • owned_context -- structural leaf that owns one literal context.
  • group -- structural container.
  • flatten -- structural container whose children flatten into the parent menu level.
  • selector -- structural dynamic owner from context_prefix or collect_plugin.
  • context -- display widget that references contexts.
  • grid -- display container with positioned child widgets.
  • first_available -- ordered display alternatives.
  • view_switch -- whole-body replacement for multi-node vs single-node rendering.
  • string shorthand appears only in authoring; generated output normalizes it to owned_context.

All public content sections consumed by downstream renderers must be markdown strings in the generated artifacts, even when the source metadata.yaml stores them as structured YAML objects or arrays. Examples: collector-like metrics, alerts, functions, overview, setup, troubleshooting, and related_resources must pass through the renderer before reaching integrations.js / integrations.json. Leaving raw objects or arrays in these fields breaks the website Hugo renderer and produces blank tabs or link-check failures in cloud-frontend.

The dashboard's renderer interprets the {% details %} / {% /details %} markers embedded in the rendered text (the clean=False variant is the one written into the .js file). This is why the dashboard receives rich-text content with collapsible sections, while GitHub-rendered .md files use the clean=True variant where these markers are stripped.

Special case: deploy integrations

deploy entries are NOT written to disk as per-integration .md files (see per-type-matrix.md). They live ONLY inside integrations.js, sorted by quick_start integer. The dashboard's "Add Nodes" dialog is the consumer.

quick_start: -1 (or any negative) hides the entry from the dialog. Positive values define the sort order.

Drift detection

The cloud-frontend's checkIntegrations.js is the only end-to-end drift check between the two repos. It runs in the dashboard's CI and surfaces a failure if the local copy diverges from netdata/master. There is no symmetric check in this repo (this repo doesn't know what version of integrations.js the dashboard currently has baked in).

In practice this means:

  • A PR in this repo that changes metadata.yaml does NOT break the dashboard immediately. The dashboard rebuilds on its own schedule (or when its developers re-run their sync-to-s3 workflow).
  • A breaking change to the integrations.js shape (e.g. a removed top-level field) WILL break the dashboard on the next sync. There is no shape-versioning today; both repos assume the JS export shape is stable.
  • A breaking change to taxonomy.json must bump taxonomy_schema_version and coordinate with downstream consumers. Additive fields are acceptable only when old consumers can ignore them safely.

What is OUT of scope for integrations-lifecycle

  • The React renderer in cloud-frontend that turns integrations.js into UI. Not documented here.
  • The Integrations page UX, search behavior, filtering, navigation. Cloud-frontend territory.
  • The "Add Nodes" dialog flow beyond the quick_start sort contract.
  • Per-platform install commands rendering (the deploy entries' methods[].commands[]). The dashboard renders them; the metadata produces them.

Maintainer rules

  1. Treat integrations.js as a published artifact. Its shape (the two named exports, the per-integration object keys) is a contract. Avoid breaking changes; coordinate with the cloud-frontend team if a key must be renamed or removed.
  2. Treat taxonomy.json as a versioned published artifact once consumed. Keep v1 authoring closed: section_id:, ordered items:, explicit item type values, selector keys (context_prefix:, context_prefix_exclude:, collect_plugin:), widget contexts:, and sparse single_node: overrides.
  3. Render structured metadata before publication. A new integration type that reuses collector-style sections must include every structured content key in its render-key list. Do not publish raw metrics objects, alerts arrays, or similar YAML structures under the public markdown section names.
  4. Custom Jinja markers in metadata ({% details %}, {% relatedResource %}, {% if %}) are part of the contract. The dashboard's renderer interprets them. Test any new marker against both surfaces before relying on it.
  5. Do not commit integrations.js to this repo. It is gitignored on purpose; the dashboard pulls fresh on each build.