.agents/skills/learn-site-structure/mapping.md
The single most important fact to internalize about Learn:
source filesystem path is irrelevant for routing. A page's
URL on learn.netdata.cloud is computed from frontmatter that
ingest.py INJECTS from <repo>/docs/.map/map.yaml. Without
this mental model, every other rule seems arbitrary.
<repo>/docs/.map/map.yaml is the canonical declaration of
which files are published on Learn and where they appear. It is
a hierarchical tree of nodes; each leaf node represents one
published page.
Schema: <repo>/docs/.map/map.schema.json. Authoring guide:
<repo>/docs/.map/README.md.
Each meta block declares one publication node:
| Field | Type | Required | Notes |
|---|---|---|---|
meta.label | string | yes | Display name. Becomes sidebar_label AND drives the destination filename. |
meta.edit_url | string | yes (for doc-emitting nodes) | Must match ^https://github\.com/netdata/<repo>/edit/<branch>/.+\.(md|mdx)$. The unique key ingest uses to look up source files. Optional ONLY for category-only nodes that contain integration placeholders. |
meta.path | string | no | Single-segment override of the URL path component if it should differ from the label. |
meta.description | string | no | Frontmatter description; powers the page's meta description. |
meta.keywords | array<string> | no | Frontmatter keywords; powers in-app search and Learn page metadata. |
The schema sets additionalProperties: false. Unknown keys
fail validation and abort the run with exit code 2
(ingest/ingest.py:2815-2819).
For all repos in the source list (see pipeline.md), the
edit_url branch is master -- with one exception: the
netdata/.github repo uses main branch instead of master
(ingest.py:1232-1235).
integration_placeholder: true rows in map.yaml are
substituted at ingest time by integration pages
auto-discovered from the source repos. The populate_integrations
step (ingest.py:752-1003) finds every .md carrying the
DO NOT EDIT THIS FILE DIRECTLY marker
(INTEGRATION_MARKER, ingest.py:113), parses its hidden
<!--startmeta...endmeta--> block, buckets by category
(collectors / exporters / secretstore / functions /
authentication / cloud-notifications / agent-notifications /
logs), sorts by (learn_rel_path, sidebar_label), and
inserts the rows in place of the matching placeholder.
Result is written to
${NETDATA_REPOS_DIR}/learn/ingest/generated_map.yaml.
So integration pages do NOT need explicit map.yaml rows;
they are pulled in via the placeholder mechanism.
When ingest matches a source file's custom_edit_url to a
map.yaml row, it writes a hidden HTML-comment metadata block
at the top of the source file. After sanitize_page runs, the
comment delimiters become real YAML frontmatter (<!-- ->
---, --> -> ---).
Per ingest.py:1221-1299 plus ingest.py:1832-1897:
| Field | Source | Notes |
|---|---|---|
custom_edit_url | meta.edit_url from map.yaml | Used by the "Edit this page" link. For autogenerated grid pages, this is null. |
sidebar_label | meta.label | Drives sidebar AND destination filename. |
learn_status | "Published" or "AUTOGENERATED" | Anything other than Published is excluded from to_publish. |
learn_rel_path | reconstructed from map.yaml hierarchy | Drives the destination directory tree. root means write to docs/ directly. |
keywords | meta.keywords | Normalized to inline YAML array. CSV strings, brackets, and Python lists all accepted. |
description | meta.description | Skipped when empty/None to minimize diffs. |
sidebar_position | computed from map traversal order | Always written as quoted string sidebar_position: "10". |
slug | computed (/<learn_rel_path>/<label> lowercased) | Author-supplied slug: in source is preserved as override (ingest.py:1890-1895). |
learn_link | https://learn.netdata.cloud/docs<slug> | Updated each ingest. Validated daily by check_learn_links.py. |
Special case: paths containing Collecting Metrics get
toc_max_heading_level: 6 and toc_collapsible: true injected
(ingest.py:1251-1253).
For files marked learn_status: Published, the destination is
computed by create_mdx_path_from_metadata
(ingest.py:1140-1204):
docs/<learn_rel_path>/<sanitized_sidebar_label>.mdx
sidebar_label is sanitized for the filename (ingest.py:1159-1175):
', :, /, (, ), ,, backtick.For the URL slug (different from filename), additionally:
-.// -> /.Filename keeps original case; slug is lowercase. So
docs/Welcome to Netdata/Welcome to Netdata.mdx exists with
spaces and capitals.
Functions integrations -- if the source custom_edit_url
is under /integrations/functions/, the filename uses the
function slug from the URL stem (with - -> ) instead of
sidebar_label (ingest.py:1153-1157). Avoids collisions
when many integrations share a "Top Queries" label.
Category overview pages -- if the last two segments of
the slug are equal (e.g. /collecting-metrics/collecting-metrics),
create_mdx_path_from_metadata returns the slug with the
duplicate trimmed (ingest.py:1178-1191). On disk the file
is still written as docs/<X>/<X>.mdx; Docusaurus then
routes it to the parent URL /<X> because it has the same
slug.
Frontmatter slug: override -- if the source file
already has slug: declared in frontmatter
(custom-authored, like docs/ask-nedi.mdx), that value wins
(ingest.py:1890-1895). For these files, the
author-supplied slug must be set BEFORE the
<!-- ... --> metadata block so it survives the rewrite.
id: in mapDict -- if the metadata in mapDict carries
an id, the last segment of the URL is replaced by the id
(ingest.py:2120-2126).
README.md, index.md, _index.md, index.mdx, hidden
directories: NONE of these are special-cased in ingest.py.
The Python pipeline never publishes a file unless it appears
in map.yaml (or comes from an integration placeholder), so
filename has no routing effect.map.yaml.meta.label + the node's position in the
hierarchy, not by source path depth.map.yaml matters.<repo>/docs/getting-started-netdata/quick-tour.md.
The exact path doesn't matter for routing -- use a
reasonable location.<repo>/docs/.map/map.yaml:
- meta:
label: Quick Tour
edit_url: https://github.com/netdata/netdata/edit/master/docs/getting-started-netdata/quick-tour.md
description: A 5-minute tour of Netdata.
keywords: [tour, getting-started, intro]
https://learn.netdata.cloud/docs/<parent-slug>/quick-tour.docs/<parent>/Quick Tour.mdx (with spaces and capitals).map.yaml, move the node to its new parent.meta.edit_url unchanged -- still points to the same
source file.learn_rel_path is computed from the
new tree position; the diff between previous and current
target produces an automatic Netlify redirect from the old
URL to the new (see redirects.md).Same as a move -- the URL slug is derived from sidebar_label,
so changing meta.label changes the URL. Old URL is
auto-redirected.
The map.yaml side is just removing the node and the source
file. There is one manual step on the learn-repo side
(redirect surgery in
LegacyLearnCorrelateLinksWithGHURLs.json); see
redirects.md and recipes/delete-doc-page.md.
If two to_publish entries collide case-insensitively
(<learn_rel_path>, <sidebar_label> matches), ingest emits a
warning at ingest.py:2891-2908. On case-insensitive
filesystems (macOS / Windows), one file silently overwrites
the other. On Linux they coexist as separate files.
The lookup key is the canonical edit URL:
https://github.com/netdata/<repo>/edit/<branch>/<repo-rel-path>
(ingest.py:1232-1235). Branch is master for everything
except .github, which uses main. This is the join key
between ingest's filesystem walk and map.yaml.
If a source file's edit URL doesn't match any map.yaml row,
the file is skipped. No warning -- silent skip.
Markdown links between Learn pages must use the GitHub-relative
path with .md extension — NOT the learn.netdata.cloud URL,
and NOT a bare slug without extension. The ingest pipeline
rewrites these links to the correct Learn URLs during processing.
Correct:
See [Configuration](/docs/npm/network-flows/configuration.md) for details.
Wrong (will NOT resolve on Learn or GitHub):
See [Configuration](/network-flows/configuration) for details.
The link path is the repo-relative path to the source .md file,
prefixed with /docs/. The ingest pipeline matches it against the
map.yaml tree and rewrites it to the final Learn URL.
This also means links work natively on GitHub — readers browsing the source repo can click through to the linked file.