for-ais-only/metadata_docs/IMPLEMENTATION_NOTES.md
This document captures lessons learned from implementing auto-generated metadata features for Redis documentation pages, including:
These insights should help guide future metadata feature implementations.
Lesson: Always check what Hugo provides before building custom solutions.
Context: Initial attempts tried to manually extract headers from page content using custom partials. This was complex, error-prone, and required parsing HTML/Markdown.
Solution: Hugo's .TableOfContents method already generates HTML TOC from page headings. Using this as the source was much simpler and more reliable.
Takeaway: For future metadata features, audit Hugo's built-in methods first. They often solve 80% of the problem with minimal code.
Lesson: Simple regex transformations can convert between formats more reliably than complex parsing.
Context: Converting HTML to JSON seemed like it would require a full HTML parser or complex state machine.
Solution: Breaking the conversion into small, sequential regex steps:
<nav>, </nav>)<ul> → [, </ul> → ])<li><a href="#ID">TITLE</a> → {"id":"ID","title":"TITLE")Takeaway: For format conversions, think in terms of sequential substitution patterns rather than parsing. This is often simpler and more maintainable.
Lesson: Hugo template whitespace and comments generate output that affects final formatting.
Context: Generated JSON had many blank lines, making it less readable.
Solution: Use Hugo's whitespace trimming markers ({{- and -}}) to prevent unwanted newlines.
Takeaway: When generating structured output (JSON, YAML), always consider whitespace. Test the final output, not just the template logic.
Lesson: Hugo's markdown template processor (.md files) behaves differently from HTML templates.
Context: Initial attempts to include metadata in markdown output failed because the template processor treated code blocks as boundaries.
Solution: Place metadata generation in the template itself, not in content blocks. Use safeHTML filter to prevent HTML entity escaping.
Takeaway: When targeting multiple output formats, test each format separately. Markdown templates have unique constraints that HTML templates don't have.
Lesson: Create the schema before or immediately after implementation, not after.
Context: Schema was created last, after implementation was complete.
Better approach: Define the schema first, then implement to match it. This:
Takeaway: For future metadata features, write the schema first as a specification.
config.tomlLesson: Language and client identifiers should be centralized in configuration, not hardcoded in templates.
Context: When implementing per-language metadata, we initially considered hardcoding language/client mappings in templates. This would have been error-prone and difficult to maintain.
Solution: Created a centralized clientsConfig in config.toml with:
[params.clientsConfig.Python]
langId = "python"
clientId = "redis-py"
clientName = "redis-py"
Then referenced this in templates via index $.Site.Params.clientsConfig $tabTitle.
Takeaway: For metadata that maps display names to stable identifiers, use config.toml as the single source of truth. This enables:
Lesson: For metadata that applies to individual DOM elements (not page-level), use data attributes instead of separate metadata blocks.
Context: When implementing per-codetabs metadata, we could have created separate metadata blocks for each codetabs container. Instead, we used a data-codetabs-meta attribute on the container itself.
Solution: Store JSON metadata directly in data attributes:
<div class="codetabs" data-codetabs-meta='{"Python": {"language": "python", "client": "redis-py"}, ...}'>
Benefits:
element.getAttribute()Takeaway: For element-level metadata, prefer data attributes over separate metadata blocks. This is more efficient and easier to access at runtime.
Lesson: When metadata is duplicated in multiple locations, explicitly mark which is primary and which is fallback.
Context: We embed page metadata in both <head> (script tag) and <body> (hidden div) for redundancy. Without clear marking, downstream tools couldn't determine which to use.
Solution: Added two fields to every metadata instance:
location: "head" or "body" - indicates where this copy is locatedduplicateOf: "head:data-ai-metadata" - references the primary copy (only in duplicates)Benefits:
Takeaway: When duplicating metadata for redundancy, always include location markers. This enables intelligent handling by tools and AI agents.
Lesson: When multiple metadata sources exist, document which takes precedence and why.
Context: With head and body metadata, per-codetabs metadata, and per-panel attributes, tools need to know which to use.
Solution: Added a "Metadata Precedence" section to documentation that clearly states:
duplicateOf field (indicates duplicate)Takeaway: Always document metadata precedence explicitly. This prevents tools from making incorrect assumptions and enables consistent behavior across different implementations.
Lesson: Metadata features must work across different page types with different content.
Context: Implementation was tested on data types pages and command pages, which have different metadata fields.
Takeaway: Always test on at least 2-3 different page types to ensure the feature is robust and handles optional fields correctly.
Lesson: Optional metadata fields require clear documentation about when to use them and what they mean.
Context: The buildsUpon field was added to code examples to indicate learning progression, but without clear guidance, content authors didn't know when to use it.
Solution: Created comprehensive documentation including:
Takeaway: For optional metadata fields, provide:
Lesson: Different audiences need different documentation.
Context: The buildsUpon feature needed documentation for:
Solution: Created separate documentation files:
PAGE_METADATA_FORMAT.md - Metadata structure and examplestcedocs/README.md - Content author guide with patternsBUILDSUPON_AI_AGENT_GUIDE.md - AI agent consumption patternsBUILDSUPON_VALIDATION_RULES.md - Validation rules and constraintsTakeaway: For complex features, create multiple documentation files targeting different audiences:
When implementing new metadata features, follow this order:
Identify the metadata scope
Centralize configuration (if needed)
config.toml under paramsDefine the schema (static/schemas/feature-name.json)
location and duplicateOf fieldsfor-ais-only/metadata_docs/FEATURE_NAME_FORMAT.md)
Implement the feature
Handle optional fields gracefully
if statements to only include fields when presentValidate the output
Document implementation notes
safeHTML filter when outputting HTML/JSON in markdown templates{{- and -}} to trim whitespaceconfig.tomllocation and duplicateOf fields when duplicating metadataThe Redis documentation now has a comprehensive, multi-layered metadata system:
<script type="application/json" data-ai-metadata> in <head>location: "head"<div hidden data-redis-metadata="page"> in <body>location: "body", duplicateOf: "head:data-ai-metadata"data-codetabs-meta attribute on codetabs container<div> elementsdata-lang, data-binder-id, data-codetabs-idconfig.toml for mappings that apply across pagesreplaceRE, jsonify, safeHTMLconfig.toml for centralized mappingsjsonschema library for schema validationgrep and head to inspect generated output