ContextQMD
Back to Formatjs

FormatJS Documentation - Typesense Search Integration

docs/README.md

4.5.011.6 KB
Original Source

FormatJS Documentation - Typesense Search Integration

This documentation site uses Typesense for fast, typo-tolerant search across all documentation pages.

Quick Start

1. Install Dependencies

bash
pnpm install

2. Configure Environment

bash
# Create .env file
cd docs
cp .env.example .env

Edit docs/.env and add your Typesense admin API key:

env
TYPESENSE_ADMIN_API_KEY=your_admin_key_here

Note: The search-only API key (zsQzkAsJQkHdtXxra3IzxOuhxEA4J8Qj) is hardcoded in SearchBar.tsx. This is intentional and safe - it only allows searching, not modifications.

3. Build and Upload Index

bash
# Build the Typesense index
bazel build //docs:typesense_index

# Upload to Typesense Cloud
bazel run //docs:upload_typesense_index_tool -- \
  --file=$(bazel info bazel-bin)/docs/typesense-index.jsonl

4. Test Locally

bash
# Start dev server (search key already hardcoded)
bazel run //docs:serve

# Open http://localhost:5173
# Press Cmd+K (Mac) or Ctrl+K (Windows/Linux) to test search

5. Configure GitHub Actions

Add the TYPESENSE_ADMIN_API_KEY secret to your GitHub repository:

  1. Go to Settings → Secrets and variables → Actions
  2. Click "New repository secret"
  3. Name: TYPESENSE_ADMIN_API_KEY
  4. Value: Your Typesense admin API key
  5. Click "Add secret"

The workflow will automatically update the index when docs are changed.

Architecture

Components

  1. Index Builder (scripts/build-typesense-index.ts)

    • Scans all MDX documentation files in src/docs/
    • Extracts content, headings, and metadata
    • Generates JSONL file with structured documents
    • Each heading section becomes a separate searchable document

  2. Index Uploader (scripts/upload-typesense-index.ts)

    • Connects to Typesense Cloud (cluster: 6e4uts1pzdy7wm3fp)
    • Creates/recreates the docs collection
    • Uploads all documents to Typesense
    • Runs automatically via GitHub Actions

  3. Search UI (src/components/SearchBar.tsx)

    • React component with command palette (Cmd+K / Ctrl+K)
    • Real-time search with 300ms debounce
    • Typo-tolerant search (1 typo per word)
    • Result highlighting with snippets

Document Schema

Each document in Typesense has the following fields:

typescript
{
  // Core fields
  title: string           // Document title (includes heading: "Doc Title > Heading")
  content: string         // Plain text content for this section
  url: string            // Full URL to the document/section (with anchor)

  // Metadata fields
  section: string        // Top-level section (e.g., "getting-started", "react-intl")
  page_title: string     // Document title without heading
  doc_type: string       // "guide", "api", "reference", or "example"
  tags: string[]         // Extracted keywords (e.g., ["react", "hooks", "i18n"])
  heading_level?: number // 1-6 for h1-h6 (optional, only for sections)
}

Note: Each heading section in a document becomes a separate searchable entry. The title field contains both the document title and heading (e.g., "React Intl > Installation") for better search context.

Search Configuration

  • Query fields: title,tags,page_title,content
  • Field weights: title (10x), tags (8x), page_title (5x), content (1x)
  • Typo tolerance: 1 typo per word
  • Results per page: 10
  • Highlighting: Enabled for all fields
  • Faceting: Enabled for section and doc_type (for filtering)

Search Features

Enhanced Discovery:

  • Smart tagging: Automatically extracts relevant keywords (react, i18n, hooks, etc.)
  • Document classification: Every doc is tagged as guide, API, reference, or example
  • Section filtering: Filter results by documentation section
  • Weighted search: Matches in titles and tags rank higher than body content

Visual Indicators:

  • Doc type badges: Color-coded badges show document type at a glance
    • 🔵 Guide - Tutorial and how-to content
    • 🟣 API - API reference documentation
    • 🟢 Reference - Technical reference material
    • 🟠 Example - Code examples and samples
  • Section labels: Shows which part of docs each result is from

Adding Documentation Pages

When adding new MDX pages to src/docs/, you must also:

  1. Add the page to the sidebar navigation in src/utils/navigation.ts
  2. Regenerate docs metadata: bazel run //docs:docs_metadata

File Structure

docs/
├── scripts/
│   ├── build-typesense-index.ts     # Index generator
│   └── upload-typesense-index.ts    # Index uploader
├── src/
│   ├── components/
│   │   └── SearchBar.tsx            # Typesense search UI
│   ├── utils/
│   │   └── navigation.ts           # Sidebar navigation tree
│   └── docs/                        # MDX documentation files
├── .env.example                     # Environment variable template
├── .env                            # Your keys (gitignored)
├── BUILD.bazel                      # Bazel build configuration
└── README.md                        # This file

.github/
└── workflows/
    └── typesense-index.yml          # Auto-update workflow

Bazel Targets

bash
# Build the JSONL index file
bazel build //docs:typesense_index

# Upload index to Typesense Cloud
bazel run //docs:upload_typesense_index_tool

# Build tools (used internally)
bazel build //docs:build_typesense_index_tool
bazel build //docs:upload_typesense_index_tool

# Dev server
bazel run //docs:serve

# Build production site
bazel build //docs:dist

Common Commands

Force Index Update

bash
bazel build //docs:typesense_index && \
bazel run //docs:upload_typesense_index_tool -- \
  --file=$(bazel info bazel-bin)/docs/typesense-index.jsonl

Manual Upload with Node

bash
export TYPESENSE_ADMIN_API_KEY=your_admin_key
node docs/scripts/upload-typesense-index.ts \
  --file=bazel-bin/docs/typesense-index.jsonl

Check Index Contents

bash
# View generated JSONL
cat bazel-bin/docs/typesense-index.jsonl | jq '.'

# Count documents
wc -l bazel-bin/docs/typesense-index.jsonl

Check API Connection

bash
# Test connection
curl -H "X-TYPESENSE-API-KEY: $TYPESENSE_ADMIN_API_KEY" \
  "https://6e4uts1pzdy7wm3fp.a1.typesense.net/health"

# View collection info
curl -H "X-TYPESENSE-API-KEY: $TYPESENSE_ADMIN_API_KEY" \
  "https://6e4uts1pzdy7wm3fp.a1.typesense.net/collections/docs"

Environment Variables

Frontend (Hardcoded)

All settings including the search-only API key are hardcoded in SearchBar.tsx:

  • Host: 6e4uts1pzdy7wm3fp.a1.typesense.net
  • Port: 443
  • Protocol: https
  • API Key: zsQzkAsJQkHdtXxra3IzxOuhxEA4J8Qj (search-only, safe to expose)

No environment variables needed for the frontend.

Backend/Uploads (in docs/.env or CI)

bash
TYPESENSE_ADMIN_API_KEY=xxx  # Admin key (keep secret!)

Optional overrides (defaults are fine):

bash
TYPESENSE_HOST=6e4uts1pzdy7wm3fp-1.a1.typesense.net
TYPESENSE_PORT=443
TYPESENSE_PROTOCOL=https

CI/CD

The index automatically updates when:

  • Documentation files change (docs/src/docs/**/*.mdx)
  • Index scripts change
  • Workflow file changes
  • Manually triggered (Actions → "Update Typesense Search Index" → Run workflow)

See .github/workflows/typesense-index.yml for details.

Search Features

Query Capabilities

  • Full-text search: Searches title, heading, and content
  • Weighted results: Title has 10x weight, headings 5x, content 1x
  • Typo tolerance: Automatically corrects 1 typo per word
  • Highlighting: Matching terms are highlighted in results
  • Snippets: Shows context around matching terms
  • Faceting: Results can be filtered by section

UI Features

  • Command palette: Cmd+K / Ctrl+K to open
  • Real-time search: Results update as you type (300ms debounce)
  • Section context: Shows which section each result is from
  • Direct navigation: Click to jump to exact heading
  • Loading states: Shows spinner while searching
  • Empty states: Clear messaging when no results

Search Implementation

This documentation site uses Typesense for search - a fast, typo-tolerant, server-side search engine.

Key Features

  • Fast: Server-side search with sub-millisecond response times
  • Typo-tolerant: Automatically handles typos and spelling mistakes
  • Small bundle size: ~10KB client code (vs ~150KB for client-side alternatives)
  • Built-in highlighting: Automatic result highlighting and snippets
  • Faceting: Filter by section and document type
  • Near real-time: Index updates automatically via CI/CD

Typesense Cloud

Cluster Configuration:

  • Cluster ID: 6e4uts1pzdy7wm3fp
  • Collection: docs
  • Region: Auto-selected
  • Protocol: HTTPS (port 443)

Dashboard: https://cloud.typesense.org/

Troubleshooting

Build Errors

bash
# If dependencies are missing
pnpm install

# Check Bazel targets
bazel query //docs:all

Upload Errors

"TYPESENSE_ADMIN_API_KEY environment variable is required"

  • Make sure .env file exists with TYPESENSE_ADMIN_API_KEY set
  • For scripts, export: export TYPESENSE_ADMIN_API_KEY=your_key

"Collection not found"

  • The upload script auto-creates the collection, this is normal on first run

"Invalid API key"

  • Check your API key in Typesense Cloud dashboard
  • Make sure you're using the admin key (not search-only key)

Search UI Issues

No results showing

  • Check browser DevTools → Network tab for Typesense API calls
  • Verify the collection has documents in Typesense dashboard
  • Try uploading the index again

Search not working at all

  • The search-only key is hardcoded, no config needed
  • Check browser console for errors
  • Verify network connectivity to Typesense Cloud

GitHub Actions Issues

Workflow fails on "Upload to Typesense" step

  • Make sure TYPESENSE_ADMIN_API_KEY secret is set in GitHub
  • Check workflow logs for specific error message
  • Verify the secret has the correct admin API key value

Stale Results

The index is only updated when documentation changes. To force an update:

bash
# Rebuild and upload
bazel build //docs:typesense_index && \
bazel run //docs:upload_typesense_index_tool -- \
  --file=$(bazel info bazel-bin)/docs/typesense-index.jsonl

Or trigger the GitHub Actions workflow manually.

API Keys

Getting Your Keys

  1. Log in to Typesense Cloud
  2. Select your cluster (6e4uts1pzdy7wm3fp)
  3. Go to "API Keys"
  4. You'll see:
    • Admin API Key: Use for TYPESENSE_ADMIN_API_KEY (keep secret!)
    • Search-only API Key: Already hardcoded as zsQzkAsJQkHdtXxra3IzxOuhxEA4J8Qj

Security Notes

  • Admin key: Can create/delete collections and documents. Keep this secret!
  • Search-only key: Can only perform searches. Safe to expose in frontend code.
  • The search-only key is intentionally hardcoded and visible in the browser - this is the standard pattern for client-side search.

Resources

Support

For issues with: