@lowdefy/build

The Lowdefy configuration compiler. Transforms YAML/JSON config files into optimized build artifacts for the runtime.

Purpose

This package is responsible for:

• Parsing YAML/JSON configuration files
• Resolving _ref imports to compose config from multiple files
• Validating config against the Lowdefy schema
• Evaluating build-time operators
• Generating optimized build artifacts for production

Why Build-Time Compilation?

Lowdefy apps are fast because expensive operations happen once at build time:

Build Pipeline

The build function orchestrates 31 steps in sequence:

async function build(options) {
  const context = createContext(options);

  // Phase 0-1: Module processing
  await fetchModules({ context });                       // Fetch GitHub tarballs, resolve local paths
  await buildModuleDefs({ components, context });        // Local resolve → validate deps → full resolve

  // Phase 2: Parse and compose configuration
  const components = await buildRefs({ context });       // Resolve _ref imports (incl. module refs)

  // Phase 3: Module integration
  buildModules({ components, context });                 // Scope IDs, resolve _module operators, merge

  // Phase 4: Schema warnings + validate
  testSchema({ components, context });                   // Emit schema warnings (non-blocking)

  // Phase 5: Build specific domains (each has its own focused validation)
  buildApp({ components, context });                     // Process app config
  validateConfig({ components, context });               // Business rule validation
  addDefaultPages({ components, context });              // Add 404, etc.
  buildAuth({ components, context });                    // Auth providers/adapters (wildcard matching)
  buildConnections({ components, context });             // Connection configs
  buildApi({ components, context });                     // API endpoints
  buildPages({ components, context });                   // Page definitions
  buildMenu({ components, context });                    // Navigation menus
  buildJs({ components, context });                      // Custom JS functions

  // Phase 6: Finalize
  addKeys({ components, context });                      // Add unique keys
  buildTypes({ components, context });                   // Resolve block/operator types
  buildImports({ components, context });                 // Track plugin imports

  // Phase 7: Write output files
  await cleanBuildDirectory({ context });
  await writeApp({ components, context });
  await writeAuth({ components, context });
  await writeConnections({ components, context });
  await writeApi({ components, context });
  await writeRequests({ components, context });
  await writePages({ components, context });
  await writeConfig({ components, context });
  await writeGlobal({ components, context });
  await writeMaps({ components, context });
  await writeMenus({ components, context });
  await writeTypes({ components, context });
  await writePluginImports({ components, context });  // includes writeGlobalsCss
  await writeJs({ components, context });
  await updateServerPackageJson({ components, context });
  await copyPublicFolder({ components, context });
}

Key Modules

Core Build Steps

Reference Resolution (buildRefs/)

The _ref operator enables modular configuration:

# lowdefy.yaml
pages:
  _ref: pages/   # Load all YAML files from pages/ directory

# pages/home.yaml
id: home
type: PageHeaderMenu
blocks:
  _ref: ../components/header.yaml  # Relative path reference

Ref types:

• Directory refs: _ref: pages/ - loads all .yaml/.json files
• File refs: _ref: header.yaml - loads single file
• JSON5 refs: _ref: config.json5 - supports JSON5 format
• Nunjucks refs: _ref: template.njk - template with variables

Walker Architecture (walker.js)

Ref resolution uses a single-pass async tree walker that handles _ref, _var, _module.var, and _build.* operators in one traversal, eliminating the JSON round-trips from the old serializer.copy-based pipeline.

Key components:

• resolve(node, ctx) — core recursive walk function
• resolveRef(refNode, parentCtx) — full 12-step ref handling (load, walk content, transform, tag ~r)
  • For _ref: { module, component/menu } — calls getModuleRefContent() to look up exports in resolved manifests
• resolveVar(node, ctx) — variable substitution with ~r provenance
• resolveModuleVar(node, ctx) — lazy module variable resolution: reads consumer value from ctx.moduleEntry.consumerVars, otherwise walks the raw default expression via resolveEffectiveVar / resolveVarDefault and caches on ctx.moduleEntry.resolvedVarCache
• cloneVarValue(value, sourceRefId) — deep clone for var values (needed because same var may appear at multiple _var sites)
• tagRefDeep(node, refId) — in-place ~r tagging (replaces old createRefReviver + serializer.copy)
• WalkContext — immutable context with child(segment) for path tracking, forRef() for entering ref files, and moduleEntry (carrying consumerVars, varDefs, resolvedVarCache) for module-scoped state

Traversal order: Top-down for _ref/_var/_module.var detection, bottom-up for _build.* evaluation. Children always resolve before their parent.

evaluateStaticOperators uses evaluateOperators from @lowdefy/operators (in-place walk, no serializer.copy).

Schema Validation (testSchema.js)

Validates config against the Lowdefy JSON schema and emits warnings (not errors). Schema validation is non-blocking — it surfaces helpful hints like typos caught by additionalProperties and property type mismatches, but does not stop the build.

Critical structural checks (required id/type, correct types) are handled by focused validations in each build step (validateBlock, buildConnections, buildEvents, etc.), which provide better error messages with full context (pageId, blockId, eventId).

Validation ownership:

Page Building (buildPages/)

Processes page definitions:

• Validates block hierarchy
• Resolves area/slot definitions
• Processes skeleton configurations
• Handles page-level properties

Each block goes through these build-time transforms in buildBlock.js:

moveAreasToSlots (deprecation transform)

At build time, if a block has areas, it is copied to slots and the areas key is deleted. If both areas and slots are present, the build throws a ConfigError. A ConfigWarning is emitted when areas is encountered, advising migration to slots.

// Before build:
{ areas: { content: { blocks: [...] } } }
// After build:
{ slots: { content: { blocks: [...] } } }

normalizeClassAndStyles (styling normalization)

Normalizes class and style config into a canonical shape. Both use . (dot) prefix for slot targeting:

1. properties.style is merged into style['.element'] (deprecation migration — the component's own style).
2. block.style plain CSS keys (no . prefix) are moved to the block slot. . prefixed keys (e.g., .element, .label) have their prefix stripped. Responsive breakpoint keys are rejected with a ConfigError.
3. block.class as a string or array is normalized to { block: value }. Object keys with . prefix have the prefix stripped.

The breakpointKeys used for validation are: ['xs', 'sm', 'md', 'lg', 'xl', '2xl']. Note: xxl has been replaced with 2xl to align with Tailwind v4 conventions.

Menu Building (buildMenu.js)

Generates navigation from:

• Explicit menu definitions
• Auto-generated from page structure
• Role-based menu filtering

Output Structure

Build artifacts go to .lowdefy/build/:

.lowdefy/build/
├── app.json           # App-level configuration
├── auth.json          # Auth providers/callbacks
├── config.json        # Runtime configuration
├── global.json        # Global state defaults
├── menus.json         # Navigation menus
├── types.json         # Type definitions map
├── keyMap.json        # Config key → source location mapping (for error tracing)
├── refMap.json        # Ref ID → source file mapping (for error tracing)
├── api/               # API endpoint configs (one per endpoint)
├── connections/       # Connection configs (one per connection)
├── pages/             # Page configs (one per page)
├── requests/          # Request configs (one per request)
├── plugins/           # Plugin import manifests + schema maps
│   ├── actionSchemas.json   # Action param schemas (for runtime validation)
│   ├── blockSchemas.json    # Block property schemas (for runtime validation)
│   └── operatorSchemas.json # Operator param schemas (for runtime validation)
└── js/                # Compiled JavaScript functions

Error Tracing Artifacts

keyMap.json and refMap.json enable config-aware error tracing:

// keyMap.json - maps internal keys to config locations
{
  "abc123": {
    "key": "pages.0.blocks.0.type",
    "~r": "ref1",    // Reference to refMap entry
    "~l": 15         // Line number in source file
  }
}

// refMap.json - maps ref IDs to source files
{
  "ref1": { "path": "pages/home.yaml" }
}

Used by resolveConfigLocation() to produce human-readable errors:

[Config Error] Block type "Buton" not found.
  pages/home.yaml:15 at pages.0.blocks.0.type
  /Users/dev/myapp/pages/home.yaml:15   ← clickable VSCode link

See architecture/error-tracing.md for details.

Design Decisions

Why JSON Output?

Build output is JSON (not YAML) because:

• Faster to parse at runtime
• No YAML parser needed in production bundle
• Consistent serialization format

Why Separate Files?

Each page/request/connection is a separate file:

• Load only what's needed per request
• Better caching granularity
• Smaller memory footprint

Why Build-Time Operator Evaluation?

Some operators run at build time:

• _ref - must compose config before runtime
• _var - build-time variables
• _build - environment-specific config

This keeps runtime fast and allows static analysis.

Plugin Type Resolution

Block and operator types are resolved at build time:

• Validates types exist in configured plugins
• Generates import map for code splitting
• Catches typos early (build fails, not runtime)

writeGlobalsCss (CSS Generation)

Replaces the old writeStyleImports. Generates globals.css which is imported by the server's _app.js. The generated file includes:

1. Layer order declaration — @layer theme, base, antd, components, utilities; to lock CSS cascade priority.
2. Tailwind CSS v4 import — @import "tailwindcss";
3. Grid CSS import — @import "@lowdefy/layout/grid.css"; for the layout grid system.
4. Optional user styles — imports public/styles.css if present (in the components layer).
5. Content source — @source "../lowdefy-build/tailwind/*.html" for Tailwind to scan per-page content files and block JS content.
6. Trigger import — @import './tailwind-candidates.css' that is rewritten on page changes to trigger CSS recompilation.
7. Antd-to-Tailwind theme bridge — @theme inline block that maps --ant-* CSS variables to Tailwind design tokens (colors, radius, font-size, font-family). Users can override these via theme.tailwind in lowdefy.yaml.

Also writes:

• Per-page tailwind HTML files from context.tailwindContentMap to lowdefy-build/tailwind/{pageId}.html
• Block plugin JS content via collectBlockSourceContent() to lowdefy-build/tailwind/_blocks.html (follows pnpm/yarn symlinks to resolve real paths)

If public/styles.less is detected, a ConfigError is thrown, advising migration.

Plugin Schema Map Generation

During writePluginImports, schema maps are generated for runtime validation:

Files: packages/build/src/build/writePluginImports/write{Action,Block,Operator}SchemaMap.js

Each function:

1. Groups used plugin types by package
2. Imports metadata/schemas from each package
3. Prioritizes custom schemas from context.typesMap.schemas over package schemas
4. Writes a JSON map: { "TypeName": { type, properties/params, ... } }

Block schemas are generated from meta.js files: writeBlockSchemaMap imports the metas export from each block package (e.g., @lowdefy/blocks-antd/metas), then calls buildBlockSchema(meta) from @lowdefy/block-utils to generate full JSON Schemas. It falls back to importing from /schemas for backward compatibility with older plugin packages. In addition to plugins/blockSchemas.json, it also writes plugins/blockMetas.json with runtime metadata (category, valueType, initValue) from context.typesMap.blockMetas or the meta objects.

Action and operator schemas are imported from a /schemas entry point:

// package.json exports
{
  "./schemas": "./dist/schemas.js"
}

// src/schemas.js
export { default as SetState } from './actions/SetState/schema.js';
export { default as Request } from './actions/Request/schema.js';

These schema maps are consumed by logClientError in @lowdefy/api for runtime validation. See api.md for details.

Integration Points

• lowdefy CLI: Calls this package for lowdefy build
• @lowdefy/api: Consumes build output files
• @lowdefy/client: Loads page configs from build output
• Plugin packages: Provide type definitions for validation

Configuration Files Supported

Build Utilities

Located in packages/build/src/utils/:

Error handling pattern:

import { ConfigError, ConfigWarning } from '@lowdefy/errors';

// Fatal error — collected via collectExceptions, build continues
collectExceptions(context, new ConfigError({
  message: `Block type "${type}" not found.`,
  configKey: block['~k'],
  checkSlug: 'types',
}));

// Warning — suppression, dedup, location resolution via handleWarning
context.handleWarning(new ConfigWarning({
  message: `Deprecated feature used.`,
  configKey: obj['~k'],
  checkSlug: 'state-refs',
  prodError: true,
}));

context.handleError and context.handleWarning are explicit functions wired in createContext.js — not logger methods. context.logger is plain pino with zero monkey-patching.

Errors and warnings can be suppressed using ~ignoreBuildChecks in config. See architecture/error-tracing.md for details.

Entry Points

Production Build

import build from '@lowdefy/build';
import { createPluginTypesMap } from '@lowdefy/build';

// Run full build (resolves all refs, all pages)
await build({
  configDirectory: './app',
  outputDirectory: './.lowdefy/build',
  // ... other options
});

// Create types map for validation
const typesMap = createPluginTypesMap(plugins);

Dev Build (JIT)

import { shallowBuild, buildPageJit, createContext } from '@lowdefy/build/dev';

// Phase 1: Skeleton build (resolves everything except page content)
const { components, pageRegistry, context } = await shallowBuild({
  customTypesMap,
  directories,
  logger,
  refResolver,
  stage: 'dev',
});

// Phase 2: Build a single page on demand
const buildContext = createContext({ directories, logger, stage: 'dev' });
Object.assign(buildContext.refMap, refMap);    // Restore from skeleton
Object.assign(buildContext.keyMap, keyMap);
buildContext.jsMap = jsMap;

await buildPageJit({
  pageId: 'tasks',
  pageRegistry,          // From shallowBuild or deserialized from pageRegistry.json
  context: buildContext,
});

Dev Build Modules

Shallow Build Stop Paths

The walker's shouldStop callback uses isPageContentPath() for semantic path matching. Any path under pages. containing a page content key segment is stopped:

const PAGE_CONTENT_KEYS = ['blocks', 'areas', 'slots', 'events', 'requests', 'layout'];

function isPageContentPath(jsonPath) {
  if (!jsonPath.startsWith('pages.')) return false;
  const segments = jsonPath.split('.');
  for (let i = 1; i < segments.length; i++) {
    if (PAGE_CONTENT_KEYS.includes(segments[i])) return true;
  }
  return false;
}

This replaces the old regex-based pathMatcher and correctly handles _build.array intermediate paths (e.g., pages._build.array.concat.0.blocks).

Page Registry Structure

// createPageRegistry extracts from shallow-built components:
registry.set(pageId, {
  pageId: page.id,
  auth: page.auth,
  type: page.type,
  refId: page['~r'] ?? null,    // For file dependency tracking
  rawContent: serializer.copy({  // Deep copy of unresolved content
    blocks: page.blocks,         // Contains _shallow markers
    areas: page.areas,
    events: page.events,
    requests: page.requests,
    layout: page.layout,
  }),
});

File Dependency Map

Maps config file paths to the page IDs that depend on them:

// createFileDependencyMap builds:
// Map<filePath, Set<pageId>>
//
// Sources of dependencies:
// 1. Page's own source file: pageEntry.refId → refMap[refId].path
// 2. Child _ref paths: collected from _shallow markers in rawContent