Back to Beads

Federation Setup Guide

FEDERATION-SETUP.md

1.0.34.7 KB
Original Source

Federation Setup Guide

Federation enables peer-to-peer synchronization of beads databases between multiple workspaces using Dolt remotes. Each workspace maintains its own database while sharing work items with configured peers.

Overview

Federation uses Dolt's distributed version control capabilities to sync issue data between independent teams or locations. Key benefits:

  • Peer-to-peer: No central server required; each town is autonomous
  • Database-native versioning: Built on Dolt's version control, not file exports
  • Flexible infrastructure: Works with DoltHub, S3, GCS, local paths, or SSH
  • Data sovereignty: Configurable tiers for compliance (GDPR, regional laws)

Prerequisites

  1. Dolt backend: Federation requires the Dolt storage backend (the only supported backend)

Configuration

Enable Federation-Compatible Sync

Edit .beads/config.yaml or ~/.config/bd/config.yaml:

yaml
federation:
  remote: dolthub://myorg/beads          # Primary remote (optional)
  sovereignty: T2                        # Data sovereignty tier

Or via environment variables:

bash
export BD_FEDERATION_REMOTE="dolthub://myorg/beads"
export BD_FEDERATION_SOVEREIGNTY="T2"

Data Sovereignty Tiers

TierDescriptionUse Case
T1No restrictionsPublic data
T2Organization-levelRegional/company compliance
T3PseudonymousIdentifiers removed
T4AnonymousMaximum privacy

Adding Federation Peers

Use bd federation add-peer to register remote peers:

bash
bd federation add-peer <name> <endpoint>

Peer Name Rules

  • Must start with a letter
  • Alphanumeric, dash, and underscore only
  • Reserved names not allowed: origin, main, master, HEAD

Supported Endpoint Formats

FormatExampleDescription
DoltHubdolthub://org/repoDoltHub hosted repository
Google Cloudgs://bucket/pathGoogle Cloud Storage
Amazon S3s3://bucket/pathAmazon S3
Localfile:///path/to/backupLocal filesystem
HTTPShttps://host/pathHTTPS remote
SSHssh://host/pathSSH remote
Git SSHgit@host:pathGit SSH shorthand

Examples

bash
# Add a staging environment on DoltHub
bd federation add-peer staging dolthub://myorg/staging-beads

# Add a cloud backup
bd federation add-peer backup gs://mybucket/beads-backup
bd federation add-peer backup-s3 s3://mybucket/beads-backup

# Add a local backup
bd federation add-peer local file:///home/user/beads-backup

# Add a partner organization
bd federation add-peer partner-town dolthub://partner-org/beads

JSON Output

For scripting, use the --json flag:

bash
bd --json federation add-peer staging dolthub://myorg/staging-beads
# {"name":"staging","endpoint":"dolthub://myorg/staging-beads","status":"added"}

Verify Configuration

Check stored peers:

bash
bd config list | grep federation.peers

Architecture Notes

How It Works

  1. Each workspace has its own Dolt database
  2. add-peer registers a Dolt remote (similar to git remote add)
  3. Push/pull operations sync commits between peers
  4. Conflict resolution follows configured strategy

Multi-Repo Support

Issues track their SourceSystem to identify which federated system created them. This enables proper attribution and trust chains across organizations.

Connectivity

Remote connectivity is validated on first push/pull operation, not when adding the peer. This allows configuring remotes before infrastructure is ready.

Planned Features

The following operations have infrastructure support but are not yet exposed as commands:

  • bd federation list-peers - List configured peers
  • bd federation push <peer> - Push to specific peer
  • bd federation pull <peer> - Pull from specific peer
  • bd federation sync <peer> - Bidirectional sync

Troubleshooting

"requires direct database access"

Federation commands require the Dolt backend with direct database access. Ensure you have the Dolt backend configured for federation operations.

"peer already exists"

A peer with that name is already configured. Use a different name or check existing peers with bd config list | grep federation.peers.

Invalid endpoint format

Ensure your endpoint matches one of the supported formats above. The scheme must be one of: dolthub://, gs://, s3://, file://, https://, ssh://, or git SSH format (git@host:path).

Reference

  • Configuration: See docs/CONFIG.md for all federation settings
  • Source: cmd/bd/federation.go
  • Storage interfaces: internal/storage/versioned.go
  • Dolt implementation: internal/storage/dolt/store.go