packages/chord/PLANNING.md
Status: Active implementation plan. Context, strict JSON, replicated state, service publication/consumption, the facet host/loader, and initial Node facet bundling and generation loading now live in Chord. Symmetric RPC and structural generation replacement remain planned. This is not a stable public API contract yet.
Chord will be the application-neutral foundation for:
The current Pi experiments prove many required behaviors, but Chord will be implemented from scratch. Existing source may be used as test and design evidence, not copied into this package. Compatibility with experimental APIs or wire messages is not a requirement.
The dependency direction is strict:
@earendil-works/chord
↑
Pi agent, protocol, server, coding agent, TUI, and future applications
Chord must:
@earendil-works/pi-* or relative paths outside packages/chord;A generic third-party dependency is not prohibited, but each dependency must be justified. The runtime should initially prefer standard JavaScript APIs. The bundler may use one pinned implementation behind a Chord-owned adapter.
The following terms must not become Chord concepts: Session, Harness, AgentLane, server, client, attachment, TUI, model, tool, hook, provider credential, or workspace. Those belong to consumers.
The names below are provisional, but the distinctions are required.
A product feature may ship multiple plugin-module entries for different application environments. Chord does not group those entries into a cross-process runtime object and does not interpret their entry names.
The implementation should be divided into these layers:
plugin loader and bundler
↓
plugin host, lifecycle, and dependency graph
↓
service tokens, providers, facades, and keyed instances
↓
replicated state and service subscriptions
↓
symmetric RPC peer and transport adapter
↓
strict JSON, invocation context, cancellation, and errors
The local service path must not require RPC serialization. The remote path must use the same service semantics through a strict wire boundary.
Current experiments depend on Pi's Harness Context, which Chord cannot import. Chord therefore needs a small neutral invocation context.
The initial context should provide:
AbortSignal for cancellation of one invocation or observation;Applications may define their own context keys. A Pi adapter can carry telemetry and authenticated identity through such keys without Chord knowing their types.
A context object never crosses RPC as a business value. The calling peer sends cancellation control and, if configured, an opaque strict-JSON metadata carrier. The receiving adapter constructs a fresh local context. The adapter, not remote business arguments, installs authenticated local identity.
Chord owns the static JsonValue contract and provides JsonRepresentation<T> plus isJsonValue() for adapter boundaries. The service runtime deliberately performs no automatic recursive validation; concrete serializers remain responsible for rejecting unsupported values. Remote arguments, results, errors, snapshots, updates, catalogues, and RPC envelopes are expected to be finite strict JSON:
undefined, sparse arrays, symbols, prototypes, cycles, classes, functions, Map, or Set; andnull, rather than undefined, for business-level absence.Application schema validation remains the application's responsibility. Chord validates structural control envelopes, but runtime enforcement of the strict-JSON boundary is currently deferred to serializers.
The intended author model is equivalent to:
interface Plugin {
readonly id: string;
setup(environment: PluginEnvironment): void;
}
Setup is synchronous declaration. It may:
Setup must not:
The host records setup calls in a private generation ledger. Plugin authors do not maintain a parallel requires/provides manifest.
After every plugin has completed setup, the host must:
A plugin may provide and consume the same token without creating a self-cycle. Optional dependencies are not part of the first contract; they require a distinct acquisition API later.
Every plugin generation owns:
Disposal is idempotent. Consumers deactivate before providers, and each plugin's resources dispose in reverse registration order. Cleanup continues after individual failures and reports one error or an AggregateError after all cleanup attempts finish.
Keyed observation handlers receive a fresh cancellable context. Closing or replacing the instance aborts only that handler task. A handler failure is reported through host policy unless its context was cancelled as normal cleanup.
Module loading and plugin activation are separate ownership domains:
interface LoadedPlugins {
readonly plugins: readonly Plugin[];
dispose(): Promise<void>;
}
interface PluginLoader {
load(): Promise<LoadedPlugins>;
}
Required loaders:
The loader owns one source generation. The host owns active plugin lifecycles. A coordinator must deactivate a retired generation before disposing its LoadedPlugins.
Node's default ESM loader retains every imported module generation in its process-wide cache. Chord therefore bundles Node facets as CommonJS and compiles each generation directly with node:vm, without inserting plugin code into either Node module cache. Chord's unload contract remains deactivation, removal of service reachability, cleanup, and release of loader-owned references/resources. Once no plugin-created timer, listener, callback, or other escaped reference remains, V8 can garbage-collect the compiled generation; collection timing is not deterministic.
Host updates are serialized and have two forms.
A targeted replacement is shape-preserving when each replaced plugin keeps the same:
The sequence is:
load replacement module generation
→ run replacement setup
→ validate replacement shape and remote implementations
→ activate replacements in dependency order while the old providers remain routed
→ replace each local or remote singleton directly without withdrawing it
→ deactivate replaced plugins in reverse dependency order
→ dispose the retired loaded generation
Properties:
There is no rollback after cutover. Terminal cleanup revokes every facet handle before best-effort disposal; only pre-cutover candidate cleanup and ordinary host disposal guarantee dependency-ordered cleanup. Committed application effects are outside the reload transaction.
Adding or removing plugins, changing service shape, or changing connection selection is structural. The first implementation should replace the complete host generation rather than attempt a partial affected-subgraph update:
load and synchronously set up the complete desired generation
→ resolve and validate its graph without activation effects
→ begin cutover
→ withdraw old providers and deactivate the old graph
→ install and activate the new graph
→ dispose the old loaded generation
Rules:
This full-generation path supplies actual plugin load and unload semantics without first implementing a complex partial-graph transaction.
Replacement never leaves a singleton facade without a target: an invocation resolves either the old implementation or its replacement. Work already running in a retired facet is not drained; later use of that facet's revoked handles may fail as stale work. A future killable-isolate host terminates such work directly. Transport disconnect still cancels only connection-owned invocations; it does not imply application-level cancellation or rollback.
An update result must distinguish:
Multiple failures are aggregated without hiding the first transition failure. Concurrent update, unload, and host-dispose requests are serialized or rejected with a stable lifecycle error.
A service token has:
Chord reserves a prefix for control-plane IDs. Duplicate IDs in one catalogue are invalid.
Two modes are required:
One token has one mode in one host graph. Mixing singleton and keyed use is an assembly or protocol error.
Process-local services:
A local singleton consumer receives a stable lazy facade, not the provider object. This removes setup-order dependence and lets provider replacement update captured methods.
Remotely exposable services use the provider/binding path even when provider and consumer share a host. An internal loopback binding keeps replacement, replicated-state, and keyed-generation semantics independent of placement. Only explicitly process-local services bypass that path.
A remotely exposable implementation may contain only own data properties classified as:
The provider derives a runtime member table from the implementation. Plugin authors do not maintain a second method/state descriptor. Unsupported accessors, fields, or member kinds are rejected before publication.
Type-level checks should reject obviously invalid remote contracts. Runtime checks remain mandatory because types do not authenticate peers or survive JavaScript consumption.
Remote methods:
void;Arbitrary returned object references, callbacks, function serialization, and general object-graph remoting are outside the initial scope.
use(token) returns a facet-owned capability view over a source-independent host service slot. Repeated acquisition within one facet returns the same view; different facet lifecycles receive different views. Member slots are created lazily on property access and validated against provider metadata when bound.
Required behavior:
undefined while unhydrated;provideMany(token) declares one keyed owner during setup and returns a generation-owned ServiceSpawner. While active, spawn(key, implementation):
An instance address is (service ID, key, generation). Reusing a closed key creates a new generation. Stale facades cannot call the replacement.
observe(token, handler):
Replicated state is authoritative one-writer latest-value replication.
The source API has an initialized value, set(value, context), and subscriptions. A remote or disconnected replica has value === undefined until hydration.
Required semantics:
State identity is structural:
provider binding + service ID + optional keyed address + member name
There is no separate state ID.
Explicit non-goals:
Chord exposes an intent-preserving JSON delta primitive and uses its operation batches internally for remote replicated state. Initial hydration and reconnection carry a complete root replacement; producers mutate tracked state and flush compact operations on publication. Replicated-state sources do not select reducers or interact with path encoders. Every client/state pairing owns an independent encoder, and sequence handling rejects gaps before a later operation can be applied.
Chord must not expose Client, Server, SessionConnection, or similar topology classes. It exposes a symmetric peer over an application-supplied duplex channel. Either endpoint may register handlers, provide services, call methods, or subscribe to state.
Application adapters own:
Chord sees one connected peer and already-decoded strict-JSON messages.
The generic RPC peer should own:
AbortController;The channel contract should be small: send one strict-JSON message, receive messages in order, observe close, and close. send() should be awaitable so adapters can provide backpressure.
A peer disconnect:
The service layer adds transport-neutral operations for:
The wire protocol contains no server ID, session ID, attachment ID, route, user identity, or host kind. A Pi router may wrap or forward Chord envelopes using its own control fields without parsing service business payloads.
Provider catalogue and subscription state must come from actual service provisions, not a handwritten application inventory.
Chord needs stable generic error codes for:
Unexpected provider exceptions become a sanitized internal error by default. Stack traces and arbitrary exception fields do not cross the wire. Applications may register or map additional stable error codes through an adapter, but Chord does not own application error taxonomies.
The first implementation must version Chord's RPC/service envelope independently of any application protocol. Version negotiation can be a peer handshake or an adapter-guaranteed constructor parameter, but incompatible peers must fail before service calls are admitted.
Member and DTO compatibility across plugin-generation skew is an application responsibility. Chord guarantees only its generic envelope semantics.
The bundler turns one or more application-declared ESM or TypeScript plugin entries into independently loadable Node CommonJS artifacts. It is not a package manager or plugin registry.
Each entry is built independently. Chord does not assume names such as server, session, tui, or web; entry names are opaque application data.
The initial bundler should accept:
The package-level API additionally accepts a plugin package directory, derives identity and version from package.json, applies application-supplied conventional entry paths when those files exist, and lets chord.facets override or disable conventions. Package discovery does not install dependencies or run lifecycle scripts.
It should emit:
Writes should use a temporary output directory followed by an atomic rename so a loader never sees a half-written generation.
@earendil-works/chord must be externalized so a plugin uses the host's one runtime and branding symbols.require, and unresolved externals must be reported deterministically.The concrete bundler engine is an implementation detail. Select it with a spike covering TypeScript and ESM inputs, CommonJS output, source maps, externals, content hashing, and programmatic diagnostics before adding a dependency.
The bundle loader must:
node:vm as a fresh generation outside Node's module caches;LoadedPlugins; andPlugin discovery, installation, version resolution, download, signature trust, and update policy remain application responsibilities.
The following existing files describe behavior that should become Chord responsibility through a rewrite:
| Existing area | Chord responsibility |
|---|---|
packages/agent/src/plugins/services/types.ts | service tokens, modes, remote contract checks, strict JSON, snapshots, updates, connection interfaces |
packages/agent/src/plugins/services/replicated-state.ts | authoritative replicated state and delivery semantics |
packages/agent/src/plugins/services/provider.ts | provider classification, calls, singleton replacement, keyed generations, snapshots |
packages/agent/src/plugins/services/namespace.ts | stable remote facades, hydration, state updates, keyed observation |
packages/coding-agent/src/experimental/facets.ts | plugin environment, dependency ledger, lifecycle graph, host, reload |
packages/coding-agent/src/experimental/facet-loader.ts | static and combined loaders plus loaded-generation ownership |
generic service sections of packages/protocol/src/protocol.ts | Chord-owned versioned service/RPC envelope |
The following must remain outside Chord:
| Existing area | Downstream responsibility |
|---|---|
packages/coding-agent/src/experimental/services/connection.ts | Pi connection state, selected-session attachment, route rebinding, Pi client adapter |
packages/coding-agent/src/experimental/services/server.ts | server-wide session directory and management implementations |
packages/coding-agent/src/experimental/services/worker.ts | Session worker host construction and Pi protocol publication adapter |
packages/server, packages/client, and process managers | framing, routing, authentication, attachment, process lifecycle, reconnect policy |
| slash-command, model, account, transcript, TUI, and agent-controller services | application contracts and plugin implementations |
source-resolver.ts and Pi internal process entrypoints | Pi source execution and process policy |
packages/agent/docs/plugins.md, packages/agent/docs/rpc.md, the experimental service tests, and the remote plugin fixture are behavioral input. They are not normative Chord APIs. Once migration finishes, generic semantics should be documented in Chord and Pi documents should cover only their host-specific contracts and adapters.
Migration should happen only after Chord passes its standalone conformance suite:
No compatibility shim is required unless separately requested.
This is a planning aid, not a requirement to create all files immediately.
packages/chord/
src/
index.ts platform-neutral public API
api.ts root-exported functions
types.ts root-exported types, including strict JSON values
context/
index.ts invocation context constants and functions
errors.ts lifecycle, service, and RPC errors
services/
types.ts tokens, contracts, modes, snapshots
state.ts source and replica state primitives
state-internals.ts private replicated-state metadata
provider.ts singleton/keyed provider runtime
facade.ts stable local and remote facades
host-bindings.ts graph-facing service slots
rpc/
peer.ts symmetric request/cancel plumbing
protocol.ts versioned generic envelopes and parsing
services.ts service protocol over a peer
loopback.ts deterministic in-memory duplex transport
plugins/
types.ts plugin and environment types
lifecycle.ts activation and resource ownership
graph.ts validation and ordering
host.ts start, update, reload, dispose
loader.ts static and combined loaders
node/
bundle.ts Node CommonJS bundler
bundle-loader.ts manifest validation and generation loading
test/
...
test-fixtures/
bundled-plugin/
README.md
PLANNING.md
If Node-only APIs are exported, they should use a separate package export such as @earendil-works/chord/node or @earendil-works/chord/bundler; importing the main runtime must not load Node-only modules.
Deliver:
Exit condition: every later work package can target explicit behavior without importing experimental implementations.
Deliver:
Tests include cold/hydrated subscriptions, immediate delivery, source update order, snapshot/update races, sequence gaps, cancellation, invalid JSON, and listener failure isolation.
Deliver:
Tests include requests initiated from both peers, crossed concurrent requests, duplicate/unknown IDs, pre-aborted requests, cancellation isolation, malformed messages, send failure, disconnect during calls, and handler exceptions.
Deliver:
RpcPeer;Tests include local and loopback paths, captured method stability, provider withdrawal/replacement, mode errors, unsupported members, strict-JSON argument/result checks, local-only isolation, keyed reuse, and concurrent callers.
Deliver:
Tests include late subscribers, updates racing hydration, multiple state members, provider replacement, keyed hydration before handlers, disconnect, reconnect, sequence gaps, stale frames, and observer errors.
Deliver:
Tests include consumer-before-provider setup, missing/duplicate/ambiguous providers, mode mismatch, cycles, asynchronous setup rejection, activation order, reverse cleanup, cleanup aggregation, connection hydration failure, and service access guards.
Deliver:
Tests include load failure cleanup, setup failure with old generation retained, unload with retained hard consumer rejection, provider gaps, captured local and remote methods, replacement snapshots, activation failure after cutover, disposal failure aggregation, concurrent update/dispose, and old-generation late publication.
Deliver:
Tests bundle an application-neutral fixture with two opaque entries and a third-party dependency, load each independently, activate it, reload changed source, prove unchanged source hashes are stable, reject corrupt manifests/integrity, and prove output resolves no Pi packages.
This work is downstream of Chord rather than an implementation dependency.
Deliver:
Exit condition: Pi depends on Chord, while Chord remains independently packable and contains no Pi imports.
The standalone suite must cover at least:
void results;Race tests should control exact points rather than use timing: subscription capture versus state update, request admission versus cancellation, provider withdrawal versus invocation, instance close versus call, reload cutover versus update publication, and host dispose versus activation.
The following decisions should be recorded in this document or small ADRs before WP1/WP2/WP7 begins:
Plugin versus Facet, Host, Peer, and Connection.None of these decisions should introduce Pi concepts into Chord.
Chord's initial scope is complete when: