Back to Copilotkit

Persistence and scaling

showcase/shell-docs/src/content/docs/channels/persistence-and-scaling.mdx

1.64.27.2 KB
Original Source

Intelligence durably owns provider ingress and canonical Thread history. Redis-backed claims and ordered packets coordinate active provider delivery. Your Channels process still owns SDK workflow state. Those are separate persistence boundaries.

Know what is process-local by default

Without createChannel({ store: { adapter } }), the SDK uses MemoryStore. These values disappear when the process restarts:

  • thread.state()
  • registered component snapshots used to restore callbacks
  • SDK transcript lists
  • SDK locks, deduplication windows, and queues

Managed conversation history is different: Intelligence reconstructs recent provider turns even when the SDK store is in memory.

<Callout type="warn" title="Online does not mean restart-safe"> The Intelligence **Online** state proves that the listener is connected. It does not prove that a pending approval or application workflow survives a deployment. </Callout>

Configure typed workflow state

Use a Standard Schema to type reads and validate complete replacement values:

bash
npm install zod
<FrontendOnly frontend="slack">
ts
import { createChannel } from "@copilotkit/channels";
import { z } from "zod";
import { durableStateStore } from "./state-store.js";

const workflowState = z.object({
  incidentId: z.string(),
  stage: z.enum(["triage", "awaiting-approval", "resolved"]),
});

const channel = createChannel({
  name: required("CHANNEL_CODE"),
  provider: "slack",
  agent: makeAgent,
  store: {
    adapter: durableStateStore,
    state: workflowState,
  },
});

channel.onMessage(async ({ thread, message }) => {
  const current = await thread.state();

  await thread.setState({
    incidentId: current?.incidentId ?? "INC-421",
    stage: "triage",
  });

  await thread.runAgent({ prompt: message.text });
});
</FrontendOnly> <FrontendOnly frontend="teams">
ts
import { createChannel } from "@copilotkit/channels";
import { z } from "zod";
import { durableStateStore } from "./state-store.js";

const workflowState = z.object({
  incidentId: z.string(),
  stage: z.enum(["triage", "awaiting-approval", "resolved"]),
});

const channel = createChannel({
  name: required("CHANNEL_CODE"),
  provider: "teams",
  agent: makeAgent,
  store: {
    adapter: durableStateStore,
    state: workflowState,
  },
});

channel.onMessage(async ({ thread, message }) => {
  const current = await thread.state();

  await thread.setState({
    incidentId: current?.incidentId ?? "INC-421",
    stage: "triage",
  });

  await thread.runAgent({ prompt: message.text });
});
</FrontendOnly>

setState(value) replaces the whole value. It does not merge a partial patch. Store JSON-serializable values; remote backends do not preserve Date, Map, class instances, or functions.

Implement every StateStore facet

A complete production store implements:

FacetRequired behavior
kvGet, set with optional TTL, and delete
listOldest-first append/range, cap, trim, delete, and list TTL
lockAtomic acquire plus token-checked release
dedupAtomic first-seen check with TTL
queueFIFO enqueue/dequeue/depth with bounded overflow policy

Locks, deduplication, and queues must be atomic across every process sharing the backend. Namespacing must prevent one project or Channel from reading another one's keys.

Run the SDK's Vitest conformance suite against your adapter:

bash
npm install -D vitest
ts
import { runStateStoreConformance } from "@copilotkit/channels/testing";
import { createRedisStateStore, closeRedisStateStore } from "./redis-store.js";

runStateStoreConformance(
  "redis",
  () => createRedisStateStore(),
  (store) => closeRedisStateStore(store),
);

Run it against the real backend:

bash
npx vitest run state-store.test.ts

The SDK ships MemoryStore and the conformance suite; it does not ship a fully durable Redis or Postgres implementation in the umbrella package.

Preserve interactive callbacks across restarts

A callback survives only when both conditions are true:

  1. The JSX comes from a named component registered in createChannel({ components }).
  2. The action snapshot is stored in a durable StateStore.
<FrontendOnly frontend="slack">
tsx
const channel = createChannel({
  name: required("CHANNEL_CODE"),
  provider: "slack",
  agent: makeAgent,
  components: [ApprovalCard],
  store: { adapter: durableStateStore },
});
</FrontendOnly> <FrontendOnly frontend="teams">
tsx
const channel = createChannel({
  name: required("CHANNEL_CODE"),
  provider: "teams",
  agent: makeAgent,
  components: [ApprovalCard],
  store: { adapter: durableStateStore },
});
</FrontendOnly>

Keep registered component props JSON-serializable and derive the handler from those props. Test by posting a card, restarting the process, then clicking the old card.

Match the managed concurrency model

Delivery claims (multi-replica)

Each prepared delivery has one claim winner. Every connected Runtime that declares the matching Channel can receive the invitation, check local capacity, and attempt the claim. Different replicas can serve different conversations at the same time; there is no Channel-wide leader.

Keep each replica's Channel declarations and agent code aligned. Set maxConcurrentDeliveries to a local bound each process can sustain. A Runtime that has reached that bound declines new invitations before claiming them.

Provider effects use one ordered packet at a time. The SDK retries that exact packet after a known retry wait or reconnect. If a provider call becomes uncertain, Gateway records an uncertain terminal result instead of sending it again. Application tools still need their own idempotency keys when the application may call them more than once; use message.eventId, message.turnId, or a durable operation ID.

Turn concurrency (same conversation)

Managed admission keeps one canonical Thread exclusive on a Runtime. A second invitation for that Thread is not claimed while the first delivery is active. Unrelated Threads continue in parallel up to maxConcurrentDeliveries.

This delivery guard is separate from the SDK store.concurrency setting used by other adapter paths. Choose that setting for your application-state rules; do not use it to raise the managed same-Thread delivery limit. See StoreConfig.

Production checklist

  • Prefer an agent factory; singletons are cloned per run automatically.
  • Choose store.concurrency deliberately if you need serial workflow state.
  • Provide a durable store before promising restart-safe state or callbacks.
  • Run the StateStore conformance suite against the real backend.
  • Test a restart while a card is awaiting a click.
  • Make external writes idempotent.
  • Keep Channel declarations aligned across replicas and set a local delivery concurrency bound.
  • Alert on Intelligence Conflict, Offline, and Delivery failing.
  • Stop the listener on SIGINT and SIGTERM.

Continue with history and transcripts, or use the StateStore reference.