Back to Mem0

Pipeline mechanics

skills/mem0-integrate/references/pipeline.md

2.0.1916.0 KB
Original Source

Pipeline mechanics

Full step-by-step for mem0-integrate. Read this when you are executing a step. The one-line-per-step overview and every non-negotiable rule live in ../SKILL.md, which is loaded on every run; this file is loaded on demand.

Verbatim subagent system prompts for steps 8 and 10 are in subagent-prompts.md.

1. Language detection

SignalTrack
package.json + TypeScript configNode / TypeScript
package.json (no TS config)Node / JavaScript
pyproject.toml or requirements.txtPython

Monorepo with both, ask which subdirectory to operate in, then recurse.

2. Repo comprehension: what does this repo do, and where is the backend?

Before any decision (product, goal, plan), understand the repo enough to locate where in the backend the integration belongs. This is not fit-surveying, the user already decided Mem0 fits. This is mechanics: you cannot write a plan without knowing what files matter.

Read, in order, with a token budget. Do not scan the whole tree.

  1. README.md (root) plus the first page of any README_*.md variants.
  2. CONTRIBUTING.md / AGENTS.md / CLAUDE.md at root if present. These often spell out architecture and entry points.
  3. package.json / pyproject.toml scripts and entry points.
  4. The layout of the top two directory levels, not recursive.
  5. Key config files: docker-compose.yml, Dockerfile, Makefile, langgraph.json, next.config.*, nuxt.config.*.

Produce .mem0-integration/repo-summary.md:

# Repo comprehension

**What this repo does:** <one paragraph in plain English. Who is
the end user? What does the app do for them? What LLM / agent
behavior is central? Do not list dependencies, describe behavior.>

**Architecture at a glance:**
  - Backend: <path(s), framework, primary entry point>
  - Frontend: <path(s) if any, framework, for context only; no
    integration here>
  - Agent loop / orchestration: <LangGraph? custom? none?>
  - Existing memory/session/state systems: <name them, these are
    what step 6 Coexistence must preserve>

**Candidate backend integration surfaces** (ranked, best first):
  1. `<backend-file>:<line_range>` <function> <one-sentence
     reason this is where write/read could slot in without
     replacing anything existing>
  2. ...
  3. ...

**Not a fit here:** <list anything the skill considered but ruled
out, e.g. "frontend chat component: client-side, excluded by
backend-only rule"; "existing memory subsystem X: would require
replacement, excluded by additive principle">

**Sources read:** <list the files actually opened, with line counts,
so reviewers can verify coverage.>

Show the user the rendered summary and ask: "Is this understanding correct? Which of the candidate surfaces (1, 2, 3 ...) should step 3 forward target?"

Gate rules:

  • No backend surface found, exit code 1. Preconditions should already have caught frontend-only repos; reaching this point means a subtler miss (for example the "backend" is actually just a static build). Do not force a fit.
  • Every candidate surface would require replacing an existing memory or session system, exit code 1 with the additive-principle rationale. The user can point at a non-conflicting location manually and re-run.
  • User corrections update repo-summary.md and re-confirm. Max 3 rounds, beyond that exit code 1.

The user's chosen surface index is baked into product.json as preferred_site and referenced by steps 5 and 6.

3. Product selection: Platform vs OSS

Read the ## Identify the User's Setup block in https://docs.mem0.ai/llms.txt for the Platform-first routing rules, then apply the heuristics below. Ask, but never blank.

  • Other managed-service SDKs present (@clerk/*, stripe, @supabase/*, openai, @upstash/*, posthog-*), 3 or more, recommend Platform.
  • Local-infra signals (docker-compose.yml with postgres / redis / qdrant / neo4j, ollama configs, self-hosted auth), 2 or more, recommend OSS.
  • No strong signal, default recommendation Platform: lower integration cost, and migration later is supported.

Example:

I see stripe, @clerk/nextjs, and @supabase/supabase-js, managed services throughout. I recommend Mem0 Platform (4-line integration). Override and use open source?

Bake the choice into the goal doc in step 5. Do not re-decide later.

4. API key check (env first, then ask)

TrackKeyWhere to find
PlatformMEM0_API_KEYhttps://app.mem0.ai
OSS (default LLM)OPENAI_API_KEYhttps://platform.openai.com/api-keys

Present in env, continue.

MEM0_API_KEY missing and the track is Platform, default to Agent Mode: run mem0 init --agent --agent-caller <your-name> --json (after pip install mem0-cli or npm install -g @mem0/cli), substituting your agent identity such as claude-code, cursor, codex. If you forgot --agent-caller, run mem0 identify <your-name> after init. Cache the key to .env with user consent and continue. Tell the user to claim it later with mem0 init --email <their-email>: same key, no agent disruption.

Missing and CI mode (MEM0_INTEGRATE_CI=1), exit code 2 with the name of the missing key.

Never echo key values into trace.jsonl. Persist to .env only with explicit user consent, and append .env to .gitignore if it is not there already.

If the user is on OSS and wants a non-OpenAI LLM, route them to the components/llms/* docs and re-run this step with the chosen provider's key.

5. Goal doc, the hard gate

Write .mem0-integration/goal.md and require user approval before step 6.

# Mem0 Integration Goal

**What gets stored:** <one sentence. User utterances? Extracted
preferences? A specific domain fact like "dietary restrictions"?>

**When it gets retrieved:** <one sentence. On each user turn? Before a
specific tool call? At session start?>

**Why:** <one sentence, the user-visible behavior change. "Assistant
remembers previous orders across sessions," not "we added memory.">

**Product:** Platform | OSS  (locked from step 3, do not change)

**Delegated skill:** <raw URL of the published skill being used
from the delegation table in SKILL.md, or "none, custom integration
against `skills/mem0`">.

**Out of scope:** <anything explicitly excluded: "no graph memory,"
"no multimodal," "no migration from existing store">

Rules:

  • The user must approve explicitly. If they edit the doc, reload and re-confirm.
  • goal.md is the contract the test suite is written against. Never rewrite it after step 6 starts.
  • Max 3 rejection rounds. On the 4th, exit code 3 with the rejection notes: the integration is not well-specified enough to proceed.

6. Integration plan, where and how (hard gate)

goal.md is what and why. This step produces where and how, and gets explicit sign-off before any code is written.

Do a scoped read of the repo, no wide survey:

  • Grep for the LLM call sites that match the goal (openai.chat., anthropic.messages., model.generateContent, ChatOpenAI, createLLM).
  • Grep for the user-identity source (req.user, session.user, auth(), ctx.userId, cookies).
  • Check package.json / pyproject.toml / requirements.txt for conflicts, for example an existing mem0ai at a different version.

Then write .mem0-integration/plan.md:

# Mem0 Integration Plan

**Write pattern:** <one sentence, e.g. "After each assistant reply,
call client.add([user_msg, assistant_msg], user_id=<source>).">

**Read pattern:** <one sentence, e.g. "Before building the LLM prompt,
call client.search(query=latest_user_msg, user_id=<source>, limit=5)
and inject results as a system message.">

**User identifier source:** <code path, e.g. `req.auth.userId`,
`session.user.email`, `ctx.params.user_id`. If none, ask the user.>

**Session scoping:**
  - user_id: <source>
  - agent_id: <static slug | null>
  - run_id:   <source | null>

**Write call site:** `<file:line_range>` inside `<function>`
**Read call site:**  `<file:line_range>` inside `<function>`

**Dependencies to add:**
  - `<package>@<version pinned in frontmatter>`

**Preserved behavior:** <list the existing repo behaviors that must
keep working after this edit, e.g. "existing OpenAI streaming still
works," "existing Redis session store still used," "existing tests
still pass unchanged.">

**Coexistence:** <one bullet per existing system the integration sits
alongside. Name the files/classes. Example: "The existing
`agents/memory/storage.py` MemoryStorage class remains untouched and
keeps its LangGraph SummarizationEvent flow. Mem0 is added as a
parallel long-term-facts store, in a new file, invoked only when
MEM0_ENABLED=1 is set.">

**Feature flag:** <the exact mechanism and the default. Required.
Example: `env MEM0_ENABLED=1`, default unset / off; `config.mem0.enabled`,
default false. With the flag in its default state, the repo must
behave exactly like `main`.>

**Sources consulted:** <minimum 2 URLs from "Canonical sources" in
SKILL.md that informed this plan. At least one `docs.mem0.ai` URL and
one delegated-skill URL. Cite the specific section or heading.>

**E2E recipe:** <how the verification skill should drive the app
end-to-end. Omit only if the repo is a pure library with no runnable
entry point, in which case the E2E step skips with a warning.>

    start:               <shell command to launch the app locally,
                          using $PORT for any network port>
    ready_probe:         <one of: url=<URL> status=<code>  /
                          log="<substring to wait for>"  /
                          sleep=<seconds, last resort>>
    compose_services:    <optional: whitespace-separated service
                          names in docker-compose.yml to start first;
                          use label mem0-e2e: "true" to mark them>
    write_call:          <command that triggers the Mem0 write path
                          exactly once; 60s runtime or less>
    write_async_wait_ms: <milliseconds to wait after write_call for
                          async memory flush; default 0>
    read_call:           <command that triggers the Mem0 read path,
                          typically a fresh session / new request>
    read_assert:         <substring, regex, or jsonpath=<expr>=<value>
                          that MUST appear in read_call's output for
                          the E2E to pass. Derived from goal.md's
                          "What gets stored.">

**Rejected alternatives:** <briefly, 1 or 2 bullets. Patterns the skill
considered but did not pick, and why. Helps the user decide.>

Rules:

  • Show the user the proposed call sites with 10 lines of context around each before asking for approval.
  • If no plausible call site exists for either write or read, exit code 5 and ask the user to name the files manually. That is the "no fit here" signal, do not guess.
  • Max 3 rejection rounds on the plan. On the 4th, exit code 5 with the last plan and the user's notes.
  • If the user edits plan.md by hand, reload and re-confirm.

plan.md, not goal.md, is the contract the subagent implements against in step 8.

7. Tests first (TDD)

The main agent writes failing tests against goal.md in the repo's native test framework:

TrackDefault framework
Pythonpytest
TypeScriptvitest if detected, else jest
JavaScriptsame

Test assertion shapes must match the canonical signatures:

  • Platform method signatures: https://docs.mem0.ai/openapi.json, the request body schemas for /v1/memories/ and /v1/memories/search/.
  • OSS method signatures: the delegated skill named in plan.md (fetched from its raw URL), or skills/mem0/SKILL.md as the default.
  • Do not hand-roll request shapes. If the delegated skill has an example block, lift it verbatim.

Minimum two test files, paths taken from plan.md call sites:

  • test_mem0_write.<ext> asserts add() is called at the write call site with the right payload shape (Platform messages-array vs OSS string) and the right user_id source.
  • test_mem0_read.<ext> asserts search() runs before the read call site and the result is wired into the LLM prompt or response path.

Tests MUST be importable with MEM0_API_KEY unset. This is the design pressure that forces step 8's lazy MemoryClient() / Memory() construction: eager module-level init hits the API on import and breaks pre-existing test collection when the key is missing.

Run the tests. They must fail. If they pass before any implementation, the tests are wrong. Rewrite them.

8. Implementation (subagent, fresh context)

Spawn a subagent with:

  • Inputs: the repo, goal.md, plan.md, the two test files, and direct URLs to the delegated skill (from plan.md), the SDK source (pinned per mem0_tested_versions), https://docs.mem0.ai/llms.txt, and https://docs.mem0.ai/openapi.json.
  • No access to the main agent's reasoning trace or scratchpad.
  • System prompt: use the implementation prompt in subagent-prompts.md verbatim.

The subagent returns a diff. The main agent reviews it against plan.md (the mechanical contract) and goal.md (the intent):

  • Approved, apply the diff and commit.
  • Rejected, return with specific actionable feedback, not "try again."
  • Max 3 review loops. Beyond that, exit code 4 with the last diff and the reviewer feedback.

9. Commit and handoff

Create branch mem0-integrate/<short-goal-slug> and commit in separate commits so reviewers can cherry-pick:

  1. mem0: add gated dependency, just the pyproject.toml / package.json change.
  2. mem0: add integration module, the new files.
  3. mem0: wire into <call site>, the call-site edits, still gated.
  4. mem0: add tests, the new test files.

With --no-heal, print Run /mem0-test-integration to verify. and exit. Otherwise proceed to step 10.

10. Self-healing loop (default ON, disable with --no-heal)

Run /mem0-test-integration --ci in a subprocess. If scorecard.json reports overall: pass, done, exit 0.

Otherwise loop:

  1. Categorize the failing check from scorecard.json and route:

    • install / static_checks, dependency or import fix.
    • unit_tests, wiring or assertion fix.
    • smoke_test, API key or SDK call-shape fix.
    • e2e_test, recipe, flag-wiring, or integration-point fix.
    • Pre-existing test failure (test skill exit code 7, non_invasive: false in the scorecard), STOP. This is a non-invasiveness violation. Do NOT attempt to fix it, that breaks principle 3. Exit code 6 with a rationale.
  2. Spawn a remediation subagent with fresh context. Inputs: plan.md, goal.md, scorecard.md, scorecard.json, the last committed diff, and the relevant log for the failing category (test-stdout.log / smoke-stdout.log / e2e-app.log / e2e-calls.log). Use the remediation prompt in subagent-prompts.md verbatim.

  3. Apply the diff and commit on the same branch as mem0-heal: <category> attempt <N>. Do NOT amend earlier commits, reviewers need the heal trail.

  4. Re-run /mem0-test-integration --ci:

    • overall: pass, done, exit 0.
    • Same check still failing, increment the attempt counter and loop.
    • A different check now failing, that is a regression. Revert the heal commit (git revert HEAD --no-edit), record it in .mem0-integration/heal-trace.md, exit code 6.
  5. Bounded iterations. Default 3 attempts per failing category, override with --heal-max N (hard cap 10). On exhaustion, exit code 6 with the full attempt trace: each diff, each scorecard, final log tail.

  6. Post-loop summary written to .mem0-integration/heal-trace.md: which category failed, how many attempts, each diff's intent, final status, and on success the delta from the initial scorecard to the final one.