Design: Agentic Memory & Multi-Agent Orchestration
Multi-agent memory patterns against the knowledge graph.
Thesis#
Context windows are ephemeral. Multi-agent systems need a shared, durable memory that:
- Workers can write (new entities/relations with provenance)
- Evaluators can fact-check against (cite edges; reject ungrounded claims)
- Overnight / long loops can persist across sessions
That memory is the knowledge graph. The process that fills and uses it is the workflow graph.
Agent patterns × knowledge graph#
| Pattern | KG role |
|---|---|
| Augmented LLM | Graph traversal as a tool for multi-hop questions |
| Prompt chaining | Between steps, check new entities against existing nodes |
| Routing | Entity type / degree can route without an LLM call |
| Orchestrator–workers | Shared memory — workers R/W graph; orchestrator stays small |
| Evaluator–optimizer | Grounding layer — claims checked against triples + provenance |
The governing idea is simple: a session is not the context window. Durable memory, grounding, and run state live outside the model.
Roles on the shared graph#
| Role | Reads | Writes | Notes |
|---|---|---|---|
| Extractor workers | Source docs | Candidate entities/relations | Fan-out; cheap model; schema out |
| Resolver | Candidates + descriptions | Alias map / canonical ids | Stronger model; barrier stage |
| Assembler | Canonical triples | Graph store | Deterministic code |
| Researcher / analyst | Serialized subgraph | Optional notes (not facts) | Judgment; must cite |
| Verifier / skeptic | Finding + graph neighborhood | Pass/fail + reason | Fresh context; no worker chat |
| Orchestrator | Run state | Workflow progress only | Code barriers, caps, routing |
Composition pattern#
Memory write rules#
- Never write free-text “memories” as first-class facts. Write typed triples (or reject).
- Every write carries provenance (
source_doc, chunk, time, extractor version). - Verified-only promotion: candidates stay staging until a verifier majority (or human gate) promotes them to canonical.
Memory read rules#
- Queries return a bounded subgraph serialization, not the whole graph.
- Answers must cite edges. Ungrounded generations are dropped or flagged.
- Vector recall seeds passages, not edges. Ask embeds library chunks to find the span extract dropped; neighbors and citations still come from typed triples.
Multi-agent orchestration rules (applied)#
- Fake-edge audit on the ingest→query pipeline itself — extractors are independent per doc; do not serialize them.
- Diamond for user questions: plan angles → parallel subgraph queries/research → code reduce → verify → synthesize.
- Checkers are mandatory before user-facing answers and before canonical write-back.
- Anchors: unit tests on extract schemas; golden Q&A; optional human confirmation for high-impact merges.
- Caps: max docs per run, max agents, max hops, max spend — especially on discovery loops.
- Isolation: parallel extractors do not share mutable graph handles; assemble is a single-writer stage (or transactional merge). Local code writers use fail-closed OS sandboxing and unique temporary workspaces; external mutations use exact-input approval plus shared resource locks. These are deliberate security boundaries rather than implementation shortcuts.
Failure modes specific to agentic memory#
| Mode | Symptom | Fix |
|---|---|---|
| Context-as-memory | “Forgot” overnight | Persist triples; never rely on chat history |
| Self-consistent fiction | Graph agrees with itself, wrong vs world | Anchors + external checks |
| Write storms | Duplicate/conflicting triples | Resolve + staging + provenance |
| Merge poisoning | One bad over-merge collapses entities | Review high-degree clusters; undo aliases |
| Uncited synthesis | Fluent lies | Enforce citation schema on answer node |
Overnight / long-running loops#
- Discovery loop pattern: finders → dedupe vs
seen→ verify → confirm → until K dry rounds. - State that must survive:
seenkeys, confirmed triple ids, budget counters, workflow run id — store outside the model. - Resume = reload graph + run state; do not replay entire chat.
Doso implementation#
Doso is not “chat with notes.” It is:
- a knowledge-graph write/read pipeline
- executed by a workflow graph
- with shared-memory discipline above
The architecture shows how those layers form the product.