Design: Agentic Memory & Multi-Agent Orchestration

Multi-agent memory patterns against the knowledge graph.

Thesis#

Context windows are ephemeral. Multi-agent systems need a shared, durable memory that:

  1. Workers can write (new entities/relations with provenance)
  2. Evaluators can fact-check against (cite edges; reject ungrounded claims)
  3. Overnight / long loops can persist across sessions

That memory is the knowledge graph. The process that fills and uses it is the workflow graph.

Agent patterns × knowledge graph#

PatternKG role
Augmented LLMGraph traversal as a tool for multi-hop questions
Prompt chainingBetween steps, check new entities against existing nodes
RoutingEntity type / degree can route without an LLM call
Orchestrator–workersShared memory — workers R/W graph; orchestrator stays small
Evaluator–optimizerGrounding layer — claims checked against triples + provenance

The governing idea is simple: a session is not the context window. Durable memory, grounding, and run state live outside the model.

Roles on the shared graph#

RoleReadsWritesNotes
Extractor workersSource docsCandidate entities/relationsFan-out; cheap model; schema out
ResolverCandidates + descriptionsAlias map / canonical idsStronger model; barrier stage
AssemblerCanonical triplesGraph storeDeterministic code
Researcher / analystSerialized subgraphOptional notes (not facts)Judgment; must cite
Verifier / skepticFinding + graph neighborhoodPass/fail + reasonFresh context; no worker chat
OrchestratorRun stateWorkflow progress onlyCode barriers, caps, routing

Composition pattern#

Memory write rules#

  • Never write free-text “memories” as first-class facts. Write typed triples (or reject).
  • Every write carries provenance (source_doc, chunk, time, extractor version).
  • Verified-only promotion: candidates stay staging until a verifier majority (or human gate) promotes them to canonical.

Memory read rules#

  • Queries return a bounded subgraph serialization, not the whole graph.
  • Answers must cite edges. Ungrounded generations are dropped or flagged.
  • Vector recall seeds passages, not edges. Ask embeds library chunks to find the span extract dropped; neighbors and citations still come from typed triples.

Multi-agent orchestration rules (applied)#

  1. Fake-edge audit on the ingest→query pipeline itself — extractors are independent per doc; do not serialize them.
  2. Diamond for user questions: plan angles → parallel subgraph queries/research → code reduce → verify → synthesize.
  3. Checkers are mandatory before user-facing answers and before canonical write-back.
  4. Anchors: unit tests on extract schemas; golden Q&A; optional human confirmation for high-impact merges.
  5. Caps: max docs per run, max agents, max hops, max spend — especially on discovery loops.
  6. Isolation: parallel extractors do not share mutable graph handles; assemble is a single-writer stage (or transactional merge). Local code writers use fail-closed OS sandboxing and unique temporary workspaces; external mutations use exact-input approval plus shared resource locks. These are deliberate security boundaries rather than implementation shortcuts.

Failure modes specific to agentic memory#

ModeSymptomFix
Context-as-memory“Forgot” overnightPersist triples; never rely on chat history
Self-consistent fictionGraph agrees with itself, wrong vs worldAnchors + external checks
Write stormsDuplicate/conflicting triplesResolve + staging + provenance
Merge poisoningOne bad over-merge collapses entitiesReview high-degree clusters; undo aliases
Uncited synthesisFluent liesEnforce citation schema on answer node

Overnight / long-running loops#

  • Discovery loop pattern: finders → dedupe vs seen → verify → confirm → until K dry rounds.
  • State that must survive: seen keys, confirmed triple ids, budget counters, workflow run id — store outside the model.
  • Resume = reload graph + run state; do not replay entire chat.

Doso implementation#

Doso is not “chat with notes.” It is:

  • a knowledge-graph write/read pipeline
  • executed by a workflow graph
  • with shared-memory discipline above

The architecture shows how those layers form the product.