Design: Knowledge Graph Pipeline

Problem statement#

You have unstructured documents and need questions that span them. No single document holds the answer; RAG won’t chain the facts. You need:

• Entities as nodes
• Typed relations as edges
• Traversal for multi-hop reasoning

Classical path: train NER, train a relation classifier, maintain string-similarity resolution — brittle as data shifts. With Claude, each stage is a prompt + structured schema.

Pipeline#

Summarization and formal evaluation are first-class stages, not optional follow-up work.

1. Extract#

• One structured call per document (chunk in production; Wikipedia summaries in the demo).
• Schema (Entity, Relation, ExtractedGraph) via client.messages.parse — validated typed objects, no regex JSON salvage.
• Entity fields: name, type, description (one sentence grounded in this document — required later for resolution).
• Relation fields: source, predicate, target (short verb-phrase predicates; endpoints must be extracted entities).
• Model: Haiku-class (volume, schema-constrained).
• Claim: the Pydantic schema is the only “training data.”

Demo entity types: PERSON | ORGANIZATION | LOCATION | EVENT | ARTIFACT.

2. Resolve#

• Per-type clustering of surface forms into {canonical, aliases[]}.
• Uses descriptions to avoid merging namesakes; handles zero string overlap (Edwin Aldrin → Buzz Aldrin).
• Model: Sonnet-class (judgment across evidence).
• Failure modes called out in the guide:
  • Dropped alias → silent node loss → production fallback: singleton cluster.
  • Over-merge (e.g. Gemini 12 into Project Gemini) → precision loss → spot-check.

3. Assemble#

• Rewrite endpoints through alias → canonical.
• networkx.MultiDiGraph: multiple predicates between a pair; direction matters.
• Node attrs: type, description, source_docs[], mentions.
• Edge attrs: predicate, source_doc.

4. Summarize#

• For hub nodes, pool every mention across the corpus.
• Structured profile (guide uses models like time ranges + key facts).
• Sonnet-class; yields richer node cards than any single extract.

5. Query (grounded)#

• serialize_subgraph(center, hops) — frontier-style expansion to triples text.
• Prompt Claude over only that serialization; require answers from those edges.
• Grounded vs ungrounded: ungrounded may be factually OK from pretraining but untraceable; on a private corpus only grounded works.
• Product rule for doso: citations to specific edges are mandatory.

6. Evaluate / repeat#

• Gold set: data/sample_triples.json (+ alias map).
• Script: evaluation/eval_extraction.py → precision / recall / F1 for entities and relations.
• Use eval failures to revise prompts/schemas — not ad-hoc string hacks.

Cost & scale (cookbook + playbook §IX)#

For Doso, SQLite/Postgres adjacency tables preserve the entity, alias, directed multi-relation, and provenance contracts without requiring NetworkX or a specialized graph database.

Contracts to implement#

1extract(doc) -> ExtractedGraph2resolve(type, entities) -> Cluster[]3assemble(raw, alias_map) -> MultiDiGraph4summarize(graph, node_ids) -> EntityProfile[]5serialize_subgraph(center, hops) -> triple_text6query_grounded(graph, question, hops) -> { answer, citations[] }7evaluate(predicted, gold) -> { entity_f1, relation_f1, ... }

Doso implementation#

This pipeline is the memory write/read core. Workflow orchestration fans out extraction and gates promotion to canonical facts. The architecture shows how the pipeline fits into the full system.

Predicates (Doso)#

Relation strings are open. Preferred names live in the user ontology; Memories filters with a noise denylist, not an allowlist. See Predicates and ontology.