Workflows

Product surface#

/workflow is a master-detail hub with two filters:

• Workflows lists persisted workflow definitions. Selecting one opens the editor.
• Runs lists live and recent executions. Selecting one opens its trace. /workflow?id=<runUuid> or ?run= lands on this filter with that episode selected, even before the list snapshot includes it.

The hub keeps definitions and executions separate: a workflow is reusable structure; a run is one attempt with its own status, events, output, and error.

Capabilities, episodes, and the library#

Connection work uses three layers:

Three verbs#

Chat (and the hub) choose among:

1. Reuse — same capability, new run with a new objective. No new library row. Hub chat uses this for the first ask on a toolkit.
2. Chain — continue from a prior episode for follow-on work (draft/reply/update). Hub chat attaches parent_run_id only then; content asks (summarize/explain/…) run the capability DAG fresh: search → schema → agent fetch → library_upsert_v1 → library_artifact_v1. In the editor dock, Chain also appends a toolkit step onto the open workflow draft. Promote later if you want the result durable under a new name.
3. Build — compose a multi-step graph, or explicitly save as … / make this a workflow to promote an episode into the library.

Toolkit capability episodes discover matching tools, then run a bounded agent loop over those schemas (call / done). The model may invoke multiple tools in one episode; consequential calls still require Approve & run. After the agent loop, capability DAGs run generic Library skills (library_upsert_v1 → library_artifact_v1) so readable content can land in the knowledge graph and chat can render a Library artifact card. Those skills soft-skip when there is nothing to save.

Promote systematizes a successful episode into a named library workflow via POST /v1/runs/{id}/promote. For capability episodes, the server lifts successful calls[] from the agent loop into an explicit linear chain of composio_toolkit_call nodes (each with pinned_tool_slugs for replay), then keeps the Library upsert/artifact tail. The reusable conn_cap_* capability template is left unchanged for Reuse. When no calls are available, promote falls back to cloning the capability template.

Legacy per-ask chat_conn_* defs are hidden and cleaned up; episodes live under Runs.

Promote lift (trace → library DAG)#

Promote turns an episode into a library definition without inventing per-app product logic:

Lineage is stored on the new def’s ontology_bindings: promoted_from_run_id, lift (agent_calls | clone_def | goal_skeleton), and source_workflow_id.

Pinned replay: node config.pinned_tool_slugs tells the Composio discovery path to skip search and fetch schemas for those slugs only. Chat save as … and the run observer Save as workflow both call the promote API for a concrete run id.

Freeze-DAG prompt tuning#

Library workflows that reference Prompt skills can improve wording without changing topology:

1. Improve prompts… in the workflow editor (or POST /v1/eval/from-runs) builds a WorkflowEvalCorpusV1 from recent succeeded runs of that workflow_id.
2. Metrics are generic: verdict_complete, verifier_passed, json_path_equals, autophagy_score.
3. POST /v1/eval/optimize-prompts freezes the DAG, proposes impl_ref variants for Prompt skills, scores them on the corpus, and opens ImprovementEffectKind::ReviseSkillPrompt proposals.
4. Applying a proposal updates only SkillDef.impl_ref (+ version). Builtin, tool, and code skills are skipped.

Corpora live in store KV (workflow_eval_corpus:{id}); they are user/workflow data, not hardcoded job logic in the harness.

Build and edit#

A persisted workflow definition contains an id, title, description, version, source, DAG, capability limits, and optional backend fields. Definitions are stored in workflow_defs and exposed through /v1/workflows/defs.

The editor has three working areas:

1. Chat dock — describe a workflow or ask for a change. The composer can reuse visible skills, mint Prompt skills, and return a persisted definition with source=chat.
2. Visual canvas — add skill nodes, move them, connect them, and edit the workflow title. Canvas changes are converted back to the persisted DAG.
3. Skills rail — add a skill to the canvas or open its editor.

Edits autosave after a short debounce and flush when the selected workflow changes. Empty definitions may be saved as drafts, but a run validates the completed graph before execution.

The editor header exposes Run and, when the canvas includes Prompt skills, Improve prompts… (freeze-DAG prompt tuning against recent runs of this workflow).

Each node references a skill. Edges carry labels and contracts; save and run validation reject missing skills, invalid edges, cycles, incompatible contracts, and unsafe code configuration.

Knowledge workflows use hidden engine-owned V1 stages (kg_chunk_v1, kg_extract_v1, kg_resolve_v1, kg_assemble_v1, kg_summarize_v1, and kg_grounded_query_v1). Fixed product ingestion and persisted DAGs call the same typed Rust services. The registry supplies their schemas, model roles, effects, and graph-write lock; workflow config can only narrow those contracts. Extract, resolve, assemble, and summarize must form a direct trusted lineage, and candidate assembly never promotes facts to verified. Grounded query reads a bounded 1–3-hop verified subgraph and returns only citations matching retrieved edges; unsupported answers are explicitly flagged with low confidence.

Declarative execution policy#

Node config carries optional engine policy. Existing version-1 definitions remain valid; missing or null config becomes {}. Security policy is not author-controlled: effect, required approval, isolation, trusted contracts, timeout ceiling, and at least one mutation resource key are re-derived recursively from the skill registry before every run.

• join: object with mode (all, any, or quorum), optional quorum, and optional expected predecessor count
• failure: fail, skip, or null
• input_schema, output_schema, and child_output_schema: bounded local JSON Schemas; remote/file references are rejected
• retry: object with max_attempts and backoff_ms
• timeout_ms (whole node), optional attempt_timeout_ms, cache, cache_ttl_secs, and idempotency_key: bounded execution controls
• kind: task, router, map, loop, verifier, subgraph, approval, plan, expand, or goal
• model_tier (fast, balanced, strong) or model_role (extract, resolve, skeptic, etc.): routing hints for model-backed prompt/builtin skills

Example bounded map:

1{2  "kind": "map",3  "items_path": "/sources",4  "max_items": 64,5  "map_concurrency": 8,6  "retry": { "max_attempts": 2, "backoff_ms": 100 },7  "child_output_schema": {8    "type": "object",9    "required": ["finding"]10  }11}

Routers use routes[].label, routes[].when.path, routes[].when.equals, default_route, and route_path. Loops use max_rounds plus until.path/until.equals. Verifiers use attempts, quorum, and verdict_path. Map, loop, verifier, and subgraph nodes may provide a nested body DAG; recursive node, depth, invocation, timeout, and shared-concurrency caps still apply.

Goal-seeking operators close the long-run loop:

• plan — runs a planner skill (default contract: WorkPlan JSON: status continue|done|blocked|retry, artifacts, subgoals)
• expand — executes WorkPlan.subgoals concurrently (inline body, method_id, or workflow_id; optional isolation: run spawns a child workflow run)
• goal — durable plan→expand loop until done / approval blocked / max_plan_rounds

Workflow caps also include max_plan_rounds and max_subgoals_per_wave. Accumulated artifacts and waves live on the durable checkpoint goal_state and appear in the run observer.

The ready-set scheduler starts every dependency-ready node up to the workflow concurrency cap. Resource keys serialize conflicting writers without blocking unrelated work. Pure/read cache entries are stored locally with a TTL; idempotency_key memoizes those safe results but does not promise provider-side idempotency for external mutations.

Code skills run only through an opt-in OS-sandboxed subprocess with a temporary working directory, bounded output, cleared environment, server enablement, content-and-capability-bound allowlisting, and confirmation. Editing source, version, or security caps invalidates the operator token. Hosts without the supported sandbox fail closed. This is not a git worktree. Connection mutations retain their existing short-lived, exact-input approval digest; external effects are never served from the DAG cache.

The visual editor exposes these settings in the selected node’s inspector. It also includes Diamond, Layered fan-in, Loop until dry, Goal-seeking loop, Evaluator–optimizer, and Human-gated publishing templates. Client preflight mirrors the engine’s structural, join, nested-body, route/resource-limit, and schema-safety checks before a run; the server remains authoritative. Trusted pipeline skills and consequential effects keep effect, approval, and isolation controls read-only.

Skills#

The registry supports four skill kinds:

The workflow palette shows user-visible Prompt, Tool, and Code skills. /skill?id=… edits the title and description for any visible skill, plus the instruction body for Prompt skills. Other implementations are managed by the backend or integration that owns them.

Connection capabilities are discovered under Settings → Connections and projected into the skills registry with structured provider, connection, runtime, and risk metadata. The execution router sends Composio capabilities to a scoped hosted session and Google Workspace CLI capabilities to the paired outbound runtime assigned to that connection.

Connection reads are graph-backed. Every successful provider result is first stored as a provenance document and recursively materialized as candidate external-object nodes before the workflow receives it. A matching recent read is served from that stored observation unless the node requests a refresh. Semantic extraction then adds candidate people, organizations, and relationships over time. This is progressive discovery—like moving a torch through unknown context—not an assertion that provider data is already verified truth.

Run and observe#

Running a persisted definition calls the DAG interpreter with the workflow id and objective. The interpreter validates the graph and node policy, applies caps, stores a run, and emits node events. Independent nodes execute concurrently rather than in topological list order.

The run view:

• renders the saved topology read-only;
• updates node state from /v1/events while work is live;
• shows step detail, final status, and errors;
• offers Edit when the run maps back to a persisted definition;
• offers Run again after completion when that workflow can be launched through the generic runner;
• offers Save as workflow for succeeded capability / ad-hoc episodes (server-side promote lifts observed tool calls into editable steps).

The run view calls POST /v1/runs/{id}/pause, /resume, and /cancel for live controls.

Rerunning creates a new run. It does not overwrite the earlier trace. Some fixed catalog jobs, including ingest, require their owning product endpoint and cannot be relaunched by the generic runner.

Two workflow systems#

Doso currently has two related sources of workflow topology:

Persisted workflow definitions#

These are user-facing, database-backed DAGs assembled from skills. They are created and edited in the Workflows hub, authored visually or through the chat dock, and executed by the skill DAG interpreter.

Fixed Rust catalog workflows#

These are versioned topologies implemented in crates/doso-core/src/workflows/: ingest, ask, research_desk, discovery, promote_review, and memory_groom. They encode product-critical orchestration in Rust and are not editable definitions on the visual canvas.

The composer may reference selected catalog workflows as builtin skills when a full fleet topology is appropriate. A catalog run can still appear in Runs, but it should not be presented as a user-authored definition.

Review flows#

“Review” covers two distinct paths:

• Proposed automation review shows pending thought proposals associated with a workflow. A user can choose once, always, or never. “Always” permits future matching proposals to autorun only when the workflow has no confirmation-required skill and cascade limits allow it.
• Promote review is a fixed catalog workflow. It samples candidate knowledge triples, asks a skeptic panel to vote, then promotes accepted candidates to verified or marks the rest rejected.

Connection reads may run after the capability is enabled. Write, delete, external-send, local-file, and admin effects require a short-lived approval whose digest includes the connection, runtime, capability, effect, and exact input. The workflow review page changes Run to Confirm run before requesting those one-use approvals. Changing the input or replaying an approval invalidates it.

The standalone workflow review route renders a definition read-only, links back to editing, can run it, and exposes related pending proposals.

Backend-only automation#

The data model and API include schedules, tension rules, system events, thought proposals, approvals, and guarded autorun. These capabilities are available to integrations and operators, but the Workflows hub does not yet expose a complete scheduling or automation editor.

• A workflow definition may carry a cron-like schedule field.
• Schedule ticks and event/count tension evaluation can create proposals.
• Approval decisions control one-time, persistent, or blocked execution.
• Confirmation-required Tool or Code skills never silently autorun. Code execution also depends on server-side enablement and allowlisting.