Graph Execution

Execution model

Graphs are executed by topological level. The translator groups nodes into levels using Kahn’s algorithm — nodes at the same level have no dependencies on each other and can run in parallel.

Level 0: [Input]                     → single node, sequential
Level 1: [FileRead]                  → single node, sequential
Level 2: [Expert LLM, VectorDB]     → 2 nodes, PARALLEL (tokio::JoinSet)
Level 3: [Evaluator LLM]            → single node, waits for level 2
Level 4: [Structured Output]        → single node, sequential
Level 5: [FileWrite]                → single node, sequential
Level 6: [Output]                   → single node, sequential

Parallel execution

When a level has 2+ nodes and all are parallelizable types (LLM, Tool, VectorDB, StructuredOutput, FileRead/Write, Merge, SubAgent, AgentComm), they execute concurrently:

1. Each node gets a snapshot of the current context store (clone, not lock)
2. Tasks are spawned into a tokio::task::JoinSet
3. The executor waits for ALL tasks to complete
4. Results are written back to the context store
5. A checkpoint is saved after the level completes

Nodes that require mutable context (Input, Loop, Condition, Human) always execute sequentially, even if at the same level as other nodes.

Loop body parallelism

Loop nodes use topological_levels_subset() to group their body nodes by dependency level. Independent body nodes (e.g., an Expert LLM and a VectorDB Query) run in parallel within each loop iteration:

Iteration 1:
  Body level 0: [Expert LLM, VectorDB Query]  → PARALLEL
  Body level 1: [Evaluator LLM]               → waits for both
  Body level 2: [Structured Output]            → sequential
  Body level 3: [File Write]                   → sequential
Iteration 2:
  ...same pattern...

Context store

The context store is a HashMap<String, String> that accumulates all node outputs:

Key	Value
input	User’s input text (set by Input node)
node_id	The node’s output
node_label	Same output, aliased by the node’s display label
loop_item	Current item in a Loop iteration
loop_item.{field}	Dot-notation access to JSON fields of the current loop item
loop_index	Current iteration index (0-based)
loop_total	Total number of iterations
{node_id}.{field}	Dot-notation access to JSON fields of any node’s output
{node_id}_error	Error message (set when following a fail edge)

Template expansion

Any string field in a node’s config can use {{variable}} syntax to reference context store values:

Prompt: "Evaluate this response:\n\n{{Expert LLM}}\n\nAgainst:\n\n{{Ground Truth}}"

Variables are matched by both node_id and label. If a template has no variables and the predecessor produced output, it’s appended automatically.

Dot-notation for JSON fields

When a node outputs a JSON object, its top-level fields are automatically expanded as separate variables using dot-notation. This works for all nodes, not just loops:

# If VectorDB node outputs: {"title": "Topic 1", "content": "...", "similarity": 0.95}
# The following variables are available:
{{VectorDB.title}}       → "Topic 1"
{{VectorDB.content}}     → "..."
{{VectorDB.similarity}}  → "0.95"

# In a loop iterating over CSV rows:
{{loop_item.title}}      → the title column of the current row
{{loop_item.content}}    → the content column

Context Store inspector

The Studio includes a collapsible Context Store panel on the left side of the canvas. It shows all variables currently in the store in real-time during execution, with a search filter. The store is updated via WebSocket after each loop iteration and at execution completion.

Fail edges & error routing

Fail edges enable error-driven retry cycles. They are excluded from topological sorting, so they can point backward without triggering cycle detection.

How it works

Evaluator LLM ──data──► Structured Output ──data──► File Write
                              │
                              └──fail──► Evaluator LLM (retry with error context)

When a node fails and has fail edges:

1. The error is stored as {{node_id_error}} in the context store
2. The executor follows each fail edge and re-executes the target node
   • The target node can use {{node_id_error}} in its prompt template for feedback
3. The failed node is retried with the updated context store
4. This repeats up to error_policy.retries times
5. On success: the fail edge cycle breaks and execution continues via data edges
6. On final failure: respects continue_on_fail policy or aborts the graph

Use cases

• Structured Output → LLM: if JSON parsing fails, re-ask the LLM with the parse error as feedback
• Tool → Error Handler: route tool failures to a notification or logging node
• Any node → Fallback: connect fail to an alternative computation path

Error handling

Each node has an ErrorPolicy:

Field	Default	Description
retries	0	Number of retry attempts (including fail-edge retries)
continue_on_fail	false	Continue graph execution with [Error: ...] as output
timeout_secs	120	Maximum execution time per node

When a node in a parallel level fails with continue_on_fail=false, the error is propagated after all tasks in the JoinSet complete.

Checkpointing & resume

After each level completes, the executor saves a checkpoint:

INSERT INTO graph_checkpoints (run_id, node_id, ctx_store, completed_nodes)
VALUES ($1, $2, $3, $4)

To resume a failed run, call resume_from_checkpoint() which:

1. Loads the last checkpoint’s context store and completed node list
2. Computes remaining nodes from the topological sort
3. Resumes execution from where it left off

WebSocket events

During execution, the backend streams events to the frontend:

Event	Description
RunStarted	Graph execution begun
NodeStarted	A node is about to execute
NodeToken	Streaming token from an LLM node
NodeCompleted	Node finished with output preview
NodeError	Node execution failed
NodePaused	Human node waiting for input
EdgeFollowed	An edge was traversed (visual feedback)
CtxStoreUpdated	Context store snapshot (after each loop iteration and at completion)
LoopPaused	Loop paused by user (includes iteration/total)
LoopResumed	Loop resumed after pause
RunCompleted	Graph execution finished successfully
RunFailed	Graph execution failed

Debugging: Pause, Step & Stop

During execution, the Studio provides debug controls via WebSocket commands:

Button	Command	Behavior
Pause	pause_graph	Pauses before the next loop iteration
Step	step_graph	Executes one loop iteration then pauses again
Resume	resume_graph	Continues normal execution
Stop	stop_graph	Aborts execution immediately

When paused, the Context Store panel shows all current variables, allowing you to inspect intermediate state between iterations. The execution header shows the current iteration (e.g., “paused 3/50”).