Build agents.
Prove what they can do.

Conversation

The last twenty messages, so the agent inherits the thread instead of restarting cold.

Researcher

The ReAct loop itself. Reason → tool → observe, capped at the agent's iteration budget, with its tools resolved from the workspace's MCP servers and sandboxes.

Needs sign-off?

An LLM judge reads the proposed action and routes anything that writes, sends or spends down the human arm. It answers in one word, because the arm is picked by matching that answer against the branch.

Approval

The run pauses here. It survives a page reload, and resumes with whatever the reviewer typed.

Answer

Both arms land on the same output, so the caller never has to know which one ran.

Query rewrite

Turns the question into a retrieval query: expands acronyms, strips pleasantries, keeps the entities.

Retrieve

Semantic search over the workspace’s own context items — five passages, nothing below 0.35 similarity.

Evidence enough?

A regex on the retriever’s output. Empty result set, no answer — the branch is explicit rather than buried in a prompt.

Grounded answer

Answers from the retrieved passages and cites them. Temperature 0.2: this is a reading task, not a writing one.

Abstain

Returns a plain “not in the corpus” instead of a confident guess. Cheaper to read than a hallucination is to catch.

Draft

First attempt, warm. This one is allowed to be wrong.

Critique

A different model, temperature 0, told to list defects and nothing else — or to reply APPROVED if it finds none.

Approved?

A regex over the critique. Clean drafts skip the rewrite entirely instead of paying for a second generation.

Revision

Sees the task, its own draft and every objection, and has to address them point by point.

Final text

Whichever version survived. The trace keeps both, so you can see what the critique bought you.

Supervisor

Plans, delegates, and decides when the goal is met — up to six delegations, re-planning after each one.

Researcher

A member agent, enrolled by the dashed ◇ edge. Its own tools, memory and comms policy come from its definition in the library.

Analyst

Same enrolment, different specialisation. The supervisor chooses between them per step, not per run.

Writer

Members never call each other here; everything routes through the supervisor, which is what makes the delegation tree readable in the trace.

Report

The supervisor's synthesis, with the per-member token counts and cost attached to each branch of the span tree.

Case conference

The moderator: opens each round, decides whether another one is worth it, and synthesises the conclusion. Four rounds is the ceiling, not the target.

Generalist

Argues from the whole picture. Members speak in turn and see what the others said in the previous round.

Specialist

Narrow and deep. Enrolled the same way — a ◇ membership edge from the panel.

Devil’s advocate

Instructed to attack the emerging consensus. Its dissent is what the schema below preserves.

Verdict

Structured output with a schema: conclusion, confidence, and the dissent that did not make it into the conclusion. Retried against the schema if the model wanders.

Dataset

A CSV read in batch, one record per row. TREC, XML, JSON and whole folders read the same way.

Per row

For-each over the rows. Each iteration gets {{loop_item}} and its fields; a sample size lets you rehearse on twenty rows before paying for two thousand.

Answer

The system under test — here a local Ollama model, so a sweep costs GPU time instead of API credit.

Score

Exact match and cosine similarity between the answer and the row’s reference. ROUGE, BLEU, NDCG, MRR, F1 and the correlation metrics are all one checkbox away.

Scores

Accumulates every row’s metrics across iterations instead of overwriting them.

Results

A dashboard tab in the run view. The same numbers land in the experiment’s trials, next to token counts and cost.