Experiments & Trials

An experiment is a systematic run over an agent graph. You bind it to a graph and (optionally) a dataset, define a parameter space, and let a sampler explore that space. Each parameter configuration becomes one trial, and each trial runs the graph once per dataset item.

Use an experiment when you want to compare configurations rigorously — sweep models, prompts, and hyperparameters, score the outputs, and see which settings win.

Building an experiment

The ExperimentBuilder walks you through binding the graph and dataset and defining the space.

1. Bind a graph. Choose the agent graph the experiment runs.

2. Bind a dataset (optional). Each trial runs the graph once per dataset item; without a dataset, a trial is a single run.

3. Define the parameter space. For each tunable field, pick a mode (see below). Parameters include model and prompt pickers as well as numeric/graph fields.

4. Choose a sampler to explore the space (grid, random, or TPE).

5. Preview, then run.

Parameter modes

Each field in the space uses one of four modes:

Mode	Meaning
fixed	One constant value for every trial
values	An explicit list of values to try
range	A numeric range the sampler draws from
off	The field is excluded from the sweep

Samplers

The sampler decides which configurations from the space become trials:

Sampler	Strategy
Grid search	Every combination in the space
Random	Random configurations from the space
TPE (Bayesian)	Tree-structured Parzen Estimator — optimizes toward better scores

Preview before a full run

Use Preview to launch a small number of sample trials before committing to the full sweep. This catches prompt/graph mistakes and gives you a feel for cost and output shape before you spend on the entire space.

Running & live progress

A running experiment streams progress over SSE. You will see events as work proceeds:

• TrialStarted — a trial began
• TrialCompleted — a trial finished (with its score and metrics)
• Finished — the experiment is done

What a trial records

One experiment produces N trials — one per parameter configuration. Each trial tracks:

Field	Description
param_overrides	The exact configuration for this trial
score	Primary score from the graph’s Eval node
eval metrics	Full metric set from the Eval node
token counts	Tokens consumed across the run
cost	Estimated cost
error	Any failure for the trial

Tip

Scores and metrics come from the graph’s Eval node. If your graph has no Eval node, trials run but carry no score. See the node reference.

Reading results & plots

The Results view aggregates across trials. ExperimentPlots visualize how score and metrics move with each parameter, so you can spot the best configurations and trade-offs (e.g. score vs. cost).

Trial inspector

Open the TrialInspectorDrawer on any trial to drill into a single run. It shows the node-level execution timeline and the run events, so you can see exactly what each node did, where time and tokens went, and where a failure occurred.

Append outputs to a dataset

From the results, the AppendFromTrialModal appends a trial’s outputs back into a dataset as new rows — useful for curating generations into gold data or feeding the next round of work.