Labeling jobs

A labeling job is a human-annotation workflow over a dataset. The job owns the questions people answer, who is assigned to which records, and the agreement analysis — while the dataset stays purely data.

Use a labeling job when you need humans to rate, classify, or otherwise annotate records: building gold labels, evaluating model outputs, or measuring how much annotators agree.

Note

Annotation is modeled as a job, not a property of the dataset. The dataset is data-only; the job owns its questionnaire, its assignees, and its agreement metrics. This lets several jobs run over the same dataset with different questions and rosters.

Create a job

Use the CreateJobModal to start a job.

1. Bind a dataset. The job annotates records from this dataset.

2. Define the questionnaire. Add questions — they reuse the same type/widget vocabulary as dataset columns (Number, Ordinal, Text, Boolean × Input, Textarea, Select, Radio, Checkbox).

3. Choose visible fields. Pick which dataset columns annotators see while labeling (hide columns that would bias them).

4. Assign annotators to records.

Questionnaire & schema

The questionnaire is the set of questions annotators answer per record. Questions reuse the dataset column type/widget vocabulary, so an Ordinal question can render as a Radio scale, a Text question as a Textarea, and so on.

Visible fields

A job chooses which dataset columns are visible to annotators. Show the columns that provide context for the task and hide anything (such as existing labels) that should not influence the annotation.

Roles

Access is role-based, set per assignment, and drives which tabs each person sees:

Role	Can do
Admin	Manage the job, schema, assignments, and insights
Annotator	Label assigned records only
Viewer	Read-only access to records and insights

Jobs are scoped both personally and to a team.

Assignments

The AssignmentsPanel maps annotators to the records they should label. Assign overlapping records to multiple annotators when you want to measure agreement.

Annotating

Annotators work through their assigned records, answering the questionnaire for each. A response exists per (job, item, annotator) and has a status:

Status	Meaning
Draft	In progress, not yet submitted
Submitted	Completed and recorded
Discarded	Set aside, not counted

Insights & agreement

JobInsights rolls up results across annotators. For each item it shows the collected responses and their distribution, and across overlapping assignments it computes inter-annotator agreement metrics. Use these to gauge label quality and decide whether the questionnaire or guidelines need tightening.