Skip to main content
Annotations turn important runs into labeled examples. Use them when a run is a clear success, a clear failure, or useful evidence for future evaluation.

Queue a run

It is safe to enqueue the same run again after appending more steps.

List annotation work

annotations list always requires a project. The selector accepts a full project ID, an unambiguous ID prefix, or an exact project name. Queue filters also cover name/run ID, text query, time, runtime, novelty score, analysis state, and tag IDs. Run sovara tags list --project-id <project> to map tag names to IDs. Responses retain scores, statuses, tags, and distinct_code_versions, which can be fed back into later filters.

Inspect before labeling

annotations inspect returns the complete run-annotation response. The persisted AnnotationRecommendation is under recommendation and may be null when the run has no recommendation. It includes the analysis verdict and hint, scoring/status fields, every completed artifact-adjudication pair, closest retained failure evidence, and most-novel evidence. Evidence locators expose run_id, step_uuid, display step_ref, field, and zero-based chunk_index. In evidence.adjudication_pairs, failure_embedding_artifact is true for an embedding artifact, false for a completed verdict that retained the pair as plausible failure evidence, and null when the pair was not checked. Use step refs from probe when you want the label to point at specific evidence.

Save a success

Add optional ground truth when it helps future reviewers:

Save a failure

Failure annotations require ground truth:
Focus the failure on specific steps:

Remove an annotation

Only remove labels when they were saved accidentally or are no longer valid.