> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sovara-labs.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Annotations

> Queue, inspect, and label runs from the CLI.

Annotations turn important runs into labeled examples. Use them when a run is a
clear success, a clear failure, or useful evidence for future evaluation.

## Queue a run

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
sovara annotations enqueue <run-id>
```

It is safe to enqueue the same run again after appending more steps.

## List annotation work

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
sovara annotations list --project-id <project-id-or-name> --status queue --limit 10
sovara annotations list --project-id <project-id-or-name> --status queue --sort failureScore --dir desc --failure-min 0.5
sovara annotations list --project-id <project-id-or-name> --status annotated --limit 10
sovara annotations list --project-id <project-id-or-name> --status annotated --label failure --code-version <short-sha>
```

`annotations list` always requires a project. The selector accepts a full
project ID, an unambiguous ID prefix, or an exact project name.
Queue filters also cover name/run ID, text query, time, runtime, novelty score,
analysis state, and tag IDs. Run `sovara tags list --project-id <project>` to
map tag names to IDs. Responses retain scores, statuses, tags, and
`distinct_code_versions`, which can be fed back into later filters.

## Inspect before labeling

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
sovara annotations inspect <run-id>
sovara probe <run-id>
sovara probe <run-id> --step <step-ref> --preview
```

`annotations inspect` returns the complete run-annotation response. The persisted
`AnnotationRecommendation` is under `recommendation` and may be `null` when the
run has no recommendation. It includes the analysis verdict and hint,
scoring/status fields, every completed artifact-adjudication pair, closest
retained failure evidence, and most-novel evidence.

Evidence locators expose `run_id`, `step_uuid`, display `step_ref`, `field`, and
zero-based `chunk_index`. In `evidence.adjudication_pairs`,
`failure_embedding_artifact` is `true` for an embedding artifact, `false` for a
completed verdict that retained the pair as plausible failure evidence, and
`null` when the pair was not checked.

Use step refs from `probe` when you want the label to point at specific
evidence.

## Save a success

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
sovara annotations set <run-id> --label success
```

Add optional ground truth when it helps future reviewers:

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
sovara annotations set <run-id> \
  --label success \
  --groundtruth "The answer cites the contract renewal policy and gives the correct date."
```

## Save a failure

Failure annotations require ground truth:

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
sovara annotations set <run-id> \
  --label failure \
  --groundtruth "The answer should use the current refund policy and refuse unsupported exceptions."
```

Focus the failure on specific steps:

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
sovara annotations set <run-id> \
  --label failure \
  --groundtruth "The retrieval step missed the enterprise policy page." \
  --steps 4,6.2
```

## Remove an annotation

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
sovara annotations remove <run-id>
```

Only remove labels when they were saved accidentally or are no longer valid.