Annotate a trace - Sovara

1. Find a trace

Open Traces, then Filters → Annotation → Needs annotation. Sort by Failure resemblance or Novelty to prioritize review. Completed traces appear here automatically.

2. Inspect the evidence

Optionally select traces and choose Deep Analysis from Actions or the right-click menu. Read Analysis and Evidence to inspect, then choose Inspect trace to check the full trace. Sovara chat can help locate relevant steps. Compare the recorded inputs, tool results, and response with the expected behavior.

3. Save the annotation

In the analysis panel, choose a Trace label:

Success: the trace did the right thing; ground truth is optional.
Failure: the trace was wrong; ground truth is required.
No label: save ground truth without judging success or failure.

Write Ground truth describing what should have happened and why, then choose Save. For a failure limited to part of a trace, identify the relevant steps. The CLI also accepts step references with --steps.

Edit or clear

Reopen the trace, update its label or ground truth, and save. To clear the annotation, choose No label, empty Ground truth, and save. The recording is kept and becomes eligible for review again. Annotations help refine recommendations for other traces. Follow up with a lesson proposal or an issue when needed.