# Efficient annotation Source: https://docs.sovara-labs.com/annotations/annotate Review surfaced runs and save useful labels. Sovara keeps the annotation queue focused. It separates runs to annotate from already annotated runs, shows why a run was surfaced, and points reviewers to the trace steps worth checking first. Sovara annotation queue with surfaced runs and reviewer guidance Sovara annotation queue with surfaced runs and reviewer guidance ## Add a run manually Sovara recommends runs automatically, but you can also add a run from the Runs view when you already know it deserves review. Select the run, open **Actions**, then choose Add to annotation queue. Add a selected run to the annotation queue from the Runs view Add a selected run to the annotation queue from the Runs view ## Review the run Click Inspect run to inspect the trace, read the recommendation, and save a label. For failures, include the ground truth or correction that should guide future behavior. Annotation review view opened into the run trace Annotation review view opened into the run trace Use annotation to: * Mark a run as success or failure * Preserve the ground truth for important failures * Capture recurring domain knowledge as a lesson * Build a reviewed set of examples for future regression checks # Why annotate? Source: https://docs.sovara-labs.com/annotations/overview Why annotation matters for improving agent behavior. Annotation is where a run becomes training signal for the team. It records whether the agent succeeded, why the run mattered, and what a better answer or fix should preserve. ## The problem Manual annotation is expensive. It takes domain expertise, context switching, and careful reading of traces. A naive queue fills up quickly with noise: duplicates, obvious mistakes, easy successes, and runs that are already covered by earlier examples. The hard failures are different. They are often hidden in domain assumptions, financial conventions, internal policy, or multi-step tool use. These are the cases where annotation is most valuable, because the missing knowledge is not visible from the final answer alone. ## What's wrong with random sampling? Random sampling gives you a rough sense of the agent's failure rate. It is a reasonable starting point when the system is new and failures are everywhere. It becomes inefficient once the agent is already reasonably good. Reviewers spend too much time on obvious successes, repeated mistakes, and behavior already covered by earlier annotations. The more capable the agent gets, the more valuable it is to spend review time on the few runs that reveal something new. ## Why not just use LLM-as-a-judge? LLM judges are useful for straightforward checks. They can catch clear format errors, direct mismatches, or failures that are obvious from the local context. They are weaker when the failure depends on domain knowledge the agent also missed. If a general judge can reliably infer the issue from the same context, the agent often could have avoided the mistake in the first place. The most valuable failures are the ones that require a human to notice the hidden assumption and turn it into reusable guidance. That is where annotation gives teams an edge. It captures the domain judgment behind the failure, not just the fact that the output was wrong. # Recommendation algorithm Source: https://docs.sovara-labs.com/annotations/recommendation-algorithm How Sovara decides what should be reviewed. Sovara's recommendation algorithm keeps annotation focused. It does not try to replace the reviewer. It tries to decide which runs are worth a reviewer's time. ## What it optimizes for The queue favors runs that may add new information. A run is more useful when it shows behavior not already covered by nearby annotated examples, exposes a partial gap in the agent's capability, or contains a failure that is hard to judge from the final output alone. The algorithm also reduces repeat work. Runs that look similar to already reviewed successes should not keep coming back unless they reveal a new pattern. ## Priority labels Sovara groups surfaced runs by why they deserve attention. The UI shows five labels: * **Repeated failure**: the run matches a failure pattern already seen in related traces. * **Failure risk**: the run is likely wrong, but it is not yet established as a repeated failure pattern. * **Novel behavior**: the run is meaningfully different from reviewed examples. * **Partially covered**: related examples exist, but the run still tests a gap. * **Covered**: successful references already cover the behavior, so review is optional and lower priority. The label is a starting point, not a verdict. The reviewer still decides whether the run should be marked as success or failure. ## Reviewer guidance When Sovara surfaces a run, it includes a short explanation and links to the trace steps worth checking first. Start there, then open the full run when the case needs more context. Inspect run button in the annotation queue Inspect run button in the annotation queue Click Inspect run, then open Run chat and start with: ```text theme={"theme":{"light":"github-light","dark":"github-dark"}} What should I look at first? ``` Run chat can point you to the final answer, retrieved evidence, related failed behavior, and the steps cited by the recommendation. Use that as the starting point for the label and ground truth. Good annotations make future recommendations better. They tell Sovara which behaviors are already covered, which failures matter, and which domain lessons should become lessons. # Annotations Source: https://docs.sovara-labs.com/cli/annotations Queue, inspect, and label runs from the CLI. Annotations turn important runs into labeled examples. Use them when a run is a clear success, a clear failure, or useful evidence for future evaluation. ## Queue a run ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara annotations enqueue ``` It is safe to enqueue the same run again after appending more steps. ## List annotation work ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara annotations list --project-id --status queue --limit 10 sovara annotations list --project-id --status queue --sort failureScore --dir desc --failure-min 0.5 sovara annotations list --project-id --status annotated --limit 10 sovara annotations list --project-id --status annotated --label failure --code-version ``` `annotations list` always requires a project. The selector accepts a full project ID, an unambiguous ID prefix, or an exact project name. Queue filters also cover name/run ID, text query, time, runtime, novelty score, analysis state, and tag IDs. Run `sovara tags list --project-id ` to map tag names to IDs. Responses retain scores, statuses, tags, and `distinct_code_versions`, which can be fed back into later filters. ## Inspect before labeling ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara annotations inspect sovara probe sovara probe --step --preview ``` `annotations inspect` returns the complete run-annotation response. The persisted `AnnotationRecommendation` is under `recommendation` and may be `null` when the run has no recommendation. It includes the analysis verdict and hint, scoring/status fields, every completed artifact-adjudication pair, closest retained failure evidence, and most-novel evidence. Evidence locators expose `run_id`, `step_uuid`, display `step_ref`, `field`, and zero-based `chunk_index`. In `evidence.adjudication_pairs`, `failure_embedding_artifact` is `true` for an embedding artifact, `false` for a completed verdict that retained the pair as plausible failure evidence, and `null` when the pair was not checked. Use step refs from `probe` when you want the label to point at specific evidence. ## Save a success ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara annotations set --label success ``` Add optional ground truth when it helps future reviewers: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara annotations set \ --label success \ --groundtruth "The answer cites the contract renewal policy and gives the correct date." ``` ## Save a failure Failure annotations require ground truth: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara annotations set \ --label failure \ --groundtruth "The answer should use the current refund policy and refuse unsupported exceptions." ``` Focus the failure on specific steps: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara annotations set \ --label failure \ --groundtruth "The retrieval step missed the enterprise policy page." \ --steps 4,6.2 ``` ## Remove an annotation ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara annotations remove ``` Only remove labels when they were saved accidentally or are no longer valid. # Assistant skill Source: https://docs.sovara-labs.com/cli/assistant-skill Install Sovara guidance for Codex and Claude Code. The Sovara skill teaches coding assistants how to set up an agent repository, record and inspect runs, rerun LLM steps, annotate examples, and manage lessons. Its **Repository Setup** workflow is the source of truth used by the guided launcher. ## Guided repository setup Run the zero-option launcher from the repository that contains the agent: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} cd agent-dir sovara setup ``` The CLI: 1. detects Codex and Claude Code; 2. asks which assistant to use when both are installed; 3. globally installs or refreshes the skill for the selected assistant; and 4. launches that assistant in the current directory with the setup prompt. The assistant then inspects the repository, installs the matching SDK, adds a project-owned run and useful trace boundaries, and verifies a representative run. It asks before a paid provider call and finishes with an explanation and exact next command. `sovara setup` accepts no options. Run it from the directory you want the assistant to modify. ## Install the skill without launching setup Use `install-skill` for manual, multi-target, or project-local installation: | Target | Command | | --------------------- | -------------------------------------- | | Codex and Claude Code | `sovara install-skill --target both` | | Codex only | `sovara install-skill --target codex` | | Claude Code only | `sovara install-skill --target claude` | By default, the skill is installed globally. For a project-local install: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara install-skill \ --target codex \ --level project \ --project-dir /path/to/project ``` ## Useful prompts after setup ```text theme={"theme":{"light":"github-light","dark":"github-dark"}} Use the Sovara CLI to inspect why the latest run failed. ``` ```text theme={"theme":{"light":"github-light","dark":"github-dark"}} Find the LLM step that produced the wrong answer and rerun it with a lower temperature. ``` ```text theme={"theme":{"light":"github-light","dark":"github-dark"}} Queue this run for annotation and create a lesson only if the failure is reusable. ``` ## Refresh the skill ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara install-skill --target both ``` # Claude Code Source: https://docs.sovara-labs.com/cli/claude-code Use the Sovara CLI with Claude Code. ## Guided setup Run the launcher from the repository that contains the agent: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} cd agent-dir sovara setup ``` If Claude Code is the only supported coding assistant installed, it launches automatically. If Codex is also installed, choose Claude Code when prompted. The skill is installed globally at `~/.claude/skills/sovara` before launch. ## Install or refresh the skill only ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara install-skill --target claude ``` For project-local installation and example follow-up prompts, see [Assistant Skill](/cli/assistant-skill). # Codex Source: https://docs.sovara-labs.com/cli/codex Use the Sovara CLI with Codex. ## Guided setup Run the launcher from the repository that contains the agent: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} cd agent-dir sovara setup ``` If Codex is the only supported coding assistant installed, it launches automatically. If Claude Code is also installed, choose Codex when prompted. The skill is installed globally at `~/.agents/skills/sovara` before launch. ## Install or refresh the skill only ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara install-skill --target codex ``` For project-local installation and example follow-up prompts, see [Assistant Skill](/cli/assistant-skill). # Inspect runs Source: https://docs.sovara-labs.com/cli/inspect Use probe, step-overview, and logs to understand a recorded run. Start every investigation with the run overview. It returns the first 20 immediate steps in the root run, including persisted summaries when available. ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara probe ``` Use `--range` for another zero-based, end-exclusive page: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara probe --range 20:40 ``` Subruns stay collapsed in their parent overview. Open one with `--scope`; the range applies to that subrun's immediate steps, not to a flattened tree. ```text theme={"theme":{"light":"github-light","dark":"github-dark"}} root ├── 1 ├── 2 └── 3 subrun ├── 3.1 └── 3.2 subrun ├── 3.2.1 └── 3.2.2 ``` ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara probe --scope 3 sovara probe --scope 3.2 ``` The first command returns `3.1` and `3.2`; the second returns `3.2.1` and `3.2.2`. Nesting can continue to any depth. ## Inspect one step Use the `step_ref` from the overview: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara probe --step 3 --preview ``` `--preview` keeps large fields readable. Use it before fetching full input or output. ## Inspect selected keys Use `--input`, `--output`, and `--key-regex` when you know the field you need: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara probe --step 3 --input --key-regex "body.max_tokens$" sovara probe --step 3 --output --key-regex "content" ``` For LLM steps, inspection uses the actual input sent to the model, including the supplementary user message containing lessons applied to that step. Use `--key-regex` to select only the fields you need instead of dumping full prompts, tool arguments, or response payloads. ## Get a semantic summary When the exact payload is less important than the behavior, ask Sovara for a step overview: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara step-overview --step 3 ``` ## Read logs Use logs for captured stdout and stderr: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara logs --tail 100 sovara logs --grep "Cache miss" --context 2 --line-numbers ``` ## Good inspection sequence ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara runs --project-id --limit 20 --sort timestamp --dir desc sovara probe sovara probe --scope 6 --range :20 sovara probe --scope 6.2 --range :20 sovara probe --step 6.2.1 --preview sovara logs --tail 80 ``` Use step refs such as `6.2`, not internal IDs. # Install Source: https://docs.sovara-labs.com/cli/install Install the Sovara CLI and verify that it can reach Sovara. The Sovara CLI is a standalone binary named `sovara`. It is not installed by the Python SDK or TypeScript runner. Installing the CLI does not require the desktop app. `sovara status` checks the active app-server connection: keep the desktop app running for **Local**, or add and select a remote app-server connection in the desktop app. The CLI has no separate app-server setting. SDK runs can also start the local execution server when needed. ## macOS, Linux, and WSL ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} curl -fsSL https://apps.sovara-labs.com/cli/install.sh | sh ``` Open a new terminal after installation, then verify: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara --version sovara status ``` ## Windows PowerShell ```powershell theme={"theme":{"light":"github-light","dark":"github-dark"}} irm https://apps.sovara-labs.com/cli/install.ps1 | iex ``` Open a new terminal and run `sovara --version` and `sovara status`. ## Windows CMD ```cmd theme={"theme":{"light":"github-light","dark":"github-dark"}} curl -fsSL https://apps.sovara-labs.com/cli/install.cmd -o install.cmd && install.cmd && del install.cmd ``` Open a new terminal and run `sovara --version` and `sovara status`. ## Update the CLI Check for a newer release without installing it: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara update --check ``` Install the latest release and refresh the assistant skill: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara update ``` ## Set up an agent repository After installing the CLI and connecting to a Sovara environment, let a coding assistant add and verify the SDK integration: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} cd agent-dir sovara setup ``` To install guidance without launching an assistant, use: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara install-skill --target both ``` See [Assistant skill](/cli/assistant-skill) for advanced installation options. # Project Issues Source: https://docs.sovara-labs.com/cli/issues Read, create, comment on, and close Sovara project issues from the CLI. Use project issues to track concrete follow-up work discovered while inspecting or improving an agent. Every command requires `--project-id`. A selector may be a full project ID, an unambiguous ID prefix, or an exact project name. ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} PROJECT="support-agent" sovara issues list --project-id "$PROJECT" --status open sovara issues get 42 --project-id "$PROJECT" sovara issues create --project-id "$PROJECT" \ --title "Retry behavior is unclear" \ --body "The timeout path should follow lesson #17." sovara issues comment 42 --project-id "$PROJECT" \ --body "Reproduced in the latest run." sovara issues close 42 --project-id "$PROJECT" ``` Issue reference arguments are positive numbers without a leading `#`. `list` defaults to open issues and supports `--status open|closed`, `--view all|assigned|created`, `--query`, `--limit`, and `--offset`. Issues and lessons share a project reference namespace so Markdown bodies and comments can mention a lesson with `#N`. The `issues get` and `issues comment` commands reject references that resolve to lesson comments. Commands print JSON for reliable use by coding assistants. Creating, commenting, and closing are user-attributed mutations and should only be run when the user clearly requests the change. # Lessons Source: https://docs.sovara-labs.com/cli/lessons Read and manage Sovara lessons from the CLI. Lessons are reusable instructions, domain facts, and operating rules that Sovara can retrieve for future agent runs. Every `sovara lessons` subcommand requires `--project-id`. The selector accepts a full project ID, an unambiguous ID prefix, or an exact project name. Examples below use a shell variable to keep commands readable: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} PROJECT=support-agent ``` ## List and read lessons ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara lessons ls --project-id "$PROJECT" sovara lessons ls --project-id "$PROJECT" "support/refunds/" sovara lessons ls -R --project-id "$PROJECT" "support/" sovara lessons get --project-id "$PROJECT" sovara lessons get , --project-id "$PROJECT" ``` `ls` returns lessons and immediate child folders. Omit the path or pass `""` to list the root folder. Add `-R` / `--recursive` to include compact lesson records from descendant folders; full lesson content is omitted. `get` returns complete lesson details. Multiple comma-separated IDs are returned under a `lessons` array in the requested order. ## Retrieve lessons for context ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara lessons retrieve --project-id "$PROJECT" "Customer asks for a refund exception" ``` Use `retrieve` to see which lessons match concrete agent context. Optionally scope retrieval to a folder with `--path` / `-p`. ## Create a lesson Creation requires a title, content, and retrieval-oriented usage hint: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara lessons create \ --project-id "$PROJECT" \ --title "Retry rate-limited provider calls" \ --content "When an upstream API returns HTTP 429, retry with exponential backoff and jitter." \ --when-to-use "When a provider request fails because of rate limiting" \ --path "reliability/providers/" \ --run-id "" \ --step 3 ``` When a lesson comes from a specific step, pass the full source run UUID and the step ref shown by `sovara probe`. Sovara resolves the ref to a durable step UUID before creating the lesson. Omit both flags when the lesson has no concrete run provenance. ## Improve a lesson Ask for a non-mutating wording proposal: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara lessons polish --project-id "$PROJECT" ``` Apply an update: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara lessons update \ --project-id "$PROJECT" \ --content "Retry HTTP 429 responses with exponential backoff and jitter." \ --run-id "" ``` ## Manage folders ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara lessons mkdir "support/refunds/" --project-id "$PROJECT" sovara lessons mv -i "support/refunds/" --project-id "$PROJECT" sovara lessons cp "support/refunds/" "support/escalations/" --project-id "$PROJECT" ``` Use restructure commands only when intentionally reorganizing a larger lesson taxonomy. Pass the same explicit project selector to every command. # Overview Source: https://docs.sovara-labs.com/cli/overview Use the Sovara CLI to set up, report, inspect, rerun, annotate, and improve agent runs. `sovara` is the terminal interface for Sovara. It complements the Python and TypeScript SDKs: instrumented code creates runs, while the CLI helps people and coding assistants inspect and improve them. Most commands output JSON so assistants can parse the result reliably. ## Recommended path 1. Install the CLI, then use an existing Sovara environment or choose a local, remote, or headless setup in [Installation](/get-started/installation). 2. Add an SDK run with an explicit project name in code. For guided setup, run `sovara setup` from the agent repository. 3. Run the application normally, or use `sovara record -- ` when you want terminal run metadata. 4. Inspect the run with `sovara probe` and `sovara logs`. 5. Rerun an LLM step, annotate examples, manage lessons, or track project issues when the trace shows a concrete follow-up. The CLI does not infer a project from the current directory. SDK code owns project selection; project-scoped CLI commands require an explicit project selector. ## For coding assistants From the repository that contains the agent, run: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} cd agent-dir sovara setup ``` The command installs the Sovara skill for an available coding assistant and launches that assistant in the current directory. If both Codex and Claude Code are installed, it asks which one to use. For manual or advanced skill installation, use `sovara install-skill`. ## What to read next * [Install the CLI](/cli/install) * [Record and report runs](/cli/record) * [Inspect runs](/cli/inspect) * [Rerun LLM steps](/cli/rerun-steps) * [Manage project issues](/cli/issues) * [Install the assistant skill](/cli/assistant-skill) # Report runs Source: https://docs.sovara-labs.com/cli/record Run a command unchanged and print metadata for the SDK-created Sovara run. Use `sovara record` when an instrumented application should print its run ID and project metadata in the terminal: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara record -- python agent.py ``` The only supported form is: ```text theme={"theme":{"light":"github-light","dark":"github-dark"}} usage: sovara record -- [args...] ``` Everything after `--` is executed unchanged. The application SDK still owns the project name, run name, persistent client run ID, tracing, and execution timeouts. ## What `record` does 1. Gives the child process a private result-file path through `SOVARA_RUN_FILE`. 2. Runs the command with its original arguments and terminal streams. 3. Waits for the SDK to finish and publish its run metadata. 4. Prints JSON with the child status, exit code, duration, and observed run. 5. Exits with the child's exit code. When a run is observed, the output includes fields such as: ```json theme={"theme":{"light":"github-light","dark":"github-dark"}} { "status": "completed", "exit_code": 0, "duration_seconds": 1.23, "run_observed": true, "run_id": "...", "project_id": "...", "project_name": "support-agent", "appended": false, "inspect_command": "sovara probe ..." } ``` If the command creates several top-level runs, the last completed run is reported. ## What `record` does not do `record` does not instrument an arbitrary Python or Node.js program. An uninstrumented command still runs, but the result contains `"run_observed": false` and a message explaining that no SDK-created run was seen. Add the run boundary in application code first: * Python: `SovaraClient(project_name="...").run(...)` * TypeScript: `new SovaraClient({ projectName: "..." })` Run names and durable application IDs also belong in those SDK calls. ## Normal execution is supported The wrapper is optional. This records the same run without printing metadata: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} python agent.py ``` After `record`, continue with the emitted `inspect_command` or see [Inspect runs](/cli/inspect). # Reference Source: https://docs.sovara-labs.com/cli/reference CLI command reference for common Sovara workflows. Run `sovara --help` for the command list installed on your machine. Every command and subcommand accepts `-h` and `--help` for its exact usage and flags. ```text theme={"theme":{"light":"github-light","dark":"github-dark"}} usage: sovara [options] ``` ## Setup and connectivity | Command | Purpose | | ---------------------- | -------------------------------------------------------------------------------------------------------- | | `sovara status` | Check backend health and the signed-in user. | | `sovara projects` | List projects or fetch one by ID, ID prefix, or exact name. | | `sovara setup` | Install the skill and launch an available coding assistant in the current directory. Accepts no options. | | `sovara install-skill` | Install assistant guidance for Codex, Claude Code, or both. | | `sovara update` | Check for or install the latest CLI release. | | `sovara exec-server` | Start, health-check, or stop the bundled local exec server. | App commands use the active app-server connection selected in the Sovara desktop app: either **Local** or a remembered remote connection. For a remote connection, they also use its signed-in desktop session. Switch app-server connections with the selector on the **Projects** page; the CLI has no separate app-server endpoint setting. ## Local exec server ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara exec-server start sovara exec-server stop ``` `start` launches the bundled exec server when the configured local endpoint is not healthy. With an explicit `--server-url`, it only health-checks that endpoint. `stop` accepts only a local URL, stops the bundled server and its Qdrant sidecar, and succeeds when they are already stopped. ## Recording and inspection | Command | Purpose | | ------------------------------------------------- | ----------------------------------------------------------- | | `sovara record -- [args...]` | Execute a command unchanged and report its SDK-created run. | | `sovara runs` | List recorded runs, optionally scoped with `--project-id`. | | `sovara tags list --project-id ` | List tag names, colors, and IDs usable in run filters. | | `sovara probe ` | Inspect run structure and step payloads. | | `sovara step-overview --step ` | Summarize one step. | | `sovara logs ` | Read captured stdout and stderr. | Project selection for a run is defined by SDK code, not `record` or the working directory. Project-scoped run listing mirrors the Runs API. It returns the selected page plus `distinct_code_versions` and `custom_metric_columns` for discovering discrete filters: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara tags list --project-id sovara runs --project-id --limit 20 --offset 0 --sort timestamp --dir desc sovara runs --project-id --name eval --run-id 4e341aec sovara runs --project-id --label up,down --tag-id --code-version sovara runs --project-id --time-from 2026-07-01T00:00:00Z --latency-min 1.5 sovara runs --project-id --sort metric:quality --metric-filters '{"quality":{"kind":"float","min":0.8}}' ``` List-valued flags are comma-separated. Multiple `--tag-id` values are ANDed. `--project-id` is required for every run list. ## Diagnostics Create a ZIP archive of local service logs and their retained rotations: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara diagnostics sovara diagnostics --output ./sovara-diagnostics.zip sovara diagnostics --include-system-logs ``` Without `--output`, the archive is written under `~/.sovara/diagnostics/`. Normal Sovara application logs are always archived on macOS, Windows, and Linux. `--include-system-logs` additionally collects the last 30 minutes of Sovara-related macOS Unified Logs, Windows Application/Defender/Code Integrity/AppLocker events, or Linux systemd journal entries. System-log collection is opt-in because OS logs may contain sensitive machine metadata. It is best-effort, never requests elevated privileges, and writes `system/system-log-collection.txt` with `status=unavailable` when the platform source cannot be read. ## Step rerun ```text theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara rerun --step [--disable-lesson-injection] [--set KEY=VALUE] [--set-file KEY=PATH] [--set-json KEY=JSON] ``` Only keys shown by `probe` for the selected step can be edited. See [Rerun LLM Steps](/cli/rerun-steps) for examples. Reruns perform current lesson retrieval by default; pass `--disable-lesson-injection` to skip retrieving and injecting new lessons for that rerun. ## Replay keys | Command | Purpose | | --------------------------------------------------------- | -------------------------------------------------- | | `sovara replay-keys list` | Show masked previews of configured replay keys. | | `sovara replay-keys set --api-key-env ENV_VAR` | Store a provider key from an environment variable. | | `sovara replay-keys set --api-key-stdin` | Store a provider key from stdin. | | `sovara replay-keys unset ` | Remove a provider replay key. | See [Replay Keys](/cli/replay-keys) for provider naming and validation details. ## Annotations ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara annotations list --project-id --status queue sovara annotations list --project-id --status queue --sort failureScore --dir desc --failure-min 0.5 sovara annotations inspect sovara annotations enqueue sovara annotations set --label success sovara annotations set --label failure --groundtruth "" --steps 4,6.2 sovara annotations remove ``` Annotation list filters include paging, name/run ID, free-text query, time, runtime, code version, and queue-only failure score, novelty score, analysis, and tag filters. Annotated lists additionally accept `--label success,failure`. The JSON preserves scores, statuses, tags, and `distinct_code_versions`. `annotations inspect` prints the complete run-annotation response. The persisted recommendation is under `recommendation` and includes analysis, scores, statuses, adjudication pairs, closest failure evidence, and most-novel evidence. ## Project issues Every issue command requires `--project-id`: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara issues list --project-id --status open sovara issues list --project-id --status closed --view created --query "timeout" sovara issues get 42 --project-id sovara issues create --project-id --title "Retry behavior is unclear" --body "See #17." sovara issues comment 42 --project-id --body "Reproduced in the latest run." sovara issues close 42 --project-id ``` Use positive project reference numbers without a leading `#`. Commands output JSON. Issue reads and comments reject references that resolve to lesson conversations. ## Lessons Every lessons command requires `--project-id`: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara lessons ls [-R] [path] --project-id sovara lessons get [,...] --project-id sovara lessons create --project-id --title "" --content "<content>" --when-to-use "<retrieval hint>" [--run-id <full-run-uuid> --step <step-ref>] sovara lessons polish <lesson-id> --project-id <project-id-or-name> sovara lessons retrieve --project-id <project-id-or-name> "<context>" ``` `ls` lists the root folder when `path` is omitted or `""`. Add `-R` / `--recursive` for compact recursive output. `get` accepts one lesson ID or a comma-separated list. Use `--force` on lesson commands only when you intentionally want to skip model validation. # Replay keys Source: https://docs.sovara-labs.com/cli/replay-keys Manage provider API keys used when rerunning recorded LLM steps. Step rerun needs a provider API key because Sovara sends the edited LLM request to the same provider family as the recorded step. Replay keys are separate from your shell environment. Store a key once, then use `sovara rerun` without passing that key again. ## List configured keys ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara replay-keys list ``` The output shows provider names and masked previews. It does not print raw API keys. ## Set a key from an environment variable ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} export ANTHROPIC_API_KEY=sk-ant-... sovara replay-keys set anthropic --api-key-env ANTHROPIC_API_KEY ``` Common provider names include: | Provider | Example | | ---------------- | ------------------------------------------------------------------ | | Anthropic | `sovara replay-keys set anthropic --api-key-env ANTHROPIC_API_KEY` | | OpenAI | `sovara replay-keys set openai --api-key-env OPENAI_API_KEY` | | Gemini | `sovara replay-keys set gemini --api-key-env GEMINI_API_KEY` | | Azure OpenAI | `sovara replay-keys set azure --api-key-env AZURE_API_KEY` | | Palantir Foundry | `sovara replay-keys set palantir --api-key-env TWG_PALANTIR_TOKEN` | Use the provider name shown in the recorded step when in doubt. ## Set a key from stdin ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} printf '%s' "$ANTHROPIC_API_KEY" | sovara replay-keys set anthropic --api-key-stdin ``` The CLI intentionally does not accept raw keys as command-line arguments, because command-line arguments can be saved in shell history or process lists. ## Remove a key ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara replay-keys unset anthropic ``` ## Validation When possible, Sovara validates a replay key before saving it. If validation fails, the key is not saved. Check the provider name and make sure the key has permission to call the model family used by the recorded step. ## Supported replay providers Direct step rerun works for providers Sovara can call with a normal API key or bearer token. Anthropic, OpenAI, Gemini, Azure OpenAI, Palantir Foundry, and many OpenAI-compatible providers fit that model. Providers that require cloud request signing or service-account credentials may still be inspectable but not directly rerunnable from `sovara rerun`. # Rerun LLM steps Source: https://docs.sovara-labs.com/cli/rerun-steps Replay one recorded LLM step with edited input from the CLI. Use `sovara rerun` when you want to test a small change to one recorded LLM call without rerunning the entire agent. Typical uses: * Change a prompt fragment * Change an existing model parameter such as max tokens * Replace a user message from a file * Compare whether one step would have produced a better answer ## Requirements Before rerunning a step: 1. Record or locate a run. 2. Inspect the run and find the step ref. 3. Confirm the step is an LLM call. 4. Inspect the exact input keys you want to edit. 5. Configure a replay API key for the provider used by that step. ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara probe <run-id> sovara probe <run-id> --step 3 --input --key-regex "body" sovara replay-keys set anthropic --api-key-env ANTHROPIC_API_KEY ``` For a step recorded through Palantir Foundry, store the Foundry token under the `palantir` provider: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara replay-keys set palantir --api-key-env TWG_PALANTIR_TOKEN ``` See [Replay keys](/cli/replay-keys) for provider key management. ## Basic rerun ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara rerun <run-id> --step 3 ``` This reruns current lesson retrieval, replays the resulting LLM input, and returns the new output plus the retrieval snapshot. To compare the step without retrieving or injecting new lessons: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara rerun <run-id> --step 3 --disable-lesson-injection ``` The flag removes the selected step's recorded lesson injection and skips fresh retrieval for that step. ## Edit string values Use `--set KEY=VALUE` for a string edit: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara rerun <run-id> \ --step 3 \ --set body.messages.0.content="Answer using the policy excerpt only." ``` Keys must use the flattened names shown by `probe`; unknown keys are rejected: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara probe <run-id> --step 3 --input --key-regex "body.messages" ``` ## Edit JSON values Use `--set-json` for numbers, booleans, arrays, objects, or null: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara rerun <run-id> \ --step 3 \ --set-json body.max_tokens=512 ``` For arrays or objects, quote the JSON for your shell: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara rerun <run-id> \ --step 3 \ --set-json 'body.stop=["</answer>"]' ``` ## Read an edit from a file Use `--set-file KEY=PATH` for larger prompt text: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara rerun <run-id> \ --step 3 \ --set-file body.system.0.text=prompt_variant.txt ``` ## Combine edits ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara rerun <run-id> \ --step 3 \ --set-file body.system.0.text=prompt_variant.txt \ --set body.messages.0.content="Use the attached table." \ --set-json body.max_tokens=512 ``` ## Output Successful reruns return JSON with: * `status` * `run_id` * `step_ref` * `overwritten_output` * `lesson_retrieval` The Sovara app will show the rerun output for the selected step. ## Limits `sovara rerun` is for one recorded LLM step. It does not rerun tool calls, custom traced functions, or the full agent. If a provider key is missing, set it with `sovara replay-keys set`. If the provider is not replayable with an API key, inspect the step and rerun the full agent instead. # Troubleshooting Source: https://docs.sovara-labs.com/cli/troubleshooting Fix common Sovara CLI setup, reporting, inspection, and rerun issues. ## `sovara: command not found` Open a new terminal after installing the CLI so the shell reloads `PATH`. If it still fails, reinstall and verify: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} curl -fsSL https://apps.sovara-labs.com/cli/install.sh | sh sovara --version ``` ## Sovara is not reachable Check the currently configured app environment: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara status ``` Open the desktop app and verify the active app-server connection in the selector on the **Projects** page. With **Local** selected, keep the app open so its bundled backend is available. For a remembered remote connection, verify its origin under **Settings > App-server connection settings** and sign in through the app. CLI app commands automatically use the selected desktop app-server connection and, when it is remote, its session. For headless agent hosts, configure the exec-server URL and agent token directly in the Python or TypeScript SDK. That configuration applies to recording and runtime traffic, not user-facing CLI app commands. ## `record` says no SDK-created run was observed The child command ran, but it did not complete an SDK top-level run. Confirm that the executed path reaches: ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} from sovara import SovaraClient sovara_client = SovaraClient(project_name="support-agent") with sovara_client.run("answer-question"): run_agent() ``` `record` does not add this boundary automatically. ## A run appears in the wrong project Change the project name in the SDK integration: * Python: `SovaraClient(project_name="...")` * TypeScript: `new SovaraClient({ projectName: "..." })` The current directory and `record` command do not select a project. Project names are immutable; a new name selects or creates a different project. ## A module is missing during `record` Use the same environment that normally runs the application: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara record -- /path/to/venv/bin/python agent.py ``` ## A project-scoped command requires `--project-id` List accessible projects, then pass a full ID, unambiguous ID prefix, or exact project name: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara projects sovara annotations list --project-id support-agent --status queue sovara lessons ls --project-id support-agent ``` ## `rerun` says the replay API key is not configured Set a replay key for the provider shown in the error, then retry. See [Replay Keys](/cli/replay-keys). ## `rerun` says the step is not replayable Only recorded LLM calls can be rerun. Inspect tool calls and custom traced functions with `probe`, then rerun the complete application normally. ## Restricted shell or sandbox writes fail Allow writes to the global Sovara state directory, or install the assistant skill so it can configure that access. For Python package caches, use writable temporary locations when necessary: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} export UV_CACHE_DIR=/tmp/uv-cache export PYTHONPYCACHEPREFIX=/tmp/pycache ``` # Installation Source: https://docs.sovara-labs.com/get-started/installation Install Sovara and prepare your local development environment. Install the Sovara CLI first so you can check whether an existing Sovara environment is already available. Then choose a local, remote, or headless setup and install the SDK for your agent's language. ## Sovara CLI Sovara's CLI gives you terminal access to run inspection, trace queries, logs, and shared skill guidance. The Python SDK and TypeScript SDK do not install the `sovara` command. <Tabs> <Tab title="macOS, Linux, WSL"> ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} curl -fsSL https://apps.sovara-labs.com/cli/install.sh | sh ``` </Tab> <Tab title="Windows PowerShell"> ```powershell theme={"theme":{"light":"github-light","dark":"github-dark"}} irm https://apps.sovara-labs.com/cli/install.ps1 | iex ``` </Tab> <Tab title="Windows CMD"> ```cmd theme={"theme":{"light":"github-light","dark":"github-dark"}} curl -fsSL https://apps.sovara-labs.com/cli/install.cmd -o install.cmd && install.cmd && del install.cmd ``` </Tab> </Tabs> Open a new terminal window if `sovara` is not found immediately, then verify the installation: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara --version ``` Install the Sovara skill for Codex and Claude Code: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara install-skill --target both ``` ## Connect to Sovara The desktop app starts with its fixed **Local** app-server connection. To add a remote app-server connection, open **Settings > App-server connection settings** and enter the origin provided by your administrator. Choose any remembered app-server connection from the selector on the **Projects** page; remote connections prompt you to sign in. CLI app commands automatically follow the active app-server connection: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara status ``` * If `sovara status` succeeds, the CLI is using the desktop's active app-server connection and, when it is remote, its signed-in session. * To change environments, choose another app-server connection from the selector on the **Projects** page. CLI app commands do not have separate app-server configuration. * For a headless or CI agent host, configure the execution-server URL and agent token through the [Python API reference](/sdks/python/api-reference) or [TypeScript API reference](/sdks/typescript/api-reference). Headless agent configuration is for recording and runtime traffic; user-facing CLI app commands use the desktop app-server connection. ## Sovara desktop app The desktop app gives you a local workspace for recording, inspecting, and improving agent runs. Choose the installer for your operating system. <div> <div> <span>Latest installers</span> </div> <div> <section> <div> <img alt="" /> <img alt="" /> </div> <div>macOS</div> <p>For Apple Silicon Macs.</p> <a href="https://apps.sovara-labs.com/api/update/download?platform=darwin&arch=arm64"> Apple Silicon (.dmg) </a> </section> <section> <div> <img alt="" /> <img alt="" /> </div> <div>Windows</div> <p>For Windows machines.</p> <a href="https://apps.sovara-labs.com/api/update/download?platform=win32&arch=x64"> Windows (.exe) </a> </section> <section> <div> <img alt="" /> <img alt="" /> </div> <div>Linux</div> <p>For Debian/Ubuntu and Fedora/Red Hat-based distributions.</p> <a href="https://apps.sovara-labs.com/api/update/download?platform=linux&arch=x64&format=deb"> Debian / Ubuntu Intel/AMD (.deb) </a> <a href="https://apps.sovara-labs.com/api/update/download?platform=linux&arch=arm64&format=deb"> Debian / Ubuntu arm64 (.deb) </a> <a href="https://apps.sovara-labs.com/api/update/download?platform=linux&arch=x64&format=rpm"> Fedora / Red Hat Intel/AMD (.rpm) </a> <a href="https://apps.sovara-labs.com/api/update/download?platform=linux&arch=arm64&format=rpm"> Fedora / Red Hat arm64 (.rpm) </a> </section> </div> </div> After the installer finishes, open Sovara, leave **Local** selected, and verify the connection: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara status ``` Keep the desktop app running while using the local workspace. Switching the desktop app-server connection changes the app server used by the UI and CLI app commands; it does not change an SDK's execution-server URL or agent token. ## Python SDK Install `sovara` in the Python environment where your agent runs. Use the same package manager that owns your project environment. <Tabs> <Tab title="uv"> ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} uv add sovara ``` </Tab> <Tab title="pip"> ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} python -m pip install sovara ``` </Tab> <Tab title="poetry"> ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} poetry add sovara ``` </Tab> </Tabs> ## TypeScript SDK Install `@sovara/runner` in the Node.js project where your agent runs. <Tabs> <Tab title="npm"> ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} npm install @sovara/runner ``` </Tab> <Tab title="pnpm"> ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} pnpm add @sovara/runner ``` </Tab> <Tab title="yarn"> ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} yarn add @sovara/runner ``` </Tab> </Tabs> # Quickstart Source: https://docs.sovara-labs.com/get-started/quickstart Record and inspect your first Sovara run. When you open the Sovara app for the first time, you will see a tutorial using a pre-recorded example project. Complete this tutorial before continuing with this section. In this quickstart, we will record our first own run! ## Create a sample project Let's create a minimal sample project with a script that calls an LLM. Open the terminal and create a new folder for that project: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} mkdir sovara-quickstart cd sovara-quickstart ``` Select which language you want to use below. <Tabs> <Tab title="Python"> ### Create the Python project file Create a `pyproject.toml` in the `sovara-quickstart` folder. `uv run quickstart.py` uses this file to install the OpenAI client and the Sovara SDK into the project's environment. ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} cat > pyproject.toml <<'EOF' [project] name = "sovara-quickstart" version = "0.1.0" description = "Sovara quickstart example." requires-python = ">=3.10" dependencies = [ "openai", "sovara", ] EOF ``` ### Create an example script Create `quickstart.py` in the project folder. You can use the following script which simply calls OpenAI's GPT 5.4 mini. It assumes that you have set `OPENAI_API_KEY` as environment variable. If this is not the case, you need to modify the script accordingy. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} from openai import OpenAI from sovara import SovaraClient # [!code ++] openai_client = OpenAI() sovara_client = SovaraClient(project_name="quickstart") # [!code ++] with sovara_client.run("quickstart run"): # [!code ++] response = openai_client.responses.create( model="gpt-5.4-mini", input="In one sentence, describe what a good quickstart should do.", ) print(response.output_text) ``` </Tab> <Tab title="TypeScript"> ### Create the TypeScript project file Create a `package.json` in the `sovara-quickstart` folder. `npm install` uses this file to install the OpenAI client, the Sovara runner, and `tsx`. ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} cat > package.json <<'EOF' { "name": "sovara-quickstart", "version": "0.1.0", "type": "module", "private": true, "dependencies": { "@sovara/runner": "latest", "openai": "latest", "tsx": "latest" } } EOF npm install ``` ### Create an example script Create `quickstart.ts` in the project folder. You can use the following script which simply calls OpenAI's GPT 5.4 mini. It assumes that you have set `OPENAI_API_KEY` as environment variable. If this is not the case, you need to modify the script accordingly. ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} import OpenAI from "openai"; import { SovaraClient } from "@sovara/runner"; // [!code ++] const openai_client = new OpenAI(); const sovara_client = new SovaraClient({ projectName: "quickstart" }); // [!code ++] await sovara_client.run("quickstart run", async () => { // [!code ++] const response = await openai_client.responses.create({ model: "gpt-5.4-mini", input: "In one sentence, describe what a good quickstart should do.", }); console.log(response.output_text); }); // [!code ++] ``` </Tab> </Tabs> ## Record the run Run the command from the `sovara-quickstart` folder. The Sovara desktop app should stay open while the command runs. <Tabs> <Tab title="Python"> ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} uv run quickstart.py ``` </Tab> <Tab title="TypeScript"> ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} npx tsx quickstart.ts ``` </Tab> </Tabs> ## Inspect the run Go to the Sovara desktop app. A new project `quickstart` should have been created automatically. You might need to refresh the projects page at the top right corner to see it. When you open the project and go to Runs in the side bar, you should see a run called `quickstart run`. Open the run to inspect its (single) step. You should be able to find the model's input and output, and the print statement which is logged in a tab in the right most panel. # Trace your agent Source: https://docs.sovara-labs.com/get-started/tracing-your-agent Add structure and details to your logs. This guide shows how to instrument an existing agent so Sovara records its model calls, tool calls, runtime metadata, and more. To track runs with Sovara, instrument your code with the SDK. Sovara's high-level run context lets you trace your entire agent with one line of code (see [Quickstart](/get-started/quickstart)). Inside that context, Sovara records supported LLM and tool calls and makes them available for analysis and optimization. You can add more instrumentation to structure the trace and make it easier to understand. A typical workflow to implement this instrumentalization is this: 1. **Let a coding agent instrumentalize your code:** Use the turnkey setup below to launch Codex or Claude Code with the Sovara skill and instrument your codebase. 2. **Fine-tune the instrumentation if needed:** For more fine-grained control, see [Instrument Manually with the Tracing SDK](#instrument-manually-with-the-tracing-sdk). ## Instrument via coding agents and the Sovara CLI Coding agents can give you a good starting point for inserting logging statements. The Sovara CLI allows you to launch Claude Code or Codex with set up instructions to instrumentalize your code. Install the CLI if you haven't already: <Tabs> <Tab title="macOS, Linux, WSL"> ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} curl -fsSL https://apps.sovara-labs.com/cli/install.sh | sh ``` </Tab> <Tab title="Windows PowerShell"> ```powershell theme={"theme":{"light":"github-light","dark":"github-dark"}} irm https://apps.sovara-labs.com/cli/install.ps1 | iex ``` </Tab> <Tab title="Windows CMD"> ```cmd theme={"theme":{"light":"github-light","dark":"github-dark"}} curl -fsSL https://apps.sovara-labs.com/cli/install.cmd -o install.cmd && install.cmd && del install.cmd ``` </Tab> </Tabs> Run `sovara status`. If it succeeds, keep using the configured environment. If it fails, follow [Installation](/get-started/installation) to choose a local, remote, or headless setup. Then go to the project folder containing the code you want to instrumentalize. ### Turnkey instrumentalization If you want to run one command and have the coding agent instrument the code on its own, run: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara setup ``` Sovara launches Codex or Claude Code automatically. If both are installed, it asks you which one to use. ### Manually steering the coding agent Alternatively, if you want to steer the coding agent manually, do the following: 1. Refresh the assistant skill: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara install-skill --target both ``` 2. Get started by asking Claude Code or Codex something like this: ```text theme={"theme":{"light":"github-light","dark":"github-dark"}} Use the Sovara skill to set up this project. Follow its Repository Setup section from start to finish, make the required dependency and instrumentation changes, record one representative run, and show me how to inspect it. Ask before any verification that would make a paid provider call. ``` For more CLI details, see [Claude Code](/cli/claude-code), [Codex](/cli/codex), and the [CLI overview](/cli/overview). ## Instrument manually with the SDK To make the code changes yourself, follow the SDK documentation for your language: * [Python SDK](/sdks/python/quickstart) * [TypeScript SDK](/sdks/typescript/quickstart) # Manual inspection Source: https://docs.sovara-labs.com/observability/manual-inspection Inspect run steps, inputs, and outputs by hand. Manual inspection is the detailed view for a single run. It is where you check the exact step that produced a surprising result. <img alt="Sovara run inspection view showing ordered steps and selected step details" /> <img alt="Sovara run inspection view showing ordered steps and selected step details" /> The left side shows the run steps. Each step is a model call, tool call, or recorded subrun. Selecting a step opens its input and output on the right, including injected lessons, request metadata, and the content sent to the model or tool. Manual inspection helps with: * Finding the first step where the agent drifted * Checking which context the model actually saw * Reviewing tool inputs and outputs * Confirming whether a lesson affected the run # Run chat Source: https://docs.sovara-labs.com/observability/run-chat Use chat to query and reason about recorded runs. Run chat gives you a fast read on a recorded run before you inspect every node by hand. It answers from the trace context and points you toward the steps that matter. Open <strong><Icon icon="message-square-text" /> Run chat</strong> from the run side panel. Good first questions include: * What was this run trying to do? * What input did the agent receive? * Which steps mattered most? * Where did the run likely fail? When you find a meaningful issue, use the run view to inspect the exact node and decide whether the fix belongs in code, prompts, tools, or Lessons. # Runs Source: https://docs.sovara-labs.com/observability/runs Recorded executions in Sovara. <strong><Icon icon="play" /> Runs</strong> is the project timeline. It shows persisted executions for a project and keeps comparison fields in one table. <img alt="Sovara Runs view showing recorded agent runs" /> <img alt="Sovara Runs view showing recorded agent runs" /> Each row is one persisted execution. Select a run, open **Actions**, and choose <strong><Icon icon="external-link" /> Open Run</strong> to inspect the full trace. Use <strong><Icon icon="funnel" /> Filters</strong> to narrow the table by label, run name, input, output, tags, or time range. This is useful when you are comparing a batch of runs or looking for examples to annotate. Use this view to: * Find the latest execution from a project * Compare successful and failed runs * Filter or tag runs for later review * Open a run for manual inspection # SDKs Source: https://docs.sovara-labs.com/sdks/index Record agent runs from Python and TypeScript. <script /> <div> <h1>Sovara SDKs</h1> <p>Record agent runs, model calls, MCP/tool calls, custom steps, and runtime lessons from your code.</p> </div> <div> <a href="/sdks/typescript/quickstart"> <span /> <div>TypeScript</div> </a> <a href="/sdks/python/quickstart"> <span /> <div>Python</div> </a> </div> # API reference Source: https://docs.sovara-labs.com/sdks/python/api-reference The public Sovara Python SDK surface and exact client methods. <script /> The package exports `SovaraClient` and `trace`. Run-scoped helpers are methods on `SovaraClient`; module-level `run`, `subrun`, and logging helpers are not supported. ## `SovaraClient(*, project_name, base_url=None, agent_token=None, http_client=None)` Creates a client bound to one immutable project name. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} from sovara import SovaraClient client = SovaraClient(project_name="finance-agent") ``` | Parameter | Required | Purpose | | -------------- | -------- | ----------------------------------------------------------------------------------------------------------------- | | `project_name` | Yes | Stable project name for every top-level run created by this client. | | `base_url` | No | Exec server URL. Defaults to `SOVARA_EXEC_SERVER_URL` or the local exec server. | | `agent_token` | No | Project-scoped token for an agent host without a signed-in Sovara user. | | `http_client` | No | Caller-owned synchronous `httpx.Client`. Sovara uses it for lifecycle and runtime requests and does not close it. | `client.project_name` is read-only. For a remote agent host, keep the token in the host's secret manager or an excluded `.env` file and pass it explicitly: ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} import os import httpx from sovara import SovaraClient http_client = httpx.Client() # Optional: proxy, custom CA, or other transport settings. client = SovaraClient( project_name="finance-agent", base_url="https://exec.example.com", agent_token=os.environ["SOVARA_AGENT_TOKEN"], http_client=http_client, ) ``` The custom client is optional. Without it, Sovara creates and manages its normal internal HTTP client. ## `client.run(name=None, *, run_key=None, capture_logs=True, lesson_scope=<inherit>)` Creates a top-level run context usable with `with` or `async with`. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} with client.run("sync-agent"): ... async with client.run("async-agent", run_key=chat_id): ... ``` | Argument | Purpose | | --------------- | ---------------------------------------------------------------------------------------------------- | | `name` | Optional display name. Sovara generates one when omitted. | | `run_key` | Optional project-scoped correlation key. Reusing it creates another turn in the same durable run. | | `client_run_id` | Deprecated compatibility alias for `run_key`. | | `capture_logs` | Captures `stdout` and `stderr` by default. Disable for concurrent top-level runs. | | `lesson_scope` | Folder path or list for this run. Omit to inherit; `None` selects root/all; `[]` disables retrieval. | Returns `SovaraRunContext`. ## `client.subrun(name, *, lesson_scope=<inherit>)` Creates a child run under the active run. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} with client.run("batch"): with client.subrun("sample-1"): run_one_sample() ``` `name` is required. Returns `SovaraRunContext`. ## `trace(fn=None, *, name=None, meta=None)` Wraps a sync or async function as a tool-like step. It is the only top-level instrumentation helper. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} from sovara import trace @trace(name="lookup_customer", meta={"system": "crm"}) def lookup_customer(customer_id: str): return crm.get(customer_id) ``` | Argument | Purpose | | -------- | ---------------------------------------------------- | | `fn` | Function to wrap. Omit when using decorator options. | | `name` | Optional step name; defaults to the function name. | | `meta` | Optional step metadata. | The wrapper preserves the call signature, records arguments and return values, records raised exceptions, and re-raises them. ## Lesson methods ### `client.lesson_scope(scope)` Temporarily replaces the active lesson scope. Paths include descendants; `None` means root/all and `[]` disables retrieval. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} with client.lesson_scope(["conventions/", "markets/"]): call_model() ``` ### `client.disable_lesson_injection()` Temporarily prevents automatic lesson retrieval while leaving tracing active. ## `client.log_input(input)` Stores the latest user-visible input string on the active run. This value appears in the Input column of the Runs table and replaces the previously logged input. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara_client.log_input(question) ``` Outside an active run, it logs a warning and does nothing. ## `client.log_output(output)` Stores the latest user-visible output string on the active run. This value appears in the Output column of the Runs table and replaces the previously logged output. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara_client.log_output(answer) ``` Outside an active run, it logs a warning and does nothing. ## `client.log_metrics(**metrics)` Adds filterable custom metrics to the active run. Values must be booleans, integers, or finite floats; keys must be lower snake case and at most 32 characters. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara_client.log_metrics(answered=True, latency_ms=842) ``` Logging the same key again updates its latest value. Outside an active run, it logs a warning and does nothing. ## `client.get_run_id()` Returns the current run or subrun ID as a string, or `None` when called outside an active run. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} run_id = sovara_client.get_run_id() ``` ## Context and tracing controls ### `client.with_context(fn)` Captures the current context and returns a wrapper for another thread. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} executor.submit(client.with_context(eval_sample), sample) ``` ### `client.disable_tracing()` Temporarily disables supported provider, MCP, and explicit `trace` recording. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} with client.disable_tracing(): noisy_or_sensitive_work() ``` # Python SDK quickstart Source: https://docs.sovara-labs.com/sdks/python/quickstart Set up the Sovara Python SDK and record a project-bound agent run. <script /> Sovara records supported LLM, MCP, and explicitly traced tool calls inside an SDK run. Every Python integration starts with one project-bound `SovaraClient`. ## Install Follow [Installation](/get-started/installation) to use an existing Sovara environment or choose a local, remote, or headless setup. Then add the packages used by this example: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} uv add sovara openai-agents ``` Set `OPENAI_API_KEY` in the environment before running the example. ## A complete example This example uses two child agents to compare the weather in Zurich and Boston. Their model and framework tool calls are recorded automatically; `trace` records the custom comparison step. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} from agents import Agent, Runner, function_tool from sovara import SovaraClient, trace # [!code ++] sovara_client = SovaraClient(project_name="weather-agent") # [!code ++] @function_tool def get_zurich_temperature() -> int: """Get the current temperature in Zurich in Celsius.""" return 22 @function_tool def get_boston_temperature() -> int: """Get the current temperature in Boston in Celsius.""" return 27 zurich_agent = Agent( name="Zurich weather agent", instructions="Call the weather tool and return only the temperature.", model="gpt-5.4-mini", tools=[get_zurich_temperature], output_type=int, ) boston_agent = Agent( name="Boston weather agent", instructions="Call the weather tool and return only the temperature.", model="gpt-5.4-mini", tools=[get_boston_temperature], output_type=int, ) @trace(name="choose warmer city") # [!code ++] def choose_warmer_city(zurich: int, boston: int) -> str: if zurich == boston: return f"Boston and Zurich are equally warm at {zurich} C." city = "Zurich" if zurich > boston else "Boston" return f"{city} is warmer ({boston} C in Boston, {zurich} C in Zurich)." def main() -> None: question = "Which city is warmer, Boston or Zurich?" with sovara_client.run("compare city weather"): # [!code ++] # Attach the user-visible request to the top-level run. # [!code ++] sovara_client.log_input(question) # [!code ++] # Each subrun groups one delegated agent's work into an expandable child run. # [!code ++] with sovara_client.subrun("Zurich weather agent"): # [!code ++] zurich = Runner.run_sync(zurich_agent, question).final_output with sovara_client.subrun("Boston weather agent"): # [!code ++] boston = Runner.run_sync(boston_agent, question).final_output answer = choose_warmer_city(zurich, boston) # Attach the final user-visible result to the top-level run. # [!code ++] sovara_client.log_output(answer) # [!code ++] print(answer) if __name__ == "__main__": main() ``` <Note> Sovara records supported framework tools such as `@function_tool` and MCP calls automatically. Do not add `@trace` to those tools as well; reserve it for important operations that are not already captured. </Note> <Note> Subruns are optional, but provide a useful abstraction for grouping child agents or delegated phases into expandable units. </Note> ## Verify one run Save the example as `weather_agent.py`, then run it normally: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} uv run python weather_agent.py ``` Open the `weather-agent` project in Sovara to inspect the run. A run typically corresponds to one chat session or one workflow execution. `log_input()` and `log_output()` populate the Input and Output columns in the Runs table, so you can scan results without opening each run: <img alt="Sovara Runs table showing the logged weather comparison input and output" /> Open the run to see both weather-agent subruns, their LLM and tool calls, and the final `Choose Warmer City` step: <img alt="Sovara trace showing Zurich and Boston weather-agent subruns and the final city comparison" /> ## Next steps * [Use the SDK](/sdks/python/use-the-sdk) for subruns, persistent run IDs, lessons, and metadata. * [API reference](/sdks/python/api-reference) for the exact public surface. * [Troubleshooting](/sdks/python/troubleshooting) for missing runs or steps. # Troubleshooting Source: https://docs.sovara-labs.com/sdks/python/troubleshooting Diagnose missing Python runs, steps, project assignment, and context. <script /> ## Common checks | Symptom | Check | | -------------------------------------------------- | ------------------------------------------------------------------------------------------------ | | No run appears | Confirm the entrypoint enters `sovara_client.run(...)`. Check the terminal for a Sovara warning. | | Run appears in the wrong project | Check the `project_name` passed to `SovaraClient(...)`. | | An LLM or supported framework tool call is missing | Confirm the call executes before the `run(...)` context exits. | | A custom operation is missing | Wrap its shared execution boundary with `trace`. | | Threaded work attaches to the wrong run | Submit `sovara_client.with_context(fn)` to the executor. | | Logs mix between concurrent runs | Set `capture_logs=False` on concurrent top-level runs. | ## Check the installed SDK Confirm the interpreter that runs the agent can import the public API: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} python -c "from sovara import SovaraClient, trace; print('ok')" python -c "import sys; print(sys.executable)" ``` ## Keep recorded work inside the run The run context must contain the real agent task: ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara_client = SovaraClient(project_name="support-agent") with sovara_client.run("answer question"): answer = run_agent(question) ``` For async agent code, use `async with` and await the task before leaving the context. Work moved to another thread needs `sovara_client.with_context(...)` so it retains the active run. ## Trace custom operations Supported provider, framework tool, and MCP calls are recorded automatically. Use `trace` for important application operations that do not pass through one of those integrations, such as retrieval, database access, parsing, or custom tool dispatch: ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} from sovara import trace @trace def retrieve_context(question: str) -> list[str]: return vector_search(question) ``` Prefer one shared dispatch wrapper over many helper decorators. ## Inspect what was recorded ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara probe <run-id> sovara probe <run-id> --step <step-ref> --preview sovara logs <run-id> --tail 40 ``` Use visible step refs from `probe`, not internal UUIDs. # Use the SDK Source: https://docs.sovara-labs.com/sdks/python/use-the-sdk Understand project clients, runs, steps, subruns, metadata, and lessons. <script /> ## The ownership model The Python SDK exposes two top-level building blocks: * `SovaraClient` owns project identity and run-scoped helpers. * `trace` wraps application functions that should appear as steps. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} from sovara import SovaraClient, trace sovara_client = SovaraClient(project_name="support-agent") ``` All run, subrun, logging, lesson, and control operations go through that client. The project is not inferred from the working directory. ## Top-level runs Use one top-level run for one user request, conversation turn, eval sample, batch job, or other execution you want to inspect. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} with sovara_client.run("research-and-answer"): answer = run_agent() ``` The SDK registers the run with the exec server when the context opens and finalizes it when the context exits. The same context works with `with` and `async with`. For a durable conversation or workflow, pass an application-owned correlation ID. Reusing it appends new steps to the same canonical Sovara run. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} with sovara_client.run("support chat", run_key=chat_id): sovara_client.log_input(message) reply = agent.reply(message) sovara_client.log_output(reply) ``` Keep prompts, messages, and secrets out of `run_key`; use a stable ID such as a chat, ticket, or job ID. `client_run_id` remains accepted as a deprecated alias during the compatibility period. The returned `run_id` is Sovara's durable canonical UUID. ## Steps and explicit tracing Inside a run, supported provider and MCP calls become ordered steps with input, output, latency, status, and error data. `trace` adds the same visibility to important application work that automatic instrumentation does not capture. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} @trace def lookup_customer(customer_id: str) -> dict: return crm.lookup(customer_id) ``` Place explicit tracing at shared tool or dispatch chokepoints. A trace filled with miscellaneous helper calls is harder to understand than one that exposes the agent's decisions and meaningful actions. ## Subruns Subruns organize child agents, delegated branches, parallel work, and coherent multi-step phases under the active run. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} with sovara_client.run("finance-eval"): with sovara_client.subrun("sample-42"): run_one_sample("sample-42") ``` Nested top-level `client.run(...)` calls are ignored with a warning. Use `client.subrun(...)` when the child work should appear in the run tree. ## Run metadata Add the user-visible input/output and small scalar metrics from inside a run. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} with sovara_client.run("answer-question"): sovara_client.log_input(question) answer = agent(question) sovara_client.log_output(answer) sovara_client.log_metrics(answered=True, latency_budget_ms=2500) ``` Metric values must be booleans, integers, or finite floats. Keep prompts, responses, lists, dictionaries, and secrets out of metrics. ## Lessons Automatic lesson injection is project-wide by default for supported model calls. Narrow retrieval with `lesson_scope` on a run or subrun: ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} with sovara_client.run("answer-question", lesson_scope="financebench/"): answer = call_model(question) ``` Temporarily replace the active scope inside a smaller block: ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} with sovara_client.lesson_scope(["conventions/", "markets/"]): answer = call_model(question) ``` Sovara retrieves lessons independently for each supported model call and adds a supplementary user message only to the copied request sent to the provider. It does not change the conversation objects owned by your app. ## Controls and concurrency ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} with sovara_client.disable_tracing(): warm_cache_without_recording() with sovara_client.disable_lesson_injection(): answer_without_lessons() ``` Async tasks inherit context. Wrap callables submitted to a thread pool with `sovara_client.with_context(...)`. For concurrent top-level runs, set `capture_logs=False` to avoid mixing process stdout/stderr between runs. ## Run the application Run the application normally. The SDK records the run: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} python agent.py ``` Open Sovara to inspect the recorded run. Use the [API reference](/sdks/python/api-reference) for exact signatures. # API reference Source: https://docs.sovara-labs.com/sdks/typescript/api-reference The public client, run, tracing, subrun, logging, and lesson APIs in @sovara/runner. <script /> ## `new SovaraClient(options)` Creates a project-bound client. Project identity and connection configuration belong to the client, not to individual runs. ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} const sovara_client = new SovaraClient({ projectName: "support-agent" }); ``` | Option | Required | Purpose | | ------------- | -------- | --------------------------------------------------------------------------------------------------------- | | `projectName` | Yes | Stable, immutable project name selected by SDK code. | | `url` | No | Exec server URL; normally resolved from Sovara configuration. | | `agentToken` | No | Project-scoped token for an agent host without a signed-in Sovara user. Defaults to `SOVARA_AGENT_TOKEN`. | | `fetch` | No | Caller-owned `fetch` implementation used for lifecycle and runtime requests. | ## `sovara_client.run(name, fn, options?)` Creates and finalizes a top-level run around a synchronous or asynchronous callback. ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} await sovara_client.run("agent-task", async () => runAgent()); ``` | Option | Purpose | | ------------- | ---------------------------------------------------------------------------------------- | | `runKey` | Project-scoped correlation key. Reusing it creates another turn in the same durable run. | | `clientRunId` | Deprecated compatibility alias for `runKey`. | | `captureLogs` | Captures stdout/stderr unless set to `false`. | | `lessonScope` | Folder path, path list, `null` for root/all, or `[]` to disable retrieval. | Returns `Promise<T>` for the callback result and rethrows callback errors after finalization. For a remote agent host, keep the token in its secret manager or an excluded `.env` file and configure the client once. A custom `fetch` is optional: ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} const sovara_client = new SovaraClient({ projectName: "support-agent", url: "https://exec.example.com", agentToken: process.env.SOVARA_AGENT_TOKEN, fetch: customFetch, // Optional: proxy, custom CA, or other transport behavior. }); await sovara_client.run("agent-task", async () => runAgent()); ``` ## `sovara_client.subrun(name, fn, options?)` Creates a child run under the active run. The optional `lessonScope` replaces the inherited lesson scope for that child. ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} await sovara_client.subrun("research", () => research(question)); ``` Returns `Promise<T>` for the callback result and throws when no run is active. ## `trace(fn, options?)` Wraps a synchronous or asynchronous function as a step, recording arguments, return values, latency, status, and exceptions. ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} const lookupCustomer = trace( async function lookupCustomer(customerId: string) { return crm.get(customerId); }, { name: "lookup_customer", meta: { system: "crm" } }, ); ``` `name` defaults to the function or method name. `meta` adds step metadata. Stage 3 method-decorator use is also supported when the TypeScript compiler is configured for decorators. ## `sovara_client.lessonScope(scope, fn)` Temporarily replaces the active lesson scope. Paths include descendants; `null` or a root marker selects all lessons, and `[]` disables retrieval. ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} await sovara_client.lessonScope(["conventions/", "markets/"], runOneSample); ``` ## `sovara_client.logInput(value)` Stores the latest user-visible input on the active run. This value appears in the Input column of the Runs table and replaces the previously logged input. Objects and arrays are serialized as JSON. ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} await sovara_client.logInput({ question, customerId }); ``` Returns `Promise<void>`. Outside an active run, it warns once and does nothing. ## `sovara_client.logOutput(value)` Stores the latest user-visible output on the active run. This value appears in the Output column of the Runs table and replaces the previously logged output. Objects and arrays are serialized as JSON. ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} await sovara_client.logOutput({ answer, sources }); ``` Returns `Promise<void>`. Outside an active run, it warns once and does nothing. ## `sovara_client.logMetrics(metrics)` Adds filterable custom metrics to the active run. Values must be booleans or finite numbers; keys must be lower snake case and at most 32 characters. ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} await sovara_client.logMetrics({ answered: true, latency_ms: 842 }); ``` Returns `Promise<void>`. Logging the same key again updates its latest value. Outside an active run, it warns once and does nothing. ## `sovara_client.getRunId()` Returns the current run or subrun ID as a string, or `undefined` when called outside an active run. ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} const runId = sovara_client.getRunId(); ``` # TypeScript SDK quickstart Source: https://docs.sovara-labs.com/sdks/typescript/quickstart Set up the Sovara TypeScript SDK and record an agent run. <script /> Sovara records supported LLM, MCP, and framework tool calls inside an SDK run. Use `trace` for important custom operations that are not already captured. ## Install Follow [Installation](/get-started/installation) to use an existing Sovara environment or choose a local, remote, or headless setup. Then add the packages used by this example: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} npm install @sovara/runner @openai/agents zod npm install --save-dev tsx ``` Set `OPENAI_API_KEY` in the environment before running the example. ## A complete example This example uses two child agents to compare the weather in Zurich and Boston. Their model and framework tool calls are recorded automatically; `trace` records the custom comparison step. ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} import { Agent, run, tool } from "@openai/agents"; import { SovaraClient, trace } from "@sovara/runner"; // [!code ++] import { z } from "zod"; const sovara_client = new SovaraClient({ projectName: "weather-agent" }); // [!code ++] const temperature = z.object({ celsius: z.number() }); const getZurichTemperature = tool({ name: "get_zurich_temperature", description: "Get the current temperature in Zurich in Celsius.", parameters: z.object({}), execute: async () => 22, }); const getBostonTemperature = tool({ name: "get_boston_temperature", description: "Get the current temperature in Boston in Celsius.", parameters: z.object({}), execute: async () => 27, }); const zurichAgent = new Agent({ name: "Zurich weather agent", instructions: "Call the weather tool and return the temperature.", model: "gpt-5.4-mini", tools: [getZurichTemperature], outputType: temperature, }); const bostonAgent = new Agent({ name: "Boston weather agent", instructions: "Call the weather tool and return the temperature.", model: "gpt-5.4-mini", tools: [getBostonTemperature], outputType: temperature, }); const chooseWarmerCity = trace( // [!code ++] async function chooseWarmerCity(zurich: number, boston: number): Promise<string> { if (zurich === boston) return `Boston and Zurich are equally warm at ${zurich} C.`; const city = zurich > boston ? "Zurich" : "Boston"; return `${city} is warmer (${boston} C in Boston, ${zurich} C in Zurich).`; }, // [!code ++] { name: "choose warmer city" }, // [!code ++] ); // [!code ++] async function main(): Promise<void> { const question = "Which city is warmer, Boston or Zurich?"; await sovara_client.run("compare city weather", async () => { // [!code ++] // Attach the user-visible request to the top-level run. // [!code ++] await sovara_client.logInput(question); // [!code ++] // Each subrun groups one delegated agent's work into an expandable child run. // [!code ++] const zurich = await sovara_client.subrun("Zurich weather agent", async () => { // [!code ++] const result = await run(zurichAgent, question); return result.finalOutput!.celsius; }); // [!code ++] const boston = await sovara_client.subrun("Boston weather agent", async () => { // [!code ++] const result = await run(bostonAgent, question); return result.finalOutput!.celsius; }); // [!code ++] const answer = await chooseWarmerCity(zurich, boston); // Attach the final user-visible result to the top-level run. // [!code ++] await sovara_client.logOutput(answer); // [!code ++] console.log(answer); }); // [!code ++] } void main(); ``` <Note> Sovara records OpenAI Agents tools created with `tool()` and MCP calls automatically. Do not wrap those tools in `trace` as well; reserve it for important operations that are not already captured. </Note> <Note> Subruns are optional, but provide a useful abstraction for grouping child agents or delegated phases into expandable units. </Note> ## Verify one run Save the example as `weather_agent.ts`, then run it normally: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} npx tsx weather_agent.ts ``` Open the `weather-agent` project in Sovara to inspect the run. A run typically corresponds to one chat session or one workflow execution. `logInput()` and `logOutput()` populate the Input and Output columns in the Runs table, so you can scan results without opening each run: <img alt="Sovara Runs table showing the logged weather comparison input and output" /> Open the run to see both weather-agent subruns, their LLM and tool calls, and the final `Choose Warmer City` step: <img alt="Sovara trace showing Zurich and Boston weather-agent subruns and the final city comparison" /> ## Next steps * [Use the SDK](/sdks/typescript/use-the-sdk) for subruns, persistent run IDs, lessons, and metadata. * [API reference](/sdks/typescript/api-reference) for the exact public surface. * [Troubleshooting](/sdks/typescript/troubleshooting) for missing runs or steps. # Troubleshooting Source: https://docs.sovara-labs.com/sdks/typescript/troubleshooting Diagnose missing TypeScript runs, steps, project assignment, and noisy logs. <script /> ## Common checks | Symptom | Check | | -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | | No run appears | Confirm the entrypoint reaches and awaits `sovara_client.run(...)`. Check the terminal for a Sovara warning. | | Run appears in the wrong project | Check the `projectName` passed to `new SovaraClient(...)`. | | An LLM or supported framework tool call is missing | Confirm the call executes and completes before the `run(...)` callback returns. | | A custom operation is missing | Wrap its shared execution boundary with `trace`. | | Claude Agent SDK calls are missing | Import `query` or `startup` from `@sovara/runner/claude`. | | Logs mix between concurrent runs | Set `captureLogs: false` on concurrent top-level runs. | ## Keep recorded work inside the run The run callback must contain and await the real agent task: ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} const sovara_client = new SovaraClient({ projectName: "support-agent" }); const answer = await sovara_client.run("answer question", async () => { return await runAgent(question); }); ``` Promises started without `await` may continue after the run has closed, so their LLM and tool calls will not belong to that run. Use `sovara_client.subrun(...)` when delegated work should appear as a child run. ## Trace custom operations Supported provider, framework tool, and MCP calls are recorded automatically. Use `trace` for important application operations that do not pass through one of those integrations, such as retrieval, database access, parsing, or custom tool dispatch: ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} const retrieveContext = trace(async function retrieveContext(question: string) { return vectorSearch(question); }); ``` Prefer one shared dispatch wrapper over many helper wrappers. ## Inspect what was recorded ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} sovara probe <run-id> sovara probe <run-id> --step <step-ref> --preview sovara logs <run-id> --tail 40 ``` Use visible step refs from `probe`, not internal UUIDs. # Use the SDK Source: https://docs.sovara-labs.com/sdks/typescript/use-the-sdk Understand TypeScript project identity, runs, steps, subruns, metadata, and lessons. <script /> ## Project-owned top-level runs Create one `SovaraClient` for a stable project name. Use one top-level run for one user request, conversation turn, eval sample, batch job, or other execution you want to inspect. ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} const sovara_client = new SovaraClient({ projectName: "support-agent" }); await sovara_client.run("research-and-answer", () => runAgent()); ``` The client ensures the exec server is reachable, registers the run, executes the callback in an `AsyncLocalStorage` scope, and finalizes the run in `finally`. For a durable conversation or job, pass an application-owned correlation ID. Reusing it within the same project appends to the canonical Sovara run. ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} await sovara_client.run( "support chat", () => agent.reply(message), { runKey: chatId }, ); ``` Keep prompts and secrets out of `runKey`. `clientRunId` remains accepted as a deprecated alias during the compatibility period. The returned `runId` is Sovara's durable canonical UUID. ## Steps and explicit tracing Supported provider, framework tool, and MCP calls inside a run become ordered steps. Wrap an important uncaptured application boundary with `trace`: ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} const lookupCustomer = trace(async function lookupCustomer(customerId: string) { return crm.lookup(customerId); }); ``` Prefer shared tool or dispatch chokepoints. A trace filled with miscellaneous helper calls is harder to understand than one that exposes agent decisions and actions. ## Subruns Use `sovara_client.subrun(...)` for child agents, delegated branches, parallel work, and coherent multi-step phases: ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} await sovara_client.run("finance-eval", () => sovara_client.subrun("sample-42", () => runOneSample("sample-42")), ); ``` Nested top-level runs are ignored with a warning. Use a subrun when child work should appear in the run tree. ## Run metadata ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} await sovara_client.run("answer-question", async () => { await sovara_client.logInput(question); const answer = await agent(question); await sovara_client.logOutput(answer); await sovara_client.logMetrics({ answered: true, latencyBudgetMs: 2500 }); }); ``` Metrics accept booleans, integers, and finite numbers. ## Lessons Automatic lesson injection is project-wide by default for supported model calls. Narrow retrieval with the run's `lessonScope` option: ```ts theme={"theme":{"light":"github-light","dark":"github-dark"}} await sovara_client.run( "answer-question", () => callModel(question), { lessonScope: "support/refunds/" }, ); ``` Temporarily replace the active scope with `sovara_client.lessonScope(...)`. Sovara retrieves lessons independently for each supported model call and adds a supplementary user message only to the copied request sent to the provider. It does not change the conversation objects owned by your app. ## Logs and concurrency Log capture is enabled by default. Pass `{ captureLogs: false }` as the third argument to `run()` for concurrent top-level runs to avoid mixing process output. ## Run the application Run the application normally. The SDK records the run: ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}} npm run start ``` Open Sovara to inspect the recorded run. Use the [API reference](/sdks/typescript/api-reference) for exact signatures. # Improve Source: https://docs.sovara-labs.com/sovara/improve How Sovara turns observations into targeted improvements. Improvement starts after a run has been inspected and the failure mode is clear. Sovara helps you turn that review into a change that is specific enough to fix the problem without making nearby cases worse. The important decision is where the fix belongs. Some failures need code. Some need a better prompt. Some need a new tool. Some come from domain knowledge the agent lacks, where the right fix is a reusable lesson. ## Choose the right fix Use the smallest fix that addresses the actual failure: | Failure pattern | Better fix | | -------------------------------------------------------------------- | ----------------------------- | | The agent could not perform an action | Add or improve a tool | | The agent did not follow instructions that are not context dependent | Update the prompt or workflow | | The agent missed a domain rule or context-dependent instruction | Create a lesson | The goal is not to turn every annotation into a lesson. A lesson is useful when the lesson is reusable, conditional, and likely to matter again. Learn more in [What are lessons?](/sovaradb/lessons). ## Make domain lessons reusable When the failure comes from missing domain knowledge, open <strong><Icon icon="clipboard-check" /> Lessons</strong> and create a lesson. Capture the lesson in a form that is precise, scoped, and retrieval-friendly. Click <strong><Icon icon="plus" /> New Lesson</strong>, write the rule, and use <strong><Icon icon="sparkles" /> Suggest</strong> to draft the **Use this when** field. Before relying on the lesson, review it with <strong><Icon icon="shield-alert" /> Review</strong>. The detailed workflow is covered in [Creating and managing lessons](/sovaradb/creating-managing-lessons). ## Validate the change After applying a fix, rerun representative examples and inspect the new traces. The question is not only whether the original failure disappeared. Check whether the fix changed related behavior in a way you did not intend. Use [Runs](/observability/runs) to compare executions, [Manual inspection](/observability/manual-inspection) to inspect the changed steps, and [Run chat](/observability/run-chat) to get a fast read on the new run. For failures that should become regression tests, add repeatable coverage to the test suite that exercises the affected workflow. ## Close the loop Once a fix works, keep the annotation history. It tells Sovara which behavior is already covered and helps the recommendation algorithm avoid wasting reviewer time on examples that no longer teach anything new. If similar failures keep appearing, the fix is probably too narrow, not being retrieved at the right time, or applied at the wrong layer. In that case, go back to the reviewed runs and refine the prompt, tool, lesson, or test until the pattern is covered. # Observe Source: https://docs.sovara-labs.com/sovara/observe How Sovara observes agent behavior during development. Observation is the foundation of Sovara's workflow. Before improving an agent, look at a real run. Sovara keeps the run steps and supporting context together so the execution can be inspected after the fact. Sovara treats each agent execution as a run. A run preserves the structure of the agent's work, from the top-level task down to the step where behavior changed. The practical docs live in [Runs](/observability/runs), [Manual inspection](/observability/manual-inspection), and [Run chat](/observability/run-chat). # Philosophy Source: https://docs.sovara-labs.com/sovara/philosophy The principles behind Sovara’s approach to improving agents. Agentic systems improve through a disciplined loop: observe real runs, find the mistakes that matter, and apply the right fix. Sovara keeps that loop grounded in recorded behavior instead of guesswork. The workflow is intentionally simple. Traces show the run, annotations focus review, and lessons turn recurring domain knowledge into runtime context. <div> <img alt="Sovara workflow from observing traces to surfacing errors and improving agents" /> </div> <Steps> <Step title="Observe"> Start from real executions. Sovara keeps the run steps and runtime context together so the team can inspect the behavior that actually occurred. The concrete observability views are covered in [Runs](/observability/runs), [Manual inspection](/observability/manual-inspection), and [Run chat](/observability/run-chat). </Step> <Step title="Surface errors"> Not every run deserves manual review. Sovara prioritizes the failures, regressions, confusing outputs, and behavioral gaps most likely to teach the team something. Review surfaced runs in [Annotations](/annotations/overview). To learn how Sovara chooses which runs deserve attention, read about our [Recommendation Algorithm](/annotations/recommendation-algorithm). </Step> <Step title="Improve"> Once you understand the error, apply the right kind of fix. Some issues need a better prompt or a new tool. Others come from missing domain knowledge, where a reusable lesson is the better fix. To understand that pattern, start with [What are lessons?](/sovaradb/lessons). Then learn how to [create and manage lessons](/sovaradb/creating-managing-lessons), and how Sovara performs [auto-injection](/sovaradb/auto-injection) at runtime. </Step> </Steps> # Surface errors Source: https://docs.sovara-labs.com/sovara/surface-errors How Sovara helps expose failures and weak execution paths. Observation gives you the raw run history. The next problem is deciding where to spend attention. Most agent teams cannot review every trace. Many runs repeat behavior already covered by earlier annotations. Many failures are obvious. The valuable cases are the ones that reveal a missing capability, a weak instruction, or domain knowledge the agent lacks. Sovara's annotation workflow is built for that middle step. It surfaces runs worth reviewing, explains why they may matter, and keeps samples that are already covered by previously annotated runs out of the way. To see how we're doing that, read up on our [Recommendation Algorithm](/annotations/recommendation-algorithm). For more details, see [Why annotate?](/annotations/overview) # Sovara's auto-injection Source: https://docs.sovara-labs.com/sovaradb/auto-injection How Sovara injects relevant lessons at runtime. Auto-injection is how lessons become useful during a run. When your agent makes a model call inside a Sovara run, Sovara can retrieve relevant lessons and append them as a supplementary user message to the copied provider request. The default path is automatic. You create lessons once, keep the desktop app running, and Sovara handles retrieval and context placement during future runs. ## Why auto-injection is useful The useful lesson is often not known at the start of a run. A financial-analysis agent might begin with a broad question, retrieve filings, identify a liquidity subtask, and only then need the quick-ratio lesson. Auto-injection lets Sovara consider lessons close to the step where they matter. That keeps the guidance timely and avoids forcing the agent to carry every lesson from the beginning. <img alt="Sovara run view showing lessons injected into a model call" /> <img alt="Sovara run view showing lessons injected into a model call" /> ## Why not inject everything at the top? Top-loading all guidance is tempting, but it breaks down quickly: * The relevant context can change during a run * Early guidance can be stale by the time the agent reaches a later step * Large prompt blocks dilute attention * Unrelated lessons can push the agent toward the wrong behavior Sovara retrieves lessons at runtime so the injected context can match the current step instead of only the original user request. ## Why not always inject? Even relevant lessons have a cost. Every injected token competes with task context, retrieved evidence, tool output, and the model's own reasoning budget. Sovara performs a fresh retrieval for every eligible model step and injects only the lessons selected for that step. The message uses the provider's native user message format and is added only to the copied request sent to the model. It is not added to your application's conversation history, so lesson text does not accumulate across later calls. Sovara stores the executed request for inspection and annotation. The run UI removes the supplementary message from the displayed input and shows the applied lessons separately. ## Why not let the LLM decide? An LLM can help reason about context, but it is not the right control point for every injection decision. The model may not know which lesson is needed until after it has already missed the lesson. Sovara keeps more control by evaluating possible injection at the runtime step. That gives the system a chance to apply the right domain lesson before the model answers. ## Configure injection Open <strong><Icon icon="settings" /> Settings</strong> and go to the project's lesson injection settings. * Turn off **Enable automatic lesson injection** when a project should run without runtime lessons. * Choose a **Retrieval priority**: * **Latency** uses embedding search and reranking for the fastest selection. * **Balanced** adds an LLM selection pass, using embedding search first for large lesson sets. * **Accuracy** (default) gives the full allowed lesson tree to the configured Helper Model at medium reasoning effort, without an embedding or reranking shortlist. ## Skip lesson injection Use `disable_lesson_injection()` around Python code that should be traced but should not receive automatic lessons. ```python theme={"theme":{"light":"github-light","dark":"github-dark"}} from sovara import SovaraClient sovara_client = SovaraClient(project_name="support-agent") with sovara_client.run("answer-question"): with sovara_client.disable_lesson_injection(): answer = call_model(question) ``` # The context-problem of AI agents Source: https://docs.sovara-labs.com/sovaradb/context-problem-ai-agents Why agent context needs durable memory and retrieval. Agents often fail because they are missing the domain knowledge needed to make the right judgment. More model capacity helps, but it does not guarantee that the agent has the specific operating rule, company policy, or expert heuristic that the task requires. This knowledge usually appears during review. A reviewer notices the agent made the wrong call, explains the correction, and then needs that correction to matter the next time a similar situation appears. ## A financial-analysis example Imagine an agent that answers questions about company liquidity. It can retrieve the current assets, current liabilities, and quick ratio. It can even calculate the ratio correctly. The failure is in the judgment layer. A quick ratio below `1.0` usually deserves caution, but the agent may still call the company liquid because management used positive language elsewhere in the filing. The lesson is a domain rule: > When assessing liquidity from a quick ratio below `1.0`, do not call the > company healthy without checking whether other evidence offsets the short-term > liability risk. The agent could rediscover that rule by reading many filings, comparing ratios, and reasoning through the domain from scratch. That is slow. It is also not reliable enough when the same class of mistake can reappear in production. ## Why normal context is not enough Putting all known guidance at the top of every prompt does not scale. The agent will ignore some of it, the context window gets crowded, and irrelevant rules can distract from the rules that matter now. Leaving the guidance out is also risky. If the agent does not receive the liquidity lesson when the next liquidity question appears, it can repeat the same mistake. This is the gap Lessons handles. Sovara stores durable domain knowledge and makes it available as targeted runtime context. <img alt="Sovara run view showing the trace that revealed missing domain knowledge" /> <img alt="Sovara run view showing the trace that revealed missing domain knowledge" /> Next, learn what Sovara stores as runtime context: [lessons](/sovaradb/lessons). # Creating and managing lessons Source: https://docs.sovara-labs.com/sovaradb/creating-managing-lessons Create, organize, and maintain Sovara lessons. The Lessons page is where reviewed domain knowledge becomes reusable context. The goal is not to collect every observation. The goal is to keep a small, high-quality set of lessons that can be retrieved when they matter. Open <strong><Icon icon="clipboard-check" /> Lessons</strong> from the project sidebar. <img alt="Sovara Lessons editor for creating a lesson" /> ## Create a lesson Click <strong><Icon icon="plus" /> New Lesson</strong> to create a lesson, or <strong><Icon icon="folder-plus" /> New Folder</strong> to organize lessons by domain, workflow, or product area. Fill in the lesson: * **Title**: short and readable * **Use this when**: the situations where retrieval should find this lesson * **Content**: the rule the agent should follow The **Use this when** field matters. It is the retrieval-facing description of the lesson. If it is too vague, Sovara may miss the lesson. If it is too broad, the lesson may show up when it should not. Use <strong><Icon icon="sparkles" /> Suggest</strong> to draft the **Use this when** field from the lesson content, then edit it until it describes the situations where the lesson should be retrieved. <img alt="Sovara lesson editor with title and Use this when fields" /> <img alt="Sovara lesson editor with title and Use this when fields" /> ## Review quality Click <strong><Icon icon="shield-alert" /> Review</strong> before turning an important draft into an active lesson. Sovara checks whether the lesson is suitable for reuse. The checks focus on practical quality: * **Actionable**: it tells the agent what to do differently * **Concise**: it preserves the useful rule without unnecessary narrative * **Self-contained**: it includes the conditions needed to apply the rule * **Precise**: it avoids vague advice * **Well-scoped**: it generalizes without causing mistakes in nearby cases * **Faithful**: it does not invent facts or constraints * **Retrieval-friendly**: it uses the terms that should trigger retrieval * **Non-conflicting**: it does not contradict existing lessons Use <strong><Icon icon="sparkles" /> Polish</strong> when a draft has the right lesson but needs cleaner wording. When the draft is ready, click <strong><Icon icon="check" /> Save</strong>. <img alt="Sovara lesson review panel with quality feedback" /> <img alt="Sovara lesson review panel with quality feedback" /> ## Avoid duplicates and conflicts Two lessons can be individually reasonable and still make the system worse together. Sovara checks for overlap and conflict so the lesson library does not become a pile of competing instructions. When a new lesson overlaps with an existing lesson, prefer one of these outcomes: * Update the existing lesson if it already owns the rule * Split the lesson if two different situations are being mixed together * Narrow the **Use this when** field if the rule is firing too broadly * Delete or archive the weaker lesson if it only repeats another one The goal is a library that stays useful as it grows. ## Preserve provenance When a lesson comes from a concrete run, keep the link to that run. Provenance answers the question: "Why does this lesson exist?" That link is important during review. It lets someone inspect the original failure, confirm the lesson, and decide whether the lesson still applies after the agent changes. ## Balance recall, precision, and latency Lesson retrieval has three competing goals: * **Recall**: missing a relevant lesson can mean repeating a known mistake * **Precision**: injecting too many lessons dilutes attention * **Latency**: retrieval should add only a small overhead to the agent run Sovara is designed around that tradeoff. Recall is paramount because a missed lesson can recreate the same failure. Precision still matters because context is not free: irrelevant lessons make the prompt longer and can distract the agent from the rule it actually needs. The target is low overhead, ideally below `10%` of the run's normal latency. That is why Sovara uses retrieval-aware lesson generation, compact lesson content, and project-level latency controls instead of injecting the whole lesson library. Next, see how Sovara applies lessons automatically at runtime in [Sovara's auto-injection](/sovaradb/auto-injection). # What are lessons? Source: https://docs.sovara-labs.com/sovaradb/lessons Define lessons and how they shape agent behavior. A lesson is reusable guidance that should influence future agent behavior. In Sovara, lessons are stored in the lesson library and retrieved when they are relevant to a new run. Good lessons are not long notes, raw traces, or generic reminders. They are compact pieces of domain knowledge with a clear trigger. ## Anatomy of a lesson A lesson has three important parts: * **Title**: a short name for the lesson * **Use this when**: the retrieval trigger that describes when the lesson should apply * **Content**: the instruction or domain rule the agent should follow For example: ```md theme={"theme":{"light":"github-light","dark":"github-dark"}} Title: Quick ratio below 1.0 needs liquidity-context review Use this when: The task asks whether a company has healthy liquidity based on current assets, current liabilities, or quick ratio. Content: When a quick ratio is below 1.0, do not conclude that liquidity is healthy from positive management language alone. Check whether the company has offsetting evidence such as strong operating cash flow, available credit facilities, or temporary working-capital effects. If no offsetting evidence is present, treat the sub-1.0 ratio as a liquidity concern. ``` That lesson is specific enough to retrieve for liquidity analysis, but it does not overstep. It does not say every company with a quick ratio below `1.0` is in trouble. It tells the agent what extra judgment is required. ## Lessons are behavioral memory Lessons are most useful when they capture reviewed knowledge: * A failure the team wants to prevent * A domain distinction the agent did not know * A policy that should be applied consistently * A recurring edge case that should not be rediscovered from scratch They turn review work into reusable context. Instead of relying on the agent to learn the same lesson again, Sovara can retrieve the lesson when the next relevant step appears. <img alt="Sovara lesson editor showing title and retrieval guidance fields" /> <img alt="Sovara lesson editor showing title and retrieval guidance fields" /> Next, see how to create and maintain lessons in [Creating and managing lessons](/sovaradb/creating-managing-lessons).