Supported model providers

When an SDK captures a model request over HTTP, the Sovara server works out which API format and provider it uses. The format decides how the request and response are shown and whether lessons can be inserted; the provider decides the label on the step and which API key Rerun uses.

API formats

Format	Requests	Lesson insertion
OpenAI Chat Completions	`…/chat/completions`	Yes
OpenAI Responses	`…/responses`	Yes
Anthropic Messages	`…/v1/messages`	Yes
Gemini `generateContent` / `streamGenerateContent`	`…/models/{model}:generateContent`	Yes
Amazon Bedrock Converse	`…/model/{model}/converse`	Yes
Ollama chat	`…/api/chat`	Yes

Lessons are inserted as one temporary user message at the end of the conversation (before a trailing assistant prefill). System and developer instructions, tool calls and results, and reasoning fields the provider needs back are left untouched. Requests in other formats are still recorded, without lessons.

Providers and gateways

Provider	Recognized from	Formats	Extra fields kept
OpenAI	`api.openai.com`	Chat Completions, Responses
Anthropic	`api.anthropic.com`	Messages	thinking blocks
Google Gemini	`generativelanguage.googleapis.com`	`generateContent`	thought parts, signatures
Vertex AI	`*aiplatform.googleapis.com`	`generateContent`	thought parts, signatures
OpenRouter	`openrouter.ai`	Chat Completions, Responses	`reasoning`, `reasoning_details`
DeepSeek	`api.deepseek.com`	Chat Completions	`reasoning_content`
Z.AI (GLM)	`api.z.ai`, `open.bigmodel.cn`	Chat Completions	`reasoning_content`
Moonshot AI (Kimi)	`api.moonshot.ai`, `api.moonshot.cn`	Chat Completions	`reasoning_content`
xAI	`api.x.ai`	Chat Completions, Responses
Together AI	`api.together.xyz`, `api.together.ai`	Chat Completions
Amazon Bedrock	`*.amazonaws.com`	Converse
LiteLLM proxy	Configuration	Chat Completions, Responses	`reasoning_content`, `thinking_blocks`
vLLM	Configuration	Chat Completions, Responses	`reasoning`

A captured request to any other host in a supported format also gets lessons; its provider is taken from the model name when possible. Streamed reasoning is reassembled into one field per message.

Configure gateways and self-hosted servers

Set SOVARA_HTTP_ENDPOINTS on the Sovara servers to a JSON list. The longest matching url_prefix applies:

[
  {"url_prefix": "https://llm.internal.example/vllm", "provider": "vllm", "lessons": false},
  {"url_prefix": "https://gateway.example/llm", "provider": "litellm", "format": "openai_chat"}
]

Field	Meaning
`url_prefix`	Required. Scheme, host, optional port and path prefix.
`provider`	Provider name for labels and Rerun keys, e.g. `litellm`, `vllm`, `openrouter`.
`format`	Format for paths that don’t identify one: `openai_chat`, `openai_responses`, `anthropic_messages`, `gemini_generate_content`, `bedrock_converse`, `ollama_chat`.
`lessons`	`false` records the endpoint without inserting lessons. Default `true`.

These settings apply only to requests the SDKs capture, which are matched by path (for example /v1/chat/completions or /api/chat). A gateway path the SDKs don’t capture is not recorded.

Some vLLM chat templates require user and assistant messages to alternate and reject a request with two user messages in a row. If lessons cause such errors, set "lessons": false for that endpoint.