CrabTalkProviders
Supported LLM providers, translation types, and endpoint support matrix.
A provider is an LLM service that CrabLLM routes requests to. Each provider has its own API format and authentication mechanism. CrabLLM translates between the OpenAI-compatible format your application uses and the provider's native format.
Supported providers
| Kind | Provider | Translation |
|---|---|---|
openai | OpenAI, Groq, Together, vLLM, any OpenAI-compatible API | Pass-through |
anthropic | Anthropic Messages API | Full translation |
google | Google Gemini | Full translation |
azure | Azure OpenAI | URL + auth rewrite |
bedrock | AWS Bedrock Converse API | Full translation + SigV4 signing |
ollama | Ollama (local models) | Pass-through (OpenAI-compatible) |
Common fields
Every provider supports these fields:
[providers.name]
kind = "..." # required
api_key = "..." # API key (supports ${ENV_VAR})
base_url = "..." # base URL override
models = ["..."] # model names this provider serves
weight = 1 # routing weight (higher = more traffic)
max_retries = 2 # retries on transient errors (429, 5xx)
timeout = 30 # per-request timeout in secondsMultiple providers for the same model
When multiple providers list the same model, CrabLLM selects between them using weighted random selection. If the selected provider fails, it falls back to the next provider by weight. See Routing.
[providers.openai_primary]
kind = "openai"
api_key = "${OPENAI_KEY_1}"
models = ["gpt-4o"]
weight = 3
[providers.openai_backup]
kind = "openai"
api_key = "${OPENAI_KEY_2}"
models = ["gpt-4o"]
weight = 1Endpoint support
| Endpoint | OpenAI | Anthropic | Azure | Bedrock | Ollama | |
|---|---|---|---|---|---|---|
| Chat completions | yes | yes | yes | yes | yes | yes |
| Streaming | yes | yes | yes | yes | yes | yes |
| Embeddings | yes | — | — | yes | — | yes |
| Image generation | yes | — | — | yes | — | — |
| Audio speech | yes | — | — | yes | — | — |
| Audio transcription | yes | — | — | yes | — | — |
| Tool/function calling | yes | yes | yes | yes | yes | yes |