> ## Documentation Index
> Fetch the complete documentation index at: https://arka-agent-mintlify-f21cea86.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM orchestration with multi-provider failover

> How Arka orchestrates 24 LLM providers with automatic failover, API key rotation, per-task profiles, and per-skill model overrides.

Arka's `LlmFallbackEngine` manages all LLM calls across **25 providers** with automatic failover, API key rotation, and per-task or per-skill model overrides.

## Supported providers

Anthropic, OpenAI, Gemini, Groq, xAI, DeepSeek, Moonshot, Z.AI, MiniMax, Venice, Mistral, Cohere, Together, Fireworks, Perplexity, Hugging Face, OpenRouter, Sakana Fugu, Bedrock, Azure, LiteLLM, Ollama, LM Studio, vLLM, and vLLM Cloud.

## How failover works

The orchestrator builds a candidate list in order:

1. `AI_PREFERRED_PROVIDER` + `AI_PREFERRED_MODEL` (if set)
2. Live model lists from each provider API (when keys allow)
3. Built-in chain: Gemini 2.5/2.0 Flash → Groq Llama 3.3/3.1 → Ollama cloud/local

On 429, 401, or timeout, Arka tries the next model. API key rotation (`API_KEY_ROTATION=1`) cycles backup keys before switching providers.

```env theme={null}
AI_PREFERRED_PROVIDER=gemini
AI_PREFERRED_MODEL=gemini-2.0-flash
AI_KEEP_MODEL_ON_PROVIDER_SWITCH=1
LLM_AUTO_FALLBACK=1
API_KEY_ROTATION=1
```

When you switch providers with `arka provider set groq` (or `ai-pref groq` in fish), Arka tries to **keep the same model** on the new provider before falling back to that provider's default. To disable this, use `arka provider set groq --no-keep-model` or `AI_KEEP_MODEL_ON_PROVIDER_SWITCH=0`.

## Web chat model order

Interactive web chat, such as the [Open WebUI frontend](/guides/open-webui), uses a separate fast chain so replies keep coming when cloud free tiers hit rate limits:

1. A small local Ollama model, when Ollama is running. Arka picks the first installed model from `llama3.2:1b`, `llama3.2:3b`, `qwen3:8b`, and `llama3.1:8b`.
2. Gemini Flash models, when `GEMINI_API_KEY` or `GOOGLE_API_KEY` is set.
3. Groq `llama-3.3-70b-versatile`, when `GROQ_API_KEY` is set.

Override the chain or the local model:

```env theme={null}
WEB_CHAT_OLLAMA_MODEL=llama3.2:3b
WEB_CHAT_MODEL_CHAIN=ollama/qwen3:8b,groq/llama-3.3-70b-versatile
```

Set `WEB_CHAT_FAST=0` to use the standard failover chain instead.

<Note>
  Short acknowledgments such as "thanks" or "got it" get a brief welcome. Arka doesn't continue the previous build.
</Note>

## Per-task profiles

| Task | Used for |
| - | - |
| `route` | Skill routing decisions |
| `summarize` | Email and document summaries |
| `chat` | Conversational replies |
| `research` | Deep web research |
| `agent` | Goal agent steps |
| `pdf` | PDF question answering |
| `arka predictions` | Stock and forecast analysis |
| `compose_video` | Video script generation |
| `arka compose_slides` | Presentation slide script generation |

Override per task:

```env theme={null}
LLM_FALLBACK_ROUTE=gemini/gemini-2.0-flash,groq/llama-3.3-70b-versatile
LLM_FALLBACK_SUMMARIZE=groq/llama-3.3-70b-versatile
```

## Per-skill models

```bash theme={null}
arka ai-skill-model web_answer groq/llama-3.3-70b-versatile
arka ai-skill-model profiles
arka ai-models
```

Or edit `~/.config/arka/llm-skill-models.json`.

## Benchmark orchestration

If you want model selection to follow live benchmark winners, run a benchmark suite first and then enable orchestration:

```bash theme={null}
arka benchmark run
arka benchmark show
ARKA_BENCHMARK_ORCHESTRATE=1 arka orchestrate --benchmark "how to get free ai credits"
```

The benchmark profile follows the routed task. A request that routes to `web_answer` uses the `chat` benchmark results. Agent-style requests use `agent`.

## Shared exhaustion cache

Arka keeps one session cache. It skips exhausted models across skills until you reset the cache:

```bash theme={null}
arka ai reset-exhaustion
```

Enable stderr notification on failover: `LLM_FALLBACK_NOTIFY=1`. Debug attempts: `LLM_VERBOSE=1`.

## Local LLM servers

```env theme={null}
OLLAMA_HOST=127.0.0.1:11434
LLM_AUTO_START_SERVERS=1
LLM_AUTO_STOP_SERVERS=1
```

## Why skills stay concise

Arka does **not** ask the LLM to memorize 70 skill manuals. Skills are local programs. The model only picks the right name. On LLM route, Arka passes:

* A **compact skill catalog** (registered names only)
* **Curated routing rules** (high-signal patterns)
* **Shell aliases** via `ROUTE_ALIASES` for your machine

The route task uses low temperature and a dedicated chain so a fast, cheap model handles classification. Heavy lifting runs locally inside the skill.

## Inspect at runtime

```bash theme={null}
arka ai-models
arka ai models --task summarize
arka ai models --skill web_answer
arka ai active-model
arka ai reset-exhaustion
```

<Note>
  Set `SHOW_MODEL=1` (default) to display `Model: provider/name` under answers.
</Note>

## Related

<CardGroup cols={2} />


## Related topics

- [Arka — AI terminal agent documentation](/index.md)
- [Quickstart: install Arka and run your first command](/quickstart.md)
- [How Arka compares to other agents](/concepts/comparison.md)
- [Symbolic and LLM routing pipeline](/concepts/routing.md)
- [Sakana Fugu orchestrator](/guides/fugu.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.