Models & providers

How model routing works#

A workflow has a default model, and each agent in the crew may override it. At run time LoopLlama looks at the model id, picks the provider that serves it, makes the call with the platform's pooled credentials, and records the provider's reported token usage and its cost on the step. Tool calling, human-in-the-loop pauses and checkpoints work the same way on every provider, so you can mix models freely within one run — a cheap model to triage, a frontier model to decide, a fast one to format.

• Bare ids like claude-sonnet-4-6, gpt-5.4 or gemini-2.5-flash are routed by their naming convention.
• Prefixed ids like openai/gpt-5.4 make the provider explicit — required for models that aren't in the catalog yet, or for a custom endpoint (custom/<model>). An unlisted model is billed at its provider's highest listed rate until it's added to the catalog with its real price (unlisted Claude models are priced by family), so it can never be under-billed.
• Model ids that can't be routed are rejected with 400 Bad Request when you create or update a workflow, so a typo never reaches a run.

{
  "name": "Support triage",
  "model": "gemini-2.5-flash",
  "crew": [
    { "role": "classifier", "systemPrompt": "Label the ticket: billing, bug, or how-to." },
    { "role": "investigator", "model": "claude-sonnet-4-6",
      "systemPrompt": "Pull the relevant account facts and draft a reply." },
    { "role": "reviewer", "model": "openai/gpt-5.4",
      "systemPrompt": "Check the draft for accuracy and tone; output the final reply." }
  ]
}

Web-grounded models (Perplexity Sonar)

Perplexity's Sonar models are available as pipeline steps and behave like Perplexity's answer engine: every call searches the live web and the answer comes back with numbered citations, which LoopLlama appends to the step output as a Sources list. Use sonar for quick cited answers, sonar-pro for thorough multi-angle research, sonar-reasoning-pro for verification and analysis, and sonar-deep-research for long-form reports. The Agent Hub ships ready-made Web Answer, Pro Search, Deep Research Report, Fact Checker, Market & Competitor Monitor and Finance Brief agents built on them.

Your own fine-tuned models

GPT-4.1, GPT-4.1 mini, Mistral Small and Codestral can be fine-tuned on your own examples — including the steps your workflows have already produced — from Dashboard → Model training or POST /v1/fine-tunes. A finished job's ft:… model id is routable like any catalog id in the account that trained it, billed at the base model's fine-tuned inference rate; training tokens are billed once at the provider's training rate. See the model training guide.

Discovering available models#

The dashboard's model pickers list every model whose provider is configured on your LoopLlama deployment, grouped by provider with its rate. Programmatically, the same list is available to logged-in users at GET /api/models:

{
  "models": [
    {
      "id": "claude-sonnet-4-6",
      "label": "Claude Sonnet 4.6",
      "provider": "anthropic",
      "providerLabel": "Anthropic",
      "inPer1M": 3,
      "outPer1M": 15,
      "context": 1000000,
      "tools": true,
      "available": true
    }
  ],
  "providers": [{ "id": "anthropic", "label": "Anthropic", "configured": true }],
  "source": "live",
  "pricingAsOf": "2026-10-02"
}

Billing: at cost, no markup#

Your plan pays for the platform — orchestration, traces, connectors, approvals, scheduling — and is metered in workflow steps. Model usage is a separate, pass-through line: for every step we record the provider's reported input, output and cached tokens and price them at that provider's published list rate. The sum appears as cost_usd on each step and run, in your usage dashboard, and on your invoice at exactly that amount.

Model catalog & list prices#

Providers#

• Anthropic — Claude models via the official Anthropic SDK.
• OpenAI — GPT models via the Chat Completions API.
• Google — Gemini models via the Gemini API's OpenAI-compatible endpoint.
• xAI — Grok models.
• DeepSeek — DeepSeek V4 models.
• Mistral — Mistral models via La Plateforme.
• Perplexity — Sonar models search the live web on every call and return cited sources — Perplexity's answer engine as a pipeline step. Token rates plus a per-request search fee (medium search context), passed through at cost. No tool calling.
• Custom endpoint — Any OpenAI-compatible server (vLLM, Ollama, OpenRouter, Together, Groq…). Models are registered with EXTRA_MODELS.

Self-hosting: enabling providers

On your own deployment, a provider is enabled by setting its API key (and, for a custom OpenAI-compatible server, its base URL). Models from unconfigured providers are hidden from pickers, and a run that references one fails fast with a message naming the missing variable. With no provider configured at all, runs complete with a deterministic stub so you can test the integration first.

ANTHROPIC_API_KEY=sk-ant-…
OPENAI_API_KEY=sk-…
GOOGLE_API_KEY=…
XAI_API_KEY=…
DEEPSEEK_API_KEY=…
MISTRAL_API_KEY=…
PERPLEXITY_API_KEY=pplx-…

# Any OpenAI-compatible server (vLLM, Ollama, OpenRouter, Together, Groq…)
OPENAI_COMPAT_BASE_URL=http://localhost:11434/v1
OPENAI_COMPAT_API_KEY=
EXTRA_MODELS='[{"id":"llama-3.3-70b","label":"Llama 3.3 70B","in":0.6,"out":0.6}]'

# Correct a list price without a deploy (USD per 1M tokens)
MODEL_PRICING_OVERRIDES='{"gpt-5.4":{"in":2.5,"out":15}}'

Rates are checked against provider pricing pages regularly; see Pricing for how model usage combines with your plan.