# Models and providers A harness names the model it executes with in two fields: ```yaml model: provider: openai # a registry name id: gpt-4.1-mini # a model id that provider serves ``` `provider` is a registry name, not a hard-coded literal. hiveloom ships builtin providers for hosted services, routing aggregators, and local servers. Hosted runs need credentials; local runs need a reachable server. Inspect the registry for this installed version before choosing a model. Run `hiveloom models` to see what is registered, which environment variable each provider reads, and whether that variable is currently set: ```console $ hiveloom models provider key env key catalog models claude ANTHROPIC_API_KEY set fixed claude-haiku-4-5, claude-sonnet-5, … openai OPENAI_API_KEY unset open gpt-6.1-sol, gpt-6-astra, gpt-6-luna, … ollama - n/a open - ``` `hiveloom models ` narrows to one provider and prints its endpoint and per-model pricing. Both forms accept `--json`. Neither ever prints a key value — only whether one is present. ## Builtin providers | `provider` | Lab / service | Key variable | Endpoint | |---|---|---|---| | `claude` | Anthropic | `ANTHROPIC_API_KEY` | native SDK | | `openai` | OpenAI | `OPENAI_API_KEY` | `https://api.openai.com/v1` | | `gemini` | Google Gemini | `GEMINI_API_KEY` | `https://generativelanguage.googleapis.com/v1beta/openai` | | `mistral` | Mistral | `MISTRAL_API_KEY` | `https://api.mistral.ai/v1` | | `deepseek` | DeepSeek | `DEEPSEEK_API_KEY` | `https://api.deepseek.com/v1` | | `xai` | xAI Grok | `XAI_API_KEY` | `https://api.x.ai/v1` | | `moonshot` | Moonshot AI | `MOONSHOT_API_KEY` | `https://api.moonshot.ai/v1` | | `groq` | Groq | `GROQ_API_KEY` | `https://api.groq.com/openai/v1` | | `openrouter` | OpenRouter | `OPENROUTER_API_KEY` | `https://openrouter.ai/api/v1` | | `together` | Together AI | `TOGETHER_API_KEY` | `https://api.together.xyz/v1` | | `fireworks` | Fireworks AI | `FIREWORKS_API_KEY` | `https://api.fireworks.ai/inference/v1` | | `ollama` | Ollama (local) | none | `http://localhost:11434/v1` | | `vllm` | vLLM (local) | none | `http://localhost:8000/v1` | ### Switching a harness to another lab Use the `provider/model-id` selector — the same syntax `generate --model` and `evolve --model` accept: ```console $ hiveloom set model openai/gpt-4.1-mini --dir ./summarizer set model openai/gpt-4.1-mini ``` Do **not** set `model.provider` and `model.id` separately. They validate against each other, so whichever you write first leaves the spec inconsistent and the edit is rolled back — `set model` is the only ordering that works. The selector splits on the first `/` only, so aggregator ids keep their own slashes: `openrouter/deepseek/deepseek-r1`. `claude` uses the Anthropic SDK. Every other builtin is the stdlib-only `OpenAICompatProvider` speaking `/chat/completions`, so any other server with that API works too — LM Studio, `mlx_lm.server`, a corporate gateway — either by pointing `vllm`/`ollama` at it or by declaring your own entry (below). The key is read from the process environment or the harness's own `.env`, so a harness folder stays self-contained: ```console $ echo 'OPENAI_API_KEY=sk-...' >> my-harness/.env $ hiveloom run my-harness --input-text "…" ``` For a one-run comparison, override the runtime config instead of mutating the harness: ```console $ hiveloom run my-harness --input-file case.txt \ --provider openrouter --model qwen3.5-9b --json ``` The override is validated and included in the run's version hash, but `harness.yaml` remains byte-identical. ## Workbench providers and model offers The workbench's **Settings → Models** adds providers, selects offered models, and stores a default. Custom OpenAI-compatible endpoints are written to `$HIVELOOM_HOME/providers.yaml`; it has the same declaration format as `models.yaml`. The runtime loads `providers.yaml` first and `models.yaml` second, so hand-maintained overrides keep precedence. The workbench reloads these declarations without resetting extension-pack registrations. Keep hand-written entries in `models.yaml`: `providers.yaml` is rewritten by the UI. A provider `label` supplies its display name. The UI saves credentials in `$HIVELOOM_HOME/.env` with owner-only permissions and adopts them into the workbench process. Separate CLI processes still need an exported key or a harness-local `.env`; they do not automatically read this workbench key file. `ModelInfo` carries `legacy` and `restricted` flags. These affect default UI offers, not whether a registered ID validates. Existing specs can keep their models. YAML flags inherit builtin metadata when omitted, so repricing a legacy ID does not silently offer it again. An explicit `legacy: false` or `restricted: false` changes that presentation; it does not restore a retired endpoint or authorize restricted access. The workbench may use its default for a target run when the configured provider has no credentials here, and displays that fallback without changing the spec. CLI runs do not do this. See [Workbench](workbench.md) for the separate scopes of copilot, target, and research-director models. ## Open vs fixed catalogs `hiveloom models` labels each provider's catalog `open` or `fixed`. - **Fixed** (`claude`): only the model ids registered in-repo validate, so `claude-hiaku-4-5` fails `hiveloom validate` with a clear error instead of at runtime. - **Open** (everything else): any model id validates. Lab catalogs change far faster than hiveloom releases, and aggregators route to thousands of ids — a fixed list would make each new frontier model unusable until the next hiveloom version. An unlisted ID can pass local validation; the endpoint must still accept it, and capability/pricing metadata may be missing. The trade-off is that a typo on an open provider is only caught when the call fails. That is the right side to err on: the alternative blocks real work. ## Capability and identity probes Inspect registered declarations without contacting the endpoint: ```bash hiveloom models probe ./h --provider openrouter --model qwen3.5-9b --json ``` Add `--live` to send one small tool request and, only when the response carries replayable reasoning, one follow-up request: ```bash hiveloom models probe ./h --provider openrouter --model qwen3.5-9b \ --identity exact --live --require-compatible --json ``` `--live` may perform two billed provider calls. The JSON `plan` repeats that side-effect boundary, and the result reports call count, usage, cost source, and request IDs. Live results are cached for 24 hours; `--refresh` bypasses the cache. A provider, requested-model, or adapter-implementation change creates a new cache key. Identity policies are explicit: - `warn` accepts a mismatch but keeps a machine-readable warning; - `exact` accepts only the requested model and rejects an absent identity; - `alias` accepts the requested model or a repeated `--alias` value. Capabilities for tool calling, structured output, and reasoning replay each say whether the value was `declared`, `observed`, or remains `unknown`. The generic live probe can observe tool use and replay reasoning. Provider extensions may override `probe_capabilities()` when their API exposes a more precise structured-output check. ## Pricing and budget guardrails The registered rates describe the runtime's estimates, not the current external provider catalog. Flat per-token prices cannot express every long-context tier, service surcharge, or account discount. Use `models.yaml` for overrides when your workload or negotiated rates differ. Pricing drives cost estimation, the `max_cost_usd` guardrail, and the cost-per-success numbers in `hiveloom stats`. Resolution order for a model id: 1. An exact registration — the builtin per-lab lists, or your `models.yaml`. 2. The provider's default price. Only local providers declare one, at zero, so an unlisted Ollama model is correctly free. 3. The conservative fallback, Haiku-class `$1.00 / $5.00` per 1M tokens. Step 3 charges an unknown hosted model a nonzero estimate. It is not a safe upper bound for every model: expensive or tiered endpoints may cost more. Register accurate prices or use reported billing where available. Runtime budget checks are not a provider account spending limit. > The shipped prices are list price at release time. They are estimates for > budgeting, not billing. Verify against your provider's current pricing and > override anything that has moved. ### Prompt caching The `claude` provider always requests prompt caching: the system prompt, the tool list, and the conversation tail are marked as cache breakpoints, so the stable prefix of an agent loop is written once and read cheaply on every later turn. Cache traffic is reported separately on usage (`cache_read_tokens`, `cache_write_tokens`) and priced at 0.1x / 1.25x the input price. OpenAI-style servers cache implicitly; when they report `cached_tokens`, hiveloom splits them out of the input count and prices them the same way. Both feed the `max_cost_usd` guardrail and `hiveloom stats`, so accounting reflects reported cache usage and configured estimates. Provider billing may differ from these rates. ### Adaptive-thinking Claude models The newer Claude tiers (Opus 4.7 and later, Sonnet 5, Fable/Mythos) reject sampling parameters and reason with adaptive thinking. The provider adapts on its own — nothing to configure — but two consequences are worth knowing: - `model.temperature` is **not sent** for those IDs, even when explicitly configured. The spec default is `None`, which omits sampling temperature. Set a value only for providers/models whose API supports it. - Thinking blocks are kept on the assistant turn and replayed verbatim, which those models require across a tool-use loop. They are billed as output tokens, so budget `model.max_tokens` for thinking *plus* the answer: a harness whose responses are large should raise it (the frontier evals use `16000`). The shipped catalog now includes output capabilities and presentation metadata for newer Claude models. Inspect `hiveloom models claude --json` for the IDs registered in your installation. Register a missing ID and its pricing in `models.yaml` as below; registration does not grant provider account access. ## Customising with `models.yaml` `~/.hiveloom/models.yaml` (or `$HIVELOOM_HOME/models.yaml`) adds providers and adjusts builtin ones. Three shapes: **Correct or add models on a builtin** — omit `base_url` to extend rather than replace: ```yaml providers: openai: models: - id: gpt-4o input_cost_per_mtok: 1.11 # your negotiated rate output_cost_per_mtok: 2.22 ``` **Point a builtin name at a different endpoint** — supply `base_url` and it overrides the builtin, keeping the name your harnesses already reference: ```yaml providers: openai: base_url: https://gateway.internal.example/v1 api_key_env: INTERNAL_GATEWAY_KEY ``` **Declare a new provider** — any OpenAI-compatible server: ```yaml providers: lmstudio: api: openai_compat base_url: http://localhost:1234/v1 models: - id: qwen3-8b input_cost_per_mtok: 0 output_cost_per_mtok: 0 supports_tool_calling: true supports_structured_output: false supports_reasoning_replay: false ``` A newly declared provider is **fixed** by default: you listed the models you meant, so a typo should fail. Set `open_catalog: true` to accept any id. Omitting a model's pricing is allowed but reported by `hiveloom extensions`, and that model is priced at the conservative fallback. Set both costs to `0` to declare a model genuinely free. A malformed `models.yaml` never crashes the CLI — the error is collected and shown by `hiveloom extensions`. ## Provider request parameters `model.params` carries provider-specific JSON request fields. Both the Claude provider and the OpenAI-compatible provider forward these fields, and model overrides retain them. Extensions implementing other providers decide which parameters they support. Provider-specific settings may need an operator update when switching providers. ```console hiveloom set model.params '{"seed": 7}' --dir ./harness --json ``` Use field names accepted by the selected endpoint. The mapping is limited to 64 KiB of JSON and cannot replace harness-controlled fields: `model`, `messages`, `system`, `input`, `tools`, `functions`, `max_tokens`, `max_completion_tokens`, `max_output_tokens`, `temperature`, `stream`, `stream_options`, or `n`. `model` remains frozen from evolution, including these parameters. Parameters are request-body values, not HTTP headers or SDK options. A model registered through an extension or `models.yaml` can declare a positive `max_output_tokens`. `model.max_tokens` is validated against that capability. Unknown capabilities use the spec's absolute 1,000,000-token validation bound; this does not promise that an endpoint accepts that many tokens. The runtime never automatically raises the configured budget after a truncated response. Custom OpenAI-compatible endpoints can declare a positive `timeout_seconds`: ```yaml providers: local: api: openai_compat base_url: http://localhost:8000/v1 timeout_seconds: 600 models: - id: local-model max_output_tokens: 16384 ``` The default timeout is 120 seconds per HTTP request. Retries can consume additional time. Runtime wall-clock guardrails are checked at loop boundaries; they do not cancel a blocking HTTP request. ## Generation and evolution models `hiveloom generate` and `hiveloom evolve` use a *strong* model, separate from the small executor inside the harness. Select it with `provider/model-id`: ```console $ hiveloom generate "extract invoice totals" --model openai/gpt-4.1 -o ./invoices $ hiveloom evolve ./invoices --model ollama/qwen3:32b ``` In this 1.3.0 implementation, the default is `claude-opus-5-5` and requires `ANTHROPIC_API_KEY`. It is separate from the default Haiku executor. Proposal calls stream their response, and use a capability-derived output budget. Pass `--model` explicitly to preserve a proposing model across upgrades. Note that `model` is in `ALWAYS_FROZEN`: the registry widens what a human or a generator may *choose*, never what evolution may *mutate*. ## Compatibility notes - **Tool calling** is required for most harnesses. Provider support differs per model; a small local model may ignore tools entirely. Use a declared or live `hiveloom models probe` before a batch. `run --dry-run` validates the harness but does not test model behavior. - **Gemini** is reached through Google's OpenAI-compatible surface, which supports a narrower slice of the API than the native one. Basic tool calling works; exotic parameters may not. - **Reasoning models** (the DeepSeek-R1 family and similar) are handled: a reasoning-only turn is normalized from the response's `reasoning`/`reasoning_content` field when `content` is empty. - **Aggregators** (`openrouter`, `together`, `fireworks`) use their own id namespace, e.g. `deepseek/deepseek-r1` — pass the id exactly as that service documents it. ## Programmatic registration For anything that is not an OpenAI-compatible HTTP endpoint, register a provider from an extension: ```python def setup(hive): hive.register_provider( "mylab", lambda ctx: MyProvider(), api_key_env="MYLAB_API_KEY", open_catalog=False, models=[{"id": "mylab-small", "input_cost_per_mtok": 0.1, "output_cost_per_mtok": 0.4}], ) ``` The factory must return a `ModelProvider` (see [`models/provider.py`](../src/hiveloom/models/provider.py)). Full extension mechanics live in [extending.md](extending.md). Strong-model generation and evolution use up to 32,768 output tokens when the model declares an output capability, capped by that capability. Models without a declared capability retain the 4,096-token default. SDK callers may request a positive budget explicitly, within any declared model limit.