# Extending hiveloom

hiveloom's catalog is open: tools, guardrails, validators, loop policies,
compaction methods, event hooks, eval datasets/scorers, and model providers are
**catalog entries**,
and extensions register new entries through one API. A registered entry shows
up in `hiveloom catalog`, validates in specs like a builtin, and appears in the
generator's meta-prompt — so `hiveloom generate` can weave harnesses with a
capability the moment its pack is installed. MCP tools are the dynamic
exception: they come from declared servers at run time and appear in
`hiveloom mcp list-tools`, not `hiveloom catalog`.

## Writing an extension

An extension is any module exposing `hiveloom_extension(hive)`:

```python
# my_ext.py
from hiveloom.ext import ExtensionAPI

def hiveloom_extension(hive: ExtensionAPI) -> None:
    # a plain function tool (schema derived from type hints)
    def ping() -> str:
        """Reply with pong."""
        return "pong"
    hive.register_function_tool(ping)

    # a configurable tool: factory(params, ctx) -> Tool; params come from the
    # spec entry `{builtin: slack_post, default_channel: "#ops"}`
    hive.register_tool(
        "slack_post", make_slack_tool,
        description="Post a message to a Slack channel.",
        tags=["network", "write"],
        params=[{"name": "default_channel", "type": "str"}],
    )

    hive.register_guardrail("pii_filter", make_pii_guardrail,
                            description="Block outputs containing PII.")
    hive.register_validator("json_diff", make_json_diff,
                            description="Diff output against a golden file.",
                            params=[{"name": "golden", "type": "str", "required": True}])
    hive.register_policy("reflexion", lambda p, c: ReflexionPolicy(),
                         description="One self-critique pass before finishing.")
    hive.register_compaction("keep_files", lambda p, c: KeepFilesCompaction(),
                             description="File-aware summarization.")
    hive.register_hook("audit_log", make_audit_handler,
                       description="Record every tool call to the audit sink.")
    hive.register_dataset("local_cases", make_dataset,
                          description="Load held-out local eval cases.")
    hive.register_scorer("task_quality", make_scorer,
                         description="Score a verified run against expected data.")
    hive.register_blueprint("scraper", "Always add no_network_write. Task: $ARGUMENTS")

    # ambient: runs for every harness in this process (e.g. org-wide audit)
    @hive.on("run_finished")
    def report(event):
        ...
```

Base classes to implement: `hiveloom.tools.registry.Tool`,
`hiveloom.guardrails.base.Guardrail`, `hiveloom.verify.base.Verifier`,
`hiveloom.loop.policies.LoopPolicy`,
`hiveloom.context.manager.CompactionMethod`.
Eval factories return a loader with `load()` or a scorer with
`score(ScorerContext)`; see [evaluating.md](evaluating.md).

Validators may keep the legacy two-argument method or request structured
run-local evidence with a third argument:

```python
from hiveloom import VerificationContext
from hiveloom.verify import VerdictResult, Verifier

class SelectedRecordsExist(Verifier):
    name = "selected_records_exist"

    def validate(
        self,
        run_output: str,
        run_context: dict,
        verification_context: VerificationContext,
    ) -> VerdictResult:
        calls = verification_context.tool_calls
        return VerdictResult(passed=any(call.name == "lookup" for call in calls))
```

`VerificationContext` contains bounded, redacted `tool_calls`, structured step
receipts, declared artifacts, and the current `run_id`. Tool evidence includes
only allowed calls that executed in this run. Read the typed context directly;
do not parse a trace or retain its private payloads. The same context is also
available at `run_context["verification_context"]` for code-hook compatibility.

## Where extensions load from

| Source | Scope | Failure behavior |
|---|---|---|
| pip packages with a `hiveloom.extensions` entry point (**packs**) | everywhere | collected, never crashes (`hiveloom extensions` shows errors) |
| `~/.hiveloom/extensions/*.py` | this user | collected, never crashes |
| `extensions:` list in `harness.yaml` (paths or module names) | that harness | `SpecError` — the harness can't run without it |

A **pack** declares its entry point in `pyproject.toml`:

```toml
[project.entry-points."hiveloom.extensions"]
acme-tools = "acme_tools:hiveloom_extension"
```

`hiveloom package` records the packs a spec's entries come from in
`hiveloom.lock` (`packs:` with name + version), and a run against a missing
pack fails with a message naming it. `hiveloom extensions` lists everything
loaded plus any load errors.

The `extensions:` spec path is **always frozen** — evolution can never add or
change the code a harness loads.

## Model providers

`model.provider` is a registry name, not a hard-coded literal. Builtin
providers already cover the major labs (`claude`, `openai`, `gemini`,
`mistral`, `deepseek`, `xai`), the aggregators (`groq`, `openrouter`,
`together`, `fireworks`), and local servers (`ollama`, `vllm`) — run
`hiveloom models` to list them. **[models.md](models.md) is the full
reference**: endpoints, key variables, pricing rules, open vs fixed catalogs,
and `~/.hiveloom/models.yaml` customisation.

What belongs here is the programmatic path, for anything that is not an
OpenAI-compatible HTTP endpoint:

```python
def setup(hive):
    hive.register_provider(
        "mylab",
        lambda ctx: MyProvider(),        # must return a ModelProvider
        api_key_env="MYLAB_API_KEY",
        base_url="https://api.mylab.example/v1",
        open_catalog=False,              # only the ids below validate
        models=[{"id": "mylab-small", "input_cost_per_mtok": 0.1,
                 "output_cost_per_mtok": 0.4,
                 "supports_tool_calling": True,
                 "supports_structured_output": False,
                 "supports_reasoning_replay": True}],
    )
```

Registering a name that already exists raises, so two packs cannot silently
fight over one provider name; only `models.yaml` may override a builtin.
Pricing lives in the registry, and unknown models fall back to conservative
Haiku-class pricing so cost guardrails never under-count.

Provider adapters return the old `ModelResponse(text=..., usage=...)` minimum
or add provenance that Hiveloom can carry without knowing the vendor:

```python
from hiveloom.models import ModelResponse, Usage

return ModelResponse(
    text=answer,
    usage=Usage(input_tokens=120, output_tokens=20),
    model=raw["model"],                 # identity actually reported
    provider_request_id=raw["id"],
    billed_cost=raw["usage"]["cost"], # charge for this call
    billed_currency="USD",
    reasoning=replay_data,             # opaque JSON needed on the next turn
    provider_metadata={"route": "fast"},
)
```

If a provider bills in another currency, retain the original amount and set
`billed_cost_usd` only after the adapter performs a real conversion. Otherwise
Hiveloom keeps the charge as provenance and uses its token-price estimate for
the USD cost guardrail. Each provider call reports `cost_source` as `billed`
or `estimated` in the journal and public run result.

`reasoning` and `provider_metadata` must be JSON-safe and are size-bounded.
Metadata reaches the trace through the normal `logging.redact` policy, but it
is deliberately omitted from the run-result receipt. Do not put credentials in
metadata; use the harness environment and provider constructor.

OpenAI-compatible extensions can use supported codecs instead of copying
private helpers:

```python
from hiveloom.models.openai_compat import (
    normalize_openai_response,
    to_openai_messages,
    to_openai_tool,
)
```

The pre-1.1 underscore names remain aliases for compatibility, not extension
API. Reasoning fields normalized by this codec are replayed only to the same
OpenAI-compatible provider and are dropped by Hiveloom at a model swap.

Generate/evolve can use any provider too: `--model ollama/qwen3:32b`
(`provider/model-id`). Note `model` stays in `ALWAYS_FROZEN` — the registry
widens what a human or generator may choose, never what evolution can mutate.

Adapters may override `ModelProvider.probe_capabilities(config)` for a
provider-specific live check. Return only the generic
`ProviderCapabilityObservation`; routing policy and vendor response bodies stay
inside the extension. Core compares the reported effective model against the
caller's exact or alias policy and never treats an alias as the requested ID.

## MCP servers

A harness can declare `mcp_servers`; their tools join the loop as ordinary
tools (`mcp__<server-name>__<tool>`), discovered eagerly when the tool
registry is built (including `run --dry-run`):

```bash
hiveloom add mcp-server --name search --stdio-command npx \
  --stdio-arg -y --stdio-arg @foo/mcp-search \
  --env-from-host API_KEY=FOO_SEARCH_API_KEY --dir ./h

hiveloom add mcp-server --name jira --url https://mcp.acme.com/mcp \
  --header-env 'Authorization=ACME_MCP_TOKEN' --tool search_issues --dir ./h

hiveloom mcp list-tools --dir ./h   # see what a declared server actually exposes
```

A stdio server (`--stdio-command`) is **arbitrary local exec** — the same
trust boundary as any other code hook. `mcp_servers` is **always frozen** from
evolution, the same risk class as `extensions`. A remote tool's own
`annotations` (e.g. `readOnlyHint`/`destructiveHint`) are **self-reported by
an untrusted server and are never a security boundary** — the real boundaries
are the always-frozen set plus harness trust gating.

## Lifecycle event hooks

One event taxonomy serves the spec's `hooks:` section, ambient `hive.on(...)`
handlers, and the trace. Handlers get one dict payload; returning a dict
steers the run, returning `None` observes:

| Event | A returned dict may… |
|---|---|
| `context_assemble` | `{"messages": [...]}` — replace the message list |
| `before_provider_request` | `{"system": ...}` / `{"messages": [...]}` / `{"tools": [...]}` — patch the outgoing request (this request only; runs after guardrails) |
| `before_tool_call` | `{"block": True, "reason": ...}` or `{"input": {...}}` |
| `after_tool_call` | `{"content": ...}` / `{"is_error": ...}` — patch the result |
| `before_verification` | `{"output": ...}` — replace final output before guardrails and validators |
| `before_compaction` | `{"cancel": True}` or `{"summary": "..."}` |
| `run_started` / `before_model_call` / `after_provider_response` / `after_model_response` / `verification` / `run_finished` | observe only |

```bash
hiveloom add hook --on before_tool_call --code hooks/audit.py:audit --dir ./h
hiveloom add hook --on before_verification --builtin strip_json_fence --dir ./h
```

(In `harness.yaml` the field is `event:` — unquoted `on:` is a YAML boolean.)
Handlers must not raise; one that does is logged as a `hook_error` trace event
and skipped. Guardrails remain the frozen safety layer and always run first;
hooks are the extensible middleware layer.

## Skills, deferred tools, tool ergonomics

**Skills** (progressive disclosure): `hiveloom add skill pdf-report
--description "..."` scaffolds `skills/pdf-report/SKILL.md` and lists it in the
spec. Only name + description enter the system prompt; the model reads the full
file on demand with `file_read`.

**Deferred tools**: mark a tool `deferred: true` and it stays out of the model's
payload until the auto-added `search_tools` tool activates it — keeps context
small for harnesses with many tools.

**Tool authors** get: `guidelines` (usage rules injected while the tool is
active), `prepare(kwargs)` (normalize model-mangled args before validation),
`run_with_updates(kwargs, on_update)` (+`supports_updates`) for streaming
progress as `tool_update` trace events, and `ToolResult(terminate=True)` to
end the run without a final model call (honored when every result in the batch
terminates).

## Blueprints (generator house style)

A blueprint is a markdown fragment appended to `hiveloom generate`'s
meta-prompt — preferred tools, guardrail posture, validator patterns for a
family of tasks. `$ARGUMENTS`/`$@` expand to the task, `$1..$9` to its words.

```bash
hiveloom generate "extract top HN stories" -o ./hn --blueprint scraper
```

Lookup order: `~/.hiveloom/blueprints/<name>.md`, then pack-registered
(`hive.register_blueprint`).

## Trust

Harness folders carry executable code. Folders **built on this machine**
(via `init`/construct/generate) are trusted automatically; a foreign folder
(unzipped artifact, clone) is gated before any of its code loads:

```bash
hiveloom trust ./foreign-harness      # or: hiveloom run ./foreign --approve
HIVELOOM_TRUST=always hiveloom run .  # CI; `never` refuses instead
```

Decisions live in `~/.hiveloom/trust.json`. Frozen paths protect a harness
from its evolution; trust protects a machine from a foreign harness.

## Embedding hiveloom

**Any language** — stream trace events as JSONL over stdout:

```bash
hiveloom run ./h --input notes.txt --stream
# {"type":"run_started",...}
# {"type":"tool_call",...}
# ...
# {"type":"run_result","ok":true,"status":"success",...}
```

**Python** — the semver-stable SDK:

```python
from hiveloom import run_harness, generate_harness, Hive

result = run_harness("./h", "notes.txt", on_event=lambda e: print(e.type))
```

`$HIVELOOM_HOME` relocates everything user-level (extensions, models.yaml,
blueprints, trust.json, the default Hive DB) — useful for tests and CI.