1.3.0 release notes

Autonomous improvement, with experiments and a reviewable result. This release adds research programs: a director investigates a harness's failures, while an engine measures candidates, accounts for spend, confirms against sealed cases, and queues the result for review. The workbench exposes the program alongside runs, traces, proposals, and provider setup.

Released 4 October 2026. Runtime 1.3.0 is available on PyPI and at the v1.3.0 source tag. Workbench 0.2.0 is a separate release; npm still offers 0.1.0 at this docs release. Use the tagged source for the new UI, as shown in Install.

Research programs #

Use hiveloom evolve ./h --research --json for a bounded multi-round program, or hiveloom research for explicit initialization, stepping, status, reports, stop requests, and resume. A charter declares an eval, mutable levers, execution policy, budget pools, guards, and stop conditions.

The director reads working evidence, registers hypotheses, and designs candidates. The engine measures them against the incumbent case by case, can stop for harm or futility, and decides benefit at the planned sample size. It can combine and remeasure compatible winners. Candidates use their own program identity; their fitness data stays separate from production runs.

Research can keep a significant success gain even if the predicted mechanism was not confirmed (improved). A mechanism disappearing because runs stopped answering is shifted and is not kept. Programs stop on goals, limits, no progress, or an observed ceiling under their permitted levers.

Kept changes receive one sealed confirmation and become a three-way-merged trigger=research proposal. The live harness stays unchanged by default. evolve --research --yes explicitly permits application when a pending proposal's evidence strength is confirmed or supported; supported is positive but statistically uncertain evidence.

Read the Research programs guide for prerequisites, commands, verdicts, confirmation, and promotion.

Concepts mode #

A program can start from concepts and seed requests instead of an existing eval. The director drafts criteria and working cases; you approve the contract and samples before experiments start. Framing itself consumes model budget. A different examiner creates sealed cases independently of the working cases.

Criteria may use deterministic checks or judge rubrics. Judged criteria count only while unanimous judges meet agreement and κ floors on enough human labels. The engine asks for labels within a declared question budget and audits trust during rounds. Reports expose unmeasured criteria and working-versus-sealed gaps. A pass is evidence only for the criteria actually measured.

Execution and accounting #

• Research classifies tools as allow, supported workspace sandbox, recorded replay, or deny. Unclassified code, network, shell, and MCP tools prevent initialization. MCP discovery and code-loading boundaries still apply.
• Existing safety, redaction, confinement, and frozen-field gates remain in force. Delegation is disabled in research candidates because peers execute outside the research tool policy.
• Exploration, experiment, and reserved confirmation pools account for program spend; concepts mode adds a data pool. Provider billing and estimated token costs determine the accuracy of that accounting.
• Programs persist state, an eval manifest, and a hash-chained ledger. An interrupted program resumes its remaining work. Only one process advances a program at once.
• Research proposals now name the harness's metric objectives, so harnesses that declare evolution.objectives can pass the evolution gate. Expectations follow the hypothesis's metric target, then charter goal metrics, then all configured objectives; directions come from the objectives.
• Program status and reports read a promotion's current proposal status, so an applied or rejected proposal no longer appears permanently pending.
• evolve --experiment now charges the proposing model's reported billed cost to the experiment round as well as executor spend.

Workbench #

• Research provides charter creation, background execution, live eval progress, per-case base/candidate links into Trace, contract approval, questions, budget pools, trust, findings, reports, and ledger entries.
• Interrupted research jobs resume on server restart. A rail badge indicates that a program needs attention. The copilot can start and relay programs; Improve offers Evolve autonomously.
• Global Settings → Models manages providers, selectable models, and the default model. Custom OpenAI-compatible providers are registered through providers.yaml; hand-maintained models.yaml loads afterwards and wins.
• A harness's Settings tab edits its run model, input-token budget, temperature, and durable memory through validated framework operations.
• Shared model pickers show configured choices. Legacy and restricted model metadata filters default offers without invalidating existing IDs.
• If a target run cannot use its configured provider because credentials are missing, the workbench can use its configured default for that run and shows the fallback. The spec is unchanged. CLI runs do not acquire this fallback.
• Improve shows pending text changes against the current spec and the recorded YAML diff/version hashes for applied changes.
• Forks have an explicit Resume from turn action. Construction events record old/new version hashes so version graphs can attribute edits.
• The copilot receives compact structured tool-result data as well as the interface artifact, allowing it to act on IDs it just read. A non-Claude copilot no longer requires an unrelated Anthropic key.

See the revised Workbench guide.

Runtime and provider fixes #

Forks now copy supporting files such as data tables from the current source folder, without overwriting behavioral files restored from the journal. Hidden entries, venv, node_modules, __pycache__, symlinked files, and individual files over 50 MB are excluded. These supporting files are not checked against the parent manifest: a fork may read today's data even when its code matches the parent.

Resumed runs give validators the original user task in run_context["input"] instead of the parent run ID. Provenance remains in lineage. See Journal and forks.

Hook modules are registered before execution so postponed annotations, including TypedDict tool parameters, can resolve when building input schemas.

The generation/evolution default is now claude-opus-5-5. Claude proposal calls stream their response, with output budgets derived from registered model capabilities. Builtin model metadata has been refreshed; use hiveloom models for registered IDs rather than treating a release-note list as a provider's live catalog. Existing legacy IDs remain accepted. See Models.

Evaluations and examples #

The Python SDK's run_eval(case_ids=...) can execute a subset by EvalCase.id without changing the eval identity. Unknown IDs are rejected; the selected set is persisted for resume. This is an SDK capability, not a new CLI flag.

The active harnesses/ directory now contains ranked-retrieval, ticket-triage, and log-forensics. The scripted labs and other older examples are under tests/fixtures/harnesses/; previous paths must be updated. Ranked retrieval has additional records/cases and a research charter for vocabulary mismatch. Ticket triage's code validator checks queue coverage, duplicates, known/open IDs, allowed labels, and a nonempty reason; it does not judge whether a category or priority is semantically correct.

research-lab demonstrates the engine offline with a scripted director. evals/research-bakeoff records live-director comparisons and autonomy checks. The original 1.5× defect-fix target was not met. Later tests show useful unattended improvement on small planted tasks; they do not establish general superiority. Evaluations gives the results and limits.

Upgrading from 1.2.0 #

1. Keep existing harnesses: research is opt-in, and its charter is a separate document. There is no new required harness.yaml field for research.
2. If scripts use generate or evolve without --model, review the changed default and its budget, or pass an explicit selector to preserve a choice.
3. Update example paths and check fork data before assuming a replay is a historical reproduction. Supporting data comes from the working copy.
4. In the workbench, verify the effective run model if a provider is missing. Add its credentials or explicitly choose the intended model for comparisons.
5. Define a meaningful eval, mutable levers, and effect classifications before starting research. Concepts mode needs contract approval and may need labels; it is not fully unattended from initial prose through promotion.

Install or upgrade the published runtime:

uv tool install --upgrade 'hiveloom==1.3.0'
# In a project instead:
uv add 'hiveloom==1.3.0'
hiveloom --version

The workbench is a separate npm release. For the new UI features, use the v1.3.0 source checkout until workbench 0.2.0 is available, as shown in Install.

The 1.2.0 documentation and release notes remain available as a frozen edition.