Journal and forks

Every run writes an append-only JSONL journal into the harness's .hiveloom/traces/. From 1.0.0 that journal is progressive, self-describing, and tamper-evident — complete enough to read a run back turn by turn, and complete enough to re-enter it at any model call and continue from there against an edited harness.

This page covers what the journal records, how to check it, and the three things you can do with it afterwards: fork, resume, and swap models.

hiveloom trace <run_id>                    # summary + ordered events
hiveloom trace <run_id> --verify           # walk the append-only hash chain
hiveloom trace <run_id> --materialize 42   # the exact request sent at seq 42
hiveloom fork <run_id> --list              # the model calls you may re-enter
hiveloom fork <run_id> --at 42 --name probe
hiveloom run <harness>/.hiveloom/forks/probe --resume
hiveloom lineage <run_id>                  # the fork tree and its divergence points

The record is progressive, not snapshot-shaped #

Before 1.0.0 every model_call re-serialised the whole conversation, which is O(n²) in turns and left "the state at seq N" undefined — every reader re-derived it differently. Now the conversation is recorded once, message by message:

A model_call therefore references the folded context instead of copying it. The prompt at any turn is a fold over the events up to its context_head, and compaction is another fold operation rather than a special case.

On a 12-call run this cut the journal from 676 KiB to 184 KiB, and model_call from 85% of the file to 2.5%.

hiveloom.logging.journal is that fold, and it is the one implementation shared by trace --materialize, fork, and the workbench:

from hiveloom.logging.journal import read_events, state_at_model_call, verify_chain

events = read_events(".hiveloom/traces/run_abc123.jsonl")
state = state_at_model_call(events, 42)   # ContextState(system, messages, tools)
request = state.as_request()              # exactly what the provider was sent

Pre-1.0 traces still fold correctly: a model_call carrying inline system/messages/tools is treated as a wholesale replace.

"Read-only" is a claim you can check #

Each event carries prev: the sha256 of the line before it. --verify walks the chain and reports the first break with the position it broke at, exiting 4.

hiveloom trace run_abc123 --verify
# intact: 61 events, chain unbroken

Three outcomes, deliberately distinct:

• intact — every line commits to the one before it.
• BROKEN at line N — a line was edited, reordered, or removed. Truncating the tail is not a break: a prefix of an append-only log is always valid.
• unchained — the events carry no prev at all, because they were written before 1.0.0. "We cannot tell" is a different answer from "it was tampered with", and conflating the two would make the check useless for exactly the traces most likely to be old.

The harness is in the record #

run_started carries a harness snapshot: the dumped spec plus a path -> sha256 manifest of every local behavioural file — tools, validators, hooks, playbook prompts — the same set the version hash fingerprints. You can name and reconstruct the spec that ran without the folder being present.

Set logging.snapshot_files: true to inline the file bodies too, bounded at 256 KiB, reporting whatever it skipped. That makes a journal self-contained at the cost of size; the default records hashes only.

run_finished closes the record with the run's output, verdicts, artifacts, model_path, models_used, and the same execution envelope returned by the SDK and CLI. That envelope keeps the requested, resolved, and provider-reported model identities separate; sums provider-call usage; labels cost as billed, estimated, or mixed; and records whether verification passed on the first output, recovered, failed, or never ran.

behavior_hash is the current name of the harness version hash inside this public envelope. schema_version reflects the canonical harness document field. Legacy documents using version load with the same meaning and migrate without changing the behavior hash, so their existing journal and Hive buckets remain comparable.

Levels #

logging:
  level: journal   # or: summary
  snapshot_files: false
  trace_dir: .hiveloom/traces

The old names still load in both harness.yaml and TraceWriter (full → journal, tool_calls_only → summary), so existing harness folders keep working. The names changed to say what they cost you: the reason to pick one over the other is whether you will be able to fork.

Redaction and retention #

Redaction runs on structured values before a trace event is serialized, kept in memory, sent to an on_event stream consumer, or later ingested into the Hive:

logging:
  redact:
    keys: [email, phone, api_key]
    paths: ["result.candidates[*].cv_text"]
    patterns: ["secret-[a-z0-9]+"]
  retention:
    days: 30
    max_runs: 5000
    max_bytes: 1073741824

Keys match recursively and case-insensitively. Paths are case-sensitive and relative to each event's payload; they support dot-separated dictionary keys and [*] list wildcards. Patterns run over every remaining string. The old bare redact: [regex, ...] form still loads and serializes in the same shape, so a regex-only harness keeps its behavior hash.

Retention is absent by default. When configured, a completed ingested run prunes older raw journals after proposal drafting, while always preserving the trace returned for the current run. Preview or apply the same policy directly:

hiveloom traces prune ./h --dry-run --json
hiveloom traces prune ./h --yes --json

Hiveloom deletes only direct, non-symlinked RUN_ID.jsonl files under a trace root carrying its marker. The first event must identify the same run as the filename. Age uses the file modification time, so copying a trace starts a new local retention window. Count and byte limits remove the oldest eligible files first. An atomic rename lets readers with an open handle finish before unlink.

Pruning keeps the indexed run, verification, and outcome evidence. The Hive sets trace_path to null and records trace_pruned_at, so trace and fork commands report pruned evidence instead of following a stale path. If the same run was re-ingested from another durable location, pruning an older copy does not clear that newer reference. At-rest encryption is not part of this policy; it needs a separate key storage, rotation, and recovery design.

What confined the run #

run_started carries a confinement record — the declared mode and the backend that was actually available on the machine. A spec asking for mode: auto says nothing about whether a sandbox existed; the journal says which one ran, so "these processes were isolated" is a checkable claim about a particular run rather than a property of the configuration. See Process confinement.

Spilled tool results #

A tool result too large to inline is stored whole under trace_dir/spill/ and reaches the model as a preview plus a handle (see Large tool results). The journal is unaffected: tool_result still carries the complete content, and a tool_spilled event records the handle, the total size, and how many bytes the model did not see — so a trace shows both what the tool produced and what the run actually reasoned over. Redaction is applied to spill objects exactly as it is to the journal.

transform_result can create a further object inside a tool call — the narrowed result of an op that was itself too large to inline. Those are journaled as tool_spilled too, with name: transform_result plus derived_from and op, so the fork path (which reads minted handles off the verified journal) treats a derived object exactly as it treats a spilled one.

hiveloom fork copies the objects a fork's context still quotes into the fork's own trace directory, so a resumed fork can read them back rather than inheriting previews it can never expand. spill_inherited records what the resumed run was granted.

Notes #

A harness that declares the notes tool gives the executor a small run-private store (see Notes). Every change to it is an event:

The content is in the event, so hiveloom trace shows what the model wrote down, and the fold reconstructs the store rather than guessing at it: hiveloom fork replays notes_inherited, then note_written/note_deleted, up to the fork point, and writes the notes still held into fork.yaml as a hash-bound notes_manifest. The bytes it copies are the ones held at the fork point, taken from the journaled content and checked against its digest — so a note the parent rewrote or deleted later still reaches the fork as it was, and a fork of a fork carries what it inherited. A note written and then deleted before the fork point is not a note the fork inherits, and a name mentioned in the transcript grants nothing. Deleting an inherited note in a resumed fork drops the grant, not the fork's copy, so the next resume inherits it again.

Proposed lessons #

A harness that declares the propose_memory tool (see Memory) emits one memory_proposed event per call, carrying {id, kind, title, content, evidence, outcome, proposal_id} — the redacted lesson plus what became of it (queued, already_pending, already_known, cap_reached, memory_disabled, eval_run, refused, queue_unavailable). One event with the outcome rather than one per stage, so a run that proposed something and did not queue it — an eval cell, a spent cap — still leaves the lesson behind for a reader. The spec itself never changes here: proposal_id is what hiveloom proposals show takes.

Forking a run #

A fork re-enters a finished run at one of its model calls and replays the identical prefix against a changed harness. That is the difference between debugging the failure you had and hoping a fresh run reproduces it.

hiveloom fork run_abc123 --list
# seq  turn  phase
#  12     1  react
#  27     2  react
#  42     3  react

hiveloom fork run_abc123 --at 42 --name probe
# -> ./my-harness/.hiveloom/forks/probe

Fork points are model calls. The folded state immediately before a model_call is by construction a valid provider request; an arbitrary seq can land mid-turn with a dangling tool_use and no result, which no provider accepts. A --at that names something else snaps back to the previous model call and says so.

The fork directory holds the harness that actually ran — spec rebuilt from the journal snapshot, files verified against its sha256 manifest — plus:

.hiveloom/forks/probe/
├── harness.yaml          # the parent's spec, as it was at run time
├── tools/ validators/    # the parent's code, verified against the manifest
├── fork.yaml             # the lineage record
└── .hiveloom/context.json  # the folded conversation at the fork point

fork.yaml pins the lineage to the exact journal line it came from:

parent_run_id: run_abc123
parent_harness_version_hash: 9f2c1a4b7e05
at_seq: 42
at_turn: 3
parent_line_hash: 3b1f…          # sha256 of that journal line
created_at: 2026-08-28T09:14:22+00:00
context_file: .hiveloom/context.json
harness_version_hash: c05e8d31aa17

Change the fork through the construction CLI, then resume:

hiveloom set loop.max_turns 30 --dir ./my-harness/.hiveloom/forks/probe --json

hiveloom run ./my-harness/.hiveloom/forks/probe --resume --json

--resume accepts no new input: it continues the saved conversation prefix. Validators receive the original task in run_context["input"], recovered from the first text user message; provenance stays in lineage rather than replacing the task with a parent run ID. In the library, runner.run_harness(resume_messages=..., lineage=...) is the same seam.

Where forks live, and why #

Forks live inside the harness they came from, under .hiveloom/forks/. A fork is an experiment on a harness rather than a harness of its own, so archiving the entire folder can keep its experiments with it, a directory of harnesses stays a directory of harnesses, and the parent's file tools — rooted at the harness folder, which they do not descend into .hiveloom from — cannot reach a running experiment.

Forking a fork produces a sibling under the same original harness rather than nesting a level deeper. Depth would record generation, and fork.yaml already does. fork.fork_target() is the one resolver the CLI, the workbench, and MCP share, so they cannot disagree about where a fork goes.

--dir opts a one-off out of containment for your own shell. The workbench has no such escape hatch: a fork writes files, and a browser must never choose where.

Supporting files and reproducibility #

In 1.3.0 a fork also copies supporting files from the current source folder, such as data tables read by a validator. Files restored from the journal are not overwritten. Hidden files/directories (including .env, .git, and .hiveloom), venv, node_modules, __pycache__, symlinked files, and files over 50 MB are excluded; copying failures and oversized files produce warnings.

Supporting files are not verified against the parent manifest. A fork's spec and referenced behavioral code can match the original while its data is today's copy. Archive or version the dataset separately if historical data matters. Hash-only snapshots also need the original behavioral files to remain available; inlined snapshots are bounded and may skip files.

hiveloom package excludes .hiveloom, so a deployment package does not carry forks, research state, or run journals. Collect evidence separately.

The workbench's Resume from turn control continues a fork; Use starts a fresh run. See Workbench.

When a fork is refused #

The redaction warning matters: redaction happens before persistence, so the fork sends the marker where the parent sent a value. The run is reproducible in shape, not in content.

Trust inheritance. A fork whose files came from an already-trusted source folder, verified against the journal's manifest, inherits that trust — it is the same code at a new path. A fork built from a journal's inlined contents never does, and says so: a journal is a file someone can hand you, and its hash chain proves internal consistency, not provenance.

Lineage #

runs carries parent_run_id and forked_at_seq (migrated in place on existing Hives), so a fork and its parent are compared on the prefix they share rather than as two unrelated runs:

hiveloom lineage run_abc123

Model hot-swap #

The executing model can change while a run is in flight, through two surfaces and no others.

Declarative — a playbook may declare its own model: (and model_provider:). Profile on a cheap model, decide on an expensive one, inside one harness and one conversation. Leaving the mode restores the harness default, so a mode is a configuration and not a one-way door:

playbooks:
  - name: triage
    model: claude-haiku-4-5-20251001
    prompt: playbooks/triage.md
  - name: decide
    model: claude-opus-5
    prompt: playbooks/decide.md

Both fields are frozen from evolution, joining top-level model: evolution must not move a harness onto a pricier model, or a different lab, on its own initiative.

Imperative — RunControl.switch_model(...) and POST /runs/{run_id}/model, consumed at the loop's next turn boundary alongside stop and steer, where no model call or tool is in flight.

hiveloom.models.router.ModelRouter owns which model is current and which provider instance serves it, building cross-provider instances lazily so an unused provider's absent credentials are not a startup failure. Pass run_harness(providers={...}) to pre-register what a swap may talk to.

At the boundary, prior turns are stripped of model-internal content — thinking, redacted_thinking, anything carrying a signature — because those blocks are only valid for the model that produced them. An assistant turn stripped to nothing is dropped whole rather than left to break role alternation.

Swaps are journalled as model_swap; a swap to an unknown provider is model_swap_failed and the run continues on the model it had.

Swapped runs are held out of fitness #

A run whose model moved did not execute the harness as declared. Averaging it into "this version scores N%" would silently turn that number into a distribution over model paths, so Hive.version_stats excludes it from the per-version bucket and reports the count held out as swapped_runs. hiveloom stats says so in words and breaks the held-out runs into a per-model-path table; version_stats(include_swapped=True) returns the raw population. An empty model_path — every pre-1.0 run — means "not recorded", never "swapped".

Fork × swap: the controlled A/B #

The cleanest experiment the two features make possible is one exact prefix, replayed on two models:

hiveloom fork run_abc123 --name on-haiku  --model claude-haiku-4-5-20251001
hiveloom fork run_abc123 --name on-sonnet --model claude-sonnet-5
hiveloom lineage run_abc123

--model rewrites the fork's spec (through construct.set_model, so it is validated and rolled back like any other edit) rather than swapping mid-run. Each arm is therefore a clean sample of its own harness version, and neither is held out as swapped. A rejected model removes the half-built fork rather than leaving one behind that claims a model it does not have.

Reading the journal from the Hive #

Ingested runs are queryable:

from hiveloom import Hive

with Hive() as hive:
    hive.search_runs("invoice reconciliation")      # runs by what was asked
    hive.compare_versions("my-harness", "9f2c1a", "c05e8d")
    hive.lineage("run_abc123")
    hive.list_friction("my-harness", recovered=True)

runs carries task (the opening statement, capped at 2000 chars — a title and search target, not a shadow copy of the journal) and model_path.

compare_versions puts two versions side by side with deltas (right minus left), plus which failure signatures stopped appearing and which started. It reports underpowered when either side has fewer than five runs, because a confident delta over a sample of two is worse than no delta.

Friction is not final failure #

A run can finish successfully after a schema retry, tool error, context recovery, guardrail block, or operator steer. The Hive derives bounded friction_events rows from the already-redacted journal so those incidents do not disappear behind the final status:

hiveloom friction list ./h --category output_validation --recovered true --json
hiveloom friction list ./h --model qwen3.5-9b --since 2026-08-01T00:00:00Z --json
hiveloom stats ./h --include-friction --json

Each record names its run, journal sequence, category, phase, attempt, component, stable error fingerprint, recovery state, timestamp, and a summary capped at 500 characters. Tool bodies, model text, task input, and operator messages are not copied into the friction table. Re-ingesting a run replaces its derived rows, so counts remain idempotent. Unknown future category strings remain readable by older Hive clients.

Numeric evaluator signals #

Deferred outcomes answer an eventual yes/no question. Ranked quality, latency, cost-quality tradeoffs, and other numeric observations use a separate RunMetric record joined to the run:

hiveloom metrics schema --json
hiveloom metrics record ./h --run-id run_abc123 \
  --name recall_at_5 --value 0.4 --direction maximize \
  --unit ratio --source matching_eval_v1 --scope case --json
hiveloom metrics import ./h metrics.ndjson --json
hiveloom metrics list ./h --name recall_at_5 --model qwen3.5-9b --json

Metric writes are immutable. Without an explicit idempotency_key, Hiveloom allows one logical observation per run, name, source, and scope. Replaying the same observation is a no-op; reusing its key with different content rejects the whole batch. Supply distinct explicit keys only when a scorer deliberately emits repeated observations for the same logical slot.

NDJSON imports parse and validate every row before one database transaction. Names are user-defined, values must be finite, metadata must be JSON-safe and bounded, and the referenced run must belong to the target harness. Queries can filter by run, source, name, scope, effective model, and run finish time. Aggregates never combine scopes, units, directions, or sources and always report sample_count and missing_value_count. Case and run scopes count missing indexed runs in the filtered population; eval scope reports its observed records because Hiveloom has no external eval-population manifest yet.

See also #

• The workbench — the graphical front end to all of this
• Harness spec — logging, playbooks, and the frozen fields
• Architecture — where the journal sits in the runtime
• Deploying and evolving — the evidence loop