# Demo harnesses

Each folder is a complete harness: a `harness.yaml`, the code it declares, its
data, and a README that states **what it proves**, which capabilities it
exercises, how to run it, what evidence to look for, and what to try next. They
were built — and are changed — only through the CLI (`init`, `add`, `set`,
`memory`), never by editing `harness.yaml` by hand.

**Offline** demos run on a scripted provider shipped in their `extensions/`:
no API key, and the same journal every time. The scripts never carry the
answer — they react only to what they are actually shown (their prompt, their
tool results, the signal map), so the evidence is real. **Live** demos need an
API key for their provider (any provider works via `--provider/--model`).

| demo | proves | runs |
|---|---|---|
| [quickstart](../harnesses/quickstart) | a harness with no tools still adds a journal, a hashed spec, spending and turn ceilings, and a safety layer that keeps a credential out of the request, the trace and the answer | live |
| [example-summarizer](../harnesses/example-summarizer) | a model held to a contract — shape, content and a house style it loads as a skill — and retried with feedback until the output earns a success | live |
| [article-extractor](../harnesses/article-extractor) | code parses, the model judges, and the answer is checked against the live page so invented headings fail | live |
| [log-forensics](../harnesses/log-forensics) | an allowlisted, confined shell command produces more than the context holds and the harness still answers from its last line | live |
| [ranked-retrieval](../harnesses/ranked-retrieval) | structure beats model size: enforced phases, verify-first search and grounded ids make a 3B model reliable, measured by a local eval | live |
| [ticket-triage](../harnesses/ticket-triage) | an MCP server as the only data source, reads fanned out in parallel, one validated report with no invented ids | live |
| [routing-lab](../harnesses/routing-lab) | playbooks change the model *and* the tools mid-run on a pinned plan; forks re-enter a run at any call; an aimed evolution is confirmed by measurement | offline |
| [memory-lab](../harnesses/memory-lab) | a small executor works over more data than its context holds — narrowing it in place, keeping notes, exporting by handle — and learns only through review | offline |
| [signal-lab](../harnesses/signal-lab) | the harness finds where its failures come from, drafts an aimed fix on its own, and keeps a change only when its eval confirms it | offline |
| [delegation-lab](../harnesses/delegation-lab) | a harness hands a task to a peer only once the peer has earned it in measured runs, verifies the answer itself, and otherwise names the peer | offline |

## By capability

| capability | where to see it |
|---|---|
| journal, version hash, `stats` | every demo; [quickstart](../harnesses/quickstart) is the minimal case |
| guardrails: cost, wall clock | [quickstart](../harnesses/quickstart), [example-summarizer](../harnesses/example-summarizer), [article-extractor](../harnesses/article-extractor) |
| guardrails: `regex_output_filter`, `max_turns_hard_cap` | [quickstart](../harnesses/quickstart) |
| guardrails: `tool_allowlist` | [example-summarizer](../harnesses/example-summarizer), [article-extractor](../harnesses/article-extractor) |
| guardrails: `no_network_write` | [article-extractor](../harnesses/article-extractor) |
| redaction (`logging.redact`) | [quickstart](../harnesses/quickstart), [example-summarizer](../harnesses/example-summarizer) |
| provider egress screening | [quickstart](../harnesses/quickstart), [log-forensics](../harnesses/log-forensics) |
| confinement of spawned processes | [log-forensics](../harnesses/log-forensics) |
| verification: `output_schema`, retry with feedback | [example-summarizer](../harnesses/example-summarizer), [routing-lab](../harnesses/routing-lab), [memory-lab](../harnesses/memory-lab) |
| verification: code validators | [example-summarizer](../harnesses/example-summarizer), [article-extractor](../harnesses/article-extractor) |
| verification: `grounded_references` | [ranked-retrieval](../harnesses/ranked-retrieval) |
| verification: `regex_match` | [ticket-triage](../harnesses/ticket-triage), [signal-lab](../harnesses/signal-lab), [delegation-lab](../harnesses/delegation-lab) |
| verification: `file_exists`, `command_succeeds` | [memory-lab](../harnesses/memory-lab) |
| custom `@tool` | [article-extractor](../harnesses/article-extractor), [ranked-retrieval](../harnesses/ranked-retrieval), [signal-lab](../harnesses/signal-lab) |
| output hooks (`strip_json_fence`) | [article-extractor](../harnesses/article-extractor), [log-forensics](../harnesses/log-forensics), [ranked-retrieval](../harnesses/ranked-retrieval) |
| skills and `load_skill` | [example-summarizer](../harnesses/example-summarizer) |
| MCP servers as tools | [ticket-triage](../harnesses/ticket-triage) |
| serving a harness (`package`, `serve`, `mcp serve`) | [quickstart](../harnesses/quickstart), [delegation-lab](../harnesses/delegation-lab) |
| loop: `plan_then_act` | [routing-lab](../harnesses/routing-lab) |
| loop: `sequential_steps` | [ranked-retrieval](../harnesses/ranked-retrieval), [log-forensics](../harnesses/log-forensics) |
| loop: parallel tool execution | [ticket-triage](../harnesses/ticket-triage) |
| playbooks, per-playbook models | [routing-lab](../harnesses/routing-lab) |
| forking and `--resume` | [routing-lab](../harnesses/routing-lab), [memory-lab](../harnesses/memory-lab) |
| spill, handles, `transform_result` | [memory-lab](../harnesses/memory-lab), [log-forensics](../harnesses/log-forensics) |
| `notes` | [memory-lab](../harnesses/memory-lab) |
| `recall_runs` | [log-forensics](../harnesses/log-forensics) |
| durable memory: `memory.entries`, `propose_memory` | [memory-lab](../harnesses/memory-lab) |
| durable memory: relevance selection, `search_memory` | [signal-lab](../harnesses/signal-lab) |
| reflection (`evolution.reflect`) | [signal-lab](../harnesses/signal-lab) |
| `hiveloom signal` | [signal-lab](../harnesses/signal-lab), [routing-lab](../harnesses/routing-lab) |
| aimed proposals, `evolve --propose`, `proposals apply` | [routing-lab](../harnesses/routing-lab), [memory-lab](../harnesses/memory-lab), [signal-lab](../harnesses/signal-lab) |
| auto-propose with trace excerpts | [signal-lab](../harnesses/signal-lab) |
| `evolve --experiment`, `hiveloom assess` | [signal-lab](../harnesses/signal-lab) (offline), [ranked-retrieval](../harnesses/ranked-retrieval) (live) |
| evals, datasets, scorers, metric objectives | [ranked-retrieval](../harnesses/ranked-retrieval), [signal-lab](../harnesses/signal-lab) |
| delegation between harnesses, referrals, lineage | [delegation-lab](../harnesses/delegation-lab) |

`best_of_n` is experimental and not in a demo: its plurality vote needs
answers that can be compared verbatim, which none of these tasks produce.
`http_get` with pre-declared hosts is documented in
[spec.md](spec.md).

## Verified

`scripts/package_e2e.py` builds the wheel, installs it in a clean
environment, and drives every offline demo end to end, validating and
dry-running the live ones (a CI step). With `--live` and an OpenRouter key it
also runs the live demos on small models.