# Mixture of Harnesses (MoH): Architecture from First Principles ## Executive thesis A Mixture of Harnesses is not a larger model choosing one of several experts. It is a **meta-execution system** choosing and composing unlike execution environments. The unit of composition is a *work contract*, not a token stream or a prediction vector. The three harnesses have different affordances and different definitions of success: | Harness | Natural job | Effects/outputs | Strength | Failure if misused | |---|---|---|---|---| | `pi` | deterministic local execution | files, processes, command results | fast, inspectable, minimal | makes an architectural judgment or produces weak research | | Prime Agent | planning, reasoning, delegation, stateful analysis | plans, reports, datasets, decisions | multi-step synthesis and persistent working state | over-engineers simple edits; unbounded autonomy | | Hermes | external research and signal extraction | sources, evidence, findings, patterns | breadth, web/community intelligence | changes local state or treats weak signals as facts | The meta-harness therefore performs four different operations: **contract formation, decomposition, dispatch, and typed synthesis**. It does not average harness outputs. It composes effects, evidence, and decisions under explicit provenance and acceptance criteria. --- ## 1. First principles and the rejection of MoE assumptions ### What MoE assumes A conventional MoE normally assumes: 1. experts accept approximately the same representation and produce compatible representations; 2. a gate can select or weight experts at a point in a forward pass; 3. outputs can be combined numerically or passed into a common downstream layer; 4. the objective is usually one prediction, with one latency/cost budget; 5. experts are interchangeable implementations of a common function. Those assumptions fail here. A shell edit is not a partial report; a web citation is not a file mutation; a persistent agent's state cannot be linearly interpolated with command output. The harnesses differ in: - **execution model:** synchronous command process vs stateful Python agent vs research workflow; - **authority:** local mutation, delegated reasoning, or external read-only access; - **state:** filesystem/process state, kernel/session state, and source/evidence state; - **latency and cost:** milliseconds to many turns and network calls; - **output algebra:** effects, observations, claims, and artifacts; - **risk:** destructive local change, hallucinated interpretation, or unreliable external evidence. Thus MoH should be modeled as a **typed workflow graph** (or a small orchestration state machine), not a neural layer: ``` User intent -> contract -> work graph -> dispatch -> receipts -> synthesis/verification -> result ``` A harness is selected because a *node in the work graph* needs its affordances—not because it is globally the “best expert.” A single task can invoke all three, sequentially or conditionally. --- ## 2. The common unit: a work contract Before routing, the meta-harness normalizes the request into a contract. The contract should contain: ```yaml contract_id: moh-2025-... objective: "Add a rate-limit dashboard and recommend alert thresholds" context: repo: /workspace/app user_constraints: ["do not deploy", "cite external sources"] available_harnesses: [pi, prime-agent, hermes] requested_deliverables: - kind: code_change path_scope: dashboard/ - kind: recommendation requires_citations: true acceptance: - tests pass - recommendation distinguishes evidence from assumption authority: allowed_effects: [read_files, edit_files, run_tests] forbidden_effects: [deploy, send_message] budget: wall_clock_seconds: 600 network_calls: 20 provenance_required: true ``` The contract is not a prompt copied to each harness. It is a compact, machine-checkable statement of objective, scope, authority, budgets, and acceptance tests. It makes hidden assumptions explicit and gives each sub-harness only the context needed for its assigned node. ### The envelope/payload distinction Every invocation uses one common **envelope** and one typed, harness-specific **payload**. ```yaml invocation: envelope: contract_id: ... task_id: research-1 parent_task_id: root role: evidence_gatherer objective: "Find current guidance on API rate-limit alerting" inputs: ["service is multi-tenant", "p95 latency is available"] authority: {network: read_only, filesystem: none} budget: {calls: 8, seconds: 90} expected_output: evidence_bundle acceptance: [at least 3 primary/authoritative sources] trace: {correlation_id: ..., causation_id: ...} payload: hermes: query_plan: [...] domains: [official docs, reputable SRE sources] extraction_mode: claims_with_quotes ``` The envelope provides interoperability and safety. The payload preserves the harness's native shape, avoiding a lowest-common-denominator interface that would erase useful capabilities. A returned envelope is equally uniform: ```yaml receipt: task_id: research-1 status: succeeded output: kind: evidence_bundle value: ... artifacts: [] observations: ["one source was inaccessible"] claims: [] provenance: [{source: https://..., quote: ..., retrieved_at: ...}] effects: [] validation: {checks: [...], passed: true} costs: {seconds: 41, network_calls: 6} next_actions: [] ``` `status`, provenance, authority, validation, cost, and identity are common. `value` is typed and non-interchangeable. --- ## 3. Decomposition: from intent to heterogeneous work graph ### Decomposition is planning, not classification A classifier asks: “Which harness owns this task?” Decomposition asks: “What must become true, what evidence is needed, what transformations connect them, and which execution model can perform each transformation?” The meta-harness should produce a DAG of nodes with explicit dependencies. Each node has: - a **verb** (inspect, research, implement, test, compare, decide, verify); - input and output types; - required capabilities and authority; - risk and reversibility; - acceptance checks; - dependencies and branch conditions. For each node, choose the *smallest capable harness*. A direct read or edit belongs in `pi`; a bounded synthesis across many inputs belongs in Prime Agent; external facts belong in Hermes. Do not ask Prime Agent to run `sed` merely because it can, and do not ask Hermes to infer facts from local unshared files. A useful planning rule is: > Decompose at boundaries where either the required capability, authority, state, output type, or verification method changes. ### Example: “Investigate a traffic drop, fix the dashboard query, and explain likely causes” A heterogeneous graph might be: ```text A pi: inspect repo/config and reproduce query B Hermes: research external incidents/API changes for the relevant dates C Prime Agent: correlate A + B + supplied analytics; rank hypotheses D pi: implement query fix E pi: run tests/query fixtures F Prime Agent: synthesize report from receipts and validation ``` `C` depends on evidence from `A` and `B`; `D` depends on the chosen fix, but should receive a narrow implementation contract rather than the entire research transcript. `F` depends on validated changes and retains the distinction between observed facts and hypotheses. ### Planning algorithm (conceptual) 1. Parse objective, deliverables, constraints, and implicit risks. 2. Identify required terminal artifacts/effects and their acceptance tests. 3. Expand each deliverable into capability-changing steps: acquire, transform, mutate, validate, communicate. 4. Type every edge (`repo_snapshot`, `evidence_bundle`, `implementation_plan`, `test_receipt`, etc.). 5. Assign harnesses by capability/authority fit and minimize privilege. 6. Add verification nodes, rollback points, and escalation conditions. 7. Execute independent read-only nodes in parallel; serialize mutations and conflicting state access. 8. Re-plan only from receipts and changed facts, not from opaque conversational state. The planner should support loops, but bounded ones: research may refine a hypothesis, and tests may cause one repair iteration. An unbounded “agentic” loop is not decomposition; it is loss of control. --- ## 4. Routing: dispatcher plus orchestrator, not a gating network The MoH router has two layers. ### Dispatcher The dispatcher is a deterministic capability/constraint matcher. It answers: *which harness is eligible for this node?* It checks: - capability required (`local_edit`, `stateful_synthesis`, `external_search`); - input/output type compatibility; - authority and sandbox boundaries; - availability and health; - latency/cost budget; - idempotence and concurrency constraints. A simple score may rank eligible choices, but it must not conceal hard constraints: ``` eligible(h, n) = capability(h,n) AND authority_allows(h,n) AND types_match(h,n) AND budget_possible(h,n) ``` Scores can then optimize speed, cost, or reliability. This is a policy engine, not learned gating. ### Orchestrator The orchestrator understands graph dependencies and lifecycle. It: - materializes inputs (snapshot, selected artifacts, or references); - invokes the dispatcher; - retries only safe/idempotent nodes; - records receipts and state transitions; - unlocks dependent nodes; - pauses on missing authority or ambiguous acceptance; - triggers verification and rollback; - asks Prime Agent for re-planning only when the graph is invalidated. Routing can be dynamic. If `pi` cannot reproduce a bug because the fixture is missing, the orchestrator may dispatch a bounded Prime Agent node to diagnose the fixture issue, then return to `pi`. If Hermes returns weak evidence, it should not silently route to “another expert”; it should mark evidence insufficient and invoke a specified fallback or ask the user. ### Harness health and admission Each harness exposes a capability manifest and health heartbeat: ```yaml name: pi version: ... capabilities: [read, write, shell, test] output_types: [file_diff, command_result, test_receipt] side_effects: [filesystem, subprocess] requires_confirmation: [delete, network, deploy] ``` The router should admit no task that lacks a declared output type, authority boundary, and acceptance check. This prevents “route by vibes.” --- ## 5. Synthesis: typed joins, not voting or averaging There is no single universal synthesis operation. MoH needs a small algebra of joins: 1. **Effect join:** combine non-conflicting file/process effects after validation; conflicts require serialization or explicit merge. A file diff is never “averaged” with another diff. 2. **Evidence join:** union claims while deduplicating sources and preserving quotes, timestamps, confidence, and contradiction edges. 3. **Analysis join:** Prime Agent transforms observations/evidence into hypotheses, rankings, plans, or decisions; these remain claims with uncertainty, not facts. 4. **Decision join:** apply policy and acceptance criteria to determine an action. The policy, not majority vote, resolves conflict. 5. **Artifact join:** package a report, patch, dataset, or receipt with lineage to its inputs. The key primitive is a **typed evidence/causality graph**: ```text source -> observation -> claim -> decision -> effect -> validation ``` Each edge carries provenance and confidence. A recommendation supported by one blog post cannot override a failed local test. A successful test cannot establish the external truth of a pricing claim. Different output types meet only through explicit transformation nodes. ### Example synthesis Hermes says three vendors document a 429 backoff policy. `pi` shows our client retries immediately. Prime Agent proposes exponential backoff with jitter. `pi` applies the patch and tests it. The final result should say: - **observed:** current client retries immediately (file/line and command receipt); - **external evidence:** sources and retrieval times; - **inference:** likely reduction in retry storms, with assumptions; - **effect:** exact diff; - **validation:** test names/results; - **residual risk:** production behavior not measured. Synthesis is therefore a controlled transformation and audit trail, not consensus among outputs. ### Conflicts Conflicts are first-class typed states: - file-vs-file conflict: stop and merge with a patch-aware tool; - source-vs-source contradiction: present both, rank by authority/date, request adjudication; - analysis-vs-acceptance conflict: acceptance test wins or task is blocked; - effect-vs-authority conflict: refuse, never negotiate authority implicitly. --- ## 6. Minimalist meta-harness: pi-like control plane Complexity should live in **data and policies**, not in a large always-on agent. A minimalist MoH can be a small executable with five primitives: ```text plan(contract) -> graph run(node, envelope) -> receipt join(receipts) -> typed artifact check(artifact, acceptance) -> verdict stop(reason) -> safe terminal state ``` Its core state is an append-only event/receipt log plus references to artifacts. It should not duplicate harness runtimes, hold all transcripts in memory, or make free-form decisions on every edge. Design principles: - **thin waist:** stable envelope, capability manifests, typed receipts; - **least privilege:** every node gets scoped authority and a short-lived context; - **lazy activation:** start Hermes/Prime Agent only when a graph node requires them; - **explicit policy:** routing, retry, confirmation, and conflict rules in declarative config; - **streaming artifacts:** pass snapshots, diffs, and evidence references rather than giant prompts; - **deterministic replay:** event log can reconstruct why a dispatch occurred; - **human gates:** require confirmation before irreversible mutation or low-confidence decisions; - **fail closed:** missing provenance, type mismatch, budget exhaustion, or failed checks stop the graph; - **bounded autonomy:** time, calls, depth, and repair iterations are hard limits. Prime Agent should be used as a *planner/synthesizer node* when needed, not embedded as the permanent supervisor of every command. For simple “rename this symbol and run tests,” the meta-harness should reduce to a `pi` invocation plus a test receipt—the same minimalist path as pi itself. --- ## 7. Concrete end-to-end example: “Research and implement a privacy-safe analytics change” User request: “Find current guidance on retaining event data, update our retention configuration, and give me a cited rationale. Do not deploy.” 1. **Contract formation:** deliverables are `evidence_bundle`, `file_diff`, and `decision_report`; authority permits network read, local read/write, tests; deploy is forbidden. 2. **Decomposition:** - `pi` inspect current config and tests -> `repo_snapshot`; - Hermes research regulator/vendor guidance -> `evidence_bundle`; - Prime Agent reconcile the snapshot and evidence into an `implementation_plan` with assumptions; - `pi` edit retention config -> `file_diff`; - `pi` run config validation/tests -> `test_receipt`; - Prime Agent produce final report -> `decision_report`. 3. **Dispatch:** Hermes and initial pi inspection run in parallel because both are read-only. The plan node waits for both typed outputs. 4. **Synthesis:** Prime Agent must not copy an external recommendation directly into config. It proposes a value and cites which requirement it satisfies. `pi` applies only the approved plan. 5. **Verification:** tests validate syntax and invariants. If tests fail, the orchestrator creates one repair node with the failure receipt; if still failing, stop with no claim of completion. 6. **Final package:** report includes diff, tests, citations, assumptions, and explicit “not deployed.” This sequence demonstrates why a single router choice is inadequate: the task is intrinsically a pipeline of unlike operations. --- ## 8. Safety, observability, and evaluation Every node should emit structured telemetry: queue/start/end times, harness/version, input artifact hashes, output hashes, tool calls, authority used, retries, and validation results. Secrets and full user data should be redacted at the envelope boundary. Evaluate MoH on more than task success: - decomposition quality (necessary nodes present; unnecessary work avoided); - type/authority violations (must be zero); - artifact correctness and test pass rate; - provenance completeness and citation quality; - conflict detection and safe stopping; - latency/cost per deliverable; - replayability and explanation of routing; - mutation containment and rollback success. A useful invariant is: **no effect is considered complete without a receipt and no claim is considered established without provenance or an explicitly labeled assumption.** ## Conclusion MoH is a heterogeneous orchestration architecture. Its intelligence lies in discovering a typed work graph, dispatching each node to the smallest harness with the required capability, and synthesizing through explicit transformations that preserve provenance and authority. `pi`, Prime Agent, and Hermes are not interchangeable experts. They are distinct execution substrates connected by a thin contract-and-receipt protocol. Thinking in MoE terms would produce a gate, a shared representation, and an illegitimate expectation of blendable outputs. Thinking in MoH terms produces a dispatcher, an orchestrator, typed artifacts, evidence lineage, policy-based conflict handling, and a minimalist control plane that invokes complexity only when the task actually requires it.