MAKE THE DECISION CONTRACT EXPLICIT: OBJECTIVE, CONSTRAINTS, EXAMPLES, AND OUTPUT.
PROMPT vs CONTEXT vs HARNESS vs LOOP vs GRAPH
LAYER
ENGINEERING
DESIGNING AND DEBUGGING THE AGENT STACK
five places a system can succeed or fail
They are responsibilities, not competing products. Trace the first incorrect artifact, then fix the smallest layer that produced it.
ONE DECISIONOBSERVED SYSTEM
COMPLEXITY IS A COST
NOT A CAPABILITY SCORE
Layer engineering outcomes
GIVE THE CALL A BOUNDED, CURRENT WORKING SET—NOT A DOCUMENT DUMP.
TRACE ARTIFACTS AND EVALUATE OUTCOMES BEFORE EXPANDING THE SYSTEM.
USE A GRAPH ONLY WHEN ROUTING OR HANDOFFS CREATE MEASURED VALUE.
Machine
(five responsibilities · one trace · compose only as needed)
└─ calls, tools, loops, deterministic steps, specialists, and humans connect through explicit handoffs when orchestration is justified
MODEL CALL
LAYER 2 · CONTEXT BREAKS WHENwrong, stale, or simply too much
FICTIONAL REFUND AGENT CURRENT ARTIFACT
- decision contract
- Determine eligibility; return decision, reason, and next safe action.
- working evidence
- Order, customer, policy version, payment state, and recent actions.
- runtime tools
lookup_order()and permission-gatedissue_refund().- control policy
- Look up, decide, act once, verify the receipt, then stop or escalate.
- exception route
- Normal refund path; fraud and policy exceptions go to specialist or human review.
RESPONSIBILITIES COMPOSE AS NEEDED — THEY ARE NOT A REQUIRED NESTING LADDER
LAYER 1PROMPT · THE DECISION CONTRACT
- Definition
- Instructions for the current model decision: objective, constraints, examples, and required output.
- Receives
- The current objective and available context.
- Owns
- THE DECISION CONTRACT: the objective, constraints, examples, and output schema.
- Produces
- A model decision or structured request for action.
- Does not own
- Missing facts, unavailable tools, or multi-step execution state.
- Refund example
- Decide refund eligibility and return decision, reason, and next safe action.
- Failure
- The answer uses supplied facts but solves the wrong task or violates the output contract.
- Inspect first
- Exact instructions and expected output for the failing call.
- Smallest fix
- Clarify the decision contract or schema; do not add orchestration.
LAYER 2CONTEXT · THE WORKING SET
- Definition
- Runtime information available for the current decision: retrieved evidence, state, memory, history, and observations.
- Receives
- Source records, current state, and retrieval results.
- Owns
- THE WORKING SET: facts, state, retrieved policy, and observations.
- Produces
- A bounded working set for one decision.
- Does not own
- Tool execution, retry policy, or routing among agents.
- Refund example
- Supply the order, customer, applicable policy version, payment state, and recent actions.
- Failure
- The model reasons plausibly from missing, stale, contradictory, or overwhelming facts.
- Inspect first
- Exact context sent on the failing call.
- Smallest fix
- Correct retrieval, freshness, selection, or representation.
LAYER 3HARNESS · RUNTIME & TOOLS
- Definition
- Software connecting model decisions to real operations. Harness is a useful engineering label; Runtime & Tools is the explicit name used here.
- Receives
- A structured tool request and the caller’s permission context.
- Owns
- RUNTIME & TOOLS: schemas, permissions, execution, and errors.
- Produces
- Validated execution result or explicit error.
- Does not own
- The agent’s global stopping strategy or multi-agent topology.
- Refund example
- Expose typed lookup and permission-gated refund operations.
- Failure
- The model chooses the right action but cannot express, authorize, execute, or interpret it.
- Inspect first
- Tool request, validated arguments, permission decision, result, and error.
- Smallest fix
- Repair the schema, permission, executor, timeout, or returned error.
LAYER 4LOOP · THE CONTROL POLICY
- Definition
- The cycle that observes, decides, acts, checks, and chooses whether to continue, stop, or escalate.
- Receives
- Current state plus the latest action result.
- Owns
- THE CONTROL POLICY: observe, decide, act, verify, and stop.
- Produces
- Updated state and a continue, stop, or escalate decision.
- Does not own
- Connections among several agents or workflows.
- Refund example
- Look up, decide, issue at most one refund, verify the receipt, then stop.
- Failure
- Repeated calls, premature completion, missing verification, or endless retry.
- Inspect first
- Trajectory, state transitions, step budget, and termination reason.
- Smallest fix
- Add or correct verification, state transition, budget, or stop condition.
LAYER 5GRAPH · Orchestration Topology
- Definition
- Connections among calls, tools, loops, specialist agents, deterministic steps, and humans. This is an orchestration graph—not a neural-network graph and not a knowledge graph.
- Receives
- A task or completed node artifact plus routing state.
- Owns
- ORCHESTRATION TOPOLOGY: route, branch, parallelize, join, and hand off.
- Produces
- A chosen route, parallel branches, joined result, handoff, or global completion.
- Does not own
- Ambiguous work inside a node, missing context inside a call, or a broken tool contract.
- Refund example
- Route normal refunds directly and send fraud or policy exceptions to qualified review.
- Failure
- Wrong specialist, duplicate branches, serial work, lossy joins, or incomplete handoffs.
- Inspect first
- Route decision, edge payloads, node outcomes, join, and global termination.
- Smallest fix
- Correct the route or handoff; simplify the topology when a graph adds no measured value. Compare the four topologies.
FOUR USEFUL TOPOLOGIES
(choose the smallest structure that earns its cost)
SINGLE CALL
What TASK → MODEL → RESULT.
Fits a well-specified transformation or decision.
Cost no autonomous recovery.
SINGLE AGENT LOOP
What OBSERVE → DECIDE → ACT → CHECK ↺.
Fits a coherent multi-step task sharing one state.
Cost needs budgets, verification, and a stop condition.
ROUTED WORKFLOW
What ROUTER → ONE SPECIALIST PATH.
Fits materially different domains or permissions.
Cost routing errors and handoff contracts.
FAN-OUT + REDUCE
What TASK → PARALLEL BRANCHES → REDUCER.
Fits independent work where coverage or latency improves.
Cost duplication, joins, global budgets, and compounding errors.
BUILD ORDER · NOT A MATURITY LADDER
ONE MODEL CALL
Start with the decision contract and working set.
RETRIEVAL + TOOLS
External facts or actions required?
CONTROLLED LOOP
Iterative action and verification required?
GRAPH
Routing, specialization, or parallelism measured to help?
A simpler architecture can be the correct final design.
CROSS-CUTTING SYSTEMS
(reliability belongs across every layer)
GUARDRAILS
Validate inputs, permissions, consequential actions, and outputs; require human review where impact demands it.
TRACING
Record prompt, context references, tool calls, state transitions, routes, handoffs, and termination reasons.
EVALUATIONS
Test components and end-to-end outcomes, including regressions in quality, tokens, latency, and cost.
TRACE THE FIRST INCORRECT ARTIFACT
(symptom → owning layer → smallest repair)
PROMPT
inspect exact task and output contract
fix repair instructions or schema
CONTEXT
inspect context sent on the call
fix retrieve the correct policy version
HARNESS
inspect arguments, permission, executor error
fix repair schema or permission boundary
LOOP
inspect state transition and termination reason
fix verify receipt and stop
GRAPH
inspect route decision and edge payload
fix correct route or handoff contract