AI Harness Engineering

Section 1: Anatomy of a Harness

• How a harness differs from a framework, and who owns the control flow

• The components of an agentic AI system: model, loop, tools, context and policy

• Dissecting commercial coding agents to see what each part does

• Configuring the Claude Code and Cursor setups used in class to work from specs

Section 2: The Agent Loop

• The observe, decide, act and verify cycle of an agent

• Setting step budgets and token budgets per task

• Stop conditions and detecting loops that never finish

• Structured tracing so you can replay what the agent decided at each turn

Section 3: Workshop: Building the Agent Loop from a Spec

• Write a loop spec with explicit step and token ceilings

• Define the stop conditions and the trace format you need

• Have a coding agent implement the loop in async Python

• Review the result, find design flaws the tests miss, and direct the fixes

Section 4: The Tool Layer

• Tool descriptions are part of the prompt and must be designed deliberately

• Schemas precise enough that the model never has to guess parameters

• Error surfaces that state the cause and the fix so the agent can self-correct

• Idempotency for side-effect tools, and truncating oversized results before they return to the context

Section 5: Workshop: File Read, File Edit and Run Command Tools

• Write specs for file read, file edit and run command tools

• Design error messages the agent can act on

• Have the coding agent build the tools and wire them into the loop from the first workshop

• Test failure cases and very large outputs

Section 6: Hooks and Middleware

• Policy belongs in the runtime, not in the prompt

• Deterministic lifecycle hooks that run before and after tool calls

• Ordering a middleware chain and what changes when the order changes

• Workshop: have the coding agent build a middleware chain from a spec without touching the loop code

Section 7: Context Engineering

• Why long-running agents gradually degrade

• Tool-result clearing: removing old results that are no longer needed

• Compaction: condensing history into structured blocks

• External memory that survives a context reset

• Why all three are needed together, and how to measure the trade-offs

Section 8: Workshop: Implementing Context Engineering

• Write specs for tool-result clearing, compaction and external memory

• Have the coding agent build all three and connect them to the harness

• Measure before and after with traces, both token counts and task quality

Section 9: Subagents and Context Isolation

• Isolating the context of each subagent

• Short return contracts with pointers to the full details

• The risks when each part reduces context on its own without coordination

• Deciding when to split work into subagents and when not to

Section 10: Sandboxing and Recovery

• Limiting filesystem, network and process access with Docker

• Retries that do not repeat work that already succeeded

• Checkpoint and resume with SQLite for long-running tasks

• Cost caps and kill switches when an agent exceeds its limits

• Exporting traces with OpenTelemetry to follow failed runs

Section 11: Agent SDK Scope and Moving to Other Domains

• Comparing an Agent SDK with a custom-built harness

• Criteria for extending what exists versus rebuilding

• Evaluating and adapting Agent SDK components

• Carrying the knowledge to customer service, data and RAG agents

Section 12: Capstone: A Coding Agent on a Real Repository

• Use the harness and a coding agent on a real repository

• Review the agent's work from an architectural point of view

• Present three decisions that contradict the agent's suggestions, with reasons

• Summarize how to bring the harness into the team's own work