#209 — Harness engineering

Harness engineering is the practice of designing feedforward constraints (guides) and feedback loops (sensors) around coding agents so they produce reliable, production-ready code. Smarter LLMs increase generation speed, but they do not possess institutional context, architectural intuition, or accountability. As documented by teams at OpenAI and Stripe, raw intelligence shifts your engineering bottleneck from writing code to human review toil unless the execution environment automatically enforces codebase conventions and self-correction.

Prompt engineering optimizes instructions inside a prompt window, and context engineering retrieves relevant tokens for that window. Harness engineering builds the persistent, cybernetic operating environment across both CPU and GPU tooling. It combines deterministic static tools (linters, AST codemods, language server protocols) with semantic evaluators to govern code before generation and trigger automated self-healing loops after execution.

Computational controls are deterministic, execute in milliseconds on a CPU, and cost virtually nothing (e.g., TypeScript compilers, ArchUnit boundary tests, ESLint). Inferential controls are probabilistic, run on GPUs or NPUs, and handle subjective nuance (e.g., LLM-as-a-judge reviews, semantic diff inspections). Early-stage engineering teams prioritize computational controls because they provide cheap, immediate failure signals that agents can reliably parse and fix in-session.

Allowing an agent to write both application logic and the corresponding test suite creates a circular validation loop where the AI writes flawed tests to validate its own flawed assumptions. Engineering teams mitigate this by separating the spec writer from the test author, introducing approved fixtures (curated snapshots of known good inputs and outputs), and using mutation testing (like Stryker or PIT) to verify whether test suites detect deliberately injected bugs.

Harnessability is the degree to which an environment provides ambient affordances (legibility and tractability) to an autonomous agent. Stacks with strong static typing (such as TypeScript, Go, or Rust), explicit module boundaries, and mature scaffolding tooling provide immediate feedback channels for agent self-correction. Dynamically typed, unconstrained codebases lack built-in computational rails, forcing you to rely on expensive and probabilistic inferential judges.

Harness templates are pre-packaged bundles of scaffolding, architectural fitness functions, and feedback sensors tailored to common system topologies like CRUD APIs or asynchronous event workers. Under Ashby's Law of Requisite Variety, an unconstrained agent will explore infinite ways to build a service. Templates restrict the solution space to a vetted pattern, allowing startups to scale microservices or features without accumulating inconsistent boilerplate.

OpenAI built layered architectures validated by custom linters and structural tests, backed by automated background 'janitor' agents that scan repositories for architectural drift and open remediation PRs. Stripe implemented Minions, an end-to-end coding agent framework using blueprint-driven feedforward constraints and predictive pre-push linter heuristics that feed diagnostics back to the agent before code reaches human code review.

Coined by Ned Letcher, ambient affordances describe structural properties of an environment that make code legible, navigable, and tractable for an autonomous agent. Monolithic frameworks with opinionated directory conventions (like Ruby on Rails or Django) and explicit language server protocols provide strong ambient affordances out of the box, whereas bespoke, loosely coupled microservices force teams to build heavy custom tooling just to make the repository interpretable.

Ashby's Law states that a control system can only regulate what it has a model of, and its regulator must possess at least as much variety as the disturbances it governs. Because modern LLMs can generate an infinite variety of code patterns, an unconstrained agent inevitably escapes quality control. Harness engineering acts as a variety-reduction mechanism, using rigid templates, API schemas, and architectural boundaries to keep the agent's problem space strictly within the regulator's capacity.

Instead of line-by-line syntax reviews, human engineers transition into control systems designers and architectural arbiters. Senior developers externalize their implicit taste, security assumptions, and domain constraints into explicit guides and sensors, while reserving manual review exclusively for high-stakes business logic, product trade-offs, and behavioral intent that automated tools cannot inspect.

Traditional unit test coverage metrics are easily gamed by LLMs that write low-value assertions simply to make coverage turn green. Mutation testing (which injects intentional bugs to see if tests fail) and structural tests (like ArchUnit or Dependency-Cruiser) provide objective computational verification that agent-authored tests actually catch regressions and strictly adhere to layer isolation boundaries.

Harness drift occurs when competing prompt instructions, stale configuration files, and outdated linter rules give contradictory guidance to an agent. High-performing engineering teams treat the harness itself as code: rules files (AGENTS.md) and sensor suites are versioned in Git, tested continuously against synthetic benchmark tasks, and pruned periodically using automated cleanup sweeps.