Harness Engineering in legacy: agent writes code

Harness Engineering: How an AI Agent Writes Production Code in a Legacy Monorepo

In Harness Engineering, humans define architectural boundaries, test scenarios, and review results, while the AI agent generates all the production code. An experiment on a legacy monorepo with 310,000 lines of Scala/Java proved the approach works for common tasks like extracting a module without changing behavior. The engineer's role shifts to managing context and process, with the agent handling the mechanical grunt work.

Monorepo: 25 SBT modules, 8 years of history, high-load system. Task: Extract Channel X webhook processing into a standalone service, following the pattern of existing adapters.

Experiment Setup and Roles

Humans don't write production code. Roles are clearly divided:

• Human: Architectural boundaries, task breakdown, requirements, tests, reviews, final validation.
• Agent: Code creation/changes, moving components, running scenario-based tests, infrastructure setup, bug fixes.

Typical task: Mechanical extraction with wiring and dependency adaptations, no business logic refactoring. Verifiable via unit and E2E tests.

Context Prep: Documentation as the Control Interface

Documentation becomes the tool: ADRs outline options, plans define stages. The agent sticks to the given frames, no guesswork.

Example ADR:

# ADR-42: Extract Channel X Webhook Processing from the Monolith

## Context
Channel X is currently handled inside the core. Other channels already run as separate services behind a message broker.

## Options
1. Thin wrapper: Move existing code with minimal tweaks (packages, wiring).
2. Rewrite the handler from scratch on a different stack.
3. Leave as is.

## Decision
Option 1: Minimizes loss of implicit behavior during extraction; aligns architecture with other channels.

## Consequences
+ Consistent pattern across all channels  
− Temporary code duplication; refactoring as a separate task

Staged plans cut errors: From skeleton to full E2E loop.

Stages and Prompts: Step-by-Step Agent Control

Breaking into steps ensures oversight. Prompt examples:

• Architectural Blueprint:

Study how Channel A and B adapters work: How they subscribe to the broker and publish incoming events.
Locate Channel X webhook handling in the monolith.
Write an ADR with three options: wrapper / rewrite / keep in core. Recommend wrapper and detail consequences.
Place files in architecture/adr/, don't touch code yet.

• Tests as Contract:

In WebhookHandler class, there's branching by chat type. Extract it to a pure function routeEvent(...).
Don't improve behavior—copy branches as-is. Write unit tests: private chat, group, unknown type, empty body.
Success: `sbt test` for the module passes green.

• New Module:

Create sbt module channel-x-adapter per SPEC in ADR-42.
Copy listed files from monolith; change only packages, imports, and deps for clean compile without `dependsOn(core)`.
Rule: No business logic refactoring this stage.

• E2E Loop:

Add e2e/ dir with pytest + testcontainers: Broker container, PostgreSQL, external API mock on separate port.
Write one test: POST webhook with "ping" text → Broker gets event on expected subject with text=ping.

Tests drive the agent: It aims for green sbt test or pytest.

Agent's Strengths: Routine Under Control

Agent shines at:

• Code extraction and adaptation.
• Replicating patterns from examples.
• Generating boilerplate.
• Building test infrastructure.
• Batches of similar edits.

Example E2E test with pytest + testcontainers:

# Fixtures: Broker container, adapter, external HTTP API mock

def test_incoming_webhook_publishes_to_broker(e2e):
    e2e.http.post("/webhook/channel-x", json={"message": {"text": "hello", "chat": {"id": 42}}})
    msg = e2e.broker.await_message("inbound.channel-x", timeout_s=5)
    assert msg["text"] == "hello"
    assert msg["chat_id"] == 42

def test_outgoing_command_reaches_external_api(e2e):
    e2e.broker.publish("outbound.channel-x", {"chat_id": "42", "text": "hi"})
    calls = e2e.external_mock.wait_for_requests(method="POST", path_contains="sendMessage")
    assert any(b"hi" in c.body for c in calls)

Verifies not just compilation, but integrated behavior: Broker, DB, API mock.

Limits and Boundaries

Agent doesn't replace architectural decisions but speeds up mechanics. Risks: Masked regressions (green tests hide issues), need for strict bounds. Suits legacy with patterns, not greenfield or R&D.

Devs may differ: Routine savings vs. context overhead.

Key Takeaways

• Harness Engineering: Controlled agent environment—rules, tools, feedback.
• Wins on code extraction/adaptation and test infra in legacy projects.
• Tests: Core control interface, not ritual.
• Boundaries: Routine tasks with examples, no architectural innovation.
• Industry: Anthropic (90% AI code), Meta, Amazon—shifting to AI-native dev.

— Editorial Team