From LLM Calls to Agent Harnesses

Source and evidence

This lesson summarizes the engineering essay 从一次 LLM 调用到完整 Harness，Agent 到底经历了什么？, credited to ivanxxie and davoszhang / 腾讯程序员 (Tencent), dated September 7, 2026 by the republication. Original WeChat article · Credited republication used for reading.

The original WeChat page was unavailable during preparation. The summary was prepared from the credited republication. This is an engineering essay, not an arXiv paper. The tutorial, diagram and executable fixture below are Token Lab teaching material; they are not measurements of the systems discussed in the essay. Technical references checked October 25, 2026.

The article in 150 words

A useful agent is more than a language model that can call a tool. The article describes how a single model call grows into an operational system as tasks become longer and actions acquire consequences. Context must be assembled; tools need interfaces; observations must feed subsequent decisions; and work must survive beyond one context window. Permissions, isolation, persistent state and recovery then become engineering responsibilities rather than prompt-writing details.

The central distinction is between a model that proposes what to do next and a harness that organizes and constrains execution. The essay compares different agent implementations as different choices about where that surrounding machinery belongs. Its practical lesson is to understand the failures each component addresses instead of treating a framework’s feature list as an architecture. For a learner, the useful question is: which state, boundary or recovery mechanism is missing when this agent fails? Benchmark figures in the republication are not reproduced here.

1. Before you start

Prerequisites: functions, JSON, basic tool calling and the difference between model context and application state. You do not need model weights or an API key.

• Explain which decisions belong to the model and which checks belong to the runtime.
• Implement an execution gate and test denied actions, budget exhaustion and replay.
• Design a fair harness comparison with fixed model, task set and success criteria.

Predict first: an approved edit succeeds, then the client retries it with the same ID. Should the harness execute it again? What if the arguments changed?

2. A proposal is not an execution

Imagine a coding assistant that inspects a failing test, proposes a file edit and runs the test again. “The model decided to patch” and “the patch was authorized and applied” are different events. Tool output is untrusted input for the next decision; it must not grant new permissions simply because it contains instructions.

3. How this connects to real systems

Long-running coding: Anthropic describes an initializer and subsequent coding sessions that use progress artifacts, repository state and end-to-end testing. The useful engineering connection is an explicit handoff: a fresh context should recover what was done and what remains. This does not establish that more agents always perform better. Read the Anthropic engineering account →

One runtime, different interfaces: OpenAI’s Codex platform article describes a harness exposed through app-server concepts including threads, turns, events and approvals. A CLI and a custom application can sit above the same execution machinery. For your own application, separate UI state from the durable task state and permission decisions the runtime must own. Read OpenAI’s platform explanation →

Policy is explicit: OpenCode documents allow, ask and deny permission decisions. These are useful categories for a runtime gate. A policy decision alone is not an operating-system isolation boundary: restricting what a process can access is a separate mechanism. OpenCode permissions → · OpenAI’s sandbox and execution discussion →

4. Run a bounded harness locally

This deterministic fixture runs entirely in your browser. It does not call an LLM, upload inputs, execute shell commands or modify real files. A patch increments an in-memory counter. No dependencies or randomness are required; the same inputs give the same results.

Expected result: seven PASS checks, one executed action and one simulated write. Replay returns the recorded identity without performing the effect again.

Ready. Predict the replay outcome, then run the experiment.

Download complete dependency-free implementation (.mjs)

// Deterministic teaching fixture. No model, network, filesystem or OS sandbox.
export function createHarness(snapshot) {
  const state = snapshot ? JSON.parse(JSON.stringify(snapshot)) : {used:0, writes:0, cache:{}, events:[]};
  function step(call, {budget=3, approved=false}={}) {
    const emit = status => {state.events.push({id:call?.id,status}); return status;};
    if (!call || !/^[a-z0-9-]+$/.test(call.id || '') ||
        !['read','patch','shell'].includes(call.tool) || typeof call.input !== 'string') return emit('invalid');
    const fingerprint=JSON.stringify([call.tool,call.input]);
    if (Object.hasOwn(state.cache,call.id)) {
      return emit(state.cache[call.id]===fingerprint ? 'replayed' : 'id-conflict');
    }
    if (call.tool==='shell') return emit('denied');
    if (call.tool==='patch' && !approved) return emit('approval-required');
    if (!Number.isInteger(budget) || budget<0 || state.used>=budget) return emit('budget-exhausted');
    state.used++;
    if (call.tool==='patch') state.writes++;
    // Both effect and cache live in the same synchronous in-memory fixture.
    state.cache[call.id]=fingerprint;
    return emit('executed');
  }
  return {state,step};
}
export function run() {
  const h=createHarness(), results=[];
  const check=(label,ok)=>{if(!ok)throw Error(label);results.push({check:label,result:'PASS'});};
  const patch={id:'edit-1',tool:'patch',input:'fix fixture'};
  check('Unapproved patch has no effect',h.step(patch)==='approval-required' && h.state.writes===0);
  check('Denied tool consumes no execution budget',h.step({id:'cmd-1',tool:'shell',input:'anything'})==='denied' && h.state.used===0);
  check('Explicit approval permits one effect',h.step(patch,{approved:true})==='executed' && h.state.writes===1);
  const resumed=createHarness(h.state);
  check('Checkpoint replay does not duplicate effect',resumed.step(patch,{approved:true})==='replayed' && resumed.state.writes===1);
  check('Reused ID with changed arguments is rejected',resumed.step({...patch,input:'different'},{approved:true})==='id-conflict');
  check('Budget checked before execution',resumed.step({id:'read-1',tool:'read',input:'fixture'},{budget:1})==='budget-exhausted' && resumed.state.used===1);
  check('Unknown tools fail validation',resumed.step({id:'x',tool:'unknown',input:''})==='invalid');
  return {checks:results, state:resumed.state};
}

Reproduce with Node.js 22 or later, from the folder containing the downloaded file:

node --input-type=module -e "import {run} from './agent-harness.mjs'; console.log(run())"

Boundary: the fixture counts executed actions, not model tokens or wall time. It does not implement an autonomous model loop, real storage, concurrent requests or an OS sandbox. Its JSON checkpoint models a clean handoff. A crash after a real external effect but before a durable cache write can still cause duplicate execution. Exactly-once effects require support such as transactional state changes or idempotency keys at the external service.

5. Exercises: explain, modify, investigate

A graduate-level ablation

Hold model version, prompt, tool implementations, task fixtures and total budget fixed. Compare a baseline loop against a loop with checkpoints, then add permission enforcement. Pre-register acceptance tests and inject failures at specified boundaries. Use development tasks for tuning and held-out tasks for the final report.

Report task counts, seeds where applicable and uncertainty across repeated runs. Do not infer a general capability improvement from these seven deterministic unit checks. Memory compression can lose constraints; delegation can introduce coordination failures. These are research questions to test, not benefits guaranteed by adopting a framework.

Continue learning

Inference budgets and verification → · Other frontier lessons →