research / Published
The system around the model deserves its own tests
A model does not work alone. The surrounding system decides which tools it can use, where it keeps state, how it hands work to another agent and what counts as finished.
Reading: Natural-Language Agent Harnesses, v2
Natural-Language Agent Harnesses asks whether that system's policy can become an explicit document rather than stay buried in controller code. The paper pairs those documents with a runtime that turns them into agent calls, handoffs, checks and saved outputs.
What the paper reports
The authors report comparable task outcomes across several benchmark families while making the policy easier to inspect.
Interpretation
The distinction I find useful is between policy and mechanism. A document can describe how work should proceed. Code still needs to enforce exact behavior such as parsing, tool execution and isolation. Writing a clear rule is not proof that the system follows it.
For our own work, this suggests a useful test: can we show where a rule affects the run, and can we detect when it fails? That seems more valuable than judging an agent workflow by how thorough its instructions look.
Limits and open questions
Natural-language ambiguity remains a risk. Exact mechanisms need code enforcement.
Sources and context
- Paper
- Natural-Language Agent Harnesses, v2
- Paper authors
- Linyue Pan, Lexiao Zou, Shuo Guo, Jingchen Ni, Hai-Tao Zheng
- Read at