Opera: A Verbal Critic Framework for Long-horizon Coding Agents

arXiv · Kai Mei, Zhiyuan Hu, Yutong Dai, Juntao Tan, Yifan Zhang, Dingjie Song, Dimitris N. Metaxas, Silvio Savarese, Ran Xu, Zeyuan Chen · 2026-09-27

Opera tracks a critic’s corrections until an issue is resolved, separating compliance from resolution.

What it does

Opera is a verbal critic that keeps a diagnosed issue as a persistent note and tracks the agent’s subsequent work. Periodic/event-driven review, typed diagnosis and evidence checks distinguish acting on feedback from resolving the issue.

Why it matters

The paper reports gains of up to 12.4, 15.0 and 8.9 percentage points on its Terminal-Bench 2.1, SWE-Bench Pro subset and DeepSWE v1.1 comparisons across four policies. These results support a tested tracking mechanism, not a universal solution to agent reliability.

How it applies

Compare a persistent-note critic against the current review loop with the same tasks and policy. Track resolution, not only compliance. The reported approximately on-policy training rollouts improve Qwen3.5-9B by 10.2 percentage points on held-out SWE-Bench Pro repositories without a critic at inference.

The short version

Did the agent follow a correction, or resolve the issue?

My take

Evaluate whether persistent tracking improves actual resolution in the target loop. The training result concerns Opera-guided rollouts; it does not prove that bad corrections are useful.

How it connects

• Stateful issue tracking
• Evidence-checked feedback
• Inference critique versus training rollouts

Bottom line

Measure resolution of tracked issues against the existing critic at a matched budget.

Takeaways

• Persistent notes distinguish delivered feedback from resolution.
• Review and diagnosis use visible evidence.
• Benchmark gains depend on the tested task and policy.
• Training rollouts are approximately on-policy, not a guarantee about arbitrary corrections.

Topics: agents, coding, evals, self-correction