CLASHarness: Benchmarking Multi-Agent Conflict in Computer Use via a Controllable and Verifiable Generation Pipeline

arXiv · Yuxiang Lai, Jike Zhong, Chengzhi Liu, Ming Li, Haonan Wang, Xiaofeng Yang, Liang Zhao, Youbao Tang, Ruixi Fan · 2026-10-13

Introduces CLASHarness, a framework for generating and benchmarking verifiable multi-agent conflicts in computer-use workflows.

What it does

Creates 600 conflict rounds using hierarchical scenario planning and an eight-step verified micro-pipeline, and evaluates 15 models. The paper reports that the tested GPT and Claude configurations highlighted in its analysis fail over 58% of required closing-action checks. That is a benchmark result, not a production incident rate.

Why it matters

Targets conflict-heavy scenarios that cooperative benchmarks can miss. It evaluates logs and workspace traces rather than live multi-agent conversations, so it does not establish production-wide failure rates.

How it applies

Test if a multi-agent system can recover when a sub-agent provides a state that contradicts the actual visual environment; limit this test to tasks where ground truth is visually verifiable.

The short version

Investigating conflicting reports was easier in these scenarios than completing the required state changes.

My take

The closing-action failures are the finding I would turn into an evaluation case. I would check whether a system completes its required handoff after diagnosing a conflict, rather than treat this benchmark's percentage as a production failure rate.

How it connects

• When a workflow makes authoritative evidence available only in screenshots, test whether agents actually inspect and use it instead of trusting a peer report.
• A remediation-focused training trial is one possible follow-up; the paper does not demonstrate that fine-tuning fixes the observed gap.

Bottom line

Test closing handoffs and time-window compliance separately. Closing actions were a weak point in this benchmark, but time-window checks had higher failure proportions.

Takeaways

• Within this benchmark, stronger models were more likely to consult visual evidence when peer reports conflicted with it.
• In the treated benchmark scenarios, task pass rates were lower when agents had to resolve conflicts than when they only investigated them.
• The tested GPT and Claude configurations highlighted by the authors failed over 58% of required closing-action checks; time-window compliance was also a major weakness.
• CLASHarness checks executed reference solutions and screens textual leakage before accepting generated rounds.

Topics: Multi-Agent Systems, VLMs, Benchmarking, Computer Use