Explicit Context Compaction Outperforms Naïve Large Windows
AutoCompact improved two coding benchmarks by 9.2 and 5.0 percentage points in one controlled scaffold, using jointly trained coding and compaction policies.
On 1 Oct 2026, AutoCompact reported a 9.2-percentage-point gain on SWE-bench Verified and a 5.0-point gain on SWE-PolyBench Verified. These are controlled coding-agent results, not a general ranking of compaction against larger context windows.
The researchers use Qwen3-Coder-30B-A3B-Instruct with a compact() action. During training-data collection, a judge corrects compaction timing, summaries and subsequent actions before execution. Supervised fine-tuning is followed by outcome-based reinforcement learning. The judge is removed at evaluation; the result combines model training with the compaction mechanism.
The paper's main table reports 30.4% to 39.6% on SWE-bench Verified and 19.5% to 24.5% on SWE-PolyBench Verified. The original version of this post incorrectly substituted approximately 57% to 66% and 61% to 66%. Those absolute pass rates were not the paper's results.
The authors test a 256K context setting where trajectories do not reach forced compaction, and a 16K forced-compaction threshold shared by the compared systems. Learned proactive compaction improves results across their evaluated inference-cost budgets. That does not mean a 16K budget simply runs out of memory, or that every larger context window is a losing engineering choice.
I think the useful design question is when an agent should replace exploration history with a working summary. Throwing away evidence too early can be as damaging as preserving stale hypotheses. This study gives a reason to test proactive compaction alongside an existing fallback, rather than treat context size as the only variable.
I would keep the reproduction narrow: the same scaffold, task distribution and cost accounting first. The paper leaves broader harness integration for future work. It does not establish that learned compaction will improve every non-coding task, or that developers relying on a million-token context will necessarily be disappointed.