ESPO beats GEPA on prompt length and accuracy by stabilizing selection

2026-09-04

On seven NLP benchmarks, ESPO improves average accuracy by 3.76 points over GEPA with prompts 47% shorter. Its Tweet diversity-only ablation drops 1.20 percentage points; the largest Qwen3 result starts from a deliberately weak prompt.

ESPO (arXiv 2609.04197) was submitted September 3, 2026, and it attacks a problem every practitioner who has tuned prompts knows too well: evolutionary optimizers like GEPA keep appending rules, caveats, and constraints until the prompt is three times longer and no more accurate.

The authors trace this bloat to three specific deficiencies — incomplete error observation, limited search diversity, and unreliable selection — and propose a three-phase pipeline. Diagnose clusters all training errors into structural patterns in a single pass. Propose generates candidates through four complementary strategies, each with independent biases. Select applies bootstrap stability selection to filter out noise before committing to a prompt. On seven public benchmarks — Tweet, MMLU, GSM8K, HotpotQA, ScoNe, HoVer, and PUPA — ESPO hits 74.67% average accuracy against GEPA's 70.91%, a 3.76-point gain, while cutting prompt length from 1,878 characters down to 1,004.

Here is what I think actually matters: in the Tweet ablation, the diversity-only configuration scored 67.40% versus the GEPA baseline’s 68.60%, a 1.20-percentage-point drop. That tested configuration cautions against assuming extra candidates will help without reliable selection. It is not evidence that exploration is always harmful or that every pipeline without this stability gate wastes compute. I would compare selection methods under the intended task and evaluation budget.

The cross-model results show the best average accuracy among the tested optimizers on Gemma 3 12B, Mistral 14B, Qwen3 32B, and Claude Haiku 4.5. On Qwen3 GSM8K, the deliberately weak default prompt scored 15.00%, GEPA scored 35.40%, and ESPO scored 91.40%. The 15.00-to-91.40 comparison is against the default prompt; the GEPA-to-ESPO gain is 56.00 percentage points. Retaining that baseline matters when estimating the benefit over an existing tuned prompt.

I do not know whether these prompt-level optimizations survive deployment in production systems where the data distribution shifts daily. The paper's generalization bound lives in the appendix and grounds each phase in a corresponding term of the test-time gap, which is mathematically clean, but I would want to see this tested against real traffic, not held-out benchmark splits.

The bigger open question is whether Diagnose-Propose-Select generalizes beyond prompts. If the core insight is that structured error clustering plus a stability filter beats unstructured search, that pattern could apply to code generation, tool calling, or any domain where an optimizer is appending noise to a template. But that is my speculation, not the paper's claim. They tested prompts on NLP benchmarks and that is the scope of what I can defend.