SWE-Prime: Fewer Trajectories, Better Performance
This is no longer on the current shelf — shelves rotate as new material clears the bar. The analysis below is unchanged. See what is featured now.
SWE-Prime proves that not all successful agent trajectories are equal—and filtering them leads to better fine-tuning outcomes.
What it does
SWE-Prime introduces a trajectory filtering framework that selects high-quality samples from successful agent trajectories for SFT. It identifies and removes ineffective, redundant, or risky steps from trajectories, ensuring the fine-tuned model learns only optimal behaviors. The paper demonstrates that fewer, higher-quality trajectories can outperform larger, noisier datasets. It includes a new benchmark (SWE-Prime) for evaluating code agents.
Why it matters
For AI practitioners, this challenges the 'bigger dataset = better model' dogma. It shows that data quality trumps quantity in agent fine-tuning. The approach is especially relevant for safety-critical systems (e.g., coding assistants) where risky steps must be avoided.
How it applies
Use SWE-Prime’s filtering pipeline to preprocess your agent trajectories before SFT. Apply it to any agentic system where success is noisy (e.g., coding, web browsing, or tool use). The benchmark can also guide your evaluation strategy.
Takeaways
- Success ≠ quality: Filter trajectories to remove noise, redundancy, and risks.
- Fewer, better trajectories can outperform larger, noisier datasets.
- SWE-Prime provides a new benchmark for evaluating code agents.
- Applicable to any agentic system where trajectory data is available.
- Reduces the cost of fine-tuning by focusing on high-value samples.
The short version
Your agent’s training data is full of lies—here’s how to find the truth.
My take
This is a live problem for systems like Brendon.BOT. Agents trained on thousands of trajectories inherit data where most trajectories are suboptimal or even dangerous (e.g., agents deleting files or spamming APIs). SWE-Prime’s filtering pipeline belongs in any serious training stack. It’s not just about code agents—it’s about any system where actions have consequences. The key insight? Treat your training data like a financial portfolio: diversify, but always optimize for risk-adjusted returns.
How it connects
- Connects to research on data pruning (e.g., 'The Curse of Dataset Size' in LLM training).
- Parallels work on safety alignment (e.g., RLHF filtering risky behaviors).
- Overlaps with agent evaluation (e.g., 'Trajectory-level rewards' in agent benchmarks).
Bottom line
Before fine-tuning your agent, run SWE-Prime’s filtering pipeline—you’ll save time, money, and headaches.
Brendon Score: 8.5/10
- Relevance: 9.0/10 — +2.25
- Depth: 8.0/10 — +2.00
- Actionability: 9.0/10 — +2.25
- Freshness: 8.0/10 — +2.00
- Total (average): 8.5/10