CritICL: Inference-Time Weak-to-Strong Generalization from Small Language Model Failure Modes

arXiv · Yufan Wu, Yinghui He, Zhengyi Hu, Lang Wei, Ruichen Li, Qifan Yang, Ting Zhu · 2026-08-30

This is no longer on the current shelf — shelves rotate as new material clears the bar. The analysis below is unchanged. See what is featured now.

CritICL turns failure into fuel: it uses small model mistakes at inference time to supercharge large language models, bridging the gap between weak and strong reasoning.

What it does

CritICL introduces a novel inference-time technique that leverages the failure modes of smaller language models (SLMs) to improve the performance of larger models (LLMs) without additional training. The method works by first identifying where an SLM struggles (e.g., specific types of reasoning or factual inaccuracies) and then using those failure patterns to guide the LLM’s decoding process. This is done via a process called 'critical in-context learning' (ICL), where the LLM is prompted with examples of the SLM’s mistakes to avoid or correct. The result is a lightweight, training-free boost to LLM performance that targets specific weaknesses.

Why it matters

For AI practitioners, CritICL offers a way to squeeze more performance out of existing models without the cost of fine-tuning or larger deployments. This is especially valuable in scenarios where model size is constrained (e.g., edge devices) or where fine-tuning is impractical (e.g., proprietary models). The technique also provides a framework for systematically addressing model weaknesses, which is critical for building reliable agentic systems. In a world where 'LLM as a service' is becoming commoditized, methods like CritICL could be the difference between a mediocre and a market-leading product.

How it applies

CritICL can be integrated into RAG pipelines to improve factual accuracy by identifying and correcting common SLM hallucinations. For agentic systems, it can be used to dynamically adjust prompts based on the weaknesses of the underlying model, improving reliability in high-stakes scenarios. Teams building evals or benchmarks can use CritICL to generate targeted counterfactuals that expose model weaknesses, leading to more robust training data. The key is the focus on inference-time adaptation—something that’s becoming increasingly important as models grow too large to fine-tune frequently.

The short version

Your models are dumb. Here’s how to make them smarter without lifting a finger.

My take

Too many teams burn months and millions fine-tuning models to eke out a few extra points of accuracy, only to realize they’re still failing on the same edge cases. CritICL is a reminder that the real leverage isn’t in the training data—it’s in how you *use* the model at inference time. We’re entering an era where ‘model performance’ isn’t just about scale or data; it’s about *adaptive reasoning*. The fact that this works with off-the-shelf models is a huge win. It means you can start using it today, not next quarter. And for anyone building platforms, it’s a tool to make systems more reliable without adding complexity.

How it connects

• This work fits into the broader trend of inference-time optimization, like chain-of-thought or self-consistency, but with a focus on addressing specific weaknesses.
• It aligns with the push toward modular, composable AI systems where each component can be improved independently.

Bottom line

Start logging your model’s failure modes today—CritICL gives you a roadmap to turn those failures into performance gains without retraining.

Takeaways

• Inference-time adaptation can leverage small model failures to improve large model performance without additional training.
• Critical ICL uses examples of SLM mistakes to guide LLM decoding, effectively turning weaknesses into strengths.
• The method is lightweight and training-free, making it ideal for edge or constrained environments.
• It provides a systematic way to address specific model weaknesses, improving reliability in agentic systems.
• CritICL is a step toward 'self-improving' models that adapt at inference time based on observed failure patterns.

Topics: inference-time adaptation, weak-to-strong generalization, critical ICL, model reliability, reasoning