Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation

arXiv · Hermann Kumbong, Ishan S. Khare, Iddah Mlauzi, Andrew Woen, Charles Ding, E. Kelly Buchanan, Simran Arora, Azalia Mirhoseini, Christopher Ré · 2026-10-09

Coarse-to-fine generation improves speed against the tested diffusion baselines, with quality gains or parity depending on the task.

What it does

Next-Concept Modeling (NCM) encodes text into a hierarchy of discrete codes via a multiscale residual VQ-VAE, then predicts all codes within a level in parallel. On LM1B, it achieved better generative perplexity than the strongest DLM baseline while running 21x faster; on XSum, it matched the strongest DLM's ROUGE-L while being 31x faster.

Why it matters

The reported speedups concern the tested benchmarks. Dataset-specific tokenizers and fixed hierarchy depth remain practical constraints, including for short continuations.

How it applies

Use the authors’ evaluation code at https://github.com/HazyResearch/ncm as a starting point. Compare quality, latency and sample diversity on a relevant task before judging deployment fit.

The short version

Parallel coarse-to-fine generation shows benchmark-specific speed gains, with quality gains or parity.

My take

My takeaway is to compare complete generation pipelines, including tokenizer cost, output quality and sample diversity. Sudoku results motivate replication; they do not establish broader reasoning gains.

How it connects

• The method builds on hierarchical image-generation ideas; transfer to other workloads still needs evaluation.
• A useful trial could compare NCM with speculative decoding on the same workload and quality target. That comparison is not demonstrated here.

Bottom line

Evaluate research-scale NCM on your own workload before treating its benchmark results as deployment advantages.

Takeaways

• Reported speedups against particular diffusion baselines range from 10x to 31x across the tested datasets.
• Outperforms both autoregressive and diffusion baselines on Sudoku reasoning tasks.
• Uses a two-stage process: hierarchical encoding followed by level-by-level parallel generation.
• Quality gains or parity depend on the task and comparison; faster sampling does not establish uniformly better quality.

Topics: Language Modeling, Diffusion Models, Parallel Generation, VQ-VAE