One Example Is All You Need: How 1-Shot RLVR Matches 7,500 Math Problems
Researchers from UW and Microsoft discover that RLVR on a single problem unlocks frontier reasoning, revealing the strange physics of post-saturation generalization.
The dominant narrative in frontier AI post-training is that scaling reasoning requires vast synthetic datasets: tens of thousands of verified step-by-step proofs, Monte Carlo tree searches, and curated math-and-code problem banks. A research paper from the University of Washington and Microsoft (Wang et al.) challenges this fundamental assumption.
Applying Reinforcement Learning with Verifiable Reward (RLVR) to an LLM using a single training example (1-shot RLVR) unlocks virtually the same mathematical reasoning capabilities as training on thousands of problems. On Qwen2.5-Math-1.5B, running RLVR on just one math prompt catapults MATH500 performance from 36.0% to 73.6%—matching the score achieved by training on the 1,200-example DeepScaleR subset, and trailing fully supervised baselines by less than two percentage points.
Adding just a second example pushes performance to 74.8% on MATH500 and 36.6% average across six competitive benchmarks, exceeding the 1.2k-example dataset. This revelation forces a fundamental rethinking of what reinforcement learning actually does during post-training.
The Benchmark Shock: 1 Problem vs. 7,500 Problems
To evaluate the limits of data efficiency in reasoning, the researchers evaluated models across six common mathematical reasoning benchmarks.
Using Group Relative Policy Optimization (GRPO) on Qwen2.5-Math-1.5B, they trained the policy exclusively on one duplicated question-answer pair. The results were striking:
- Zero-Shot Base Model: 36.0% on MATH500, 17.6% 6-benchmark average.
- Format Reward Baseline: 65.0% on MATH500 (fixing extraction tags and output formatting).
- 1-Shot RLVR (Single Example): 73.6% on MATH500, 35.7% 6-benchmark average (an 8.6% absolute gain beyond format correction).
- 1,200-Example DeepScaleR Subset: 73.6% on MATH500, 35.9% 6-benchmark average.
- 2-Shot RLVR (Two Examples): 74.8% on MATH500, 36.6% 6-benchmark average.
This outcome was not an isolated quirk of one base model or algorithm. The team observed identical dynamics when scaling to Qwen2.5-Math-7B, switching to Llama-3.2-3B-Instruct and DeepSeek-R1-Distill-Qwen-1.5B, and swapping between GRPO and Proximal Policy Optimization (PPO).
MATH500 Benchmark Performance Comparison:
[Base Model] 36.0% |====================
[Format Reward Only] 65.0% |====================================
[1-Shot RLVR (1 Example)] 73.6% |=========================================
[DeepScaleR (1.2k Ex.)] 73.6% |=========================================
[2-Shot RLVR (2 Examples)] 74.8% |==========================================
[Full MATH Train (7.5k)] 75.2% |==========================================
The Discovery of "Post-Saturation Generalization"
In standard Supervised Fine-Tuning (SFT), training on a single data point causes immediate catastrophic overfitting. The loss drops to zero, the model memorizes the exact sequence, and downstream generalization collapses within a handful of gradient steps.
In 1-shot RLVR, the exact opposite occurs through a phenomenon the authors call post-saturation generalization:
- Early Phase (Steps 0–50): The model explores candidate rollout paths for the single prompt. Training reward reaches 100% (accuracy saturation on the training example).
- Post-Saturation Phase (Steps 50–500+): Even though the training accuracy on that single example is completely saturated at 100%, test accuracy across unseen benchmarks (calculus, geometry, number theory) continues to climb steadily for hundreds of subsequent steps.
Why does this happen? Unlike SFT, which forces the model to memorize a specific token stream, RLVR with group normalization computes advantages across K sampled trajectories per batch. Because generation remains stochastic (non-zero temperature), the model continues exploring alternative reasoning topologies for that same question—discovering more robust verification routines, error correction strategies, and optimal calculation steps.
The Engine: Exploration and Entropy Regularization
The researchers proved that 1-shot RLVR is fundamentally different from grokking (the delayed generalization phenomenon seen in overparameterized supervised networks via weight decay). Instead, the engine powering 1-shot RLVR is the interaction between the policy gradient objective and entropy regularization.
- Without Entropy Loss / Exploration: When training without an entropy bonus or with greedy sampling, the policy rapidly collapses into a single deterministic trajectory. Exploration ceases, and test performance plateaus immediately.
- With Entropy Regularization: Adding an explicit entropy loss bonus (encouraging rollout diversity) forces the model to continuously sample alternate reasoning paths even after finding a valid solution. The policy gradient systematically rewards structural reasoning behaviors—such as branching, intermediate consistency checks, and self-reflection—while suppressing degenerate reasoning patterns.
Crucially, training on a single algebra problem spontaneously improved the model's performance on geometry, combinatorics, and logic puzzles. The model began naturally generating reflection tokens—such as "Wait, let me double-check this step..." and "Alternatively, let's verify if the denominator can be zero"—without receiving a single human demonstration of Chain-of-Thought (CoT) self-correction.
Isolating True Reasoning from Format Elicitation
A common skepticism in RLVR research is whether reinforcement learning merely acts as an expensive format parser—teaching the model to place answers inside \boxed{} tags rather than enhancing actual intellect.
To rigorously test this, the researchers constructed a format-only reward baseline that rewarded proper syntax and extraction tags without scoring mathematical validity. While format alignment lifted the base model from 36.0% to 65.0%, 1-shot RLVR added an additional +8.6% absolute gain purely in mathematical precision, step-by-step logic, and computational accuracy.
Key Takeaways for AI Architects and Post-Training Teams
- RLVR Elicits, It Does Not Teach World Knowledge: RLVR does not inject external facts into an LLM; it is an optimization operator that restructures the latent reasoning pathways already embedded during web-scale pretraining. A single verifiable anchor problem provides sufficient gradient signal to activate those dormant circuits.
- Data Quantity Is Overrated in Reasoning Post-Training: For teams spending millions curating massive synthetic RL datasets, the bottleneck is rarely problem volume. It is the exploration mechanism, the reward verification harness, and the entropy schedule.
- 1-Shot RLVR Is the New Post-Training Baseline: Any future RLVR algorithm or dataset claiming breakthrough reasoning gains must be benchmarked against a 1-shot baseline to prove that its multi-thousand-example dataset provides actual value beyond raw latent elicitation.
Sources
The stories that matter for people who build with AI, with the source for every claim. Free, no spam.
Feed, daily deep-dive and bytes — readable offline, with push alerts for the topics you follow.