7M Parameters Beat Frontier LLMs: Inside the Tiny Recursive Model
Samsung SAIL's TRM strips away billions of parameters and verbal Chain-of-Thought, matching frontier models on ARC-AGI using a 2-layer network with recursive latent updates.
The dominant paradigm in artificial intelligence assumes that genuine reasoning requires scaling parameters into the hundreds of billions, training on trillions of tokens, and generating voluminous Chain-of-Thought (CoT) traces. A striking paper from Alexia Jolicoeur-Martineau at Samsung SAIL Montréal, titled "Less is More: Recursive Reasoning with Tiny Networks," delivers a direct challenge to that consensus.
Introducing the Tiny Recursive Model (TRM), the research proves that a single two-layer neural network with just 7 million parameters can outperform massive frontier models—including DeepSeek-R1, OpenAI's o3-mini, and Google's Gemini 2.5 Pro—on the Abstraction and Reasoning Corpus (ARC-AGI), using less than 0.01% of their compute and parameter footprint.
TRM does not use pretraining corpora, does not output English reasoning steps, and requires only ~1,000 training examples. Instead, it re-engineers how computational depth is delivered during inference.
The Pathology of Autoregressive LLM Reasoning
Frontier LLMs reason through autoregressive token generation. This architecture suffers from three structural flaws when tackling abstract logic puzzles and formal spatial reasoning:
- Error Cascading: Autoregressive generation is strictly left-to-right. If a model generates an erroneous token at step 12 of a 500-token CoT trace, the entire downstream computation condition builds upon that hallucination with no native mechanism to roll back or correct intermediate latent states.
- Verbalization Bottleneck: Forcing a model to serialize non-verbal, multidimensional logic (such as spatial transformations, grid rotations, and constraint satisfaction) into natural language tokens introduces severe representational overhead and brittle heuristic parsing.
- Compute-Parameter Coupling: In standard transformers, computational depth is strictly bound to parameter count. To apply 80 sequential layer transformations, a model must instantiate and load 80 distinct parameter layers into VRAM.
While recent breakthroughs like Wang et al.'s Hierarchical Reasoning Model (HRM) demonstrated that small recurrent models (27M parameters) could solve complex Sudoku and ARC grids using two coupled sub-networks operating at different frequencies, HRM's dual-network architecture was complex, hard to stabilize, and relied heavily on biological analogies of brain wave oscillations.
TRM strips that complexity down to its mathematical core.
How TRM Works: Recursive Latent Refinement
TRM eliminates the dual-model hierarchy entirely. It uses one single, compact 2-layer network that iteratively refines both an internal latent representation z and a predicted output tensor y across two nested loops.
Input: Problem x, initial latent z₀, initial answer y₀
For step = 1 to N_sup (Deep Supervision / Refinement Steps):
# Inner Loop: Recursive Latent Reasoning
For iter = 1 to n (e.g., n = 6):
z = f_θ(x, y, z)
# Outer Update: Answer Generation
y = g_θ(y, z)
# Gradient Detach
Detach computation graph of z and y
1. The Inner Loop: Pure Latent Reasoning
Given problem tensor x, current answer guess y, and current latent state z, the network executes n consecutive recursive passes (typically n = 6):z_{k+1} = f_θ(x, y, z_k)
Because the weights θ are shared across every recursion pass, the model achieves deep algorithmic computation without adding a single extra parameter. Rather than expressing thoughts in text tokens, the network updates its hypothesis directly in continuous latent space.
2. The Outer Loop: Deep Supervision and Truncated Residuals
Once the latent state z has stabilized after n recursive steps, an output head updates the concrete solution y based on the refined (y, z) representations. This outer process repeats for up to N_sup = 16 supervision steps.
Crucially, TRM applies deep supervision with detached computation graphs: gradients do not backpropagate all the way back to step 1. By detaching z and y at each supervision boundary, TRM:
- Prevents gradient explosion and vanishing across deep recurrent unrolling.
- Drastically caps peak training memory, allowing training on modest consumer hardware.
- Emulates a residual network hundreds of layers deep (2 layers × 6 recursions × 16 supervision steps = 192 effective layer passes) while maintaining a strict 7M parameter footprint.
Benchmark Performance: Beating Giants on ARC-AGI
ARC-AGI (Abstraction and Reasoning Corpus), designed by François Chollet, measures an AI system's ability to acquire new skills and solve visual-logical transformation tasks out-of-distribution without task-specific priors. While standard language models easily memorize standard benchmarks (MMLU, GSM8K), they consistently hit a wall on ARC-AGI.
On the public ARC-AGI-1 evaluation and the hardened ARC-AGI-2 benchmark, TRM produced unprecedented parameter efficiency:
- ARC-AGI-1 Accuracy: TRM achieves 45% test accuracy, matching or surpassing leading large language models (such as DeepSeek-R1 and OpenAI o3-mini in direct evaluation settings).
- ARC-AGI-2 Accuracy: On ARC-AGI-2, TRM hits 8.0% test accuracy.
- Combinatorial Logic Tasks: On deterministic combinatorial logic tasks, such as complex 9×9 Sudoku completion and large labyrinth maze pathfinding, TRM achieved 87.4% on Sudoku-Extreme and 85.3% on Maze-Hard, where standard LLMs routinely fail due to cumulative autoregressive drift.
Why Recurrence Prevents Overfitting
One of the most notable findings in the paper is that large models fail on abstract puzzle benchmarks primarily due to representational overfitting. When a 70B+ parameter model is exposed to a few hundred example transformations, its capacity is so massive that it memorizes arbitrary superficial token correlations rather than discovering the underlying transformation rule.
By forcing the architecture down to 7M parameters and two layers, the network simply lacks the capacity to memorize brute-force patterns. The only way it can minimize training loss across the 16 deep supervision steps is to learn reusable, generalized computational primitives—such as symmetry detection, topological connectivity, color-mapping rules, and flood fills.
Model Capacity Comparison:
┌──────────────────────────────┬──────────────┬───────────────────┐
│ Model │ Parameters │ ARC-AGI-1 Test │
├──────────────────────────────┼──────────────┼───────────────────┤
│ Gemini 2.5 Pro (w/ TTC) │ >1 Trillion │ ~40-42% │
│ DeepSeek-R1-Distill-70B │ 70 Billion │ ~35-40% │
│ HRM (Wang et al., 2025) │ 27 Million │ 40.3% │
│ TRM (Jolicoeur-Martineau) │ 7 Million │ 45.0% │
└──────────────────────────────┴──────────────┴───────────────────┘
Key Takeaways for AI Architecture Design
- Recursion Over Scale for Pure Reasoning: Scaling parameters solves world knowledge and linguistic fluency, but algorithmic logic and spatial planning benefit exponentially more from recurrent computational iterations in latent space.
- The Death of CoT Verbosity: Verbal Chain-of-Thought is an expensive workaround for feed-forward limitations. Recurrent latent updates provide self-correction and iterative refinement without generating tokens.
- Hybrid Future: Future foundation models will likely adopt hybrid architectures: large autoregressive encoders for world-knowledge retrieval and user interaction, paired with compact, deeply recurrent latent engines like TRM to handle rigorous mathematical, logical, and code-verification bottlenecks.
Sources
The stories that matter for people who build with AI, with the source for every claim. Free, no spam.
Feed, daily deep-dive and bytes — readable offline, with push alerts for the topics you follow.