This paper introduces Recursive INference Scaling (RINS), a method that recursively applies a model block to exploit language’s self-similarity, achieving significant performance gains in language and multimodal tasks under compute-matched conditions while offering inference flexibility through stochastic training and linear adapters.
Large Language Model, Multimodal Systems, Inference Scaling, Recursive Architecture, Parameter Sharing, Stochastic Training
Ibrahim Alabdulmohsin, Xiaohua Zhai
Google DeepMind Zürich, Switzerland
Generated by grok-3
Background Problem
The research is motivated by the need to enhance inference-time performance in language and multimodal systems without increasing model size or training compute. Recent studies have highlighted the fractal, self-similar nature of language, suggesting that recursive, scale-invariant strategies could exploit this property for better performance. Additionally, scaling inference compute has shown promise in improving model capabilities (e.g., through chain-of-thought prompting), yet prior recursive methods like ‘repeat-all-over’ (RAO) in Mobile LLM have not been rigorously evaluated under compute-matched conditions, leaving uncertainty about their true efficacy. The key problem addressed is identifying an optimal recursive architecture that maximizes performance gains for a fixed compute budget, extending beyond language to multimodal tasks, while offering flexibility at inference time.
Method
Recursive INference Scaling (RINS) is a plug-in inference scaling strategy that partitions a model into two blocks (A and B) and recursively applies block A on its own output multiple times (signature A^rB, r > 1) before passing to block B. This leverages the self-similar structure of language by iteratively refining intermediate representations. Key steps include: (1) defining a taxonomy of recursive architectures using ‘signature’ (block arrangement) and ‘degree’ (nested recursion); (2) training models under a compute-matched regime to ensure fair comparison; (3) introducing stochastic RINS, where recursion rounds are randomly dropped during training with probability p^s, enhancing flexibility; and (4) adding lightweight linear adapters (<1% parameters) to mitigate performance trade-offs when recursion is disabled at inference. RINS aims to increase computational path length without altering model size, complementing other techniques like chain-of-thought prompting.
Experiment
Experiments were conducted on decoder-only transformer language models (300M, 600M, 1B parameters) using datasets like C4 and SlimPajama, and on multimodal SigLIP-B/16 models with image-text pairs. The setup matched training compute FLOPs across recursive and baseline models, training on up to 500B tokens for language tasks. Over 59 architectures were tested, focusing on various signatures and degrees. Results showed RINS (A^rB) consistently outperformed baselines and other recursive methods like RAO, with performance gaps widening as compute increased (e.g., better log-perplexity in Figure 2). Downstream tasks (e.g., OpenBookQA, HellaSwag) confirmed gains (Table 1). In multimodal tasks, SigLIP-RINS-B/16 improved 0-shot ImageNet accuracy by +2% (77.3% to 79.6%) over baseline. Stochastic RINS with adapters provided a ‘no-regret’ strategy, maintaining performance even without recursion at inference. However, the experimental design, while comprehensive in compute matching, lacks robustness analysis (e.g., sensitivity to hyperparameters or dataset shifts), and multimodal gains seem modest relative to added complexity. Vision-only tasks showed no improvement, supporting the language-specific hypothesis but limiting generalizability. Overall, results align with expectations but may overstate practical impact due to limited discussion of failure cases or deployment challenges.
Further Thoughts
The concept of leveraging language’s fractal nature through RINS opens intriguing avenues for future research, particularly in exploring whether other domains with hierarchical or self-similar structures (e.g., biological sequences or social network data) could benefit from similar recursive strategies. The ‘no-regret’ aspect of stochastic RINS with adapters suggests potential synergies with parameter-efficient fine-tuning methods like LoRA, where minimal additional parameters could adapt recursive depth to specific tasks or hardware constraints—could this be a path to democratize powerful inference scaling for edge devices? However, the paper’s focus on compact models raises concerns about scalability to much larger models (e.g., 100B+ parameters), where recursive depth might exacerbate issues like vanishing gradients or memory bottlenecks, even with KV cache sharing. Additionally, connecting RINS to cognitive science’s System 2 thinking (deliberation) is fascinating, but it prompts a question: does recursive inference risk overthinking, akin to human over-deliberation leading to suboptimal decisions, especially in time-sensitive applications? Cross-referencing with works on iterative refinement like ReAct or Self-Refine, RINS might benefit from incorporating explicit feedback loops rather than blind recursion, potentially enhancing robustness. These considerations suggest that while RINS is a promising step, its real-world impact hinges on addressing scalability, domain adaptation, and integration with complementary inference strategies.