Apple Machine Learning Research · 10/15/2026, 08:00:00
SSR Framework Reformulates Multimodal Search Reasoning as Selection, Cutting Latency by 90% for Small Models
Researchers from CMU and UIUC introduce Selection-based Structured Reasoning (SSR) to address the inefficiency of freeform reasoning in small multimodal agents. By reformulating reasoning as selecting from pre-specified natural-language candidates via likelihood scoring, SSR avoids costly generation steps. Experiments on 2B and 4B models show SSR reduces per-turn reasoning latency by over 90% and total inference latency by 28–54% while maintaining competitive success rates.
SOURCE COVERAGEFull Original Report
Multimodal agents commonly generate freeform reasoning before each action. For small models, limited model capacity can result in lengthy reasoning that provides little useful guidance for action generation while incurring substantial inference cost. To address this challenge, we introduce Selection-based Structured Reasoning (SSR), a framework that reformulates reasoning as selection instead of open-ended generation. SSR represents recurring high-level reasoning as pre-specified, reusable natural-language candidates. At each turn, the model selects from the reasoning candidates based on their likelihoods given the current context, without an auxiliary task head. Using pre-specified reasoning traces enables parallel scoring, where teacher-forced prefilling computes token likelihoods concurrently within and across candidates using a shared context KV cache. We evaluated SSR on seven multimodal search benchmarks using 2B and 4B models. Across multiple reinforcement learning objectives and supervised fine-tuning, SSR delivers significant efficiency gains without sacrificing task performance. SSR achieves an average success rate competitive with leading search agents of the same scale, while reducing per-turn reasoning latency by over 90% and total per-question model inference latency by 28–54%.
- † Carnegie Mellon University
- ‡ University of Illinois Urbana-Champaign
- ** Work partially done while at Apple
- *** Work done while at Apple