Apple Machine Learning ResearchT1·Models
62 pts
A NeurIPS 2026 PALM workshop paper introduces weighted state souping, which learns query-dependent weights to recombine cached state-space model encodings. The method substantially improves long-context performance on single- and multi-hop QA, needle-in-a-haystack retrieval, variable tracking, and tool use, while a sparse variant achieves near-constant encoding cost with respect to context length.