The authors ran five small models (Llama 3.2 3B, Mistral 7B, Gemma 2 2B, Phi-3 mini, Qwen 2.5 3B) locally on a single Apple M3 Pro, each retrieving passages and citing sources. Since plain majority voting cannot catch shared errors, they propose 'Verify, Then Vote': verify each model's cited sources first, then apply a fixed rule to select the answer, keeping answering, verification and selection as separate reusable steps. Paper, code and saved runs are open-sourced.
Model Ensemble
模型集成
Combining predictions from multiple models to improve overall performance
Curated Intelligence
A new Max Planck paper tested nine Qwen and GPT models on three prediction tasks built from US census surveys, with reasoning on and off. Enabling reasoning raised cross-model agreement, but the share of people all nine models got wrong also rose by 2.0 to 2.4 points; for the longest reasoning traces, adjusted agreement fell to roughly 0.31-0.39. The authors warn that ensemble-style error correction assumes uncorrelated mistakes, and reasoning may make errors more correlated.
vLLM Semantic Router introduces the 'Fusion' primitive to address single-model limitations in production. It enables requests to be routed to a panel of diverse models (fast, cheap, private), where a judge model analyzes agreement and gaps to synthesize a unified answer. This shifts multi-model coordination from offline research to a programmable, observable online serving standard.