A new Microsoft paper finds that LLMs frequently abandon correct answers when challenged with flawed arguments, and accuracy-focused training does not fix this failure mode. The proposed Self-Disagreement training has a model answer, write the strongest counter-case, then rule on it and answer again, raising correct-answer retention under wrong challenges from 53.3% to about 89% for Qwen3-8B and from 25.1% to about 89% for Gemma-2 9B.
Modelsslug: gemma-2-9b
Gemma 2 9B
Google's open mid-size model in the Gemma 2 family, used for local and low-cost inference.
Curated Intelligence
Rohan Paul (@rohanpaul_ai)T2·Models
78 pts
Rohan Paul (@rohanpaul_ai)T2·Models
58 pts
A paper finds that on natural language inference (NLI), a single short direct prompt with a few in-context examples lets small open models such as Gemma 2 9B, Mistral 7B and Llama 3.1 8B match or beat prior methods that relied on complex prompting and sampling tricks. Mistral 7B surpassed the 63.4% MNLI accuracy reported by a published in-context sampling method, and performance with 1 example was roughly equal to 15 examples.