DeepMind 论文：强化道德论证提升模型一致性但未改善判断力
Google DeepMind 最新研究发现，通过强化学习训练 Gemma-4 模型撰写道德论证，能显著降低其在选项重排时的回答翻转率（从 14.3% 降至 6.6%），表明模型变得更加“果断”和一致。然而，这种训练并未提升其实际道德判断的准确性，甚至在道德寓言测试中准确率略有下降。该研究提示开发者在评估 AI 建议类任务时，需将一致性指标与正确性指标分开考量。
报道正文
Rohan Paul (@rohanpaul_ai) · 2026-10-29 16
New Google DeepMind paper finds that training a model to write better moral arguments makes it more decisive, but no better at moral judgment.
LLMs are more and more often asked for moral advice. A model that keeps its answer when the options are reordered is often taken as a sign of sound judgment.
They trained Gemma-4 4B with reinforcement learning to argue dilemmas taken from classic novels. The model judged its own arguments in pairs, and the winners set the reward.
Answer flips when options were swapped fell from 14.3% to 6.6% on ambiguous MoralChoice cases. Yet siding with the user barely moved, from 45.3% to 44.0%, and accuracy on moral fables dropped from 56.5% to 53.4%.
If you evaluate models for advice or judgment tasks, test consistency and correctness separately, since a steadier model can still be wrong.
