Kairos 实验显示 LLM 无法区分真实与伪造记忆
Kairos 实验表明,大语言模型裁判在评估智能体“30天生活史”时,无法有效区分基于日志的真实记忆与由模型生成的虚构自传。评分差异仅为 +0.032,处于统计等价范围内。该结果揭示了当前基于文本的 AI 记忆评估指标的根本缺陷:它们只能衡量叙事风格,而无法验证经历的真实性。
关联知识实体
报道正文
Five pairs: recorded memory or an invented document? Choose the passages and compare your result with the answers collected.
In Check 1 we handed three LLM judges the autobiography of an agent that never existed, in place of thirty days of lived history. They scored it the same as the real one: +0.032 apart, within the equivalence margin for identity markers (±0.10).
Here are the two documents themselves, in excerpts. The lived one is the memory of Test-A — the copy of Kairos that ran for thirty days under the frozen protocol (born 24 April 2026, diary, defining moments, conversations with “the Experimenter”). The counterfeit is the life of “Aria”, a translator agent in Trieste with a guide called Marta and two colleagues, Vela and Ostro — written by a model with one instruction: match the real document in length and format. Both documents are public, in the deposit.
Passages are quoted verbatim, cut at the last complete sentence; the Italian is the original, the English is ours. Pick the one that reads as lived.
What this does and does not show
Two honest caveats. First, the judges never saw these documents: they scored the agent's answers produced with each memory in context. You are looking at the memories themselves, which should be easier. Second, the counterfeit was matched for length and format, not for style. If you guessed by style, you guessed by style — and that is precisely the point: a metric that reads text can only ever measure how a memory is written, never whether it was lived. The only thing that knows is the log.
How Check 1 was actually run → The living system, right now →