谷歌发布 SymptomAI:面向日常症状评估的对话智能体

原标题:SymptomAI: Towards a conversational AI agent for everyday symptom assessment
68AI 研判分

谷歌研究团队推出 SymptomAI,一种旨在通过自然语言交互进行日常症状评估的对话式 AI 智能体。该模型针对真实世界中患者非结构化、低医学素养的症状描述进行了优化,弥补了现有大模型在临床诊断评估中依赖合成病例数据的不足。

报道正文

本文目录10 个章节

大量临床诊断仅凭基于语言的访谈即可得出。这些诊断访谈通常由临床医生在面对面或远程就诊期间,通过医患互动进行。尽管此类互动是症状评估的金标准,但往往受到财务、地理和系统性障碍的限制,从而影响其可及性。当前的语言模型(LM)在经筛选的医学病例研究中已展现出强大的鉴别诊断评估能力,凸显了其在辅助诊断过程中的潜力。然而,现有的评估大多依赖于经过精心筛选、高度详细甚至有时为合成的患者案例描述,这可能无法反映真实世界的体验以及临床表现的多样性。这些评估未能捕捉日常患者如何报告其健康症状,例如患者具备不同的医学素养水平、提供不完整的信息,以及在自然对话中出现的其他复杂情况。这构成了一个关键缺口,导致我们不确定语言模型在真实世界情境中的表现究竟如何。

为弥补这一空白,我们开展了一项 原位比较 研究,针对一组实验性对话式 AI 智能体原型,探索对话式 AI 如何执行端到端的症状访谈和鉴别诊断评估,以用于研究基准测试。在我们最近的研究论文 “SymptomAI: Towards a Conversational AI Agent for Everyday Symptom Assessment” 中,我们分享了一项随机全国规模研究(n=13,917)的结果,其中经同意的研究参与者与五个可能的 Gemini Flash 2.0 SymptomAI 智能体之一进行交互。研究期间生成的所有诊断、标签和疾病关联仅用于研究分析,不构成确诊的临床诊断或官方医疗评估。在与 AI 智能体交互两周后,我们要求研究参与者报告他们在医疗保健提供者就诊时收到的任何诊断。利用这些数据,我们进行了一项临床专家标注研究,将 SymptomAI 的诊断性能与真实临床医生的医疗评估进行比较。

在评估 SymptomAI 的鉴别诊断(DDx)准确性后,我们进一步将 SymptomAI 的诊断结果与参与者在与其对话之前通过 Fitbit 可穿戴设备采集的生物信号数据进行对比。结果显示,导致传染病病因诊断的 SymptomAI 对话与可能指示免疫反应的生理趋势相吻合,这为 SymptomAI 的性能提供了进一步的证据。

工作原理

我们招募了 13,917 名签署知情同意书的研究参与者,每位参与者向五个随机分配的 SymptomAI 智能体之一描述其症状,这些智能体在进行症状问诊时的灵活程度各不相同。在这些对话中,参与者描述自己的症状,SymptomAI 则提出后续问题,对话最终生成一份鉴别诊断(DDx,即一系列可能的诊断列表)以及下一步行动建议。随后,参与者可以前往医疗机构就诊,并在两周后通过问卷调查分享就诊结果。为了评估并建立 SymptomAI 诊断能力的基线,我们开展了一项临床专家标注研究,由三位委员会认证医师组成的评审小组审阅对话记录,并提供他们自己的评估结果(即鉴别诊断)。接着,每位医师以双盲方式对 SymptomAI 提供的 DDx 以及其他医师提供的 DDx 进行排名。

关键结果

临床专家更偏好 SymptomAI 生成的鉴别诊断

我们发现,在超过 50% 的案例中,临床专家更偏好 SymptomAI 生成的鉴别诊断,而非其他临床专家提供的诊断。这表明 SymptomAI 的鉴别诊断与我们临床专家的医学评估一致的程度,不低于甚至高于其他临床专家之间的匹配度。

临床专家认为 SymptomAI 的鉴别诊断更准确

同样地,我们通过 Top-5 准确率(即参与者个人医疗保健提供者给出的真实诊断是否出现在鉴别诊断列表的五种可能诊断之中)来比较 SymptomAI 生成的鉴别诊断与真实临床专家提供的鉴别诊断的准确性。我们要求每位临床专家判断所提供的诊断是否包含在每个鉴别诊断列表中,包括 SymptomAI 生成的列表和其他临床专家提供的列表。结果显示,临床专家认定 SymptomAI 生成的鉴别诊断准确的频率高于其他临床专家提供的鉴别诊断。

获取更多信息可提升性能

作为本研究的一部分,我们评估了进行病史采集访谈的不同方法。参与者被随机分配到五个研究组,每组采用不同的提示策略。其中两组(Dynamic Live 和 Dynamic Final)拥有完全自主权,可以提出不受限制的后续问题;另外两组(Fixed Canonical 和 Flexible Canonical)分别基于医学院教授的一套标准病史采集问题进行提问;最后一组是未经特定提示的基础语言模型(Base unprompted LM),代表了当前用户查询 LLM 聊天机器人时完全由用户主导的体验现状。我们发现,所有由智能体驱动的提示策略(即 SymptomAI 主动提出后续问题的情况)的表现均显著优于基础条件,这证明了从参与者那里获取信息对于提高鉴别诊断准确性的价值。

SymptomAI 在低置信度案例中的表现

我们发现,当临床医生对自己做出的鉴别诊断(DDx)信心最低时,SymptomAI 相对于临床基线的性能优势最为显著。

SymptomAI 的诊断结果与生物信号相关

鉴于 SymptomAI 相较于临床基线的高准确性,我们还可以探索其在大规模应用中的潜力。目前,获取临床标签的成本阻碍了对人群规模数据集的真实世界分析。像 SymptomAI 这样准确的症状检查系统,有望实现具有临床质量的诊断的自动参考标注,从而开启对生理数据的大规模分析——这是一项目前在大规模场景下无法完成的任务。

[注:本文篇幅超长,以上为核心前篇精译,后续内容保留原文呈现]

One such example is correlating wearable biosignals with different categories of illness. The most notable changes in wearable biosignals are observed for acute respiratory infections. To study this at population scale, we collected daily biometric data from our consenting participants for up to 30 days prior to their interaction with SymptomAI. We find clear biosignal shifts indicating symptom onset in the days approaching the user's symptom reporting. Importantly, the separation between cohorts was derived through categorizing SymptomAI's top-1 candidate diagnosis and grouping diagnoses that were classified as respiratory infections. This cohort excludes non-infectious respiratory illnesses like allergic rhinitis or chronic obstructive pulmonary disease. The correlation of wearable biosignals shift peaks aligning with the date of symptom reporting for these participants serves as observational physiological evidence that align with their reported symptoms.

Utility alongside biosignals

AI-based assessment of symptom presentations opens the door to new research. By using SymptomAI to analyze a large volume of symptom reports and pairing those with real-time Fitbit data, we can explore digital biosignal phenotypes across a wide range of diseases. Our analysis revealed distinct shifts in physiological metrics — including cardiovascular function, respiration, skin temperature, and sleep quality — in the days leading up to a user's SymptomAI conversation. These objective changes align closely with the timing of the symptom conversation, offering a potential way to validate patient-reported symptoms or provide passive data to help inform a differential diagnosis alongside their symptom conversation. Additionally, this real-time accessibility highlights a core benefit of AI symptom checkers. Unlike traditional clinical appointments that can suffer from scheduling delays, participants could take part on the SymptomAI research study contemporaneously while symptoms are fresh. This potentially could improve the accuracy of patient-reported onset timelines — a crucial detail for population-scale health analysis.

Limitations

SymptomAI is an exploratory research effort that could represent a significant research advancement in AI-based symptom assessment and demonstrates the potential it could provide for the general public seeking understanding of their symptoms. While a population deployment evaluation reveals the accuracy of symptom assessment through remote patient interviews, there are nuanced limitations when comparing against clinician’s assessments.

Firstly, differential diagnosis itself is an ambiguous task and even reported diagnoses may change and develop longitudinally. A symptom assessment is a snapshot in time and captures the symptoms as they present in that moment. Due to the scale of our deployment, we were unable to control for frequency and timing of symptom reporting. As a result, some participants may have reported their symptoms well before more representative indicators developed, while others may have reported obvious indicators from an informed context after years of experience with chronic illness. Future work may focus on specific illnesses at specific points during symptom development such as early-onset metabolic syndrome or symptoms discussed at the start of respiratory infections. All diagnoses, labels, and disease associations generated during the study are AI-derived for research analysis only and do not constitute confirmed clinical diagnoses or official medical assessments.

Secondly, in our evaluation the clinicians reviewed static chat transcripts and were not given agency to ask their own follow-up questions. Clinicians may have intuitively sourced different information had they directed the symptom interview. Moreover, while recent research has shown that conversational AI systems can source clinical data with a clinician-level of detail and accuracy, such systems may miss alternative signals like body language, visual assessment, medical records, or in the context of primary care, existing rapport with the patient.

In conclusion, we introduce SymptomAI, an investigational conversational AI agent for conducting real-world patient interviews and symptom assessments. We demonstrate SymptomAI’s end-to-end real-world performance through DDx accuracy on a population sample, and show how SymptomAI diagnoses can enable analysis of population-scale signals like wearable biosignals for identifying associations in physiological signals with reported illness.

Acknowledgements

This work is the result of equal contributions from Joe Breda, Jake Sunshine and Daniel McDuff. We would like to thank our co-authors and collaborators from Google Research and Google DeepMind for their contributions to this work.