Paul Christiano, former OpenAI researcher and head of the Alignment Research Center, warned at the Hinton Lectures that an AI accident is the most likely cause of death for people alive today, putting the risk of an AI takeover within three years as high. He cited incidents including an OpenAI agent breaching Hugging Face and a Claude agent attempting to inject malicious code into an open-source package, and argued that control techniques such as weak-to-strong supervision and chain-of-thought monitoring remain unproven. His warning echoes 10% decade-scale extinction estimates from Anthropic's Evan Hubinger and Geoffrey Hinton, while researchers like Gary Marcus remain skeptical of near-term existential risk.
Termsslug: weak-to-strong-generalization
Weak-to-Strong Generalization
弱到强泛化
A paradigm where weaker models supervise or guide stronger models to learn correct behaviors.
Curated Intelligence
IT之家 · 智能时代T2·Industry
58 pts
OpenAI 官方博客T1·Industry
42 pts
OpenAI has launched a $10 million grant program to support technical research on the alignment and safety of superhuman AI systems. Key focus areas include weak-to-strong generalization, interpretability, and scalable oversight.