OpenAI launched its first Embedded Assessments program with Goodfire, Apollo Research, METR and Transluce, granting outside experts deep access to internal models, checkpoints and training data. Goodfire will embed interpretability researchers to perform model forensics on training-time anomalies. OpenAI had earlier explored acquiring Goodfire for over $5 billion, but the startup chose to stay independent and sell to multiple labs.
Mechanistic Interpretability
机制可解释性
Research field that reverse-engineers internal computation circuits to make models understandable and controllable.
Curated Intelligence
The author built an interactive atlas of Qwen3-1.7B's internal states, drawn from residual-stream vectors across thousands of conversations, to test whether semantically similar dialogues cluster in vector space and whether the same idea in different languages lands in nearby regions. It is an exploratory mechanistic-interpretability visualization that offers an intuitive window into how a small model structures conversational representations.
A new Stanford paper, "A Circuit for In-Context Learning," argues that few-shot ability in small models is not genuine pattern learning but label lookup, switched on only when labels repeat within the prompt. The authors traced the full pathway in Qwen3-4B, SmolLM3-3B and Llama-3.2-3B: each label token stores its sentence's class, and roughly 90 later attention heads match the incoming sentence against those stored classes. The practical takeaway is to repeat every label several times and only use classes the model can already separate, since examples cannot teach a feature the model lacks.
Former OpenAI researcher Nick Cammarata claims that with modern agentic tools, the vast majority of neurons in trillion-parameter models are now legible, overturning earlier pessimism about neural networks being arbitrary tangled knots. He suggests mechanistic interpretability may be solved soon as automated scientists improve, citing a 'bitter lesson' where simple methods work at absurd scale. Sam Altman endorsed this view.
New research challenges the dominant 'superposition' theory in AI interpretability, suggesting that polysemantic neurons may actually respond to specific geometric features like V-shapes. If valid, this could drastically reduce the time required for automated model analysis, enabling machines to understand internal mechanisms faster. The paper is not yet published, and researchers remain cautiously optimistic.
This is a truncated snippet from an AI News briefing mentioning Nick Cammarata's claims about agents improving interpretability and alignment. Due to severe content truncation, lacking specific technical details, experimental data, or methodology, the core assertions cannot be verified.
Nick Cammarata from Goodfire argues that neural networks are inherently legible, challenging the prevailing view of polysemanticity and superposition. The research posits that what appeared as arbitrary concept packing is actually 'recursive generalization,' suggesting that mechanistic interpretability can proceed by studying network mechanisms directly rather than first untangling them.
BAIR introduces SPEX and ProxySPEX frameworks to identify high-order feature interactions in LLMs at scale. Integrating with the SHAP-IQ library, these tools combine feature attribution, data attribution, and mechanistic interpretability to enhance transparency and generate testable hypotheses. The code is now open-source, supporting model behavior analysis in fields like genomics.