Cartesia Research introduces 'Hierarchical Networks' to address the reliance on hand-crafted encoding in current LLMs. The study advocates returning to the 'bitter lesson' by enabling end-to-end learning directly from raw bytes, aiming to improve generalization in data-scarce domains like science and robotics.
Bitter Lesson
苦涩教训
The observation that general methods leveraging computation ultimately outperform hand-coded heuristics.
Curated Intelligence
Former OpenAI researcher Nick Cammarata claims that with modern agentic tools, the vast majority of neurons in trillion-parameter models are now legible, overturning earlier pessimism about neural networks being arbitrary tangled knots. He suggests mechanistic interpretability may be solved soon as automated scientists improve, citing a 'bitter lesson' where simple methods work at absurd scale. Sam Altman endorsed this view.
A senior AI observer admits a significant prediction error regarding multi-agent systems, noting that the 'Bitter Lesson' has struck again. The article argues that brute-force compute and stronger model capabilities allow agent swarms to emerge with efficient collaboration spontaneously, eliminating the need for humans to construct complex organizational rules or prompt chains. This challenges the prevailing paradigm of 'managing AI like a manager,' emphasizing that underlying model capability improvements are the key to solving coordination problems.
Epoch and METR released MirrorCode, a new benchmark evaluating AI performance on long-horizon programming tasks. Claude Opus 4.7 completed a software reimplementation estimated to take humans 2-17 weeks in just 14 hours for $251, demonstrating rapid capability growth of ~30% annually. The findings support the 'bitter lesson' hypothesis that general agents can bootstrap complex industrial capabilities through black-box interaction.