Termsslug: knowledge-distillation

Knowledge Distillation

知识蒸馏

Model compression method transferring capability by training smaller models on larger outputs.

Curated Intelligence

47 items · Page 1/5
Latent SpaceT1·Models
78 pts
Google DeepMind Ships Gemini 4 Neon as Mid-Tier Model

Google DeepMind released Gemini 4 Neon, its fourth Gemini 4 model in roughly seven weeks, positioned as a faster and cheaper mid-tier option between Carbon and Pro. Thanks to distillation, RL and benchmarking effects, Neon outperforms the larger Carbon on some measures, while Argon has been retired and the legacy Pro and Flash models are gone.

Rohan Paul (@rohanpaul_ai)T2·Agents
78 pts
Meta Opens Muse AI Agent Live Video Calls to All US Users

Meta has opened live video calls with its Muse AI agent to all US users, letting them talk to a customizable animated avatar with roughly 870ms response latency. A distilled student model reproduces in two passes what a teacher model needed 120 passes for, and with 4-bit quantization, caching and dynamic batching a single Nvidia GB200 serves 12 concurrent video streams—though serving cost at million-user scale remains the key constraint.

InfoQ 中文T2·Agents
68 pts
Ant Group's Robbyant Embodied AI Stack Adopted by Fei-Fei Li Team

Ant Group's open-source embodied AI stack 'Robbyant' was showcased at the Wuzhen Summit for pharmacy sorting and confirmed to be reused in Stanford's OpenWAM framework led by Fei-Fei Li. The suite includes vision, world models, and action decision modules, with over 33k GitHub stars and 2.2M downloads, signaling its integration into global mainstream R&D pipelines.

Yuchen Jin (@Yuchenj_UW)T2·Models
78 pts
Frontier LLMs Edit Small Model Weights for 6x Math Boost

Researchers introduced 'Weight-Space Programming,' using GPT-6 Astra to directly edit Qwen3.5-0.8B weights. By modifying only 0.016% of parameters without gradient training, they achieved a 6x boost in math reasoning and 3x in instruction following, presenting a paradigm more efficient than traditional distillation.

AWS Machine Learning BlogT1·Tools
48 pts
SageMaker Introduces Knowledge Distillation for Efficient AI Agents

Amazon SageMaker AI offers a knowledge distillation solution to train smaller 'student' models that mimic larger 'teacher' models, reducing inference latency and costs for AI agents while maintaining task quality. This approach addresses ROI challenges in multi-turn agentic workflows by providing code repositories and documentation for implementation.

宝玉 (@dotey)T2·Models
45 pts
Leaked Claude Fable 5 Weights Claimed to Be Stolen DeepSeek Files

Claims of stolen weights for Anthropic's flagship model, Claude Fable 5, circulating on hacker forums have been debunked as fraud. Community analysis reveals the files match open-source DeepSeek V4.1-Flash shards exactly, alongside inconsistencies in AWS region hosting and data volume calculations. While Anthropic remains silent, the incident highlights the prevalence of scams leveraging hype around proprietary AI models.

IT之家 · 智能时代T2·Models
88 pts
Google Unveils Sleep Framework to Solve LLM Catastrophic Forgetting

Google Research introduces the 'Sleep Framework,' a method enabling LLMs to continuously learn new knowledge without catastrophic forgetting by mimicking human memory consolidation during sleep. The framework divides operations into wake and sleep phases, utilizing parameter expansion, MoE memory blocks, and knowledge distillation to stabilize long-term knowledge retention and enhance reasoning. Experiments demonstrate superior performance in long-context understanding and multi-stage continual learning compared to baselines.

The DecoderT2·Industry
78 pts
Chinese AI Labs Shift Distillation Target to OpenAI's Astra as Anthropic Tightens Defenses

As Anthropic strengthens defenses around Claude's reasoning outputs, Chinese AI labs like DeepSeek and Moonshot AI have shifted their primary distillation target to OpenAI's GPT-6 Astra. These entities leveraged Astra's outputs to train models such as Kimi K3.1 Pro, despite prohibitions on extracting hidden reasoning steps, with attackers continuously adapting tactics to bypass new security measures.