Termsslug: in-context-learning

In-Context Learning

上下文内学习

A model's ability to learn from examples in the input context and complete tasks without weight updates.

Curated Intelligence

9 items · Page 1/1
Rohan Paul (@rohanpaul_ai)T2·Models
58 pts
Simple Few-Shot Prompting Lets Small Open Models Match Complex Methods

A paper finds that on natural language inference (NLI), a single short direct prompt with a few in-context examples lets small open models such as Gemma 2 9B, Mistral 7B and Llama 3.1 8B match or beat prior methods that relied on complex prompting and sampling tricks. Mistral 7B surpassed the 63.4% MNLI accuracy reported by a published in-context sampling method, and performance with 1 example was roughly equal to 15 examples.

Apple Machine Learning ResearchT1·Models
71 pts
Gating Mechanism Found to Bias State Space Models Toward Memorization

Through theory and experiments, this work shows that the gating mechanism common in modern recurrent networks explains why State Space Models lag behind Transformers on in-context learning and precise retrieval: gating drives SSMs to first learn an in-weights memorization solution, delaying or even preventing convergence to a correct in-context learning solution, even absent architectural or memory-capacity limits. Conversely, gating often benefits generalization to long sequence lengths, revealing its dual role in training dynamics.

IT之家 · 智能时代T2·Agents
74 pts
OpenAI's GPT-6.1 Sol Performs Live Mouse Brain Surgery, Halving Positioning Error in Three Runs

OpenAI's GPT-6.1 Sol carried out three consecutive anatomical-landmark localization tasks in live mouse stereotaxic surgery at Exin Therapeutics, using surgeon feedback for in-context learning without retraining: positioning error fell from 0.6 mm to 0.3 mm and manual corrections from four to two. It signals AI moving from analyzing research data to directly operating real lab equipment, though the three-run sample is tiny and not peer-reviewed, leaving human surgery far off.

机器之心 (微信公众号)T2·Models
58 pts
GPT-6.1 Sol Joins Live Mouse Brain Surgery, Cuts Targeting Error to 0.3mm

Biotech startup Exin Therapeutics had GPT-6.1 Sol perform skull landmark targeting across three consecutive live mouse brain surgeries, with a professional surgeon correcting each round and the model retaining images and measurements for the next attempt. By round three, targeting time fell from 169s to 86s, corrections dropped from four to two, and error narrowed from 0.6mm to 0.3mm. With only three trials the result is far from proven surgical competence, but it illustrates an in-context learning loop that carries real-world feedback into subsequent tasks.

Rohan Paul (@rohanpaul_ai)T2·Models
63 pts
Stanford Paper Traces an In-Context Learning Circuit in Small LLMs

A new Stanford paper, "A Circuit for In-Context Learning," argues that few-shot ability in small models is not genuine pattern learning but label lookup, switched on only when labels repeat within the prompt. The authors traced the full pathway in Qwen3-4B, SmolLM3-3B and Llama-3.2-3B: each label token stores its sentence's class, and roughly 90 later attention heads match the incoming sentence against those stored classes. The practical takeaway is to repeat every label several times and only use classes the model can already separate, since examples cannot teach a feature the model lacks.

Marktechpost (@Marktechpost)T2·Agents
78 pts
Infini-AI-Lab Open-Sources Hermit for Model-Agnostic Agent Learning

Infini-AI-Lab released Hermit, an Apache-2.0 framework enabling AI agents to learn continuously via a trainable 'Harness Adapter' without altering model weights. Using an RL loop and Git-based versioning for instructions and skills, it improved accuracy on CL-Bench and OfficeQA, though results rely on single runs and incur higher latency costs.

Google ResearchT1·Models
85 pts
Google Research Introduces 'Sleep Phase' Framework to Fix LLM Catastrophic Forgetting

Google Research introduces a 'Sleep Phase' framework for Large Language Models (LLMs) to address catastrophic forgetting in continual learning. By simulating biological sleep mechanisms, this approach allows deployed models to offline-consolidate new facts and enhance reasoning capabilities while preserving existing knowledge, overcoming the limitations of static knowledge cutoffs.

Apple Machine Learning ResearchT1·Models
78 pts
RLTL;DR: Breaking Zero-Success Barriers via Self-Generated Feedback

To address the failure of standard RLVR in tasks with zero success rates, this paper introduces RLTL;DR, a framework where agents generate concise 'TL;DR' insights after failures to guide subsequent rollouts. By backpropagating on these in-context insights, the model internalizes a direct task-to-insight mapping. Experiments show this method boosts Qwen 3.5 9B's Pass@1 from near 0% to 12-13% on extremely difficult datasets, achieving significant performance recovery with minimal training data.

腾讯混元 (Tencent Hunyuan)T1·Models
78 pts
Tencent Hunyuan Releases CL-bench: Exposing LLM Context Learning Bottlenecks with New Benchmark and Dataset

Tencent Hunyuan introduces CL-bench to evaluate Large Language Models' ability to learn instantly from new contexts. The study reveals that while frontier models excel at static benchmarks, they struggle significantly in scenarios simulating human-like immediate learning from documentation or rules. Code, data, and the paper are open-sourced, providing a new quantitative standard for improving AI context adaptability.