ALL INTELLIGENCE
The DecoderT2·Models

Nvidia Researchers Call Huge Context Windows a Dead End

By Jonathan KemperOriginal title: Nvidia researchers argue huge AI context windows are a dead end
58AI Score

An Nvidia research team argues that ever-larger context windows are often overkill and expensive. In many tests, periodic summarization combined with targeted lookups performed about as well as stuffing entire documents into the prompt, suggesting the industry should rethink the 'longer is better' assumption.

Full Report

Nvidia researchers argue huge AI context windows are a dead end

Image description

Nano Banana Pro prompted by THE DECODER

Language models can now process entire novels at once. An Nvidia research team says that's often overkill and expensive. In many tests, periodic summaries and targeted lookups worked about as well.

A language model's context window is its working memory, covering everything it holds in view when generating a reply. Over the past few years, that window has grown rapidly. GPT-4 launched in March 2023 with 8,192 tokens. Today, GPT 5.6 Sol handles around one million tokens, about 120 times as much and enough for Tolstoy's "War and Peace."

THE DECODER+