Artificial Analysis Launches AA-3D Benchmark: Evaluating LLMs on Interactive Three.js Scene Creation via Elo Rankings

Artificial Analysis

All articles

October 12, 2026

Announcing AA-3D: a benchmark for AI models building interactive 3D experiences

AA-3D measures how well AI models build interactive 3D scenes that people want to explore. Each model works as an agent in an offline sandbox and turns a written brief into a three.js web page. Our recruited panel compares pairs of scenes built from the same brief, without knowing which model made which, and we rank the models by Elo. Claude Sonnet 5.5 is first at 1194, statistically tied with three other Anthropic and OpenAI models.

AA-3D leaderboardTry AA-3D VotingRead the methodologyThis is the original launch article and shows AA-3D v1.0 results as of launch. For live results across the latest models, see the AA-3D leaderboard.

AA-3D v1.0 leaderboard

AA-3D Elo

Elo from blind pairwise votes · 95% confidence intervals · Higher is better · Data as at 12 October 2026

Claude Sonnet 5.5 has the highest Elo, 1194, but the top four are statistically tied: their 95% confidence intervals all overlap with that of Claude Sonnet 5.5.

All four come from Anthropic and OpenAI. Meta’s Muse Spark 1.3 is seventh at 1035. The best-placed open weights models, Kimi K3 (982) and MiMo-V2.6-Pro (977), are 11th and 12th, more than 200 points behind.

For up-to-date results, see the AA-3D leaderboard.

Try it yourself

Explore scenes built by two different models for the same brief, then pick the one you prefer to reveal which model built it. Votes cast here do not count toward the leaderboard.

How AA-3D works

Each model works as an agent in an offline sandbox. It gets a written brief for one interactive scene, plus a delivery contract that every task shares. It writes code, renders the scene, looks at its own screenshots and revises. The sandbox has Blender, Rapier physics, Python, a headless browser and a library of CC0 assets. A run stops after three hours or 250 turns, and the result is an index.html page that runs the scene in a browser.

Watch a scene being built

• GPT-6 Astra has its first complete page after 6 minutes. It finishes at 68 minutes. Claude Sonnet 5.5 shows nothing until minute 42 and keeps working until 2 hours 58 minutes.
• Claude Sonnet 5.5 writes 6.2 times as many output tokens as GPT-6 Astra, but costs only 1.3 times as much. Its run costs $14.93 against$11.62, and its list prices are a fifth of GPT-6 Astra’s.

Explore the scenes

Pick a domain, a task and a model to load the scene that model built.

Task

Model

Create an interactive adult man performing weightlifting exercises, with views that expose his muscles, skeleton and organs in anatomically accurate form.

Loading the scene

21 of 21 models have a scene for this task in blind voting.

HUD

The cost of an AA-3D scene

A scene costs anywhere from $0.20 to$32.08, depending on the model. GPT-6 Luna is the cheapest and Claude Fable 5.1 the most expensive. Input makes up 69% of the spend, because agents resend their whole context on every turn. Among the top five, GPT-6 Sol costs least at $3.46 a scene. Claude Sonnet 5.5 costs 4.4 times as much, and Claude Fable 5.1 9.3 times as much.

AA-3D Cost per Scene

Mean cost per scene built by token type (USD) · Lower is better · Data as at 12 October 2026

Inkling is not shown because we have no representative cache hit rate for it.

GPT-6 Luna costs $0.20 a scene and still ranks eighth. It builds all 55 scenes. Kimi K3 spends 68 times as much per scene for a similar Elo.

AA-3D Elo vs. Cost per Scene

Mean cost per scene built (USD, log scale) · Data as at 12 October 2026Most attractive quadrantPareto line

Inkling is not shown because we have no representative cache hit rate for it.

What AA-3D tells us about building in 3D

The top models look at their own work early

The top five first look at an image of their own scene after a median of 10 to 13 turns, and by turn 20 in at least 89% of their scenes. Models ranked 14th to 20th wait until turn 15 to 40. Across the 19 models that can view images, looking early tracks Elo closely (Spearman 0.82). The top five also start testing in the browser sooner, at a median turn of 16 against 25. How often a model looks matters less than when. MiniMax-M3 looks at its own scene 54 times per scene, more than any other model, but by turn 20 in only 7% of its scenes. It ranks 19th.

AA-3D Elo vs. Early Visual Inspection

Share of scenes where the model views an image it made by turn 20 · Data as at 12 October 2026

The chart shows the 19 models with an image view tool. GLM-5.3 and Nemotron 3 Ultra are text-only and built their scenes without seeing them.

Detailed scenes win more votes, and stronger models build them

The most detailed fifth of scenes in each task has screenshots twice as large (168 KB against 84 KB) and a third as empty (22% against 60%). It wins 63% of its votes, against 31% for the least detailed fifth. The models’ Elo alone predicts 61% and 33%, so the detailed scenes come mostly from the stronger models. These data cannot show whether detail is part of what makes those models strong. Within one model, the effect is small: a scene one standard deviation more detailed than the model’s other scenes wins about 1.5 points more of its votes (1.3 to 1.8, depending on the detail measure). Of 86 scene measures, 11 pass correction: the detail measures, render resolution and antialiasing, GPU time per frame and the number of page elements. None moves votes by more than 4 points.

AA-3D Votes Won by Scene Detail

Share of votes won against the share the models' Elo predicts · Data as at 12 October 2026Scenes Ranked by Detail Within Each Task

Detail combines four screenshot measures, defined in What Detail Measures. Within the same model and task, one standard deviation more detail adds 1.5 points (p = 0.001, q = 0.011). 1,008 scenes, excluding 23 that break on the voting page.

OpenAI's GPT-6 models build better 3D scenes than their general intelligence predicts

General intelligence goes a long way in AA-3D. Each point on the Artificial Analysis Intelligence Index is worth about 18 Elo (Pearson 0.92), a closer fit than Terminal-Bench 4.0 (0.81) or SciCode (0.76). GPT-6 Luna and GPT-6 Sol beat that trend by more than any other model, by 141 and 93 Elo, and GPT-6 Astra beats it by 46. GLM-5.3 falls furthest below, by 197. It cannot view images, which may explain part of the gap: Z AI’s GLM-5.3-Flash, which can, scores 149 higher with a lower Intelligence Index. Nemotron 3 Ultra, also text-only, sits near the trend.

AA-3D Elo vs. Intelligence Index

Artificial Analysis Intelligence Index · Data as at 12 October 2026Linear fit, 18 Elo per point

We use Pearson’s r where a relation is linear on the chart’s axes, and Spearman’s rank correlation where one model would dominate r or a measure takes only a few values. We call a correlation not significant when its 95% confidence interval includes zero.

Most scenes use the asset library, but the assets do not win the votes

Three in four scenes use something from the CC0 library, as the shared instructions encourage, and models that use it more often tend to rank higher (Spearman 0.53). But a model’s library scenes win no more often than its other scenes, once we account for the opponent. GPT-6 Luna uses the library in only a third of its scenes and still ranks eighth.

AA-3D Asset Library Use

Share of scenes that ship or declare a CC0 library asset · Data as at 12 October 2026

Detailed results

Models that write more do not score higher. Output tokens and Elo show no clear link (Spearman 0.33, not significant). Claude Sonnet 5.5 writes the most, 569k output tokens per scene, about two-thirds of it reasoning. GPT-6 Sol writes 109k and ranks fifth. Input is 97% to 99% of all tokens, and most models read nearly all of it from the cache.

AA-3D Output Tokens per Scene

Mean reasoning and answer tokens per scene · Lower is better · Data as at 12 October 2026Nor do models that take more turns. Turn count and Elo show no clear link (Spearman −0.27, not significant). GLM-5.3 hits the 250-turn limit in 64% of its scenes. GPT-6.1 Sol and GPT-6 Astra need medians of only 56 and 60 turns, but each of their turns lasts about two minutes, most of it model time.

AA-3D Turns per Scene

Distribution of agent turns per scene up to the 250-turn limit · 5th percentile, 25th percentile, median, 75th percentile, 95th percentile · Lower is better · Data as at 12 October 2026Agents spend 82% of their tool time testing the page in a browser. Browser tests, which take a screenshot or check for errors, are only a third of all tool calls. At 33 to 38 seconds each, though, they fill about 25 of the 30.7 minutes of tool time per scene. Writing code is one call in six and just 6% of tool time. The top five read files and docs about half as often as the other models (9.4 against 18.7 times per scene). Across all 21 models, reading less goes with a higher Elo (Pearson −0.74).

AA-3D Tool Calls per Scene

Mean tool calls per scene by purpose · Data as at 12 October 2026

We sort every tool call into five groups by purpose, then into subgroups by command. Browser tests either save a screenshot or only read errors. Code calls write or patch files, or read files and the bundled three.js docs. Asset calls browse the CC0 library, process images with Python or run Blender. Image views show the agent its own work (mostly page screenshots, also contact sheets and textures), an asset preview or a reference image from the brief.

The top models put more on the page to click and play with. The top five show a median of 15 to 20 buttons and other controls, against 6 to 8 for the bottom three, and control count tracks Elo closely (Pearson 0.90). Speed does not separate the models: every model’s median frame takes under 1.3 ms of GPU time on our test machine, far below the 16.7 ms that a 60 fps frame allows.

AA-3D Elo vs. UI Controls

Median visible buttons and inputs per scene · Data as at 12 October 2026The same models come out on top in almost every domain. A top-five model has the highest win rate in six of the seven domains. In living bodies, GPT-6.1 Sol, sixth overall, and Claude Sonnet 5.5 are level at the top, less than a point apart. The top five are also good everywhere: in every domain, each stays within 12 points of the win rate its overall Elo predicts against the opponents it met there. Only 6 of the other models’ 112 cells stray further, 3 of them grayed out. Each cell rests on 2 to 10 scenes, a median of 7. We gray out cells with fewer than 5 scenes, and differences of less than about 12 points are within noise.

AA-3D Win Rate by Domain

Share of panel votes won in each task domain · Based on 10K+ panel votes · Data as at 12 October 2026

Explore the full results on the AA-3D leaderboard.

Inside an agent run

Pick a task to replay every model’s run on a shared clock. Hover over a lane to see each action, and click it to see where the time went.

• GPT-6 Sol is the quickest of the top five on all six tasks. Its median run takes 60 minutes, against 115 to 151 for the other four.
• The top five do not test more than the others. As a group they spend 25% of their time on browser tests, about the same as everyone else.
• Every top-five run looks at its own scene. GLM-5.3 and Nemotron 3 Ultra are text-only, so none of their 12 runs does.

What's next

We will add new models to the AA-3D leaderboard as they are released.

AA-3D resources

• The AA-3D leaderboard has the full results, updated as new models are released
• Try AA-3D Voting to compare scenes yourself
• The methodology page documents the generation settings, the scene checks and how we compute Elo
• AA-3D runs on Stirrup, our open-source agent harness

Read the latest

[

Announcing FrontierCode on Artificial Analysis: can AI models write code maintainers would merge?

Cognition's FrontierCode 1.1 tests whether AI models make code changes that maintainers of real open source repositories would merge. Leading models still fail more than 40% of the time.

October 12, 2026](https://artificialanalysis.ai/articles/frontiercode)[

Muse Wonder and Muse Wonder Full: a material jump in intelligence over previous Meta models

Meta has released Muse Wonder and Muse Wonder Full, achieving Artificial Analysis Intelligence Index v4.3 scores of 52 and 56 at max reasoning effort - each a material gain over Muse Spark 1.3 (max) at 48, with a release gap of only 6 weeks

October 12, 2026](https://artificialanalysis.ai/articles/muse-wonder)[

Announcing Harvey LAB-AA v1.1: adding hallucination checks to raise the bar for agentic legal work

Harvey LAB-AA v1.1 adds hallucination auditing with a new Hallucination-Gated All-Pass Rate headline metric, a three-judge grading panel, and Harvey's latest private dataset with improved tasks and criteria.

October 8, 2026](https://artificialanalysis.ai/articles/harvey-lab-aa-v1-1)