Code2Video-AA Benchmark Launches to Evaluate AI Video Generation via Code
Artificial Analysis launched Code2Video-AA, an independent implementation of HeyGen's Code2Video Bench, evaluating how well LLMs design short videos using HyperFrames HTML code. Claude Opus 5.5 leads with a significant Elo score advantage, ranking first across all five judge axes. This benchmark provides developers with quantitative insights into model performance in creative coding and multimodal output scenarios.
Full Report
Contents18 sections
October 26, 2026
Code2Video-AA: how well do AI models design videos in code?
Artificial Analysis' implementation of HeyGen's Code2Video Bench, evaluating how well models create short product and brand videos from 114 design briefs using HyperFrames HTML.
Code2Video-AA leaderboardRead the methodologyThis is the launch article and shows Code2Video-AA results as at 26 October 2026. For current results, see the Code2Video-AA leaderboard.
Overview
Code2Video-AA is Artificial Analysis' independent implementation of HeyGen's Code2Video Bench, a set of design briefs for short product and brand videos. Each model designs the video in code.
Key findings
- Claude Opus 5.5 (max with fallback) leads: at 1536 Elo it is 93 points ahead of the next model, and it is first on all five judge axes
- Two models are tied for second: Claude Sonnet 5.5 (max with fallback) at 1443 and GPT-6 Astra (max) at 1427 are within noise of each other. GPT-6.1 Sol (max) is fourth at 1369
- Cost per task varies more than 300x: from $0.01 to $3.89. GPT-6.1 Sol (max), fourth, costs $0.31, about a tenth of the $3.08 for Claude Opus 5.5 (max with fallback)
- Open weights models trail by at least 483 points: DeepSeek V4.1 Flash (max) at 1053, MiMo-V2.6-Flash at 1052 and MiMo-V2.6-Pro at 1052 are the top open weights models
Code2Video-AA Elo
Mean Elo across the five judge axes · Anchored to GPT-5.5 (xhigh) at 1000 · Higher is better · Data as at 26 October 2026 (dataset 0.3.0)
Example tasks
Three tasks from the public split of Code2Video Bench, each with every model's rendered video. Models are in order of Code2Video-AA Elo.
Model
UI motion graphic (product screen-recording style), 16, ~7.3s. A demo of a mobile-app wireframe builder: one black title card, then a flat, high-key light-gray workspace where blocks are added to a phone canvas one at a time. Palette: workspace/chrome light gray #F0F0F0, slightly darker panels/menus #E8E8E8, near-black primary text #1A1A1A, muted gray secondary labels #A4A4A1, white canvas #FFFFFF, and a single bright lime-yellow accent #E4E41B used ONLY on the primary button and the active/selected item.
CRITICAL — camera: after the title card the shot is a LOCKED, flat, dead-on screen recording. The framing and scale are CONSTANT throughout — there is NO camera move of any kind (no zoom, no push-in, no pan). All motion is element-level only: a pointer cursor, instant panel swaps, and blocks populating the canvas. (Ignore any impression of a "push-in" — the frame does not scale.)
Scene 1 — Title card (0.0–~0.9s): Solid black canvas #000000. Centered bold white #FFFFFF sans-serif text reads "Rapid wireframing". It holds briefly, then a HARD CUT (no fade) reveals the UI. Static.
Scene 2 — Editor base state (~0.9–2.0s): The light-gray workspace. Left sidebar, top to bottom: "Now editing" / "First screen"; a "Status" row with a small red dot + "to be done"; then a vertical block-type menu listing IN ORDER — Text, Button, Line, Spacer, Icon(s), Gallery, Placeholder, Input, Controls, Navigation, Cards — each with a small monochrome glyph. Center: a blank white mobile-proportioned (tall) canvas. Bottom-right: a "Discard changes" link + a lime-yellow "Save and exit" button. Motion: the whole UI is already present at the cut; the pointer cursor moves toward the block menu. No zoom.
Scene 3 — Add a Spacer (~2.0–3.5s): The cursor clicks "Spacer". A dashed spacer block appears in the canvas (faint "Press Enter/return to add a new block" hint). A right-hand settings panel swaps in reading "Spacer block settings" with size chips XS S M L XL (a small size selected/highlighted lime-yellow). Motion: right panel content HARD-SWAPS (no easing); block pops into the canvas. No zoom.
Scene 4 — Add Media / image placeholder (~3.5–5.0s): The cursor returns to the menu and picks the image block ("Placeholder" / "Media"). A large square light-gray image placeholder (with a wavy image glyph) pops into the canvas below the spacer. The right panel swaps to "Media block settings" showing layout thumbnails with "Image placeholder" selected. Motion: instant swap + pop-in, cursor-driven. No zoom (the framing is locked — do NOT scale the frame here).
Scene 5 — Add Text (~5.0–7.3s): The cursor selects "Text" (the menu item highlights lime-yellow). A text block is added to the canvas: it shows the placeholder "Write something down…", then a heading auto-types in character-by-character (e.g. "Welcome!"). The right panel swaps to "Text block settings" with a text input field, formatting controls (B / I / U + alignment) and a lime-yellow "Generate text" button. Motion: text auto-types onto the canvas; panels hard-swap. The clip ends on this state. No zoom.
Motion summary: black title card → HARD CUT → a locked, dead-on screen recording of a wireframe builder. A pointer cursor drives every action: pick a block type in the left menu → the block pops into the centered white phone canvas → the right settings panel hard-swaps to match it (Spacer → Media → Text). Text auto-types. Every transition is an instant hard cut/swap with no easing, and the camera never moves and the frame never scales — flat, static, dead-on the entire time.
Videos from 27 of 27 models
The briefs are from the Code2Video Bench public split by HeyGen, licensed under CC BY 4.0.
How Code2Video-AA works
We send the model a design brief and HeyGen's system prompt in one call, with no tools. It returns one HyperFrames HTML composition, index.html, with HTML for layout and GSAP for animation.
The methodology page covers how we select the briefs, generate and render the videos, judge them, sample the comparisons and fit the Elo.
Claude Opus 5.5 leads on all five axes
The lead is largest on craft, 128 points, and smallest on prompt intent, 48 points, both over Claude Sonnet 5.5 (max with fallback). Temporal consistency separates models the least: its Elo range is 1057 points, against 1222 to 1349 points on the other four axes.
Each of the five judge axes has its own comparisons and its own Elo, and the overall Elo is their mean. The order of the models is almost the same on each axis: no model moves more than six places between an axis and the overall Elo.
Code2Video-AA Elo by Judge Axis
Elo from a separate fit for each judge axis, with their mean as the overall · Anchored to GPT-5.5 (xhigh) at 1000 · Higher is better · Data as at 26 October 2026 (dataset 0.3.0)
Cost per task varies more than 300x
Cost per task ranges from 3.89 for Claude Fable 5.1 (max with fallback). Eight models are on the Elo vs. Cost per Task Pareto frontier, from MiMo-V2.6-Flash at 1052 and $0.01 to Claude Opus 5.5 (max with fallback) at 1536 and $3.08. On that frontier, Claude Haiku 5.5 (max) scores 1192 at 0.31. GPT-6 Astra (max) costs 5x as much as GPT-6.1 Sol (max), $1.62 against $0.31, for 58 more Elo points.
Output tokens per task vary 16x
Output tokens per task range from ~11k for Nemotron 3 Ultra to ~176k for Claude Sonnet 5.5 (max with fallback), and most of the variation comes from reasoning tokens. Final answers, which contain the compositions, range from ~4k to ~12k tokens across all models. The three Claude 5.5 models use the most output tokens (~152k to ~176k per task), against a mean of ~42k and a median of ~34k for the other 24 models.
Code2Video-AA: Elo vs. Cost per Task
Code2Video-AA Elo vs. average cost per task (USD) · Anchored to GPT-5.5 (xhigh) at 1000 · Data as at 26 October 2026 (dataset 0.3.0)Most attractive quadrantPareto line
Average cost per task in the evaluation. Costs are split by input, cache hit, cache write, reasoning, and answer token pricing where canonical token counts are available.
Every model renders at least 96% of its videos
A composition must render to a video before the judge can see it. 3042 of the 3078 tasks (98.8%) rendered, and 8 of the 27 models rendered all 114. GLM-5.3 (max) has the lowest render rate, with 110 of 114.
A composition that fails to render loses every comparison against a video that rendered. The three lowest-ranked models rendered 111 to 113 of 114 tasks, so their low Elo comes mostly from the judge's verdicts on videos that did render.
Code2Video-AA: Render Rate
Share of the 114 tasks whose composition rendered to a video · Higher is better · Data as at 26 October 2026 (dataset 0.3.0)
Code2Video-AA resources
- The leaderboard is on the Code2Video-AA evaluation page
- The methodology page covers the current implementation, judging and scoring
- HeyGen's announcement post describes the benchmark
- The HyperFrames renderer is on GitHub
Read the latest
[
Kimi K3.1 Pro launches at 4th on the Intelligence Index and will lead open weights models
Kimi K3.1 Pro launches at 4th on the Artificial Analysis Intelligence Index and will lead open weights models
October 23, 2026](https://artificialanalysis.ai/articles/kimi-k3-1-pro-launch-benchmarks)[
Announcing AA-PersonalityBench: measuring how AI models communicate
AA-PersonalityBench measures the personality and communication style of AI models. A panel of LLM judges compares answers pairwise on 30 personality traits, which roll up into 8 personality indexes, alongside 21 communication characteristics measured directly from every response.
October 19, 2026](https://artificialanalysis.ai/articles/aa-personalitybench)[
HealthBench and HealthBench Professional: Evaluating models in realistic health conversations
We’re launching leaderboards for HealthBench and HealthBench Professional, two benchmarks evaluating how AI models respond in realistic health conversations
October 19, 2026](https://artificialanalysis.ai/articles/healthbench-professional-health-conversations)