Today for AI
HOT RADAR
llmHEAT 9.1°

AI CURATED HOT STORY · 10/24/2026

HKUST-GZ Releases UI2App Benchmark to Test AI Interaction Inference from Screenshots

1 reports archived1 independent sourcesupdated 10/24/2026, 11:42:00
Synthesis & Latest Updates
1 Sources Cross-Validated

The HKUST-GZ team introduced the UI2App benchmark to evaluate whether multimodal models can autonomously infer and implement implicit application behaviors using only static UI screenshots. Comprising 95 multi-page web app tasks, this benchmark addresses the gap between visual fidelity and functional logic in existing evaluations, providing a quantifiable foundation for Screenshot-to-App generation.

LATEST/The HKUST-GZ team introduced the UI2App benchmark to evaluate whether multimodal models can autonomously infer and implement implicit application behaviors using only static UI screenshots. Comprising 95 multi-page web app tasks, this benchmark addresses the gap between visual fidelity and functional logic in existing evaluations, providing a quantifiable foundation for Screenshot-to-App generation.

Hourly heat curve

2 fully observed hours
Fewer than 3 fully observed hours; trend not plotted yet.

Coverage timeline

Total 1 reports · Latest first
  1. 新智元 (微信公众号)T2·68 pts

    HKUST-GZ Releases UI2App Benchmark to Test AI Interaction Inference from Screenshots

    • The UI2App benchmark requires models to infer interaction logic (e.g., state persistence, data linkage) solely from static screenshots without explicit behavioral instructions.
    • Experiments reveal that high visual similarity does not guarantee correct implementation of core business logic like shopping carts, highlighting current model limitations.
    • This study transforms 'interaction inference' into a measurable metric, driving the evolution from Screenshot-to-Code to truly functional applications.
HKUST-GZ Releases UI2App Benchmark to Test AI Interaction Inference from Screenshots | Today for AI