ALL INTELLIGENCE
Arena.ai (@arena)T1·Agents

Arena Proposes Interaction-as-a-Judge for Computer-Use Agents

Original title: Arena research intern and PhD candidate at Carnegie Mellon University, Wayne Chi, explains Interacti...
58AI Score

Arena.ai research intern and CMU PhD candidate Wayne Chi introduced Interaction-as-a-Judge, a paradigm in which computer-use agents navigate real applications and games to produce interaction traces evaluated by an LLM judge or a trained reward model. The framework spans preference evaluation drawn from real Code Arena battles and correctness evaluation based on requirements mined from user complaints in Agent Arena.

Full Report

Arena.ai (@arena) · 2026-11-12 21

Arena research intern and PhD candidate at Carnegie Mellon University, Wayne Chi, explains Interaction-as-a-Judge: letting computer-use agents play games and use apps to better improve evaluation and downstream generation.

A computer-use agent navigates an application, producing interaction traces for evaluation by an LLM judge or a trained reward model. The talk covers preference evaluation using real Code Arena battles and correctness evaluation using requirements mined from real user complaints in Agent Arena.

Watch the full talk: