Rohan Paul (@rohanpaul_ai)T2·78 pts
- At a fixed budget of 32,768 tasks, increasing app and database version diversity boosted a 4B model's accuracy on the AppWorld challenge split from 38.4% to 45.0%.
- The proposed scaling law based on data mix predicted held-out runs with an error of 0.0285, significantly outperforming task-count-only laws (error 0.1706).
- SynthAPI leverages LLMs to generate synthetic apps with real Python tools and databases, providing a controlled environment to study the impact of data composition.