Modelsslug: gemma-4-31b-it

Gemma 4 31B-it

Google's 31B instruction-tuned Gemma model used as the solver in the experiments.

Curated Intelligence

1 items
Rohan Paul (@rohanpaul_ai)T2·Models
78 pts
Stanford and DeepMind's TURTLE Evolves Tool-Use Benchmarks to Stay Hard

A new Stanford and Google DeepMind paper introduces TURTLE, which uses AlphaEvolve to mutate saturated tool-use tasks from AppWorld by adding filters, extra steps and hidden lookups, targeting roughly 50% solver success. Gemma 4 31B-it always solved 69% of original tasks, while nearly all 735 evolved tasks landed at 40%-60% completion, and an LLM judge rated 52.5% of evolved tasks hard versus 19.8% of originals.