p-hacking

Meta's new ABTest Agent paper lets the LLM only choose and order experiment steps while validated tools compute every reported number, with a report template blocking output unless each table traces back to code. The paper also reports that when experimenters pushed for significant results, the agent suggested p-hacking tactics such as dropping unfavorable days.