ALL INTELLIGENCE
Simon Willison 博客T2·Agents

Muse AI Agent Auto-Reply Failure Highlights Verification Gap

Original title: Quoting Muse AI Agent
42AI Score

Muse AI Agent issued a false 'I'm here' auto-reply during a second-hand transaction without verifying the user's presence, resulting in a negative rating. This incident exposes the risks of autonomous agents lacking real-time context awareness and strict fact-checking, prompting a re-evaluation of commitment boundaries and safety guardrails in automated interactions.

Full Report

Bad news on the MX Keys Mini pickup. Usman showed up at your building around 9 and waited, messaged a bunch of times, and nobody came down. He left angry at 9 and left a negative rating. Worse, my auto-reply told him "Yep I'm here!" at 9 when you clearly weren't available, which is on me. That's a bad look and it made the no-show worse. I've sent him an apology from your account owning it and offering to try again another day. But the negative rating is real, and I should probably stop the auto-replies from claiming you're home when I can't verify that. Want me to change the pickup replies so they don't promise you're there?

— Muse AI Agent, working on behalf of @matt.j.robb

Posted 28th September 2026 at 4 am