Amazon CEO Jassy: AI Inference Costs Must Drop 10x for Enterprise Adoption
Amazon CEO Andy Jassy argues that AI inference costs must drop by an order of magnitude to enable widespread enterprise software adoption. With compute capacity booked through 2028, Amazon is leveraging its self-developed Trainium chips to offer 30-40% better cost-performance, aiming to unlock the underserved mid-market and challenge Nvidia's dominance.
Full Report
In an interview published by venture capital firm a16z on the 23rd (local time), Jassy revealed: "In my view, for AI to be adopted as widely as possible, inference costs are still an order of magnitude too high."
Amazon plans to invest $220 billion in capital expenditures this year (IT Home note: approximately RMB 1.47 trillion at current exchange rates), yet it still cannot build enough infrastructure to meet AI demand. Jassy disclosed that most of the primary compute capacity supply for 2026 is already booked, the majority for 2027 has been reserved by customers, and a significant portion for 2028 is also sold out. In his view, market demand is so robust that even if Amazon were to invest significantly more funds, it would not be wasted.
Jassy likened AI demand to a "barbell": one end consists of AI labs, which consume unprecedented amounts of compute; the other end comprises enterprise customers, who primarily use AI for specific tasks such as customer service and fraud detection.
The truly underdeveloped segment is the middle market—the various software applications used daily by enterprises. These software products have not yet broadly integrated AI, but Jassy believes that the largest future AI demand may come from this area.
The primary factor constraining enterprise AI adoption is inference cost. Every time an AI model answers a question or completes a task, it consumes compute resources. If the benefits gained from integrating AI do not offset the higher operational expenses, enterprises have little incentive to re-architect their existing software.
Jassy drew a comparison with Amazon's experience years ago. At that time, Intel dominated the cloud server processor market. Amazon sought higher computational performance at the same cost but struggled to persuade Intel to prioritize this need, ultimately deciding to develop its own Graviton processors. In his view, "the same story is now playing out in the AI domain."
The AI chip market is currently dominated by Nvidia. Amazon has launched its proprietary Trainium chips, aiming to offer customers a lower-cost alternative. During the interview, Jassy did not directly name Nvidia, emphasizing only that the market's dominant supplier remains an important partner for Amazon.
According to reports, Trainium offers approximately 30% to 40% better price-performance ratio compared to similar processors, allowing customers to obtain more compute for the same amount of money. "Our chip business plays a crucial role in reducing inference costs," he stated.
Amazon previously disclosed that Trainium has secured over $225 billion in future revenue commitments (approximately RMB 1.5 trillion at current exchange rates), including agreements with Anthropic and OpenAI.