Nvidia can't have every data center workload

Welcome to Runtime! Today: AWS and Google suggest that the next five years of AI infrastructure strategies won't look like the last five years, GitHub sheds a little more light on its recent uptime struggles, and the latest enterprise moves.

Please forward this email to a friend or colleague! If it was forwarded to you, sign up here to get Runtime each week, and if you value independent enterprise tech journalism, click the button below and become a Runtime supporter today.

Wafer madness

While pretty much the entire enterprise AI buildout has been driven by Nvidia's GPUs to this point, demand for alternatives is starting to build as companies grow wary of relying too heavily on a single vendor. AWS and Google, two of Nvidia's biggest customers and close partners, continued to lay groundwork this week for companies that want to follow a different path in the data center.

Amazon CEO Andy Jassy devoted significant time to AWS's chip strategy in his annual letter to shareholders Thursday, revealing that AWS is on pace to record around $20 billion in annual revenue from renting chips like Graviton and Tranium to its customers. But he also suggested that Amazon's chips could play a greater role in enterprise computing outside its cloud data centers.

• If you imagined that Amazon's chip division (built around the 2015 acquisition of Annapurna Labs) was a standalone business that sold those chips to customers like AWS and others — kind of like Nvidia does — it would be on pace to record $50 billion a year in revenue, Jassy said.
• Then he dropped this very interesting suggestion: "There’s so much demand for our chips that it’s quite possible we’ll sell racks of them to third parties in the future."
• While it's a relatively small part of its business, AWS already sells hardware, and could offer companies that want or need to run workloads in-house a cheaper alternative to Nvidia's "AI Factory" template through a rack of Graviton and Trainium chips.
• However, industry analyst Ben Bajarin noted that most of the demand for Tranium comes from AWS itself to run its Bedrock inference service and Anthropic, which signed a big deal with the cloud provider in November 2024.

Google Cloud has devoted significant resources to its own Nvidia alternative, and has also discussed selling those TPUs to big customers like Meta to use in their own data centers. But as Nvidia increasingly talks about taking on a greater role in the data center with CPUs and networking chips, Google reaffirmed its commitment to its longtime dance partner Thursday.

• Ten years ago news that a major cloud provider agreed to buy Intel's latest server processor probably wouldn't have made the roundup section of this newsletter, but a lot has changed since then.
• The more interesting part of Intel and Google's announcement, however, was an agreement to collaborate on what they call "infrastructure processing units," which play a key role in modern data centers.
• IPUs (Nvidia calls them DPUs) take on networking and storage workloads that would have otherwise occupied the CPU, and help smooth out bottlenecks caused by AI workloads.
• Google's commitment to co-designing future IPUs also makes it clear that the CPU still has a very important role to play in an AI-centric data center.

Despite Nvidia's massive foothold, companies scrambling to build AI infrastructure have a much wider variety of choices compared to the cloud-computing buildout, which relied almost exclusively on Intel's CPUs. That could help address the price of building enterprise AI, which is forcing OpenAI and Anthropic to burn billions of dollars improving model performance.

• "We have a strong partnership with Nvidia, will always have customers who choose to run Nvidia, and we will continue to make AWS the best place to run Nvidia. However, customers want better price-performance," Jassy said in his letter.
• As agentic AI starts to become real after years of disappointment, companies that were willing to invest money to get up and running with AI will likely start watching the operational costs of running those systems much more closely.

Incidents and accidents

GitHub's position as one of the most important day-to-day tools used by millions of software developers around the world hasn't really been challenged in over a decade, but frustration with the service is mounting as it buckles under the weight of increased demand thanks to the AI boom. Last week COO Kyle Daigle took to X to try and quantify that demand, and on Wednesday the company shared more details about some of those recent problems in its monthly availability report.

Microsoft's code-management service owned up to four major issues in March, including a widespread issue on March 3rd that affected several different GitHub services. In that case, in trying to implement a change to handle an increase in demand "a bug caused every user’s cache to expire, get recalculated, and get rewritten," which caused a bunch of downstream problems.

"While we have made (and are making) substantial, long-term investments in how GitHub is built and operated to improve resilience, we acknowledge we have more work to do," the company said in the report. The AI boom is testing a lot of assumptions about infrastructure design honed during the cloud boom, which creates a huge opportunity for new entrants starting from scratch.

Enterprise moves

Eric Boyd is the new head of infrastructure at Anthropic, joining the company at a crucial time for its infrastructure strategy after more than 16 years at Microsoft.

Hilary Maxson is the new chief financial officer at Oracle, joining the company after finance leadership roles at Schneider Electric and AES Corporation.

Gabe Monroy is the new chief technical officer at Workday, replacing Peter Bailis, who joined Anthropic.

Matt Waxman is the new chief product officer at Precisely, joining the data-management company after product leadership roles at Veritas, Cohesity, and Puppet.

Frank O'Dowd is the new chief revenue officer at Cohere, joining the enterprise AI platform company after sales leadership roles at Cloudera and Oracle.

Julie Irish is the new chief information officer at Alteryx, joining the data analytics company after technology leadership roles at Couchbase and New Relic.

The Runtime roundup

The D.C. Circuit Court of Appeals denied Anthropic's request for an injunction against the Pentagon's supply chain risk designation, leaving the order in place despite a California court's ruling in Anthropic's favor.

Meta signed a new compute deal with CoreWeave worth $21 billion over the next several years, adding to an existing $14 billion agreement.

Canva made a pair of acquisitions Wednesday, snapping up agent-management company Simtheroy and marketing tech startup Ortto.

Julia Liuson, who has been president of Microsoft's Developer Division since 2021, announced plans to retire in June after 34 years at the company.

Thanks for reading — see you Saturday!