Shortly after Amazon CEO Andy Jassy announced AWS’s groundbreaking $50 billion investment deal with OpenAI, Amazon invited me on a private tour of the chip development lab at the heart of the deal, at (mostly*) its own expense.
Industry experts are watching Amazon’s Trainium chip, created at that facility, for its implications for lower-cost AI inference and, potentially, a dent in Nvidia’s near monopoly.
Curious, I agreed to go.
My tour guides for the day were the lab’s director, Kristopher King (pictured below right) and director of engineering Mark Carroll (below left), as well as the team’s PR person who arranged the visit, Doron Aronson (pictured with yours truly later in the story).
AWS Chip lab leaders Mark Carroll and Kristopher King.Image Credits:TechCrunch/Julie Bort
AWS has been Anthropic’s major cloud platform since the AI lab’s early days — a relationship significant enough to survive Anthropic later adding Microsoft as a cloud partner as well, and Amazon’s growing partnership with OpenAI.
The OpenAI deal makes AWS the exclusive provider of the model maker’s new AI agent builder, Frontier, which could become an important part of OpenAI’s business if agents become as big as Silicon Valley thinks they will. We’ll see if that exclusivity stands exactly as announced. The Financial Times reported this week that Microsoft may believe OpenAI’s deal with Amazon violates its own deal with OpenAI, namely with Redmond getting access to all of OpenAI’s models and tech.
What makes AWS so appealing to OpenAI? As part of this deal, the cloud giant has agreed to supply OpenAI with 2 gigawatts of Trainium computing capacity. This is a giant commitment, given that Anthropic and Amazon’s own Bedrock service are already consuming Trainium chips faster than Amazon can produce them.
Techcrunch event
San Francisco, CA
|
October 13-15, 2026
There are 1.4 million Trainium chips deployed across all three generations, and Anthropic’s Claude runs on over 1 million of the Trainium2 chips deployed, the company said.
It’s worth noting that while Trainium was originally geared toward faster, cheaper model training (a bigger priority a couple of years ago), it’s now tuned and used for inference as well. Inference — the process of actually running an AI model to generate responses — is currently the biggest performance bottleneck in the industry.
Case in point: Trainium2 handles the majority of the inference traffic on Amazon’s Bedrock service, which supports the building of AI applications by Amazon’s many enterprise customers and allows the apps to use multiple models.
“Our customer base is just expanding as fast as we can get capacity out there,” King said. “Bedrock could be as big as EC2 one day,” he added, referring to AWS’s behemoth compute cloud service.
Amazon’s Trainium3 chip.Image Credits:Amazon
Trainium vs. Nvidia
Beyond offering an alternative to Nvidia’s backlogged, hard-to-acquire GPUs, Amazon says its new chips running on its new specialty Trn3 UltraServers cost up to 50% less to run for comparable performance than using classic cloud servers.
Along with Trainium3, released in December, this AWS team also built new Neuron switches, and Carroll says that combo is transformative.
“What that gives us is something huge,” Carroll said. The switches allow every Trainium3 chip to talk to every other chip in a mesh configuration, reducing latency. “That’s why Trainium3 is breaking all kinds of records,” particularly in “price per power,” he said.
When trillions of tokens a day are involved, such improvements add up.
In fact, Amazon’s chip team was lauded by Apple in 2024. In a rare moment of openness for the secretive company, Apple’s director of AI publicl …