This post was originally published on Data Center Knowledge
Nvidia dominates the global AI race in training large language models, but AWS unveiled its Tranium3 chip at the re:Invent conference on Tuesday, aiming to challenge Nvidia’s market share.
The cloud giant said that UltraServers, which house up to 144 Tranium3 chips, offer 4.4 times more compute power than previous generations, and energy efficiency gets a fourfold boost, along with increased memory bandwidth.
AWS said early testing shows that companies can save up to 50% on costs compared to GPU training, providing enterprises ramping up AI adoption with a balance sheet-saving option. Companies testing the new chips included Anthropic, Karakuri, Metagenomi, NetoAI, Ricoh, and Splash Music.
The company’s managed service for foundation models, Amazon Bedrock, is already serving production workloads, and the chips are now ready for enterprise-scale deployment. AWS says its EC2 UltraClusters 3.0 can connect thousands of UltraServers, each equipped with up to a million Trainium chips, 10 times more than previous generations.
“This scale enables projects that simply weren’t possible before, from training multimodal models on trillion-token datasets to running real-time inference for millions of concurrent users,” the company said in a blog post today.
With the maximum chip/server
— Read the rest of this post, which was originally published on Data Center Knowledge.
