This post was originally published on Data Center Knowledge
Technological innovations that generate a lot of hype typically have to face harsh reality at some point when practitioners begin deployment. For AI, especially generative AI, that reality is starting to sink in.
“Training (a large language model) is extremely costly,” said Constantine Goltsev, partner at AI/ML solutions agency theMind, during a recent panel at Data Center World 2023 in Austin, Texas.
With ChatGPT’s 175 billion parameters, he said, it meant OpenAI had to do 175 billion calculations of the input to produce results, using up gigawatts of power overall. With GPT-4, OpenAI used 12,000 to 15,000 Nvidia A100 chips — each costing $10,000 — on Azure and ran compute for months.
The good news is companies do not have to use gigantic language models because much smaller, open source ones can deliver results that are just as good as ChatGPT or even surpass it.
“You don’t necessarily need the large language model on the industrial scale, like ChatGPT or GPT-4, to do a lot of useful stuff,” Goltsev said. “You can take smaller academic models or open source models on the order of
— Read the rest of this post, which was originally published on Data Center Knowledge.
