ChatGPT will need over 30,000 Nvidia GPUs to go commercial

3 years ago 2

ARTICLE AD BOX

OpenAI will need 30,000 top-end Nvidia GPUs to power a commercial version of its ChatGPT language model, according to an analysis by TrendForce.

The market research firm explained that the generative AI that made ChatGPT and other language models useful required a huge amount of data for training.

Deploying a large number of high-performance GPUs to process that data could shorten the training time, ensuring the models developed as fast as required to remain competitive.

“In the case of the Generative Pre-Trained Transformer (GPT) that underlays ChatGPT, the number of training parameters used in the development of this autoregressive language model rose from around 120 million in 2018 to almost 180 billion in 2020,” TrendForce said.

TrendForce estimated the number of GPUs that the GPT model needed to process that amount of training data in 2020 was already around 20,000.

That would be assuming OpenAI used Nvidia’s powerful A100 data centre GPU, which was specifically designed for large-scale machine learning.

For the full commercialisation of the GPT model, TrendForce expected OpenAI would need over 30,000 of these GPUs.

The cost of this hardware would be monumental.

While OpenAI is likely to have gotten bulk discounts on its GPUs, it is worth noting that the Nvidia A100 comes with a retail price ranging between $10,000 (R182,189) and $15,000 (R273,277.50, excl. VAT).

On Amazon, PCIe versions of the GPU with 40GB memory currently sell for $8,519 (R155,154.50).

But the latest model boasts 80GB, so it’s more likely to sell for around $15,000 (R273,277.50).

Buying 30,000 of these could cost up to $450 million, or R8.2 billion.

TrendForce anticipated that manufacturers like Nvidia and AMD would be raking in money due to the general increase in generative AI popularity.

Nvidia will probably gain the most from the rise in GPU demand, TrendForce said.

“Its DGX A100, which is a universal system for AI-related workloads, delivers 5 petaFLOPS and has nearly become the top choice for big data analysis and AI acceleration,” TrendForce said.

“AMD has also successively launched the MI00, MI200, and MI300 series of server chips that are widely adopted for AI-powered applications.”