Chinese web giants go on $5B Nvidia AI shopping spree

Chinese web giants go on $5B Nvidia AI shopping spree

Time Stamp: August 10, 2023 9:38 PM
Source Node: 2816928

China's largest web and cloud providers are lining up to buy as many Nvidia GPUs as they can while they still can get their hands on them.

Just as the White House announced fresh restrictions on US tech investments in the Middle Kingdom, a report emerged claiming that Alibaba, Baidu, ByteDance, and Tencent have collectively ordered 100,000 Nvidia A800 GPUs worth roughly $1 billion. A nerfed version of Nv's now three-year-old A100, the A800 was developed to circumvent performance limits on the export of American AI accelerators into China.

Citing "people familiar with the matter," the report adds that these same companies purchased another $4 billion worth of GPUs for delivery next year. However, at this point, it isn't clear which cards they're trying to get. If we had to guess, Nvidia's newer Hopper GPUs, like the China-spec H800 or potentially the US corporation's Ada Lovelace generation of GPUs.

Without these chips, one Baidu employee said, the web giant would be unable to train any substantial large language models. LLMs, including OpenAI's GPT-4, TII's Falcon-40B, and Meta's Llama series, are the lifeblood of generative AI. Several of Chinese organizations are in the process of developing their own LLMs to power competing services.

US export restrictions implemented last year placed limits on the IO bandwidth at no more than 600GB/sec for AI accelerators sold to countries of concern. These limitations were intended to hobble the performance of accelerators when networked together, limiting their usefulness for parameter-heavy AI workloads.

The ruling briefly barred Nvidia, AMD, and Intel from selling their latest generation of GPUs and accelerators in China. However, in the months since, and with hype around generative AI growing, chipmakers have worked to retrofit existing cards to comply with the rules. Nvidia was among the first to do so with the aforementioned A800, which halved the memory and cut the interconnect bandwidth to two thirds.

More recently, Intel announced a China-specific version of its Guadi2 AI accelerator for sale in China, while AMD, during its most recent earnings call, indicated it was working on a GPU that complied with US export laws for the Chinese market.

US lawmakers are pressing for stiffer restrictions on the export of AI accelerators to China, with some suggesting that performance caps should be set even lower.

Fully homegrown Chinese chips capable of replacing US made or developed silicon remain elusive. As we reported earlier this week, China's Loongson, a company leading the development of a homegrown CPU, is years behind Intel in terms of performance. And Alibaba Cloud likes to talk up its 128-CPU-core server-class Yitian 710 processor, though bear in mind those cores were licensed from Arm.

While GPUs get a lot of attention for their role in AI training, CPUs still play a role in inferencing as well as training.

Chinese GPU development has also been hampered by restrictions on US intellectual property used by major foundry operators. Biren Technology, a fabless Chinese chipmaker developing a TSMC-fabbed datacenter GPU, was forced to refactor its design to abide by the same 600GB/s interconnect restrictions as US chipmakers, for instance.

There are signs the Biden administration is looking for additional measures to not only curb Chinese access to American chips, but funding as well. In addition to a ten-year ban on Chinese developments for those availing themselves of the $39 billion of US CHIPS funding, President Joe Biden Wednesday signed an executive order restricting US investment in Chinese companies developing certain technologies of concern.

The order seeks to avoid US investors from supporting the development of Chinese quantum computing, networking, and AI technologies with military and intelligence-gathering uses. ®

Time Stamp:

More from The Register