Hyperscale Bitbarns Will Grow Because Of AI Demand

Republished By Plato

Followers: 0

Total capacity of hyperscale datacenters is set to grow almost threefold over the next six years on the back of AI demand, substantially increasing the amount of power required by those facilities.

With the generative AI hype cycle in perpetual motion, datacenter operators are planning ahead to anticipate the need for higher density, higher performance infrastructure to meet the processing requirements.

A new report from analyst IDC, for example, forecasts that enterprises worldwide are set to blow nearly $16 billion on generative AI in 2023. This spending, which includes software as well as related infrastructure hardware and IT/business services, is estimated to reach $143 billion in 2027.

The upshot of this, according to Synergy Research Group, is that the average capacity of any hyperscale datacenter opening over the next several years will be more than double that of current facilities.

There will also be some retrofitting of existing datacenters to boost their capacity, and the average IT load of individual bit barns continues to grow, with the result that Synergy predicts the total capacity of all hyperscale datacenters will almost triple in the next six years.

Synergy based this analysis on the operations of 19 of the world's biggest cloud and internet service firms. This includes the providers of SaaS, IaaS, PaaS, search, social networking, e-commerce and gaming.

As of 2023, those hyperscalers had a total of 926 massive bit barns in operation around the world, and Synergy said it already knows of a further 427 facilities that are in the pipeline.

Synergy says the worldwide total number of datacenters has already doubled over the last five years. It predicts these will continue to grow by well over a hundred per year.

However, the recent advances in generative AI will not necessarily speed up the construction of data dormitories, but will rather "substantially increase" the amount of power required to operate those facilities, thanks to the burgeoning number of high-wattage GPU accelerators being crammed into server nodes.

This was noted by another research outfit, Omdia, which found that demand for servers fitted with eight GPUs for AI processing work has also had the effect of pushing up average prices for datacenter systems.

Synergy is coy about how much it reckons the amount of power required will "substantially increase".

However, a recent research paper calculated that integrating generative AI into every Google search could potentially consume the same amount of power as a country the size of Ireland.

IDC senior research director for Europe Andrew Buss agreed that AI is driving demand for higher performance datacenter infrastructure.

"We do see a huge amount of accelerated compute capacity being installed," he told us. "We see hyperscalers buying a significant amount of the overall AI accelerators that are coming onto the market to support the large generative and transformer models across B2C and B2B customers, as well as many organizations trying to get some supply as well."

This is increasing the power density of the servers and creating a lot of power supply and cooling issues, Buss said. "Many datacenters are built with a power budget of between 7.5 and 15kW per rack, but now a single Nvidia DGX can use up 10kW, meaning the entire power budget is used by a single 10U box," he explained.

Synergy chief analyst John Dinsdale told us that power concerns are causing hyperscale operators to rethink some of their datacenter architecture and deployment plans to amend the layout and enable much higher power density per rack, and possibly even review the location of their data dormitories.

"It's not just about power availability and cost," Dinsdale said. "Many AI workloads are not as latency sensitive as other workloads, so can allow the operator to place datacenters in more distant and less expensive locations. For example, we were already seeing hyperscale datacenter growth in the US Midwest outpacing growth in other regions such as Northern Virginia and Silicon Valley. We fully expect that trend to continue," he added.

Just this week, Nvidia and Taiwanese electronics manufacturer Foxconn announced plans to team up and build what they call "AI factories", meaning datacenters dedicated to AI processing.

"A new type of manufacturing has emerged - the production of intelligence. And the datacenters that produce them are AI factories," Nvidia CEO Jensen Huang said in a statement, adding that Foxconn has the expertise and scale to build those AI factories globally.

Foxconn will use Nvidia's tech to develop new datacenters for generative AI services that cover a range of applications, including industrial robots and self-driving cars. Foxconn is expected to build a large number of systems based on Nvidia's CPUs, GPUs and networking for its global customer base, many of which are seeking to create and operate their own AI factories, Nvidia claimed. ®