How 6 AI Attributes Change Data Center Design Key insights for technology leaders

/live_board/v2/downloads?uid=ee546b63-472c-4c12-b484-c0f98f033822&redirect=true

Key insights for technology leaders

From large training clusters to small edge inference servers, AI is becoming a larger percentage of data center workloads. This represents a shift to higher rack power densities. AI start-ups, enterprises, colocation providers, and internet giants must consider the impact of these densities on the design and management of the data center physical infrastructure.

This paper explains the attributes and trends of AI applications that impact data center physical infrastructure. We then present key design considerations for power, cooling, and rack systems. Finally, we explain how good supply chain practices, software management tools, and services mitigate risks inherent in AI deployments and highlight future physical infrastructure.

How 6 AI Attributes Change Data Center Design

New white paper

Read it today!

https://www.se.com/us/en/download/document/SPD_WP110_EN/

This diagram shows how the six AI attributes and trends map to the three physical infrastructure systems. The lines reveal which attributes contribute to power, cooling, and rack system challenges. For example, network communication is a key reason why the cooling system must use liquid to reject over 100 kW from a rack.

Power

Cooling

Racks

AI Workloads

Accelerator network communication

Thermal design power (TDP) of accelerators

Peak power of accelerators

Synchronous computation

AI cluster size

RacksCoolingPower

AI workloads fall into two general categories: training and inference. Training workloads utilize large numbers of servers working together to process massive datasets, with some racks exceeding 100 kW. Inference workloads deploy trained models to answer user queries, often after model compression, with rack densities ranging from a few hundred watts to over 100 kW.

AI workloads

AI training relies on clusters of accelerators operating together. Interconnect performance — bandwidth and latency — directly impacts compute and power e�ciency. This is a main reason behind the high rack densities.

Accelerator network communication

Six key AI attributes and trends impact physical infrastructure:

AI accelerators have high thermal design power, creating significant heat at the rack level. Traditional cooling approaches are challenged. Direct liquid cooling and advanced thermal strategies are increasingly necessary.

Thermal Design Power (TDP)

AI workloads introduce dynamic power demand. Electrical systems must be designed to handle these fluctuations while maintaining availability and resilience across the power chain.

Peak power of accelerators

Six key AI attributes and trends impact physical infrastructure:

Training large AI models requires tightly coordinated processing across accelerators. Any latency or downtime can disrupt operations. This means that peak power draw occurs at the same time many times per second, acting like quick step loads. Power infrastructure must support this.

Synchronous computation

Most traditional data centers today can support peak rack power densities of about 10 to 20 kW. But deploying tens or hundreds of racks all greater than 40 kW in an AI cluster presents physical infrastructure challenges.

AI cluster

Six key AI attributes and trends impact physical infrastructure:

Next steps The rapid growth of AI is reshaping data center design and operations. Both inference and training workloads introduce unique challenges, with training clusters reaching over 100 kW per rack and driving the need for robust infrastructure strategies.

Engage design partners early to plan electrical provisioning and system design.

For existing sites, conduct feasibility studies using EPMS and DCIM to assess capacity, peak loads, and circuit utilization.

Perform safety and technical studies such as capacity analysis and arc flash, supported by electrical design software.

Use digital tools like iSLDs and digital twins for accurate modeling and simulation of AI cluster

POWER

Begin with a comprehensive design assessment of liquid-cooled loads and current facility conditions.

Seek expert review to avoid building constraints and manage retrofitting complexity.

Address the lack of liquid cooling standards by leveraging partner expertise and integration support.

Use thermal modeling and fluid dynamics tools to ensure e�ective, compatible cooling solutions.

COOLING

RACKS

Specify racks that are wider, deeper, taller, and rated for higher weight capacities.

Validate floor loading for weights exceeding 3,000 kg and confirm structured cabling tray capacity.

Independently verify structural capacities before deployment.

Procure needed racks early to mitigate long lead times.

SOFTWARE TOOLS AND SERVICES

Use DCIM, EPMS, BMS, and digital design tools to reduce risks in complex electrical networks.

Create a digital twin to identify power and cooling constraints and guide layout decisions.

Apply reference architectures to streamline design and deployment.

Leverage vendor services when internal expertise or bandwidth is limited.

Retrofitting Existing Power Systems for AI Clusters

Direct Liquid Cooling System Challenges in Data Centers

Navigating Liquid Cooling Architectures for Data Centers with AI Workloads

10 Ways to Harness the Energy and Water E�ciencies of Direct Liquid Cooling

How 6 AI Attributes Change Data Center Design Key insights for technology leaders

Item Type: pdf