Considerations for Retrofitting or Deploying AI Servers in Your Legacy Data Center

/live_board/v2/downloads?uid=89951c62-a7f3-4d6d-8c10-1f646c936092&redirect=true

Your ultimate goal is to deploy a private working AI model to reengineer and/or automate a business or technical process and put your company ahead of your competitors.

Your business needs are to achieve the lowest support function costs, have complete control of your IT assets and their function, and to meet your organization or government’s stringent privacy requirements. You are confident that this private AI working, or inference model will return a positive ROI and you are ready to strike out on your own to run this model on your own IT stack. When deciding on the configuration of the IT stack you want, you will need to set your priorities for accuracy, privacy, speed and ability to scale. You will also need to decide if the IT is 100% private on premise or hybrid with cloud.

Managing AI workloads is challenging for most people that are deploying for the first time, but there is very little guidance on getting started. Let’s break down some of the deployment challenges that you need to plan for.

You are a pioneer, a trailblazer — or maybe you’re just being smart about your budget. Whatever the motivation, you have already taken a meaningful first step toward deploying AI for your organization by running a successful AI pilot that you hosted in the cloud.

Seven Critical Questions to Ask Before You Deploy

1 2 3 Unlikely. Modern AI servers — especially those equipped with high-performance GPUs like NVIDIA’s Blackwell series — are typically deeper, significantly heavier and require additional infrastructure. This includes high-capacity power connections and liquid cooling manifolds. Most legacy racks were not designed to support these physical and thermal demands, so rack replacement or reinforcement is often necessary.

Probably not. AI servers require PDUs with higher input power ratings and a greater number of high-capacity outlets. You’ll likely need to upgrade to rack-mount PDUs that support three-phase power and can handle the increased electrical load per rack.

In most cases, no. Traditional data centers were designed for lower power densities or distributed workloads. AI workloads demand concentrated power delivery, which may require upgrades to: PDUs, medium-voltage switchgear, low-voltage switchgear, transformers, circuit breakers and busways or cabling. A comprehensive electrical assessment is recommended before deployment.

Will my existing IT racks be compatible with new AI servers?

Will my existing rack Power Distribution Unit (PDU) support new AI servers?

Can I use my existing power distribution infrastructure?

6 7

5 It depends on your current capacity, but not likely. For example, a single rack of NVIDIA GB200 (Blackwell) GPUs can consume up to 132 kW, equivalent to the power usage of 25 kitchen ovens running nonstop at full blast. Even deploying one or two such racks may necessitate a utility service upgrade to your facility.

Yes, but they are typically previous-generation models. Many large language models (LLMs) were trained on air-cooled or hybrid-cooled servers using GPUs like the A100 or H100. These GPU-based servers are still available but offer lower performance and efficiency and occupy more physical space; however, they are lower cost and may not require major upgrades to power or cooling infrastructure.

A liquid cooling system is a coordinated architecture that includes:

Proper design and integration of these components is critical for proper cooling effectiveness, reliability and efficiency.

No. Most next-generation AI servers are natively liquid-cooled and come with integrated cold water inlet and hot water outlet connections. These are not optional — they are required for operation.

Note: Liquid cooling does not eliminate the need for air cooling. Approximately 20–30% of the thermal load may still require air- based cooling for components like power supplies and memory.

Tip: For mid-range deployments, consider rear-door heat exchangers, which can cool up to 50–70 kW per rack depending on the model. These can usually be integrated with existing dry coolers, chillers or cooling towers.

Do I have enough utility power to support an AI compute stack?

Are there AI servers available that do not require liquid cooling?

What components are required to build a liquid cooling architecture for AI servers?

Are the latest AI servers available without liquid cooling?

CDUs (Coolant Distribution Units) Controls flow to/from servers

COMPONENT PURPOSE

Chillers/Dry Coolers Cools liquid for circulation

Piping Infrastructure Moves hot/cold water

Heat Exchangers Transfers heat from the server loop to the building loop

Sensors / Monitoring Ensures safe, optimized, and leak- free operations

Before accelerated compute AI servers, it was common for a data center to go through multiple IT refresh cycles during its useful life of 15-30 years. These refresh cycles required little or no upgrades to the IT rack and power and cooling systems.

In the AI era, with rapid advancements in accelerated compute technology, to deploy the latest GPUs you will most certainly need to upgrade the IT racks, power distribution units and add a liquid-cooled architecture consisting of manifolds, CDUs and chillers.

A more modest upgrade to lower density accelerated compute can be accomplished with the addition of rear-door heat exchangers. But any way you look at it, IT densities are getting higher and higher, making it more challenging to deploy with each new GPU generation.

Successfully moving from an AI pilot to a private, production-ready deployment is a bold but necessary step for organizations that want more control, better performance and long-term cost efficiency. But doing so isn’t as simple as buying new servers. From power density to rack compatibility and liquid cooling infrastructure, deploying AI on your own IT stack requires careful planning and the right expertise. Whether you’re pursuing the latest high-performance GPUs or starting with a more modest setup, by asking yourself the questions above, you’ll be better equipped to assess your infrastructure, map out the necessary upgrades and realize the ROI of running AI on your own terms.

Author

Steven Carlini Vice President, Innovation and Data Center Schneider Electric LinkedIn

THIS DOCUMENT IS TO BE CONSIDERED AS AN OPINION PAPER PRESENTING GENERAL AND NON-BINDING INFORMATION ON A PARTICULAR SUBJECT. THE ANALYSIS, HYPOTHESIS AND CONCLUSIONS PRESENTED THEREIN ARE PROVIDED AS IS WITH ALL FAULTS AND WITHOUT ANY REPRESENTATION OR WARRANTY OF ANY KIND OR NATURE, EITHER EXPRESS, IMPLIED OR OTHERWISE.

se.com

Schneider Electric For more information, please contact us at

https://www.linkedin.com/in/steven-carlini

Considerations for Retrofitting or Deploying AI Servers in Your Legacy Data Center

Item Type: pdf