Per-MW pricing, regional variance, and cost drivers for owners scoping hyperscale & AI builds.
Salary benchmarks across the 14 mission-critical disciplines.
I would release only the AI capacity that passes load and failure tests - not the capacity listed on equipment labels. Before testing, I’d agree on pass/fail limits, assign test owners, and define safe stopping points.
My commissioning checklist covers four areas:
The rule I’d use: <u>50% load testing does not prove 100% capacity readiness.</u> Your release decision should match what passed, with written limits for anything still untested.
AI Data Center Commissioning: Test-to-Release Workflow
Trace the full electrical path, from medium-voltage service through transformers, switchgear, protective devices, UPS systems, generators, automatic transfer switches (ATS), panelboards, busways, rack power distribution, and monitoring systems. Use factory acceptance testing (FAT) to confirm ratings, controls, protection settings, communications, and OEM performance. Check site acceptance testing (SAT) and pre-functional records for terminations, insulation resistance, grounding, phasing, labeling, and as-builts.
Increase load only after the previous stage passes. At each approved load step, measure delivered kW/MW capacity, voltage, frequency, harmonics, phase balance, breaker temperatures, UPS ride-through time, and transfer timing using representative AI load patterns. Check protective coordination against installed settings, and test normal, emergency, and redundant paths.
Use stepped and variable load profiles that reflect idle periods, ramp-up, sustained peaks, rapid swings, and uneven loading across paths. A steady load-bank run alone does not prove AI readiness.
Once the electrical path is stable, check the liquid-cooling loop that removes AI heat. The facility water system (FWS) serves the building-side loop. The technology cooling system (TCS) serves liquid-cooled IT equipment. The CDU transfers heat between the two.
Before applying load, verify flushing, cleanliness, chemistry, pressure tests, venting, sensor calibration, and leak detection. Confirm the specified passivation or material-compatibility treatment, inspect strainers and filters, and check valve positions. Isolate components that cannot withstand piping test pressure. Check quick-disconnects using manufacturer-approved procedures.
Begin with no-load and low-flow circulation checks, then add representative heat loads in approved steps. Record flow, pressure, differential pressure, and temperatures in the units required by the specification. Check branch balance, approach temperature difference, control stability, and design limits. Compare building management system (BMS) and DCIM telemetry with calibrated instruments.
Test failure response under load. Include pump failure, CDU outage, facility-water loss, loss of TCS flow, valve faults, communications loss, and loss of pump or control power. Record alarms, thermal response, and safe recovery.
After functional checks, secure the test records before retesting or turnover. During generator transfer, verify cooling continuity with representative heat applied. Start the approved utility-loss sequence and confirm that UPS-backed controls, pumps, CDUs, valves, and monitoring systems behave as designed during generator startup and ATS transfer. Align electrical and thermal records by time to show whether service continues or recovers within approved limits.
Retain instrument IDs, calibration status, measured results, exceptions, and witnessed retest outcomes. These records support final acceptance and handover. Use the matrix below to assign each test package to its system owner and link it to the required evidence and failure conditions.
Map command and status paths across BMS, EPMS, programmable logic controllers (PLCs), SCADA, DCIM, fire alarm, and IT-management interfaces. For each critical point, record ownership, units, protocol, scan rate, timestamp source, alarm priority, fail state, permissives, interlocks, delays, overrides, and operator action. Stale normal status is not proof of safe operation. Make this interface map part of commissioning coordination - not just a controls task.
Use failure-mode-and-effects analysis (FMEA) to identify shared dependencies. Redundant equipment may still depend on a single network switch, control-power supply, or sensor. Turn high-risk failures into witnessed test scripts that check cooling continuity, valid telemetry, safe restart, and required fire-alarm actions under AI loads. Record measured evidence in the last two columns, and obtain signatures from the test director, controls lead, affected trade, and owner.
Use the highest-risk failures to set the next stage gates and abort limits.
Use the interface map and FMEA to set the gate order, observers, and abort limits. Do not advance on equipment status alone. Verify power and cooling capacity before every load increase.
Before fault injection, assign the test director, safety lead, switching authority, discipline observers, and an independent recorder. Set one command channel. Brief everyone on switching boundaries, lockout/tagout, arc-flash controls, abort limits, and restoration steps.
Abort for unsafe electrical conditions, unexpected protective operation, lost cooling, thermal excursions, escalating leaks, or loss of reliable communications. Stabilize the safest approved state before authorizing another attempt.
Advance only when evidence from the prior gate supports the next load step.
Use the same representative AI load profile for recovery testing. With load banks that provide representative thermal loading or approved IT equipment, test rapid idle-to-peak and peak-to-idle ramps, uneven rack demand, and simultaneous compute/networking demand. Document how the setup reproduces planned electrical and thermal behavior. Measure detection, command, startup, and stabilization times against approved design and OEM limits, not a generic transfer-time target.
With owner, utility, OEM, and safety approval, simulate utility loss at representative load. Then verify UPS ride-through, generator start, ATS transfer, cooling continuity, load ramps, and return to automatic control. Confirm valid control states and communications alarms. Allow restart and return to automatic control only when power and cooling permissives are satisfied.
During utility restoration, verify UPS recharge and cooling stability under changing demand. Passing requires no prohibited interruption, no thermal excursion, and complete time-aligned records.
Test leak isolation, cooling lead/lag failure, and required fire-alarm responses separately before combining faults. During maintenance bypass, allow only risk-assessed combinations that the remaining capacity can support. Test only approved fault combinations with explicit load limits, observers, abort criteria, and restoration steps.
Once IST planning is set, secure the specialists who will close findings and support turnover. Owners and GCs should book electrical testing, relay, liquid-cooling, controls, TAB, and IST specialists early. Keep OEM troubleshooting support available during scheduled tests. Hire commissioning managers, discipline leads, and operations representatives with documented experience in failure testing, deficiency closure, and handover - not just credentials.[11][12]
Give each commissioning task one accountable owner and one verifier. Make turnover duties clear: trades and OEMs make corrections; the commissioning agent (CxA) verifies closure and gathers evidence; operations validates procedures; and the owner approves acceptance. Build a commissioning asset register covering every sensor, valve, control point, network address, load bank, instrument, correction, and retest.[11]
Then move directly to deficiency tracking and closure.
Group findings by life safety, availability, performance, maintainability, or documentation, then assign severity. For each finding, log the asset tag, location, discovery date, failed test step, symptom, root cause, accountable owner, corrective action, due date, retest method, and closure evidence.[11]
Address interface defects early, and reserve time in the schedule for corrections and retesting. A missing leak response, incorrect relay setting, or uncontrolled CDU shutdown blocks acceptance. Classify labeling errors as maintainability defects.[11]
Any accepted risk requires written owner approval that names the affected capacity, operating limit, compensating control, expiration date, and risk owner. Track every item to closure before turnover.[11]
Handover is a controlled transfer of records, access, and operating authority. Deliver the full commissioning package: approved as-builts, sequences, point lists, addresses, alarm priorities, setpoints, test records, logs, warranties, contacts, spares, backups, and operating procedures.[8][10]
Operations needs live system access and change authority to run, change, and maintain the facility - not just a static document set. Keep contractors and OEMs involved until the owner finishes its assessment.[8][10]
Attendance is not readiness. Before operational acceptance, operators must demonstrate approved normal, emergency, bypass, leak-response, and power-recovery procedures without contractor prompting. Record results, errors, remedial training, and supervisor sign-off.[3][10][11]
Set change-control requirements for firmware, control logic, setpoints, CDU configuration, rack deployment, and workload-related operating limits. Plan seasonal testing and post-deployment reviews of rack power, coolant flow, temperature, differential pressure, alarms, and recovery time. Major changes should trigger a capacity review or controlled recommissioning.[3][10][11]
Once power, cooling, controls, and recovery testing are complete, release only the capacity that passed.
Release requires signed-off acceptance criteria, verified power and liquid cooling, and controls that pass failure tests under representative loads. Critical deficiencies must be closed, minor exceptions formally accepted, and records complete. Operators must also demonstrate their ability to run the systems. Power-loss recovery must prove cooling continuity and stable automatic operation - not just a successful electrical transfer.[6] Anything less is not ready for production.
Partial-load evidence does not prove full-capacity readiness. Keep capacity conditional if cooling flow, controls integration, or recovery remains unproven. Require approval from the owner, Cx authority, operations, and the relevant discipline for release and any time-bound operating limits.
Resolve power-to-controls and CDU-to-BMS interfaces before final testing. Early specialist coordination prevents interface failures that block release.[7][9]
Use calibrated load banks to simulate expected electrical and thermal conditions during Level 4 (Functional Performance Testing) and Level 5 (Integrated Systems Testing).
Prepare a test plan that defines rack density, expected power draw, power profiles, and the testing sequence. Check that cooling controls respond to simulated workloads. Before deploying AI hardware, verify that power and cooling systems remain stable, maintain redundancy, and handle failover.
Set clear, measurable acceptance criteria in your Owner's Project Requirements (OPR) during planning [1][2]. Schedule commissioning by data hall or plant module to align with the master schedule. Complete each phase’s functional testing before moving to the next [3][4].
Use staged load bank increments (25%–100%) to test representative power densities - not theoretical capacity [3][5]. At startup handoff, document any remaining limits and assign an owner and resolution date to each unresolved item [5].
Adding new hardware to direct-to-chip (D2C) cooling setups requires recommissioning [1]. As AI workloads and rack densities change day-to-day demands, teams must also pressure-test infrastructure roadmaps twice a year to check that they meet current performance requirements [2].
Regularly check dynamic control logic to confirm cooling systems respond correctly to real-time workload changes. These checks help maintain efficiency and reliability as compute loads shift [2].
