Most enterprise data centers in India were architected for traditional workloads: web applications, databases, ERP, file and print. The infrastructure choices made for those workloads — moderate compute density, standard memory configurations, 1G or 10G networking — are actively wrong for AI. If you are planning AI initiatives in the next 18 months, the infrastructure conversation needs to start now.
Compute: the GPU question
AI training and large-model inference require GPUs. The H100 and L40S from NVIDIA are the current enterprise standards, but the lead times, power requirements, and per-rack costs are dramatically higher than CPU-only servers. A single 8-GPU H100 server can draw 10kW — more than an entire rack of traditional servers. Most existing data center power and cooling capacity cannot accommodate even a small GPU cluster without significant infrastructure upgrades.
Memory and storage bandwidth
AI workloads are memory-bound far more than traditional workloads. High-bandwidth memory (HBM), NVMe storage, and fast interconnects between GPUs are not optional — they are the difference between a model that trains in days and one that trains in weeks. Legacy SAN architectures, even fast ones, are often the bottleneck. Local NVMe and parallel file systems are now standard for AI clusters.
Network: east-west traffic dominates
AI workloads generate enormous east-west traffic between compute nodes. A traditional 3-tier network designed for north-south user traffic will throttle a GPU cluster regardless of how powerful the GPUs are. Spine-leaf architectures, 100G or 400G interconnects, and RDMA-capable networking (RoCE or InfiniBand) are the baseline for serious AI infrastructure.
What to do now
If AI is on your 18-month roadmap, three actions matter today. First, audit your data center power and cooling — most enterprises will need upgrades before any GPU deployment. Second, plan your network refresh to be spine-leaf and 100G-ready, even if you do not need it yet. Third, when you refresh storage, choose NVMe-first architectures. The cost of making these decisions correctly during a normal refresh cycle is a fraction of the cost of retrofitting later when AI projects are already approved and waiting.