As AI data centers scale, silicon design is being pushed by several constraints at once. For Satish Premanathan, Vice President, HCLTech, the challenge starts with power and thermal density, but quickly extends into memory, interconnection, packaging and reliability.
Premanathan said AI data center silicon can now consume power at levels that would have been difficult to imagine in earlier generations of computing.
“The first tends to be power consumption,” he said. “Gone are the days when these silicon [chips] were operating at less than 25 watts. Today we are talking about one kilowatt.”
That increase changes both chip design and the systems built around it.
Power becomes a design constraint
As power consumption rises, designers need to manage the power profile of the silicon much more carefully.
Premanathan said techniques associated with low-power design are increasingly relevant inside the data center. Rather than treating energy consumption as a secondary concern, power control becomes part of the core architecture.
The power delivery system has also become more complex. High-performance AI silicon operates at low voltages while demanding very high current, creating difficult engineering requirements around how power reaches the device.
At the system level, that means more advanced power delivery architectures and greater coordination between the silicon and the infrastructure supporting it.
Cooling moves closer to the silicon
More power also means more heat.
“The thermal profile of the silicon becomes very high, so the cooling techniques that are adopted have to ensure that the silicon is cooled well to maintain the reliability of the silicon,” said Premanathan.
That is driving a shift away from relying only on traditional heat sinks and air cooling. Liquid cooling is becoming increasingly important as data center operators look for ways to keep high-density AI systems within the right thermal envelope.
Premanathan also connected thermal management with reliability. AI silicon needs to operate under significant thermal stress for years, which places additional emphasis on reliability testing during design and validation.
Memory and interconnects grow more complex
Power and cooling are only part of the challenge.
AI workloads also require very high memory bandwidth. Premanathan pointed to the growing use of high bandwidth memory (HBM) as designers try to move enough data into and out of the compute layer.
At the same time, the interconnect architecture is becoming more demanding.
High-speed data must move across silicon, clusters and between components. Premanathan pointed to faster generations of PCI Express, high-speed Ethernet fabrics and increasingly complex interconnects as part of that evolution.
He also highlighted optical and silicon photonics-based interconnects as a way to reduce the energy required to move data, with the aim of keeping energy consumption per gigabit of transfer as low as possible.
This makes the design problem broader than the processor itself. Compute, memory and connectivity all have to evolve together.
Advanced packaging becomes part of the architecture
The final challenge is packaging.
Premanathan said advanced packaging techniques are already being adopted in current-generation AI data center silicon and will become even more important as designs move toward chiplets.
As Moore’s law slows, increasingly complex systems can no longer rely on placing everything on a single monolithic die. Splitting designs into multiple chiplets creates new requirements for how those components are connected and packaged.
“When you divide it into multiple chiplets, you have no alternative but to have 2.5D or 3D packaging,” said Premanathan.
The challenge becomes more complex when HBM stacks and optical interconnects are added into the same system. Premanathan also pointed to the emergence of co-packaged optics as another factor in shaping package design.
Testing must evolve alongside packaging.
Combining multiple chiplets, memory stacks and other components increases the difficulty of validating the integrated system and can create yield challenges. As a result, package architecture and advanced package test strategies have become important parts of overall silicon economics.
A system-level design problem
The next generation of AI data center silicon cannot be optimized in isolation.
- Power management affects thermal design
- Thermal limits influence reliability
- Memory and interconnect choices affect performance and energy efficiency
- Chiplet architectures create new packaging and testing requirements
Together, those constraints are turning silicon development into a tightly connected system-level engineering problem.
Bringing these engineering domains together also increases the value of strategic partners that can work across silicon, systems, infrastructure and operations, helping organizations manage the complexity and accelerate innovation at scale.
For AI data centers, scaling compute will depend on faster chips and how effectively power, cooling, memory, connectivity and packaging are engineered around them.
