Phil Burr, Director, Lumai, with three reasons why the total cost of datacentre ownership is rising and how 3D optics can help tackle them.
Much has been said about the vast energy consumption and environmental impact of datacentres. The UK National Grid has predicted that the boom in AI will cause datacentre power use in the UK to surge six-fold in 10 years.
But less in being said about the huge rise in costs that this increase in power demand is producing – costs which ultimately make it harder for businesses to benefit from AI.
Goldman Sachs has predicted that AI processing means the tech industry is on track to hit the $1 trillion mark for spend on datacentres and hardware in the next few years. While this spend is turbocharging the speed of innovation and pushing requirements to new levels, this is also creating a power strain at both datacentre and rack level.
This means more investment into components like hardware, cooling, power delivery and converters to match energy demand – and the cycle continues.
Without significantly reducing the energy consumption per AI calculation, the total cost of ownership (TCO) of datacentres will increase, the industry will not be able to meet surging energy demand nor improve its sustainability.
AI acceleration using 3D optics is a pioneering new approach to maximising the energy efficiency and performance of datacentres, creating cost-efficient processing that can reduce power consumption while meeting AI performance.
Here are three reasons why TCO is rising and how 3D optics can help tackle them.
- AI is rapidly increasing datacentre capacity demand
McKinsey recently released an eye-opening new study on the global demand for new data centre capacity, predicting that the overall demand for datacentre capacity “could rise at an annual rate of between 19 and 22 per cent from 2023 to 2030”. The report highlights that AI is the driving force behind this, with AI-ready datacentre capacity demand at 33% and GenAI at 39%.
To create this capacity, not only are the major cloud service providers building their new datacentres but also partnering with colocation providers to use their facilities. But while McKinsey notes the prices charged by colocation providers “fell steadily from 2014 to 2020 in most primary markets”, they “then rose by an average of 35 per cent between 2020 and 2023”, clearly reflecting the demand.
While this demand cannot be met solely by reusing or upgrading current infrastructure, it should be part of the overall mix both to reduce the need to build new infrastructure and to improve sustainability. To achieve this, the industry needs to adopt new ways of performing AI computation and reduce the power needed for AI processing, so that the processing can be done within the power limits of these datacentres. This will take vision and leadership but given the power capacity constraints and the insatiable demand, it is clear that now is the time to look at these different approaches. One such approach is to use technologies like 3D optics.
One of the limitations of datacentres is the current use of power-hungry silicon chip based – AI accelerators. These current chips are unable to efficiently scale and provide the level of capacity needed for AI’s growing compute demand within reasonable power constraints. But if datacentres can use an optical AI accelerator, the benefits of low-power and energy efficiency, which are already seen in optical communications, can be utilised for computation.
Crucially, using optics for computation leads to a far more scalable approach so that the increasing demands of AI computation can continue to be met.
- Datacentres are power hungry – and current hardware is too inefficient
The simple truth is that current hardware will not be able to efficiently match the surging performance demand required by AI models – it is either way too expensive or not efficiently possible with current chip technology.
AI is placing immense pressure on the energy consumption of servers in datacentre racks. As the McKinsey study showed, “average power densities have more than doubled in just two years, to 17 kilowatts (kW) per rack… and are expected to rise to as high as 30 kW by 2027 as AI workloads increase”. With models like ChatGPT, energy consumption can be over “80kW per rack”, while Nvidia’s latest chip may need rack densities “up to 120kW”.
The direct costs of supplying this energy and the infrastructure costs of cooling all of this power are significant; each Watt of power consumed necessitates more cooling, more energy, more infrastructure and therefore more generated emissions.
Optical AI acceleration uses photons to compute instead of electrons and performs highly parallel computing. This means that optical AI accelerators use only 10% of the power of a GPU (currently used in datacentres) while also providing the necessary leap in performance. If optical computation can enable more efficient AI accelerators, this can increase the lifespan of existing or planned datacentres and reduce the need for new ones, significantly lowering TCO.
- The latest silicon technology is very expensive
Maximising performance in current AI accelerator products is a key area of focus for the industry. However, the current approach to meet this AI demand is to add more silicon area, more power and, crucially, more cost. Earlier this year, Nvidia reported its new chip, the Blackwell GPU, would cost $30,000 to $40,000, with the costs of its development amounting to a massive $10bn. It’s a process of chasing diminishing returns.
What’s needed is a cost-effective way of using existing optical and electronic technology in datacentres. Optical processors can leverage such infrastructure, removing the need to use expensive new silicon technology. Therefore, if we combine these cost savings with reduced power consumption and less cooling, the TCO is a fraction of a GPU.
How reducing TCO can help the industry moving forward
If we look back to 2015-2019, even as workloads trebled, datacentre power demand remained relatively stable due to a focus on efficiency. AI represents a much larger challenge, but this focus on efficiency shows the industry what’s possible when it innovates. As well as reducing costs, optical AI acceleration can play a key role in reducing the TCO and the environmental footprint of AI.
The current trajectory is unsustainable, both for the planet and AI development – the projected TCO for AI datacentres is far too high to meet requirements in both areas. It’s worth reminding ourselves that a sustainable approach also aligns with a cost-efficient one. With the help of new technology like optical AI acceleration, datacentres can reduce their TCO and create a cycle of sustainable investment.
