Welcome to the Pulse of AI Infrastructure

At the intersection of AI breakthroughs, data center infrastructure, and cutting-edge technology, staying informed isn’t just smart—it’s non-negotiable. Infra Insider is your bi-weekly guide to the latest in data center trends, AI advancements, land and power updates, capacity shifts, and critical developments shaping the IT landscape.

We’re not just reporting the news—we’re also connecting you to the latest technologies, events, and partnerships that drive the future of technology.

2024 was another monster year for AI progress and spending on AI. In 2024, organizations worldwide spent an estimated $235 billion on AI, encompassing applications, infrastructure, and related services.

The Future of AI is not certain but the huge spending is for the belief that we will transform the information industries with AGI-like or even super intelligent capabilities. There is evidence that AI capabilities are providing PHD level answers and reasoning from the OpenAI O3 model and from Google’s Deep Research system. Elon Musk told Mark Penn at CES that in 3-4 years we could perform all human labor tasks and can do any cognitive task.

There is visibility where we know there is growing usefulness for AI. There is value in answers that replace a hundred billion dollars worth of Google search. There is value in creating images and video. There is value in improving a trillion dollars in human programmer productivity. Revamping the $5 trillion global IT industry is a clearly achievable objective. There will be new products and uses.

Brad Gerstner, Altimeter Capital, says that the industry is still compute and energy constrained in 2024 and will remain compute and energy constrained at the end of 2025. There is also a memory shortage.

The core of the AI hardware story of 2025 is 2+ million Nvidia  B200 chips. The ASIC chips will matter but will be tiny in the overall market.

The improved reasoning inference need perhaps 100 times more energy to get the quality answers. OpenAI O3 using $3000 per answer is about 3000 GPU hours for an answer.

Agentic AI requires agents to perform dozens or hundreds of steps and actions.

More reasoning steps, more complex reasoning and more agentic steps and actions all require more compute and energy.

The clear evidence is that AI performance and infrastructure is still scaling in a sustainable way. AI pre-training may be having a slowing curve but post-training and test-time training are just starting.

Bill Gurley said all incremental free cash flow is now going to building and developing AI. The forecasted cumulative incremental free cash flow of major companies would be an approximation of global AI spend. AI investments will need to return increased free cash flow and corporate returns to enable continued spending and revenue growth.

In 2024, generative AI accounted for 17% of global AI spending, and this share is projected to rise to 32% by 2028, with about 60% five-year CAGR. Bloomberg Intelligence estimates that generative AI spending will grow by 71% in 2025.

Real world AI is emerging with self-driving cars and trucks which could see inflection moments in 2025. There could be safer than human driving cars as soon as Q2 of 2025. Humanoid bots will start scaling at a few tens of thousands in 2025 and over a hundred thousand in 2026 and perhaps a million in 2027. 

Amazon is spending $11 billion plan to expand its cloud computing and AI infrastructure in Georgia. Microsoft announced a plan to spend $80 billion on AI in 2025.

Sequoia Capital reports the AI industry spent $50 billion on Nvidia chips for training in 2024 but generated only $3 billion in revenue in 2024.

Nvidia's AI spending is primarily focused on R&D and strategic investments. In 2024, Nvidia invested $1 billion in various AI ventures, including funding rounds for AI startups and corporate deals. Nvidia's R&D expenses have been steadily increasing, reaching $8.7 billion in fiscal year 2024.

Nvidia has applied AI to improve its own chip designs. 

There are larger projects to use AI to speed up and improve science and technological development.

In 2023, TSMC spent over $13 billion on R&D and planned to increase its R&D budget for 2024 by as much as 20%. TSMC's capex spending in 2024 was projected to be between $28 billion and $32 billion and this will increase to $36-40 billion in 2025. 

Here are estimates of the major models that have and will be released and the level of compute that will be used. This is based upon information and statements made by executives and other sources.

There is other analysis of what will be needed in the changing AI emphasis on test time training and inference. There will be new chips and systems focused on different aspects of inference.

There is an estimation of the scaling constraints. However, there will be thousands of brilliant researchers and companies that will remove or mitigate constraints.

AI is world-changing and the level and rate of change is more likely to be larger than expected instead of less than expected. 

The following newsletters will dig into the details of approximating usage of infrastructure for different companies seeking utility from their AI spend.

Liquid Cooling: The Future of AI Infrastructure

Liquid cooling is becoming a critical component of AI infrastructure discussions—and for good reason. As the AI boom accelerates and rack densities increase, data centers are transitioning from traditional air cooling to hybrid systems that incorporate liquid cooling. In 2025, this technology will be at the forefront of AI deployment strategies.

Why The Shift?

While air cooling remains essential, liquid cooling is stepping up to handle the power densities required by advanced AI hardware. For example, NVIDIA’s GB200 chip, hitting the market this year, significantly raises power demands. Racks are now the standard deployment unit, with configurations like the NVL72 featuring 72 GPUs and consuming around 130 kilowatts. Large-scale GPU clusters require megawatts of power, with racks among the GPUs dedicated to the liquid cooling system.

This evolution introduces new challenges. Data center operators must make critical decisions about their cooling infrastructure to future-proof their facilities. Key considerations include:

• Compatibility between rack designs and cooling systems.

• The ability to support liquid-to-chip cooling, rear door heat exchangers and immersion cooling.

• Whether fully enclosed liquid-cooled systems can eliminate hot air circulation entirely.

Data centers are already exploring diverse approaches:

• Some are preparing for densities of 300+ kilowatts per rack.

• Others aim for 150 kilowatts per rack.

• Many legacy facilities remain below 50 kilowatts per rack, relying on air cooling.

Matching customers with data centers that meet their unique requirements is becoming more complex but also more vital. At Ignite and Filos, we’re working closely with operators and customers to navigate these changes, ensuring compatibility and avoiding costly delays.

Liquid cooling is not just a technical upgrade—it’s a necessity for scaling AI. As the AI revolution unfolds, liquid cooling will unlock the next level of performance and efficiency in data center infrastructure. 2025 promises to be a transformative year!

Capacity Opportunities For 2025

25 MW on the West Coast of the US

• Cost of electricity: 7.5 cents per kW-hr

• PUE < 1.2

• Rack densities up to 200 kW/rack

20 MW in the South Central Region of the US

10+ MW in Iceland

• Cost of electricity: 7 cents per kW-hr

• PUE < 1.1

• 100% renewably powered

• Latency: 10 ms to the US

• Rack densities up to 140 kW/rack

The above sites are great for large-scale deployments and we have plenty of inventory for those that need a few megawatts or hundreds of kilowatts of capacity. What kind of infrastructure are you looking for? Please book time here with our team to learn about what’s available.

Your Feedback Matters

At Infra Insider, we’re here to give you an edge—exclusive insights into the evolving infrastructure world that others miss. While most newsletters skim the surface, we go deeper, spotlighting the trends shaping AI, data centers, and tech capacity. Think behind-the-scenes intel, key players driving change, and actionable updates straight from the field. It’s not just another industry briefing—it’s your front-row seat to the future.

Your feedback is crucial in helping us refine our content and maintain the newsletter's value for you and your fellow readers. We welcome your suggestions on how we can improve our offering; [email protected]. 

Nina Tusova
Director of Customer Relationship & Success // Team Ignite