Report: Agentic AI requires significant infrastructure upgrades

Compute accelerators like TPU 8t are available for training large models, while TPU 8i is designed for low-latency inference. General-purpose compute with Arm-based processors like Google Axion is also highlighted for AI control plane operations.

Agentic workloads require dynamic matching of silicon to tasks to manage scale and minimize overhead, addressing issues like data egress fees, storage bloat, and idle hardware. Fluid compute aims to reduce the "inference tax" and operational complexity associated with scaling AI.

Scaling autonomous agents necessitates a mature governance strategy, including a centralized control plane for permissions, identity, and workflows. Solutions like Agent Gateway provide visibility, audit trails, and human-in-the-loop oversight.

To enable agents to reason effectively across an organization, a unified data layer is needed. Tools like Smart Storage and the Cross-Cloud Lakehouse aim to make data searchable and accessible regardless of its location or format.

Hybrid multicloud architectures are becoming prevalent due to digital sovereignty and data gravity concerns, with 48% of leaders prioritizing data residency controls. Edge deployments are also crucial for reducing latency, ensuring operational resilience, and maintaining cost-efficiency.

The market has evolved from simple conversational AI to AI that takes action, automates workflows, and executes complex tasks autonomously. This shift unlocks new use cases but significantly stresses existing IT infrastructure.