Google’s new Virgo data center fabric demonstrates how AI workloads are significantly transforming network design in hyperscale environments. According to a recent blog post from the company, Virgo is specifically engineered for large AI clusters, involving tens of thousands of accelerators, within Google’s AI Hypercomputer architecture. This innovative fabric employs a flatter, two-layer topology aimed at reducing latency and enhancing bandwidth throughout the network.
The introduction of Virgo highlights a crucial evolution: the need for consistent performance in AI training and inference tasks—not just peak throughput. These processes require continuous data exchange over substantial east-west paths, demanding tight coordination; if one node lags, it can impede the entire operation.
Ron Westfall, vice president and analyst at HyperFrame Research, emphasized that Google’s approach considers variability as a systemic risk rather than merely a networking challenge. He noted that the Virgo fabric reimagines data centers as a "Campus-as-a-Computer," taking tail latency as a vital hardware reliability issue and isolating AI training traffic to keep large clusters in sync.
Google has designed Virgo to accommodate clusters that exceed 100,000 accelerators, prioritizing high bisection bandwidth and low latency system-wide. The fabric features multiple independent switching planes and comprehensive telemetry to monitor congestion or failures, allowing for efficient traffic rerouting without interrupting ongoing workloads. Recognizing that localized failures are anticipated at such scale, the fabric’s design aims to prevent these disruptions from affecting the entire cluster.
Fewer Layers, Less Variance
Traditional data center networks typically employ a multi-tier Clos architecture, balancing cost and efficiency through oversubscription. However, AI workloads disrupt this model by generating consistent east-west traffic, leading to link congestion. In response, Google has replaced the conventional three-tier designs with a two-layer fabric that minimizes hop counts between nodes, thereby reducing the potential for queuing delays.
Sameh Boujelbene, vice president at Dell’Oro Group, explained that a flatter architecture directly influences latency control. "Flattening reduces hop count and creates more direct, predictable paths between accelerators, crucial for synchronized workloads," she stated.
Westfall added that addressing variability takes precedence over merely mitigating congestion. By decreasing the cumulative probability of queuing delays at intermediate hops, the design ensures that synchronized workloads don’t stall due to a delayed packet. However, he cautioned that flattening alone isn’t enough in larger-scale systems; effective traffic distribution and optical interconnects remain essential to prevent congestion from accumulating as networks simplify.
A Segmented Data Center Fabric
Virgo operates within a broader, segmented architecture that distinctly categorizes internal data center traffic. Google separates tightly coupled accelerator communications, large-scale inter-cluster traffic, and north-south traffic connecting to storage and external services. This segmentation marks a shift from one-size-fits-all networking solutions to fabrics optimized for specific workload patterns.
This shift in approach is mirrored in vendor offerings catering to AI clusters. Companies like Nvidia are advancing platforms like Spectrum-X, which combines switches and DPUs to manage congestion and ensure reliable performance across GPU clusters. Likewise, Broadcom is supplying switching silicon to underpin many large Ethernet networks, and Arista Networks is focusing on AI backend networks through software tied to traffic management and load balancing across distributed clusters.
Westfall remarked that design advancements like Virgo raise expectations for various platforms: “There’s a critical need to prioritize tail-latency consistency as the key success metric,” underscoring the importance of high-radix switching and tighter integration between hardware and software to maintain workload synchronization.
Boujelbene pointed out that while vendors are making strides in congestion control and telemetry, they still face inherent limitations. "Even with improved congestion control and integration, hyperscalers retain an upper hand due to their comprehensive control over the entire stack," she explained.
A New Direction of Travel
Although Virgo is unique to Google, this trend of flattening topologies and increasing path diversity is prevalent across hyperscaler designs. Google positions their data center as a cohesive system where compute, storage, and networking operate in synchrony rather than as disjointed components.
According to Westfall, the degree of coordination established by hyperscalers is challenging for vendors to replicate. "Hyperscalers integrate the network with the AI stack, conceptualizing the data center as a unified, software-defined computer,” a level of cohesion that general-purpose vendors are unable to fully achieve.
As AI systems continue to expand, integrated design principles such as those found in the Virgo fabric are likely to reshape data center networking, starting at the hyperscale level and eventually influencing enterprise implementations.
Welcome to DediRock, your trusted partner in high-performance hosting solutions. At DediRock, we specialize in providing dedicated servers, VPS hosting, and cloud services tailored to meet the unique needs of businesses and individuals alike. Our mission is to deliver reliable, scalable, and secure hosting solutions that empower our clients to achieve their digital goals. With a commitment to exceptional customer support, cutting-edge technology, and robust infrastructure, DediRock stands out as a leader in the hosting industry. Join us and experience the difference that dedicated service and unwavering reliability can make for your online presence. Launch our website.