/overlaid

This post discusses the application of BGP EVPN/VXLAN in AI data center networking, particularly focusing on back-end GPU fabrics. It emphasizes that single-tenant setups may not require VXLAN overlays, while multi-tenant environments benefit from logical segmentation. Key insights include the preference for Symmetric IRB over Asymmetric and the relevance of Type 5 routes for subnet aggregation.

In part five of a series on AI data center networking, various interconnected networks are discussed, highlighting the differences between traditional and AI clusters. Key topics include back-end and front-end networks, the folded-Clos topology, adaptive routing, and the significance of switch design in addressing network congestion. Dragonfly topology is also mentioned for hyperscale systems.

This post, part of a series on AI data center networking, clarifies the traffic patterns arising from collective communications during GPU operations. It highlights key operations like broadcast, scatter, gather, and reduce, explaining their significance for both training and inference, especially in Mixture-of-Experts models. Understanding these patterns aids in optimizing network performance.

This article discusses essential aspects of data center networking for AI, emphasizing the unique communication dynamics between GPUs. It highlights the importance of PCIe bus architecture, NUMA locality, and NVLink technology in optimizing performance. The design strategy of one NIC per GPU and the use of SmartNICs in multi-tenant environments are also explored.

This article discusses the necessity for rethinking data center networking in the context of AI. Traditional designs based on traffic averages and oversubscription do not apply, as AI workloads produce massive, synchronized “elephant” flows, necessitating non-blocking architectures and emphasizing metrics like job completion time and GPU utilization over conventional measures.