Intel Gaudi Network Configuration

Each Intel® Gaudi® 3 AI accelerator has 21 scale-up ports and three scale-out ports running at 200Gbps. The scale-up ports are used to connect to seven other accelerators inside a single HLS-3 box in an All2All manner, essentially using 21/7=3 scale-up ports for each connection inside the box (as shown in light blue line in the picture below). The three scale-out ports from each Gaudi 3 inside the box (shown as black line) are bundled together as OSFP (4x200Gbps, shown as purple line below) links. The OSFP links connect to the leaf/TOR switches which are further connected to spine/core switches, making up the entire data center. The theoretical peak of scale-up bandwidth from each Gaudi is 21*200Gbps = 21*25 GB/s = 525 GB/s unidirectional bandwidth or 1050 GB/s bidirectional bandwidth. The scale-out bandwidth from each Gaudi 3 is 3*200Gbps = 75 GB/s in each direction or 150 GB/s bidirectional bandwidth. The scale-out bandwidth of an HLS-3 box consisting of 8 Gaudi 3s is 3*8*200Gbps = 600 GB/s unidirectional bandwidth or 1200 GB/s bidirectional bandwidth.

Most of the collectives are implemented in a hierarchical manner, thus pipelining scale-up and scale-out communication in parallel. The theoretical peak for multi-box HLS-3 for collectives such as Allreduce, Reducescatter, Allgather is 600 GB/s unidirectional bandwidth. For all2all collective, given each Gaudi 3 has unique data for every other Gaudi 3, the theoretical peak will be 75 GB/s unidirectional bandwidth.

• Configure E2E Test in L3 Switching Environment

• Expected Switch Configuration

• Monitoring Switch and Gaudi 3 Accelerator

• Collectives Performance

• Congestion Test

• How to Pick Good Nodes in the Datacenter

• Intel Gaudi RDMA PerfTest Tool

• Arista Switch Configuration Example

Note

Gaudi 2 network architecture is similar to Gaudi 3 except that the network links operate at 100Gbps. The theoretical peak of scale-up bandwidth from each Gaudi 2 is 21*100Gbps = 21*12.5 GB/s = 262.5 GB/s unidirectional bandwidth or 525 GB/s bidirectional bandwidth. The scale-out bandwidth from each Gaudi 2 is 3*100Gbps = 37.5 GB/s in each direction or 75 GB/s bidirectional bandwidth. The scale-out bandwidth of an HLS-2 box consisting of 8 Gaudi 2s is 3*8*100Gbps = 300 GB/s unidirectional bandwidth or 600 GB/s bidirectional bandwidth.