Scale-Out Topology

The following table summarizes the scale-out topology further described in DDP-based Scaling of Gaudi on PyTorch.

Scale Out Topology

Distributed Framework

Scaling Via

Flags

Notes

AWS DL1 / Host NIC Scaling

PyTorch DDP

Libfabric

RDMAV_FORK_SAFE=1 FI_EFA_USE_DEVICE_RDMA=1

For more details on AWS DL1 and scale-out with Peer Direct, refer to Scale-out Using AWS DL1/Host NICs.

Gaudi based

PyTorch DDP

RDMA

None

Default mode = RDMA