Scale-Out Topology
Scale-Out Topology¶
The following table summarizes the scale-out topology further described in DDP-based Scaling of Gaudi on PyTorch.
Scale Out Topology |
Distributed Framework |
Scaling Via |
Flags |
Notes |
|---|---|---|---|---|
AWS DL1 / Host NIC Scaling |
PyTorch DDP |
Libfabric |
RDMAV_FORK_SAFE=1 FI_EFA_USE_DEVICE_RDMA=1 |
For more details on AWS DL1 and scale-out with Peer Direct, refer to Scale-out Using AWS DL1/Host NICs. |
Gaudi based |
PyTorch DDP |
RDMA |
None |
Default mode = RDMA |