PyTorch Mixed Precision Training on Gaudi

Gaudi supports mixed precision training using either native PyTorch autocast or Habana Mixed Precision (HMP).

• Native PyTorch Autocast
• Habana Mixed Precision

Note

• Native PyTorch autocast is the recommended method for mixed precision training.

• Existing models which use Habana Mixed Precision (HMP) are still supported.