PyTorch Mixed Precision Training on Gaudi

Gaudi supports mixed precision training using either native PyTorch autocast or Habana Mixed Precision (HMP).

Note

  • Native PyTorch autocast is the recommended method for mixed precision training.

  • Existing models which use Habana Mixed Precision (HMP) are still supported.