2. Pytorch Operators

2.1. Overview

This document summarizes the SynapseAI Pytorch supported operators for Gaudi. Note that the operators listed below support only selected variants and limited optional parameters for Gaudi.

2.2. Pytorch Operators Support Summary

Pytorch Operator

FP32

BF16

INT32

INT8

BOOL

Torch.nn.Functional

Avg_pool2d

Yes

Yes

No

No

No

Batch_norm

Yes

Yes

No

No

No

Binary_cross_entropy

Yes

Yes

No

No

No

Conv2d

Yes

Yes

No

No

No

Dropout

Yes

Yes

No

No

No

Embedding

Yes

Yes

No

No

No

Embedding_bag

Yes

Yes

No

No

No

Gelu

Yes

Yes

No

No

No

Layer_norm

Yes

Yes

No

No

No

Linear

Yes

Yes

No

No

No

Log_softmax

Yes

Yes

No

No

No

Max_pool2d

Yes

Yes

No

No

No

Mse_loss

Yes

Yes

No

No

No

Nll_loss

Yes

Yes

No

No

No

Pad

Yes

Yes

No

No

No

Relu

Yes

Yes

No

No

No

Sigmoid

Yes

Yes

No

No

No

Tanh

Yes

Yes

No

No

No

Upsample_nearest

Yes

Yes

No

No

No

Binary_cross_entropy_with_logits

Yes

Yes

No

No

No

Conv_transpose2d

Yes

Yes

No

No

No

Torch

Abs

Yes

Yes

No

No

No

Add

Yes

Yes

No

No

No

Addcdiv

Yes

Yes

No

No

No

Addcmul

Yes

Yes

No

No

No

Addmm

Yes

Yes

No

No

No

Any

No

No

No

No

Yes

Arange

Yes

Yes

Yes

No

No

Batch_norm

Yes

Yes

No

No

No

Bernoulli

Yes

Yes

No

No

No

Bmm

Yes

Yes

No

No

No

Broadcast_tensors

Yes

Yes

No

No

No

Cat

Yes

Yes

No

No

No

Clamp_min

Yes

Yes

No

No

No

Clone

Yes

Yes

Yes

Yes

Yes

Constant_pad_nd

Yes

Yes

No

No

No

Conv2d

Yes

Yes

No

No

No

Div

Yes

Yes

No

No

No

Dot

Yes

Yes

No

No

No

Dropout

Yes

Yes

No

No

No

Embedding

Yes

Yes

No

No

No

Embedding_bag

Yes

Yes

No

No

No

Empty

Yes

Yes

Yes

Yes

Yes

Empty_like

Yes

Yes

Yes

Yes

Yes

Empty_strided

Yes

Yes

Yes

Yes

Yes

Eq

Yes

Yes

No

No

No

Erf

Yes

Yes

No

No

No

Erf_

Yes

Yes

No

No

No

Exp

Yes

Yes

No

No

No

Exp_

Yes

Yes

No

No

No

Fill_

Yes

Yes

Yes

No

No

Gather

Yes

Yes

No

No

No

Gt

Yes

Yes

No

No

No

Index_put_

Yes

Yes

No

No

No

Index_select

Yes

Yes

No

No

No

Layer_norm

Yes

Yes

No

No

No

Lt

Yes

Yes

No

No

No

Matmul

Yes

Yes

No

No

No

Mean

Yes

Yes

No

No

No

Mm

Yes

Yes

No

No

No

Mul

Yes

Yes

No

No

No

Mv

Yes

Yes

No

No

No

Neg

Yes

Yes

No

No

No

Norm

Yes

Yes

No

No

No

Ones

Yes

Yes

Yes

Yes

Yes

Ones_like

Yes

Yes

Yes

Yes

Yes

Pow

Yes

Yes

No

No

No

Reciprocal

Yes

Yes

No

No

No

Reciprocal_

Yes

Yes

No

No

No

Relu

Yes

Yes

No

No

No

Relu_

Yes

Yes

No

No

No

Rsub

Yes

Yes

No

No

No

Scatter

Yes

Yes

No

No

No

Scatter_add

Yes

Yes

No

No

No

Sigmoid

Yes

Yes

No

No

No

Softmax

Yes

Yes

No

No

No

Sort

Yes

Yes

No

No

No

Split_with_sizes

Yes

Yes

Yes

No

No

Sqrt

Yes

Yes

No

No

No

Sub

Yes

Yes

No

No

No

Sum

Yes

Yes

No

No

No

T

Yes

Yes

Yes

Yes

Yes

Tanh

Yes

Yes

No

No

No

Tanh_

Yes

Yes

No

No

No

Topk

Yes

Yes

No

No

No

Transpose

Yes

Yes

Yes

Yes

Yes

Zeros

Yes

Yes

Yes

Yes

Yes

Zeros_like

Yes

Yes

Yes

Yes

Yes

Max

Yes

Yes

No

No

No

Maximum

Yes

Yes

Yes

No

No

Minimum

Yes

Yes

Yes

No

No

Floor

Yes

Yes

No

No

No

Floor_

Yes

Yes

No

No

No

Rsqrt

Yes

Yes

No

No

No

Rsqrt_

Yes

Yes

No

No

No

Log

Yes

Yes

No

No

No

Log2

Yes

Yes

No

No

No

Log2_

Yes

Yes

No

No

No

Log_

Yes

Yes

No

No

No

Round

Yes

Yes

No

No

No

Round_

Yes

Yes

No

No

No

Where

Yes

Yes

Yes

No

No

Ne

Yes

Yes

Yes

No

No

Ge

Yes

Yes

Yes

No

No

All

No

No

No

No

Yes

Sgn

Yes

Yes

No

No

No

Sign

Yes

Yes

No

No

No

Bitwise_and

No

No

No

No

Yes

Bitwise_not

No

No

No

No

Yes

Bitwise_or

No

No

No

No

Yes

Bitwise_xor

No

No

No

No

Yes

Prod

Yes

Yes

No

No

No

Randperm

No

No

Yes

No

No

Isfinite

Yes

Yes

No

No

No

Nonzero

Yes

Yes

Yes

No

Yes

Argmax

Yes

Yes

No

No

No

Unique

Yes

Yes

No

No

No

Clamp

Yes

Yes

Yes

No

No

Clamp_

Yes

Yes

Yes

No

No

Torch.Tensor

Abs

Yes

Yes

No

No

No

Add

Yes

Yes

No

No

No

Add_

Yes

Yes

No

No

No

Addcdiv

Yes

Yes

No

No

No

Addcdiv_

Yes

Yes

No

No

No

Addcmul

Yes

Yes

No

No

No

Addcmul_

Yes

Yes

No

No

No

Addmm

Yes

Yes

No

No

No

Any

No

No

No

No

Yes

Bernoulli

Yes

Yes

No

No

No

Bmm

Yes

Yes

No

No

No

Clamp

Yes

Yes

Yes

No

No

Clamp_

Yes

Yes

Yes

No

No

Clamp_min

Yes

Yes

No

No

No

Clone

Yes

Yes

Yes

Yes

Yes

Copy_

Yes

Yes

Yes

Yes

Yes

Div

Yes

Yes

No

No

No

Div_

Yes

Yes

No

No

No

Dot

Yes

Yes

No

No

No

Eq

Yes

Yes

No

No

No

Eq_

Yes

Yes

No

No

No

Erf

Yes

Yes

No

No

No

Erf_

Yes

Yes

No

No

No

Exp

Yes

Yes

No

No

No

Exp_

Yes

Yes

No

No

No

Expand

Yes

Yes

No

No

No

Expand_as

Yes

Yes

No

No

No

Fill_

Yes

Yes

Yes

No

No

Gather

Yes

Yes

No

No

No

Gt

Yes

Yes

No

No

No

Index_add_

Yes

Yes

No

No

No

Index_put_

Yes

Yes

No

No

No

Index_select

Yes

Yes

No

No

No

Log_softmax

Yes

Yes

No

No

No

Lt

Yes

Yes

No

No

No

Masked_fill_

Yes

Yes

No

No

No

Matmul

Yes

Yes

No

No

No

Mean

Yes

Yes

No

No

No

Mm

Yes

Yes

No

No

No

Mul

Yes

Yes

No

No

No

Mul_

Yes

Yes

No

No

No

Mv

Yes

Yes

No

No

No

Neg

Yes

Yes

No

No

No

Norm

Yes

Yes

No

No

No

Normal_

Yes

Yes

No

No

No

Permute

Yes

Yes

Yes

No

No

Pow

Yes

Yes

No

No

No

Pow_

Yes

Yes

No

No

No

Reciprocal

Yes

Yes

No

No

No

Reciprocal_

Yes

Yes

No

No

No

Relu

Yes

Yes

No

No

No

Relu_

Yes

Yes

No

No

No

Scatter_

Yes

Yes

No

No

No

Scatter_add

Yes

Yes

No

No

No

Scatter_add_

Yes

Yes

No

No

No

Sigmoid

Yes

Yes

No

No

No

Softmax

Yes

Yes

No

No

No

Sort

Yes

Yes

No

No

No

Split_with_sizes

Yes

Yes

Yes

No

No

Sqrt

Yes

Yes

No

No

No

Sub

Yes

Yes

No

No

No

Sub_

Yes

Yes

No

No

No

Sum

Yes

Yes

No

No

No

T

Yes

Yes

Yes

Yes

Yes

T_

Yes

Yes

Yes

Yes

Yes

Tanh

Yes

Yes

No

No

No

Tanh_

Yes

Yes

No

No

No

To

Yes

Yes

Yes

Yes

Yes

Topk

Yes

Yes

No

No

No

Transpose

Yes

Yes

Yes

Yes

Yes

Transpose_

Yes

Yes

Yes

Yes

Yes

Uniform_

Yes

Yes

No

No

No

Max

Yes

Yes

No

No

No

Maximum

Yes

Yes

Yes

No

No

Minimum

Yes

Yes

Yes

No

No

Floor

Yes

Yes

No

No

No

Floor_

Yes

Yes

No

No

No

Rsqrt

Yes

Yes

No

No

No

Rsqrt_

Yes

Yes

No

No

No

Log

Yes

Yes

No

No

No

Log2

Yes

Yes

No

No

No

Log2_

Yes

Yes

No

No

No

Log_

Yes

Yes

No

No

No

Round

Yes

Yes

No

No

No

Round_

Yes

Yes

No

No

No

Where

Yes

Yes

Yes

No

No

Ne

Yes

Yes

Yes

No

No

Ne_

Yes

Yes

Yes

No

No

Ge

Yes

Yes

Yes

No

No

Ge_

Yes

Yes

Yes

No

No

All

No

No

No

No

Yes

Sgn

Yes

Yes

No

No

No

Sgn_

Yes

Yes

No

No

No

Sign

Yes

Yes

No

No

No

Sign_

Yes

Yes

No

No

No

Bitwise_and

No

No

No

No

Yes

Bitwise_not

No

No

No

No

Yes

Bitwise_or

No

No

No

No

Yes

Bitwise_xor

No

No

No

No

Yes

Prod

Yes

Yes

No

No

No

Randperm

No

No

Yes

No

No

Isfinite

Yes

Yes

No

No

No

Nonzero

Yes

Yes

Yes

No

Yes

Argmax

Yes

Yes

No

No

No

Unique

Yes

Yes

No

No

No