MediaPipe

The Intel® Gaudi® 2 AI accelerator provides hardware-based acceleration for pre-processing inputs in deep learning frameworks. It also provides highly optimized operators, based on Tensor Processing Cores, generally used for data augmentation and image manipulation during input media processing.

The MediaPipe provides an interface for implementing media operations on input elements, such as images. Its purpose is to prepare batches of processed and augmented images, as well as labels, to be fed into training or inference models. Input data may include additional input information such as labels for classes or bounding boxes. Most of the operations are implemented on HPU, enabling accelerated execution compared to operations on CPU. See Operators section for more details.

Highlights:

• Supports JPEG images.

• Scalable across multiple cards.

• Supports PyTorch framework.

• Accelerates image classification (ResNet-50).

• Supports multiple functions such as reading and decoding, as well as various data transformations including image cropping or flipping.

For further details, see the below sections:

• Creating and Executing Media Pipeline
• MediaPipe for PyTorch ResNet
• Operators