habana_frameworks.mediapipe.fn.MediaFunc

Class:

• habana_frameworks.mediapipe.fn.MediaFunc(**kwargs)

Define graph call:

• __call__(input)

Parameter:

• input - Input tensor to operator. Tensor dimensions and data types depend on the operator used. Supported inputs: minimum = 0, maximum = 4.

Description:

This operator allows user to generate new tensor data on every iteration. User has to inherit media_function class and provide implementation in __call__ function. All the required parameters for user implementation can be passed as part of priv_params.

Supported backend:

• CPU

Keyword Arguments

kwargs	Description
func	Class implementing the required functionality, derived from media_function class. Type: class Default: None Optional: no
shape	Shape of data to be generated. Type: list[int] Default: [1] Optional: no
seed	Seed for randomization. Type: int Default: 0 Optional: yes
priv_params	User defined params. Type: dict Default: None Optional: yes

Note

Produces one output.

Example: MediaFunc Operator

The following code snippet shows usage of MediaFunc operator. In this example the random crop tensor is generated on every iteration and fed to decoder for random crop operation:

from habana_frameworks.mediapipe import fn
from habana_frameworks.mediapipe.mediapipe import MediaPipe
from habana_frameworks.mediapipe.media_types import imgtype as it
from habana_frameworks.mediapipe.media_types import dtype as dt
import matplotlib.pyplot as plt
from habana_frameworks.mediapipe.operators.cpu_nodes.cpu_nodes import media_function
import numpy as np

class myMediaPipe(MediaPipe):
    def __init__(self, device, dir, queue_depth, batch_size, img_h, img_w):
        super(
            myMediaPipe,
            self).__init__(
            device,
            queue_depth,
            batch_size,
            self.__class__.__name__)

        self.input = fn.ReadImageDatasetFromDir(shuffle=False,
                                                dir=dir,
                                                format="jpg")

        self.decode = fn.ImageDecoder(device="hpu",
                                        output_format=it.RGB_I,
                                        resize=[img_w, img_h])
        priv_params = {}
        priv_params['xval_min'] = 0.0
        priv_params['xval_max'] = 0.2
        priv_params['yval_min'] = 0.0
        priv_params['yval_max'] = 0.3
        priv_params['wval_min'] = 0.8
        priv_params['wval_max'] = 1.0
        priv_params['hval_min'] = 0.7
        priv_params['hval_max'] = 1.0
        mediapipe_seed = 7368592685

        self.random_crop = fn.MediaFunc(func=random_crop_func,
                                        dtype=dt.FLOAT32,
                                        shape=[4, batch_size],
                                        seed=mediapipe_seed,
                                        priv_params=priv_params)

    def definegraph(self):
        images, labels = self.input()
        crop_val = self.random_crop()
        images = self.decode(images, crop_val)
        return images, labels

class random_crop_func(media_function):
    def __init__(self, params):
        self.np_shape = params['shape'][::-1]
        self.np_dtype = params['dtype']
        self.batch_size = self.np_shape[0]
        self.seed = params['seed']
        self.xval_min = params['priv_params']['xval_min']
        self.xval_max = params['priv_params']['xval_max']
        self.yval_min = params['priv_params']['yval_min']
        self.yval_max = params['priv_params']['yval_max']
        self.wval_min = params['priv_params']['wval_min']
        self.wval_max = params['priv_params']['wval_max']
        self.hval_min = params['priv_params']['hval_min']
        self.hval_max = params['priv_params']['hval_max']
        self.rng = np.random.default_rng(self.seed)

    def __call__(self):
        a = np.empty(shape=self.np_shape, dtype=self.np_dtype)
        x_val = self.rng.uniform(self.xval_min, self.xval_max, self.batch_size)
        y_val = self.rng.uniform(self.yval_min, self.yval_max, self.batch_size)
        w_val = self.rng.uniform(self.wval_min, self.wval_max, self.batch_size)
        h_val = self.rng.uniform(self.hval_min, self.hval_max, self.batch_size)
        for i in range(self.batch_size):
            if((x_val[i] + w_val[i]) > 1):
                w_val[i] = 1 - x_val[i]
            if((y_val[i] + h_val[i]) > 1):
                h_val[i] = 1 - y_val[i]
            a[i] = [x_val[i], y_val[i], w_val[i], h_val[i]]
        return a

def display_images(images, batch_size, cols):
    rows = (batch_size + 1) // cols
    plt.figure(figsize=(10, 10))
    for i in range(batch_size):
        ax = plt.subplot(rows, cols, i + 1)
        plt.imshow(images[i])
        plt.axis("off")
    plt.show()

def main():
    batch_size = 6
    img_width = 200
    img_height = 200
    img_dir = "/path/to/images"
    queue_depth = 2
    columns = 3

    # Create media pipeline object
    pipe = myMediaPipe('hpu', img_dir, queue_depth, batch_size,
                        img_height, img_width)

    # Build media pipeline
    pipe.build()

    # Initialize media pipeline iterator
    pipe.iter_init()

    # Run media pipeline
    images, labels = pipe.run()

    # Copy data to host from device as numpy array
    images = images.as_cpu().as_nparray()
    labels = labels.as_cpu().as_nparray()

    # Display images
    display_images(images, batch_size, columns)

if __name__ == "__main__":
    main()

Images with Crop Generated by random_crop_func() 1

1

Licensed under a CC BY SA 4.0 license. The images used here are taken from https://data.caltech.edu/records/mzrjq-6wc02.