TPC Getting Started Guide

This Getting Started guide is designed to give you the basic skills and information to get you quickly up to speed on writing Tensor Processor Core™ (TPC™) kernel, integrating it into SynapseAI and then integrating it into a TensorFlow or PyTorch model.

For more details on TPC installation, code development, debugger installation and Intrinsics, refer to the following:

• TPC Tools Installation Guide - Provides installation instructions for the TPC-C compiler, assembler, dis-assembler and all necessary headers.

• TPC User Guide - Getting started guide for TPC code development.

• TPC Tools Debugger - Provides TPC debugger installation and usage instructions.

• TPC Intrinsics Guide - Provides TPC Intrinsics introduction, reference to the header in GitHub and intrinsics APIs.

Build a TPC Kernel for Shared Object (.so)

A TPC program consists of two parts – TPC execution code and host glue code. TPC code is the ISA executed by the TPC processor. Host code is executed on the host machine and provides specifications regarding how the program input/outputs can be dynamically partitioned between the numerous TPC processors in the Habana Gaudi device.

To build a TPC kernel, follow the steps below:

• Install TPC developers tools package. Follow the steps in TPC Tools Installation Guide. The package includes tools such as TPC-C Complier, TPC Disassembler, TPC Simulator and test library that help you write, compile, and run TPC kernels.

• To write a custom kernel library, implement the following components:

  • Expose two entry points: GetKernelNames and HabanaKernel. See GetKernelNames and HabanaKernel examples.

  • Include one TPC Op at least - kernel implementation in TPC-C. For more details, refer to TPC Programming Language section in TPC User Guide. See Kernel - leakyrelu example.

  • Include a glue code. For more details, refer to Glue Code. See Glue-code - leakyrelu example.

• Compile and create the shared object (.so). For more details, refer to Build lib Project section in TPC User Guide. The shared object is located under the project build directory, see the CMakeLists example.

Integrate a TPC Kernel into Habana Graph Compiler

To use your TPC custom kernel library with Habana Software Stack, integrate the library into the Graph Compiler (GC). Follow the steps below:

• Add your custom kernel library path to the environment variable GC_KERNEL_PATH. When initiating a TPC node, the GC will browse all the libraries that are specified under GC_KERNEL_PATH.

• Export export GC_KERNEL_PATH=/path/to/your_so/libcustom_tpc_perf_lib.so:/usr/lib/habanalabs/libtpc_kernels.so. Once exported, you can add new nodes from your cutom kernel lib to the graph.

Integrate a TPC Kernel into a TensorFlow Model

By integrating a TPC kernel into a TensorFlow model, you can add your custom TPC Op to a TensorFlow model. To integrate a TPC Kernel into a TensorFlow model, refer to the Basic Workflow in TensorFlow CustomOp API.

For examples on integrating a kernel into a TF model, refer to TensorFlow CustomOp Examples. See also Steps to build and run CustomDivOp with custom kernel customdiv_fwd_f32 full example.

Integrate a TPC Kernel into a PyTorch Model

By integrating a TPC kernel into a PyTorch model, you can add your custom TPC Op to a PyTorch model. To integrate a TPC Kernel into a PyTorch model, refer to the Basic Workflow in PyTorch CustomOp API.

For examples on integrating a kernel into a PyTorch model, refer to PyTorch CustomOp Examples.