vLLM Inference Server with Intel Gaudi

This document provides instructions for setting up and using vLLM with Intel® Gaudi® AI accelerator. It includes troubleshooting tips, guidelines for performance tuning, and enabling FP8 calibration and inference. Additionally, answers to common questions are provided to assist with getting started.

• Getting Started with vLLM

• FP8 Calibration and Inference with vLLM

• vLLM with Intel Gaudi FAQs