Custom Driver and Software Installation

The following section describes how to install each software package manually. This method ensures fine-grained control over the installation process.

Note

While you can install each package manually, using the habanalabs-installer.sh script is the recommended method for installation. For further details, see Driver and Software Installation.

The table below describes the software installers that are included in the installation:

Package

Description

habanalabs-graph

Installs the graph compiler and the runtime.

habanalabs-thunk

Installs the Thunk library.

habanalabs-dkms

Installs the habanalabs, habanalabs_cn, habanalabs_en, and habanalabs_ib drivers. The habanalabs_ib driver is supported on Gaudi 2 and Gaudi 3 only.

habanalabs-rdma-core

Installs IBVerbs libraries which provide Intel Gaudi’s libhlib along with libibverbs.

habanalabs-firmware

Installs the Gaudi firmware.

habanalabs-firmware-tools

Installs various firmware tools (hlml, hl-smi, etc).

habanalabs-firmware-odm

Installs the ODM firmware tools (SPI flash and hl_fw_loader).

habanalabs-qual

Installs the qualification application package.

habanalabs-container-runtime

Installs the habanalabs-container-runtime library.

habanalabs-qual-workloads

Installs qual workloads to run ResNet-50 training stress test plugin.

Note

  • Running the below commands installs the latest version only. Before installing the below packages and Dockers, make sure to review the currently supported versions and Operating Systems listed in the Support Matrix.

  • Installing the package with internet connection available allows the network to download and install the required dependencies (apt get, yum, dnf install or pip install etc.).

Note

Ubuntu 24.04 installation is available on Gaudi 3 only.

  1. Configure the repository:

    curl -X GET https://vault.habana.ai/artifactory/api/gpg/key/public | sudo apt-key add --
echo "deb https://vault.habana.ai/artifactory/debian $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/habanalabs_synapseai.list
sudo dpkg --configure -a
sudo apt-get update

  2. Install Kernel Mode Driver (KMD) dependencies:

    sudo apt install dkms libelf-dev
sudo apt install linux-headers-$(uname -r)

  3. Install firmware:

    sudo apt install -y habanalabs-firmware

  4. Install ODM firmware:

    sudo apt install -y habanalabs-firmware-odm

  5. Install driver. The habanalabs-dkms package installs the habanalabs, habanalabs_cn, habanalabs_en (Ethernet) and habanalabs_ib drivers. If automation scripts are used, the scripts must be modified to load/unload the drivers:

    Note

    habanalabs_ib driver is available on Gaudi 2 and Gaudi 3 only.

    1. Run the below command to install all drivers:

      sudo apt install -y habanalabs-dkms

    2. (Optional) When upgrading, unload the drivers in this order - habanalabs, habanalabs_cn, habanalabs_en and habanalabs_ib:

      sudo modprobe -r habanalabs
sudo modprobe -r habanalabs_cn
sudo modprobe -r habanalabs_en
sudo modprobe -r habanalabs_ib

    3. Load the drivers in this order - habanalabs_en and habanalabs_ib, habanalabs_cn, habanalabs:

      sudo modprobe habanalabs_en
sudo modprobe habanalabs_ib
sudo modprobe habanalabs_cn
sudo modprobe habanalabs

  6. Install Thunk library:

    sudo apt install -y habanalabs-thunk

  7. Install RDMA Core:

    sudo apt install habanalabs-rdma-core

  8. Install firmware tools:

    sudo apt install -y habanalabs-firmware-tools

  9. Install Graph Compiler and runtime:

    sudo apt install -y habanalabs-graph

  10. Install container runtime:

    sudo apt install -y habanalabs-container-runtime

  11. (Optional) Install qual library:**

    sudo apt install -y habanalabs-qual

    For further details, see Gaudi Qualification Library.

  12. (Optional) Install qual workloads:**

    sudo apt install -y habanalabs-qual-workloads

  13. (Optional) Install ethtool:

    sudo apt install -y ethtool

    For further details, see Bring up Accelerator Interfaces.

  14. Update environment variables and more. When the installation is complete, close and re-open the shell. Or, run the following:

    source /etc/profile.d/habanalabs.sh

source ~/.bashrc

  1. Configure the repository:

    curl -X GET https://vault.habana.ai/artifactory/api/gpg/key/public | sudo apt-key add --
echo "deb https://vault.habana.ai/artifactory/debian $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/habanalabs_synapseai.list
sudo dpkg --configure -a
sudo apt-get update

  2. Install Kernel Mode Driver (KMD) dependencies:

    sudo apt install dkms libelf-dev
sudo apt install linux-headers-$(uname -r)

  3. Install firmware:

    sudo apt install -y habanalabs-firmware

  4. (Optional) Install ODM firmware:**

    sudo apt install -y habanalabs-firmware-odm

  1. Install driver. The habanalabs-dkms package installs the habanalabs, habanalabs_cn, habanalabs_en (Ethernet) and habanalabs_ib drivers. If automation scripts are used, the scripts must be modified to load/unload the drivers:

    Note

    habanalabs_ib driver is available on Gaudi 2 and Gaudi 3 only.

    1. Run the below command to install all drivers:

      sudo apt install -y habanalabs-dkms

    2. (Optional) When upgrading, unload the drivers in this order - habanalabs, habanalabs_cn, habanalabs_en and habanalabs_ib:

      sudo modprobe -r habanalabs
sudo modprobe -r habanalabs_cn
sudo modprobe -r habanalabs_en
sudo modprobe -r habanalabs_ib

    3. Load the drivers in this order - habanalabs_en and habanalabs_ib, habanalabs_cn, habanalabs:

      sudo modprobe habanalabs_en
sudo modprobe habanalabs_ib
sudo modprobe habanalabs_cn
sudo modprobe habanalabs

  2. Install Thunk library:

    sudo apt install -y habanalabs-thunk

  3. Install RDMA Core:

    sudo apt install habanalabs-rdma-core

  4. Install firmware tools:

    sudo apt install -y habanalabs-firmware-tools

  5. Install Graph Compiler and runtime:

    sudo apt install -y habanalabs-graph

  6. Install container runtime:

    sudo apt install -y habanalabs-container-runtime

  7. (Optional) Install qual library:**

    sudo apt install -y habanalabs-qual

    For further details, see Gaudi Qualification Library.

  8. (Optional) Install qual workloads:**

    sudo apt install -y habanalabs-qual-workloads

  9. (Optional) Install ethtool:

    sudo apt install -y ethtool

    For further details, see Bring up Accelerator Interfaces.

  10. Update environment variables and more. When the installation is complete, close and re-open the shell. Or, run the following:

    source /etc/profile.d/habanalabs.sh

source ~/.bashrc

Note

Amazon Linux 2 installation is available on first-gen Gaudi only.

  1. Configure the repository:

    cat << EOF >  /etc/yum.repos.d/Habana-Vault.repo
[vault]
name=Habana Vault
baseurl=https://vault.habana.ai/artifactory/AmazonLinux2
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://vault.habana.ai/artifactory/api/v2/repositories/AmazonLinux2/keyPairs/primary/public
https://vault.habana.ai/artifactory/api/v2/repositories/AmazonLinux2/keyPairs/secondary/public
EOF

sudo yum makecache
sudo yum search habana

  2. Install Kernel Mode Driver (KMD) dependencies:

    sudo yum install kernel-devel
sudo yum install -y yum-utils

  3. Install firmware:

    sudo yum install -y habanalabs-firmware

  4. Install ODM firmware:**

    sudo yum install -y habanalabs-firmware-odm

  5. Install driver. The habanalabs-dkms package installs the habanalabs, habanalabs_cn, habanalabs_en (Ethernet) and habanalabs_ib drivers. If automation scripts are used, the scripts must be modified to load/unload the drivers:

    Note

    habanalabs_ib driver is available on Gaudi 2 and Gaudi 3 only.

    1. Run the below command to install all drivers:

      sudo yum install -y habanalabs

    2. (Optional) When upgrading, unload the drivers in this order - habanalabs, habanalabs_cn, habanalabs_en and habanalabs_ib:

      sudo modprobe -r habanalabs
sudo modprobe -r habanalabs_cn
sudo modprobe -r habanalabs_en
sudo modprobe -r habanalabs_ib

    3. Load the drivers in this order - habanalabs_en and habanalabs_ib, habanalabs_cn, habanalabs:

      sudo modprobe habanalabs_en
sudo modprobe habanalabs_ib
sudo modprobe habanalabs_cn
sudo modprobe habanalabs

  6. Install Thunk library:

    sudo yum install -y habanalabs-thunk

  7. Install firmware tools:

    sudo yum install -y habanalabs-firmware-tools

  8. Install Graph Compiler and runtime:

    sudo yum install -y habanalabs-graph

  9. Install container runtime:

    sudo yum install -y habanalabs-container-runtime

  10. (Optional) Install qual library:**

    sudo apt install -y habanalabs-qual

    For further details, see Gaudi Qualification Library.

  11. (Optional) Install qual workloads:**

    sudo yum install -y habanalabs-qual-workloads

  12. (Optional) Install ethtool:

    sudo yum install -y ethtool

    For further details, see Bring up Accelerator Interfaces.

  13. Update environment variables and more. When the installation is complete, close and re-open the shell. Or, run the following:

    source /etc/profile.d/habanalabs.sh

source ~/.bashrc

Note

RHEL8.6 installation is available on Gaudi 2 only.

  1. Configure the repository:

    cat << EOF >  /etc/yum.repos.d/Habana-Vault.repo
[vault]
name=Habana Vault
baseurl=https://vault.habana.ai/artifactory/rhel/8/8.6
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://vault.habana.ai/artifactory/api/v2/repositories/rhel/keyPairs/primary/public
https://vault.habana.ai/artifactory/api/v2/repositories/rhel/keyPairs/secondary/public
EOF

sudo dnf makecache
sudo dnf search habana
sudo dnf install -y libarchive*

  2. Install Kernel Mode Driver (KMD) dependencies:

    sudo dnf install kernel-devel
sudo dnf install -y dnf-utils

  3. Install firmware:

    sudo dnf install -y habanalabs-firmware

  4. Install ODM firmware:**

    sudo dnf install -y habanalabs-firmware-odm

  5. Install driver. The habanalabs-dkms package installs the habanalabs, habanalabs_cn, habanalabs_en (Ethernet) and habanalabs_ib drivers. If automation scripts are used, the scripts must be modified to load/unload the drivers:

    Note

    habanalabs_ib driver is available on Gaudi 2 and Gaudi 3 only.

    1. Run the below command to install all drivers:

      sudo dnf install -y habanalabs

    2. (Optional) When upgrading, unload the drivers in this order - habanalabs, habanalabs_cn, habanalabs_en and habanalabs_ib:

      sudo modprobe -r habanalabs
sudo modprobe -r habanalabs_cn
sudo modprobe -r habanalabs_en
sudo modprobe -r habanalabs_ib

    3. Load the drivers in this order - habanalabs_en and habanalabs_ib, habanalabs_cn, habanalabs:

      sudo modprobe habanalabs_en
sudo modprobe habanalabs_ib
sudo modprobe habanalabs_cn
sudo modprobe habanalabs

  6. Install Thunk library:

    sudo dnf install -y habanalabs-thunk

  7. Install RDMA Core:

    sudo dnf install habanalabs-rdma-core

  8. Install firmware tools:

    sudo dnf install -y habanalabs-firmware-tools

  9. Install Graph Compiler and runtime:

    sudo dnf install -y habanalabs-graph

  10. Install container runtime:

    sudo dnf install -y habanalabs-container-runtime

  11. (Optional) Install qual library:**

    sudo dnf install -y habanalabs-qual

    For further details, see Gaudi Qualification Library.

  12. (Optional) Install qual workloads:**

    sudo dnf install -y habanalabs-qual-workloads

  13. (Optional) Install ethtool:

    sudo dnf install -y ethtool

    For further details, see Bring up Accelerator Interfaces.

  14. Update environment variables and more. When the installation is complete, close and re-open the shell. Or, run the following:

    source /etc/profile.d/habanalabs.sh

source ~/.bashrc

Note

RHEL9.2 installation is available on Gaudi 2 only.

  1. Configure the repository:

    cat << EOF >  /etc/yum.repos.d/Habana-Vault.repo
[vault]
name=Habana Vault
baseurl=https://vault.habana.ai/artifactory/rhel/9/9.2
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://vault.habana.ai/artifactory/api/v2/repositories/rhel/keyPairs/primary/public
https://vault.habana.ai/artifactory/api/v2/repositories/rhel/keyPairs/secondary/public
EOF

sudo dnf makecache
sudo dnf search habana
sudo dnf install -y libarchive*

  2. Install Kernel Mode Driver (KMD) dependencies:

    sudo dnf install kernel-devel
sudo dnf install -y dnf-utils

  3. Install firmware:

    sudo dnf install -y habanalabs-firmware

  4. Install ODM firmware:**

    sudo dnf install -y habanalabs-firmware-odm

  5. Install driver. The habanalabs-dkms package installs the habanalabs, habanalabs_cn, habanalabs_en (Ethernet) and habanalabs_ib drivers. If automation scripts are used, the scripts must be modified to load/unload the drivers:

    Note

    habanalabs_ib driver is available on Gaudi 2 and Gaudi 3 only.

    1. Run the below command to install all drivers:

      sudo dnf install -y habanalabs

    2. (Optional) When upgrading, unload the drivers in this order - habanalabs, habanalabs_cn, habanalabs_en and habanalabs_ib:

      sudo modprobe -r habanalabs
sudo modprobe -r habanalabs_cn
sudo modprobe -r habanalabs_en
sudo modprobe -r habanalabs_ib

    3. Load the drivers in this order - habanalabs_en and habanalabs_ib, habanalabs_cn, habanalabs:

      sudo modprobe habanalabs_en
sudo modprobe habanalabs_ib
sudo modprobe habanalabs_cn
sudo modprobe habanalabs

  6. Install Thunk library:

    sudo dnf install -y habanalabs-thunk

  7. Install RDMA Core:

    sudo dnf install habanalabs-rdma-core

  8. Install firmware tools:

    sudo dnf install -y habanalabs-firmware-tools

  9. Install Graph Compiler and runtime:

    sudo dnf install -y habanalabs-graph

  10. Install container runtime:

    sudo dnf install -y habanalabs-container-runtime

  11. (Optional) Install qual library:**

    sudo dnf install -y habanalabs-qual

    For further details, see Gaudi Qualification Library.

  12. (Optional) Install qual workloads:**

    sudo dnf install -y habanalabs-qual-workloads

  13. (Optional) Install ethtool:

    sudo dnf install -y ethtool

    For further details, see Bring up Accelerator Interfaces.

  14. Update environment variables and more. When the installation is complete, close and re-open the shell. Or, run the following:

    source /etc/profile.d/habanalabs.sh

source ~/.bashrc

Note

RHEL9.4 installation is available on Gaudi 2 and Gaudi 3 only.

  1. Configure the repository:

    cat << EOF >  /etc/yum.repos.d/Habana-Vault.repo
[vault]
name=Habana Vault
baseurl=https://vault.habana.ai/artifactory/rhel/9/9.4
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://vault.habana.ai/artifactory/api/v2/repositories/rhel/keyPairs/primary/public
https://vault.habana.ai/artifactory/api/v2/repositories/rhel/keyPairs/secondary/public
EOF

sudo dnf makecache
sudo dnf search habana
sudo dnf install -y libarchive*

  2. Install Kernel Mode Driver (KMD) dependencies:

    sudo dnf install kernel-devel
sudo dnf install -y dnf-utils

  3. Install firmware:

    sudo dnf install -y habanalabs-firmware

  4. Install ODM firmware:**

    sudo dnf install -y habanalabs-firmware-odm

  5. Install driver. The habanalabs-dkms package installs the habanalabs, habanalabs_cn, habanalabs_en (Ethernet) and habanalabs_ib drivers. If automation scripts are used, the scripts must be modified to load/unload the drivers:

    Note

    habanalabs_ib driver is available on Gaudi 2 and Gaudi 3 only.

    1. Run the below command to install all drivers:

      sudo dnf install -y habanalabs

    2. (Optional) When upgrading, unload the drivers in this order - habanalabs, habanalabs_cn, habanalabs_en and habanalabs_ib:

      sudo modprobe -r habanalabs
sudo modprobe -r habanalabs_cn
sudo modprobe -r habanalabs_en
sudo modprobe -r habanalabs_ib

    3. Load the drivers in this order - habanalabs_en and habanalabs_ib, habanalabs_cn, habanalabs:

      sudo modprobe habanalabs_en
sudo modprobe habanalabs_ib
sudo modprobe habanalabs_cn
sudo modprobe habanalabs

  6. Install Thunk library:

    sudo dnf install -y habanalabs-thunk

  7. Install firmware tools:

    sudo dnf install -y habanalabs-firmware-tools

  8. Install RDMA Core:

    sudo dnf install habanalabs-rdma-core

  9. Install Graph Compiler and runtime:

    sudo dnf install -y habanalabs-graph

  10. Install container runtime:

    sudo dnf install -y habanalabs-container-runtime

  11. (Optional) Install qual library:**

    sudo dnf install -y habanalabs-qual

    For further details, see Gaudi Qualification Library.

  12. (Optional) Install qual workloads:**

    sudo dnf install -y habanalabs-qual-workloads

  13. (Optional) Install ethtool:

    sudo dnf install -y ethtool

    For further details, see Bring up Accelerator Interfaces.

  14. Update environment variables and more. When the installation is complete, close and re-open the shell. Or, run the following:

    source /etc/profile.d/habanalabs.sh

source ~/.bashrc

Note

TencentOS 3.1 installation is available on Gaudi 2 only.

  1. Configure the repository:

    cat << EOF >  /etc/yum.repos.d/Habana-Vault.repo
[vault]
name=Habana Vault
baseurl=https://vault.habana.ai/artifactory/tencentos/3/3.1
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://vault.habana.ai/artifactory/api/v2/repositories/tencentos/keyPairs/primary/public
https://vault.habana.ai/artifactory/api/v2/repositories/tencentos/keyPairs/secondary/public
EOF

sudo dnf makecache
sudo dnf search habana
sudo dnf install -y libarchive*

  2. Install Kernel Mode Driver (KMD) dependencies:

    sudo dnf install kernel-devel
sudo dnf install -y dnf-utils

  3. Install firmware:

    sudo dnf install -y habanalabs-firmware

  4. Install ODM firmware:**

    sudo dnf install -y habanalabs-firmware-odm

  5. Install driver. The habanalabs-dkms package installs the habanalabs, habanalabs_cn, habanalabs_en (Ethernet) and habanalabs_ib drivers. If automation scripts are used, the scripts must be modified to load/unload the drivers:

    Note

    habanalabs_ib driver is available on Gaudi 2 and Gaudi 3 only.

    1. Run the below command to install all drivers:

      sudo dnf install -y habanalabs

    2. (Optional) When upgrading, unload the drivers in this order - habanalabs, habanalabs_cn, habanalabs_en and habanalabs_ib:

      sudo modprobe -r habanalabs
sudo modprobe -r habanalabs_cn
sudo modprobe -r habanalabs_en
sudo modprobe -r habanalabs_ib

    3. Load the drivers in this order - habanalabs_en and habanalabs_ib, habanalabs_cn, habanalabs:

      sudo modprobe habanalabs_en
sudo modprobe habanalabs_ib
sudo modprobe habanalabs_cn
sudo modprobe habanalabs

  6. Install Thunk library:

    sudo dnf install -y habanalabs-thunk

  7. Install firmware tools:

    sudo dnf install -y habanalabs-firmware-tools

  8. Install RDMA Core:

    sudo dnf install habanalabs-rdma-core

  9. Install Graph Compiler and runtime:

    sudo dnf install -y habanalabs-graph

  10. Install container runtime:

    sudo dnf install -y habanalabs-container-runtime

  11. (Optional) Install qual library:

    sudo dnf install -y habanalabs-qual

    For further details, see Gaudi Qualification Library.

  12. (Optional) Install qual workloads:**

    sudo dnf install -y habanalabs-qual-workloads

  13. (Optional) Install ethtool:

    sudo dnf install -y ethtool

    For further details, see Bring up Accelerator Interfaces.

  14. Update environment variables and more. When the installation is complete, close and re-open the shell. Or, run the following:

    source /etc/profile.d/habanalabs.sh

source ~/.bashrc

Note

SUSE 15.5 installation is available on Gaudi 3 only.

  1. Configure the repository:

    cat << EOF >  /etc/yum.repos.d/Habana-Vault.repo
[vault]
name=Habana Vault
baseurl=https://vault.habana.ai/artifactory/sles/15/15.5
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://vault.habana.ai/artifactory/api/v2/repositories/sles/keyPairs/primary/public
https://vault.habana.ai/artifactory/api/v2/repositories/sles/keyPairs/secondary/public
EOF

sudo zypper refresh
sudo zypper search habana
sudo zypper install -y libarchive*

  2. Install Kernel Mode Driver (KMD) dependencies:

    sudo zypper install kernel-devel kernel-default-devel

  3. Install firmware:

    sudo zypper install -y habanalabs-firmware

  4. Install ODM firmware:**

    sudo zypper install -y habanalabs-firmware-odm

  5. Install driver. The habanalabs-dkms package installs the habanalabs, habanalabs_cn, habanalabs_en (Ethernet) and habanalabs_ib drivers. If automation scripts are used, the scripts must be modified to load/unload the drivers:

    Note

    habanalabs_ib driver is available on Gaudi 2 and Gaudi 3 only.

    1. Run the below command to install all drivers:

      sudo zypper install -y habanalabs

    2. (Optional) When upgrading, unload the drivers in this order - habanalabs, habanalabs_cn, habanalabs_en and habanalabs_ib:

      sudo modprobe -r habanalabs
sudo modprobe -r habanalabs_cn
sudo modprobe -r habanalabs_en
sudo modprobe -r habanalabs_ib

    3. Load the drivers in this order - habanalabs_en and habanalabs_ib, habanalabs_cn, habanalabs:

      sudo modprobe habanalabs_en
sudo modprobe habanalabs_ib
sudo modprobe habanalabs_cn
sudo modprobe habanalabs

  6. Install Thunk library:

    sudo zypper install -y habanalabs-thunk

  7. Install RDMA Core:

    sudo zypper install habanalabs-rdma-core

  8. Install firmware tools:

    sudo zypper install -y habanalabs-firmware-tools

  9. Install Graph Compiler and runtime:

    sudo zypper install -y habanalabs-graph

  10. Install container runtime:

    sudo zypper install -y habanalabs-container-runtime

  11. (Optional) Install qual library:**

    sudo zypper install -y habanalabs-qual

    For further details, see Gaudi Qualification Library.

  12. (Optional) Install qual workloads:**

    sudo zypper install -y habanalabs-qual-workloads

  13. (Optional) Install ethtool:

    sudo dnf install -y ethtool

    For further details, see Bring up Accelerator Interfaces.

  14. Update environment variables and more. When the installation is complete, close and re-open the shell. Or, run the following:

    source /etc/profile.d/habanalabs.sh

source ~/.bashrc

Set Number of Huge Pages

Some training models use huge pages. It is recommended to set the number of huge pages as shown below:

#set current hugepages
sudo sysctl -w vm.nr_hugepages=15000
#Remove old entry if exists in sysctl.conf
sudo sed --in-place '/nr_hugepages/d' /etc/sysctl.conf
#Insert huge pages settings to persist
echo "vm.nr_hugepages=15000" | sudo tee -a /etc/sysctl.conf