Firmware Upgrade

Firmware upgrade can be performed either by using hl-fw-loader tool as described in Firmware Update Tool section, or as described below.

Note

Make sure to review the New FW Features and Enhancements section as well as the FW Known Limitations section in the Release Notes.

Platform Level Components

Before upgrading the firmware, make sure the following platform level components are upgraded:

  • OAM CPLD - To upgrade the OAM CPLD, refer to the instructions included in Intel Gaudi vault and Intel RDC:

    • For Gaudi 3, use In-Band CPLD Programming guide.

    • For Gaudi 2, use HL225 Porting guide, Updating CPLD section.

  • Platform BIOS, and BMC FW - Refer to your system vendor documentation for details.

  • (HLS-2 only) PCIe switch version - Refer to your system vendor documentation for details.

Note

Contact your local support representative if you do not have access to the Intel Gaudi vault or Intel RDC.

Firmware Upgrade

The habanalabs-firmware-odm package must be installed before firmware upgrade. See Driver and Software Installation.

  1. Unload the drivers:

    sudo modprobe -r habanalabs && sudo modprobe -r habanalabs_cn && sudo modprobe -r habanalabs_ib && sudo modprobe -r habanalabs_en
    
  2. Check the existing Gaudi SPI FW version by running the following command:

    sudo hl-fw-loader -s 2> /dev/null | grep SPI
    

    Only the SPI FW needs to be updated at the system level. Refer to the Support Matrix for the exact version. The below shows an example output for the 1.19.2-32 release:

    Current SPI version: Preboot version hl-gaudi3-1.19.0-fw-56.1.0-sec-9 (Dec 10 2024 - 14:18:12)
    
  3. To start the firmware upgrade, run the following command:

    sudo hl-fw-loader
    

    Note

    Upgrading the firmware may take a while to complete. Make sure no network problems or power failure occurs during upgrade.

  4. Load the drivers:

    sudo modprobe habanalabs && sudo modprobe habanalabs_cn && sudo modprobe habanalabs_ib && sudo modprobe habanalabs_en
    
  5. Re-check the SPI FW version by running the hl-smi command below and verify it matches the supported versions listed in the Support Matrix according to the Gaudi device you are using. The below shows an example output of Gaudi 2:

         $ hl-smi -L | grep SPI
         Firmware [SPI] Version: Preboot version hl-gaudi2-1.20.0-fw-58.0.0-sec-9 (Jul 20 2023 - 17:57:23)
         Firmware [SPI] Version: Preboot version hl-gaudi2-1.20.0-fw-58.0.0-sec-9 (Jul 20 2023 - 17:57:23)
         Firmware [SPI] Version: Preboot version hl-gaudi2-1.20.0-fw-58.0.0-sec-9 (Jul 20 2023 - 17:57:23)
         Firmware [SPI] Version: Preboot version hl-gaudi2-1.20.0-fw-58.0.0-sec-9 (Jul 20 2023 - 17:57:23)
         Firmware [SPI] Version: Preboot version hl-gaudi2-1.20.0-fw-58.0.0-sec-9 (Jul 20 2023 - 17:57:23)
         Firmware [SPI] Version: Preboot version hl-gaudi2-1.20.0-fw-58.0.0-sec-9 (Jul 20 2023 - 17:57:23)
         Firmware [SPI] Version: Preboot version hl-gaudi2-1.20.0-fw-58.0.0-sec-9 (Jul 20 2023 - 17:57:23)
         Firmware [SPI] Version: Preboot version hl-gaudi2-1.20.0-fw-58.0.0-sec-9 (Jul 20 2023 - 17:57:23)
    

eROM Upgrade

When upgrading the FW, the eROM should also be upgraded if it is detected as empty. To verify the installed eROM version, run the following command:

sudo hl-smi --fw-version

If the output indicates the latest eROM version, an eROM upgrade is not necessary. See the example below:

 hl-smi --fw-versions | grep erom -A 2 | grep component
                 component               : hl-gaudi3-1.20.0-fw-58.1.1sec-2
                 component               : hl-gaudi3-1.20.0-fw-58.1.1sec-2
                 component               : hl-gaudi3-1.20.0-fw-58.1.1sec-2
                 component               : hl-gaudi3-1.20.0-fw-58.1.1sec-2
                 component               : hl-gaudi3-1.20.0-fw-58.1.1sec-2
                 component               : hl-gaudi3-1.20.0-fw-58.1.1sec-2
                 component               : hl-gaudi3-1.20.0-fw-58.1.1sec-2
                 component               : hl-gaudi3-1.20.0-fw-58.1.1sec-2

If the output is detected as empty, make sure to follow the steps after this example:

sudo hl-smi --fw-versions | grep erom -A 2 | grep component
            component               :
            component               :
            component               :
            component               :
            component               :
            component               :
            component               :
            component               :

Note

  • Upgrading eROM is required for Gaudi 3 only.

  • Refer to the Support Matrix for the latest eROM version.

  • Before running the procedure, make sure you have root privileges.

To upgrade the eROM, perform the following:

  1. Unload the drivers:

    sudo rmmod habanalabs; sudo rmmod habanalabs_cn; sudo rmmod habanalabs_ib; sudo rmmod habanalabs_en
    
  2. Upgrade the eROM by running the following command:

    cd /opt/habanalabs/qual/gaudi3/bin
    sudo hl-fw-loader -f /lib/firmware/habanalabs/gaudi3/gaudi3-agent-fw_loader-fit_erom.itb
    
  3. Load the drivers:

    sudo modprobe habanalabs && sudo modprobe habanalabs_cn && sudo modprobe habanalabs_ib && sudo modprobe habanalabs_en
    

Note

For the out-of-band upgrade instructions, refer to Gaudi 3 OOB Management Specification available the Intel Gaudi vault and Intel RDC.

Firmware Update Lock

By default, Gaudi device is open for updates (such as SPI flash, eROM etc.) via in-band (host or PCI interface). However, the Firmware Update Lock offers an option to disable in-band firmware updates and any other commands supported by hl-fw-loader tool. This option does not affect out-of-band updates executed from the BMC.

This feature locks the entire hl-fw-loader interface to the host in all directions.

Note

  • This feature is available on Gaudi 3 and Gaudi 2 only.

  • This feature is applicable only when the driver is unloaded.

  • The below commands can be only executed from the host (in-band).

  1. Download the gaudi3-sec-fit-fw-58.1.1.tgz file in /lib/firmware/habanalabs/gaudi3/. The file can be downloaded from the Intel Gaudi vault or Intel RDC. If you do not have access, contact your local support representative.

  2. Extract the gaudi3-sec-fit-fw-58.1.1.tgz file:

    sudo tar -xvf gaudi3-sec-fit-fw-58.1.1.tgz
    
  3. Execute firmware update lock using the gaudi3-fw_upd_closure.itb file:

    $ hl-fw_loader -f gaudi3-fw_upd_closure.itb
    

Note

  • Power cycling the device disables the Firmware Update Lock feature:

    • If you plan to update the firmware, make sure to power cycle the device.

    • If the device is power cycled for any reason, make sure to re-enable the Firmware Update Lock.

  • Updating, locking or unlocking the firmware from out-of-band is always available using PLDM.

  1. Download the gaudi2-sec-fit-fw-58.0.0.tgz file in /lib/firmware/habanalabs/gaudi2/. The file can be downloaded from the Intel Gaudi vault or Intel RDC. If you do not have access, contact your local support representative.

  2. Extract the gaudi2-sec-fit-fw-58.0.0.tgz file:

    sudo tar -xvf gaudi2-sec-fit-fw-58.0.0.tgz
    
  3. Execute firmware update lock using the gaudi2-fw_upd_closure.itb file:

    $ hl-fw-loader -f gaudi2-fw_upd_closure.itb
    

Note

Power cycling the device disables the Firmware Update Lock feature:

  • If you plan to update the firmware, make sure to power cycle the device.

  • If the device is power cycled for any reason, make sure to re-enable the Firmware Update Lock.