Intel Gaudi Secure Firmware Flow

Intel® Gaudi® employs several security mechanisms against remote attacks. It implements a secure FW flow used as a root of trust. The secure FW image is stored in the SPI hash and chain of trust, which U-boot and Linux use to read over PCIe.

Note

  • For first-gen Gaudi, enabling the secure FW flow is required.

  • For Gaudi 2, the secure FW is enabled by default.

  • For Gaudi 3, enabling the secure FW flow is required only when upgrading SVN. It is not required on a new Gaudi 3 device as it is enabled by default.

../../_images/enable_secure_boot.JPG

Checking Secure Firmware Status

To check if the secure FW is enabled, run the following command:

sudo hl-fw-loader -f /lib/firmware/habanalabs/gaudi3/gaudi3-security_status.itb

sudo hl-fw-loader -f /lib/firmware/habanalabs/gaudi2/gaudi2-security_status.itb

hl-fw-loader -f /lib/firmware/habanalabs/gaudi/gaudi-security_status.itb

The output of hl-fw-loader contains the secure FW update status:

Security:       Enabled [Permanent: True]
Keys Present:   Yes
        [0]: Valid
        [1]: Valid
        [2]: Valid
        [3]: Valid
        [4]: Valid
Minimal SVN:    0
Flash W/P:      On
Memory Clean:   No

Security:       Enabled
Keys Present:   Yes
        [0]: Valid
        [1]: Valid
        [2]: Valid
        [3]: Valid
        [4]: Valid
Minimal SVN:    0
Flash W/P:      Off

Enabling Secure Firmware Flow

The following steps describe how to enable secure FW flow when upgrading SVN on Gaudi 3. Make sure to use the latest FW version before performing the procedure. To upgrade the FW, follow the steps in Firmware Upgrade.

  1. Check if security is enabled:

    sudo hl-fw-loader -f /lib/firmware/habanalabs/gaudi3/gaudi3-security_status.itb

    If the security status is Disabled, proceed to Step 2. The expected output is as follows:

    Security:       Disabled [Permanent: True]
Keys Present:   Yes
        [0]: Valid
        [1]: Valid
        [2]: Valid
        [3]: Valid
        [4]: Valid
Minimal SVN:    0
Flash W/P:      On
Memory Clean:   No

  2. Enable eFuse power. eFuse power supply is controlled from the BMC. It is another security measure in which the host cannot program eFuse without having BMC access:

    1. Read the current value of CPLD register 0xE:

      sudo i2cget -y -f <i2c_bus> 0x25 0xE

    2. Write 1 to bit 6 of CPLD register 0xE:

      sudo i2cset -y -f <i2c_bus> 0x25 0xE <Val | 0x40> # Val is read in step a

  3. Program secureboot_config.itb by sending a request to enable security. During this stage, the device switches to secure FW mode and reboots. After passing the boot sequence successfully in secure FW mode, the eFuse is consequently burnt. After the eFuse is burnt, secure FW mode becomes permanently active:

    sudo hl-fw-loader -f  gaudi3-secureboot_config.itb

  4. Disable eFuse power. This is required to prevent accidental burning of eFuse values from host:

    1. Read the current value of CPLD register 0xE:

      sudo i2cget -y -f <i2c_bus> 0x25 0xE

    2. Write 0 to bit 6 of CPLD register 0xE:

      sudo i2cset -y -f <i2c_bus> 0x25 0xE <Val & ~0x40> # Val is read in step a

  5. Check if secure FW is enabled:

    sudo hl-fw-loader -f /lib/firmware/habanalabs/gaudi3/gaudi3-security_status.itb

    The expected output is as follows:

    Security:       Enabled [Permanent: True]
Keys Present:   Yes
        [0]: Valid
        [1]: Valid
        [2]: Valid
        [3]: Valid
        [4]: Valid
Minimal SVN:    0
Flash W/P:      On
Memory Clean:   No

The following steps describe how to enable secure FW flow on all available cards. To enable secure FW flow on a specific card, use -d argument. For more information, see Firmware Update Tool.

  1. Write OTP data to the device. The OTP data includes the hash of the public key which can be used with the device:

    hl-fw-loader -f /lib/firmware/habanalabs/gaudi/gaudi-otp.itb

  2. Enable eFuse power. eFuse power supply is controlled from the BMC. It is another security measure in which the host cannot program eFuse without having BMC access. The power supply is turned off automatically after 5 minutes of operation. To enable the power supply for burning the eFuse on all devices, run -d all:

    hl-i2c-util --efuse-wr-en off -d all

    Or, enable on a specific device (OAM1 for example):

    hl-i2c-util --efuse-wr-en off -d 1

  3. Send a request to enable security. During this stage, the device temporarily switches to secure FW mode and reboots. After passing the boot sequence successfully in secure FW mode, the eFuse is consequently burnt. After the eFuse is burnt, secure FW mode becomes permanently active:

    hl-fw-loader -f /lib/firmware/habanalabs/gaudi/gaudi-security_enable.itb

  4. Reset the host (hypervisor). Secured Gaudi has a different PCI ID than unsecured Gaudi, therefore, after enabling security, the host (hypervisor) should be reset. After reset, the Gaudi device is identified with the secured Gaudi PCI ID.

  5. Revoke obsolete keys and security versions. After enabling secure FW mode successfully, obsolete keys and security versions need to be revoked. By using a FIT file for revocation, the device revokes all keys and security versions sequentially until the present one:

    hl-fw-loader -f /lib/firmware/habanalabs/gaudi/gaudi-revoke_key.itb

  6. Check if secure FW is enabled:

    hl-fw-loader -f /lib/firmware/habanalabs/gaudi/gaudi-security_status.itb

    The expected output is as follows:

    Security:       Enabled
Keys Present:   Yes
        [0]: Valid
        [1]: Valid
        [2]: Valid
        [3]: Valid
        [4]: Valid
Minimal SVN:    0
Flash W/P:      Off

Note

  • Revocation steps also use eFuse, so in case of delay in the process, it is recommended to enable the eFuse power again before revocation steps.

  • Secure FW mode is irreversible.

  • Keys and security versions revocation is irreversible.

Updating SVN

Note

SVN update is not required for first-gen Gaudi.

  1. Enable eFuse power. eFuse power supply is controlled from the BMC. It is another security measure in which the host cannot program eFuse without having BMC access:

    1. Read the current value of CPLD register 0xE:

      sudo i2cget -y -f <i2c_bus> 0x25 0xE

    2. Write 1 to bit 6 of CPLD register 0xE:

      sudo i2cset -y -f <i2c_bus> 0x25 0xE <Val | 0x40> # Val is read in step a

  2. Revoke obsolete security versions. After enabling secure FW mode successfully, obsolete security versions need to be revoked. By using a FIT file for revocation, the device revokes all security versions sequentially until the present one:

    sudo hl-fw-loader -f gaudi3-update_svn.itb

    Note

    Once SVN is updated, downgrading to previous FW versions is not possible.

  3. Disable eFuse power. This is required to prevent accidental burning of eFuse values from host:

    1. Read the current value of CPLD register 0xE:

      sudo i2cget -y -f <i2c_bus> 0x25 0xE

    2. Write 0 to bit 6 of CPLD register 0xE:

      sudo i2cset -y -f <i2c_bus> 0x25 0xE <Val & ~0x40> # Val is read in step a

  1. Enable eFuse power. eFuse power supply is controlled from the BMC. It is another security measure in which the host cannot program eFuse without having BMC access:

    1. Read the current value of CPLD register 0x6:

      sudo i2cget -y -f <i2c_bus> 0x25 0x6

    2. Write 1 to bit 1 of CPLD register 0x6:

      sudo i2cset -y -f <i2c_bus> 0x25 0xE <Val | 0x2> # Val is read in step a

  2. Revoke obsolete security versions. After enabling secure FW mode successfully, obsolete security versions need to be revoked. By using a FIT file for revocation, the device revokes all security versions sequentially until the present one:

    sudo hl-fw-loader -f gaudi2-update_svn.itb

    Note

    Once SVN is updated, downgrading to previous FW versions is not possible.

  3. Disable eFuse power. This is required to prevent accidental burning of eFuse values from host:

    1. Read the current value of CPLD register 0x6:

      sudo i2cget -y -f <i2c_bus> 0x25 0x6

    2. Write 0 to bit 1 of CPLD register 0x6:

      sudo i2cset -y -f <i2c_bus> 0x25 0xE <Val & ~0x2> # Val is read in step a