Disable/Enable NICs

Disable/Enable Gaudi 3 External NICs

Disabling and enabling Gaudi 3 external NIC ports can be done either from the host (in-band) or through the BMC (out-of-band). This section provides instructions on how to manage external NIC ports using both methods, offering flexibility in network interface management.

You can disable or enable Intel Gaudi 3 external NIC ports using hl-fw-loader. Run the below commands only when the driver is not loaded. The below commands can be executed from the host (in-band).

Note

  • By default, all NIC ports are enabled.

  • The configuration persists until the next power cycle.

  • Make sure that the gaudi3-sec-fit-fw-57.2.2.tgz file has already been downloaded from the Intel Gaudi vault or Intel RDC and extracted. If not, follow Step 1 and Step 2 below.

  • The status of the Ethernet ports (whether enabled or disabled) can be retrieved from the “System Status” packet using the hl_smi_asynch tool. For further details, refer to Gaudi 3 In-Band Telemetry for Hypervisor document located in the Intel Gaudi vault and Intel RDC.

  1. Download the gaudi3-sec-fit-fw-57.2.2.tgz file in /lib/firmware/habanalabs/gaudi3/:

    wget https://vault.habana.ai/artifactory/gaudi-odm/1.19.1/gaudi3/gaudi3-sec-fit/gaudi3-sec-fit-fw-57.2.2.tgz

  2. Extract the gaudi3-sec-fit-fw-57.2.2.tgz file:

    sudo tar -xvf gaudi3-sec-fit-fw-57.2.2.tgz

  3. Disable/enable external ports:

    1. To disable external NIC ports, run the following command:

      hl-fw-loader -f /lib/firmware/habanalabs/gaudi3/gaudi3-nic-ext-dis.itb

    2. To enable external NIC ports, run the following command:

      hl-fw-loader -f /lib/firmware/habanalabs/gaudi3/gaudi3-nic-ext-en.itb

For details on the SerDes connectivity, refer to HLB-325 UBB Specification located in the Intel Gaudi vault and Intel RDC.

The BMC (out-of-band) can read and modify the configuration of external NIC ports as described below:

  • To read the current NIC ports configuration, use GetNumericEffecterValue.

  • To modify the NIC ports configuration, use the SetNumericEffecterValue command, when effecterValue is defined as <bitmask value> in which each bit represents the NIC port to be enabled or disabled.

Note

  • For further details on the read and modify commands, refer to the DMTF’s DSP0248 Specification.

  • Since bitmask is not a PLDM-defined baseUnit, it is defined as an OEM Unit using dedicated PDR. For further details, refer to Gaudi 3 RAS PLDM Sheet located in the the Intel Gaudi vault and Intel RDC.

The effecter value is 32 bits wide, with each bit corresponding to a specific NIC port. For example:

  • Setting bit 0 to 1 enables NIC port 1, while setting it to 0 disables it.

  • Setting bit 1 to 1 enables NIC port 2, while setting it to 0 disables it.

This continues until bit 23 for NIC port 24. Since there are 24 ports, the 8 most significant bits are always 0.

Note

  • By default, all NIC ports are enabled.

  • The configuration persists until the next power cycle.

  • The effecter is only available during preboot stage.

To disable external NIC ports, use the corresponding value for each OAM ID. The below describes the UBB port mask configuration to disable external NIC ports on HLB-325 UBB system. For further details on the SerDes connectivity, refer to HLB-325 UBB Specification located in the Intel Gaudi vault and Intel RDC.

effecterValue for OAM0: 32'b0000_0000_1100_1101_1111_1111_1111_1111
effecterValue for OAM1: 32'b0000_0000_1111_1111_1111_1100_1101_1111
effecterValue for OAM2: 32'b0000_0000_1111_1111_1111_1100_1101_1111
effecterValue for OAM3: 32'b0000_0000_1100_1101_1111_1111_1111_1111
effecterValue for OAM4: 32'b0000_0000_1111_0111_0011_1111_1111_1111
effecterValue for OAM5: 32'b0000_0000_1111_1111_1111_1111_0111_0011
effecterValue for OAM6: 32'b0000_0000_1111_1111_1111_1111_0111_0011
effecterValue for OAM7: 32'b0000_0000_1111_0111_0011_1111_1111_1111

To enable all NIC ports across all OAM IDs, use the following value. This will enable all 24 ports in all OAM IDs:

32'b0000_0000_1111_1111_1111_1111_1111_1111

Disable/Enable Gaudi 2 External NICs

You can disable or enable Gaudi external NICs using hl-fw-loader. The NIC state is stored in EEPROM which makes it persistent. Run the below commands only when the driver is not loaded. The below commands can be executed from host (inband) or BMC (out-of-band).

Note

Make sure that the gaudi2-sec-fit-fw-56.1.0.tgz file has already been downloaded from the Intel Gaudi vault or Intel RDC and extracted. If not, follow Step 1 and Step 2 below.

  1. Download the gaudi2-sec-fit-fw-56.1.0.tgz file in /lib/firmware/habanalabs/gaudi2/:

    wget https://vault.habana.ai/artifactory/gaudi-odm/1.19.1/gaudi2/gaudi2-sec-fit/gaudi2-sec-fit-fw-56.1.0.tgz`

  2. Extract the gaudi2-sec-fit-fw-56.1.0.tgz file:

    sudo tar -xvf gaudi2-sec-fit-fw-56.1.0.tgz

  3. Disable/enable external ports:

    1. To disable external NIC ports, run the following command:

      hl-fw-loader -f /lib/firmware/habanalabs/gaudi2/gaudi2-nic-no_ext.itb

    2. To enable external NIC ports, run the following command:

      hl-fw-loader -f /lib/firmware/habanalabs/gaudi2/gaudi2-nic-default.itb

Disable/Enable First-gen Gaudi External NICs

You can disable or enable Gaudi NICs from a machine only with privileged ELBI access. Make sure ELBI is not blocked on the machine. Run the below commands only when the driver is not loaded.

Note

Make sure that the gaudi-sec-fit-fw-32.6.6.tgz file has already been downloaded from the Intel Gaudi vault and extracted. If not, follow Step 1 and Step 2 below.

  1. Download the gaudi-sec-fit-fw-32.6.6.tgz file in /lib/firmware/habanalabs/gaudi/:

    wget https://vault.habana.ai/artifactory/gaudi-odm/1.19.1/gaudi/gaudi-sec-fit/gaudi-sec-fit-fw-32.6.6.tgz`

  2. Extract the gaudi-sec-fit-fw-32.6.6.tgz file:

    sudo tar -xvf gaudi-sec-fit-fw-32.6.6.tgz

  3. Disable/enable external ports:

    1. To disable external NIC ports, run the following command:

      hl-fw-loader -f /lib/firmware/habanalabs/gaudi/gaudi-nic-dis.itb

    2. To enable external NIC ports, run the following command:

      hl-fw-loader -f /lib/firmware/habanalabs/gaudi/gaudi-nic-en.itb