U30 FPGA FW update¶
In the Alveo™ U30 Hyperscaler only SKU, the satellite controller (SC) firmware supports an out-of-band method of FPGA flash image update, read-back and authentication. Server BMC shall initiate and perform all these operations by sending the I2C commands to the SC. FPGA flash operation commands are supported at I2C address 0x65 (0xCA in 8-bit). Currently, Satellite Controller supports only 100 KHz I2C speed for all the FPGA flash update commands mentioned in this section.
Note: The satellite controller will perform I2C clock-stretching, wherever applicable, to perform the requested actions.
Note: It is recommended for the BMC to add 2 seconds as inter-command interval.
In addition to the FPGA flash update via OoB, SC also supports other FPGA flash related operations like:
- Enable or Disable Write Protect (WP#) settings
- FPGA flash image read-back
- Flash image authentication via MAC/HASH calculation
- Copy of flash image from one flash to another (F2F copy)
The table below lists all the commands supported/needed for FPGA flash operations. Currently, the FPGA flash operation commands are supported only for Alveo™ U30.
Note: MAC in this chapter refers to Message Authentication Code and it can also be referred as HASH. MAC/HASH calculation of the entire or sometimes few select FPGA flash sectors is performed at the request of BMC, to validate the flash contents haven’t been tampered with.
Table: FPGA Flash Upgrade Commands
| Command | Command Description | Server BMC Action | SC Action |
|---|---|---|---|
| 0x40 | FPGA_RESET_DEVICE | B0 from BMC: 0x01: FPGA devices 0x02: SC FW |
See table ‘Flash Operation Return Codes’ for SC’s response |
| 0x41 | FPGA_GET_FW_VER | B0 from BMC: 0x01: FPGA1 Primary 0x02: FPGA1 Recovery 0x03: FPGA2 Primary (if present) 0x04: FPGA2 Recovery(if present) |
B0 – Valid Byte B1 - Minor revision B2 - Major revision |
| 0x42 | FPGA_SET_TARGET_DEVICE | B0 from BMC 0x01: FPGA1 Primary 0x02: FPGA1 Recovery 0x03: FPGA2 Primary (if present) 0x04: FPGA2 Recovery(if present) |
See table ‘Flash Operation Return Codes’ for SC’s response |
| 0x43 | FPGA_SET_BOOT_DEVICE | B0 from BMC: 0x01: FPGA1 Primary 0x02: FPGA1 Recovery 0x03: FPGA2 Primary (if present) 0x04: FPGA2 Recovery(if present) |
See table ‘Flash Operation Return Codes’ for SC’s response |
| 0x44 | FPGA_SC_SET_WRITE _ENABLE Provides control to SC over WP# pins and enables SC to access QSPI flash |
B0 from BMC: 0x01: FPGA1 Primary 0x02: FPGA1 Recovery 0x03: FPGA2 Primary (if present) 0x04: FPGA2 Recovery(if present) B1 from BMC: 0x01: WP enable 0x02: WP disable |
See table ‘Flash Operation Return Codes’ for SC’s response |
| 0x45 | FLASH_SET_WRITE_ENABLE Provides control to SC over WP# pins and enables FPGA to access QSPI flash |
B0 from BMC: 0x01: FPGA1 Primary 0x02: FPGA1 Recovery 0x03: FPGA2 Primary (if present) 0x04: FPGA2 Recovery(if present) B1 from BMC: 0x01: WP enable 0x02: WP disable |
See table ‘Flash Operation Return Codes’ for SC’s response |
| 0x46 | FLASH_GET_WRITE _PROTECT_STATES |
B0 from BMC: 0x01: FPGA1 Primary 0x02: FPGA1 Recovery 0x03: FPGA2 Primary (if present) 0x04: FPGA2 Recovery(if present) |
Byte 0: SC WP Status 0x01: WP enabled 0x02: WP disabled Byte 1: FPGA WP Status 0x01: WP enabled 0x02: WP disabled |
| 0x47 | FLASH_TX_DATA_BLOCK SC accumulates 252 bytes from each transaction to form 64KB sector & writes it into QSPI flash |
BMC sends data bytes D0, D1 … D251 |
N/A |
| 0x48 | FLASH_BLOCK_CRC_CHECK SC compares CRC, writes 1 sector to flash, rechecks CRC with FPGA device |
BMC sends 8 byte CRC B0, B1 … B7 |
See table ‘Flash Operation Return Codes’ for SC’s response |
| 0x49 | FLASH_SECTOR_SET_SEQ _NUM |
B0: Sector number (low byte) B1: Sector number (high byte) |
See table ‘Flash Operation Return Codes’ for SC’s response |
| 0x4A | FLASH_COPY_FIRMWARE | B0: Source flash device 0x01: FPGA1 Primary 0x02: FPGA1 Recovery 0x03: FPGA2 Primary (if present) 0x04: FPGA2 Recovery(if present) B1: Destination flash device 0x01: FPGA1 Primary 0x02: FPGA1 Recovery 0x03: FPGA2 Primary (if present) 0x04: FPGA2 Recovery(if present) |
See table ‘Flash Operation Return Codes’ for SC’s response |
| 0x4B | FPGA_GET_FIRMWARE _STATUS |
N/A | See table ‘Flash Operation Return Codes’ for SC’s response |
| 0x4C | FPGA_SET_KEY_NONCE BMC sends key and nonce that’s randomly generated. SC stores both in internal flash (Non-volatile) |
B0 from BMC: 0x01: FPGA1 Primary 0x02: FPGA1 Recovery 0x03: FPGA2 Primary (if present) 0x04: FPGA2 Recovery(if present) B1 - B16: Key B17 – B28: Nonce |
See table ‘Flash Operation Return Codes’ for SC’s response |
| 0x4D | FPGA_CALC_MAC SC increments nonce by 1, calculates hash using the existing key & new nonce, and stores MAC/hash value in SC’s internal flash |
B0 from BMC: 0x01: FPGA1 Primary 0x02: FPGA1 Recovery 0x03: FPGA2 Primary (if present) 0x04: FPGA2 Recovery(if present) |
See table ‘Flash Operation Return Codes’ for SC’s response |
| 0x4E | FPGA_VERIFY_MAC SC calculates hash using existing key & nonce, and stores MAC/hash value in SC’s internal flash |
B0 from BMC: 0x01: FPGA1 Primary 0x02: FPGA1 Recovery 0x03: FPGA2 Primary (if present) 0x04: FPGA2 Recovery(if present) |
See table ‘Flash Operation Return Codes’ for SC’s response |
| 0x4F | FPGA_GET_MAC_STATUS SC responds with status of MAC/has calculation or verification along with the 16-byte MAC value |
B0 from BMC: 0x01: FPGA1 Primary 0x02: FPGA1 Recovery 0x03: FPGA2 Primary (if present) 0x04: FPGA2 Recovery(if present) B1 from BMC: 0x01: Get CALC_MAC status 0x02: Get VERIFY_MAC status |
Byte B0 from SC: See table ‘Flash Operation Return Codes’ for response Bytes B1 - B16 from SC: 16 byte MAC/hash value |
| 0x50 | FPGA_SET_IMAGE_SIZE | B0 from BMC: 0x01: FPGA1 Primary 0x02: FPGA1 Recovery 0x03: FPGA2 Primary (if present) 0x04: FPGA2 Recovery(if present) B1 - B4 from BMC (LSB first): Size of QSPI image in bytes |
N/A |
| 0x51 | NOTIFY_WP_TO_FPGA | B0 from BMC: 0x01: FPGA1 Primary 0x02: FPGA1 Recovery 0x03: FPGA2 Primary (if present) 0x04: FPGA2 Recovery(if present) |
Byte B0 from SC: 0x01: Operation Success 0x02: Operation failure |
| 0x52 | FPGA_UART_DEBUG_CONTROL | B0 from BMC: 0x01: FPGA1 0x02: FPGA2 |
0x01: Operation Success 0x02: Operation failure 0x03: Operation unsupported |
| 0x53 | SET_FPGA_FLASH_READBACK _SECTOR_RANGE |
B0: Start sector num (low byte) B1: Start sector num (high byte) B2: End sector num (low byte) B3: End sector num (high byte) |
See table ‘Flash Operation Return Codes’ for SC’s response |
| 0x54 | FPGA_FLASH_READBACK_RX _DATA_BLOCK |
BMC sends repeated-start I2C command |
SC sends 256 data bytes: D0, D1 … D255 |
| 0x55 | GET_FPGA_FLASH_READBACK _SECTOR_CRC |
N/A | SC sends 8-bytes of CRC : D0, D1 … D7 |
Table: Flash Operation Return Codes
| Response Code | Description |
|---|---|
| 0x00 | Reserved |
| 0x01 | Operation success |
| 0x02 | Operation failed |
| 0x03 | Operation Not Supported |
| 0x04 | Flash erase failed. Abort operation, rectify error and re-initiate from start |
| 0x05 | Flash write failed. Abort operation, rectify error and re-initiate from start |
| 0x06 | Flash read failed. Abort operation, rectify error and re-initiate from start |
| 0x07 | Flash CRC failed. Abort operation, rectify error and re-initiate from start |
| 0x08 | Invalid Flash Selection |
| 0x09 | FPGA_GENERAL_ERROR |
| 0x0A | FPGA_MAC_CALCULATION_INVALID |
| 0x0B | FPGA_INVALID_IMAGE_LENGTH |
| 0x0C | QSPI SC disable WP failed. Abort operation, rectify error and re-initiate from start |
| 0x0D | QSPI wrong MCS file format. Abort operation, rectify error and re-initiate from start |
| 0x0E | Set the KEY and/or NONCE before proceeding for MAC Calculation |
| 0x0F | MAC calculation not performed. Please send command 0x4D before MAC verify |
| 0x10 | FLASH_TX_DATA_BLOCK command in-progress |
| 0x11 | FPGA1 primary flash update in-progress |
| 0x12 | FPGA1 recovery flash update in-progress |
| 0x13 | FPGA2 primary flash update in-progress |
| 0x14 | FPGA2 recovery flash update in-progress |
| 0x15–0x1F | Reserved |
| 0x20 | FLASH_BLOCK_CRC_CHECK command in-progress |
| 0x21 | FPGA_CRC_CHECK_STATUS_SECTOR_RESEND (Resend last sector (0x47)) |
| 0x22 | Reserved |
| 0x23 | QSPI_SET_UPDATE_DEVICE_NOT_SENT (send command 0x42) |
| 0x24 | QSPI_SC_SET_WRITE_NOT_ENABLED (send command 0x44) |
| 0x25–0x2F | Reserved |
| 0x30 | FLASH_COPY_FIRMWARE command in-progress |
| 0x31 | FPGA1 primary to FPGA1 recovery flash copy in-progress |
| 0x32 | FPGA1 primary to FPGA2 primary flash copy in-progress |
| 0x33 | FPGA1 primary to FPGA2 recovery flash copy in-progress |
| 0x34 | FPGA1 recovery to FPGA1 primary flash copy in-progress |
| 0x35 | FPGA1 recovery to FPGA2 primary flash copy in-progress |
| 0x36 | FPGA1 recovery to FPGA2 recovery flash copy in-progress |
| 0x37 | FPGA2 primary to FPGA1 primary flash copy in-progress |
| 0x38 | FPGA2 primary to FPGA1 recovery flash copy in-progress |
| 0x39 | FPGA2 primary to FPGA2 recovery flash copy in-progress |
| 0x3A | FPGA2 recovery to FPGA1 primary flash copy in-progress |
| 0x3B | FPGA2 recovery to FPGA1 recovery flash copy in-progress |
| 0x3C | FPGA2 recovery to FPGA2 primary flash copy in-progress |
| 0x3D–0x3F | Reserved |
| 0x40 | FPGA_CALC_MAC command in-progress |
| 0x41 | FPGA1 primary MAC calculation in-progress |
| 0x42 | FPGA1 recovery MAC calculation in-progress |
| 0x43 | FPGA2 primary MAC calculation in-progress |
| 0x44 | FPGA2 recovery MAC calculation in-progress |
| 0x45 | FPGA_KEY_NONCE update in-progress |
| 0x46–0x4F | Reserved |
| 0x50 | FPGA_VERIFY_MAC command in-progress |
| 0x51 | FPGA1 primary MAC verification in-progress |
| 0x52 | FPGA1 recovery MAC verification in-progress |
| 0x53 | FPGA2 primary MAC verification in-progress |
| 0x54 | FPGA2 recovery MAC verification in-progress |
| 0x55–0x6F | Reserved |
| 0x70 | MAC verification not performed. Please send command (0x4E) before fetching MAC verification status |
| 0x71 | FPGA MAC Calculation/Verification failed. (Abort operation, re-initiate MAC Calculation/Verification) |
| 0x72 | EEPROM read failed |
| 0x73 | EEPROM write failed |
| 0x74-0x7F | Reserved |
| 0x80 | FPGA flash data read-back in progress |
| 0x81 | FPGA flash data ready for read-back |
| 0x82 | Invalid FPGA sector range. Check and resend within the valid range (0 - 2047) |
| 0x83–0xEF | Reserved |
| 0xFF | FPGA_NO_OPERATION |
0x40 - FPGA_RESET_DEVICE¶
The BMC can send the FPGA_RESET_DEVICE command to reset FPGA device(s) or SC FW. For SC reset (warm reset), the satellite controller firmware responds with the status and reboots itself. In Alveo U30, the reset command will reset both the FPGA devices (ZYNQ1 and ZYNQ2) and internally, both PS (Processing Subsystem) and PL (Programmable Logic) will reload from flash device.
Table: FPGA_RESET_DEVICE Server BMC Request
| Server BMC Request | |
|---|---|
| Command code | 0x40 |
| Byte0 | 0x01: FPGA devices 0x02: SC FW |
Table: FPGA_RESET_DEVICE Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | B0 | See table ‘Flash Operation Return Codes’ for SC’s response |
0x41 - FPGA_GET_FW_VER¶
The BMC sends this command to fetch the FW version running in either FPGA1 or FPGA2 device. This command is currently supported only in the Alveo U30 data accelerator card. The byte 0 (validity byte) in response from SC must be read first.
Table: FPGA_GET_FW_VER Server BMC Request
| Server BMC Request | |
|---|---|
| Command code | 0x41 |
| Byte0 | 0x01: FPGA1 primary flash device 0x02: FPGA1 recovery flash device 0x03: FPGA2 primary flash device (if present) 0x04: FPGA2 recovery flash device (if present) |
Table: FPGA_GET_FW_VER Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | B0 B1, B2 |
B0 - Valid Byte 0x00 - Not supported (B1, B2 - Invalid) 0x01 - Unknown or Reduced Service 0x02 - reserved 0x03 - No error; Valid B1 & B2 bytes B1 – Minor version; B2 – Major version |
0x42 - FPGA_SET_TARGET_DEVICE¶
BMC sends the FPGA_SET_TARGET_DEVICE command to select the flash device to initiate the FW upgrade.
NOTE: This command is not persistence across SC reboots. On boot-up, SC restores the default configuration (i.e.) Primary flash as teh target device.
Table: FPGA_SET_TARGET_DEVICE Server BMC Request
| Server BMC Request | |
|---|---|
| Command code | 0x42 |
| Byte0 | 0x01: FPGA1 primary flash device 0x02: FPGA1 recovery flash device 0x03: FPGA2 primary flash device (if present) 0x04: FPGA2 recovery flash device (if present) |
Table: FPGA_SET_TARGET_DEVICE Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | B0 | See table ‘Flash Operation Return Codes’ for SC’s response |
0x43 - FPGA_SET_BOOT_DEVICE¶
The BMC sends the FPGA_SET_BOOT_DEVICE command to set the boot device. The SC FW stores this information in the internal flash memory and restores the configuration in case of server cool boot or power cycle.
Note: Primary flash device is selected as the boot device/default configuration in FPGA1 (and FPGA2 if present).
Table: FPGA_SET_BOOT_DEVICE Server BMC Request
| Server BMC Request | |
|---|---|
| Command code | 0x43 |
| Byte0 | 0x01– FPGA1 primary flash device 0x02– FPGA1 recovery flash device 0x03– FPGA2 primary flash device (if present) 0x04– FPGA2 recovery flash device (if present) |
Table: FPGA_SET_BOOT_DEVICE Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | Byte0 | 0x01– Request success 0x02– Request failed |
0x44 - FPGA_SC_SET_WRITE_ENABLE¶
The BMC sends the FPGA_SC_SET_WRITE_ENABLE command to enable/disable write protect for SPI mode from the SC point of view. This command must follow FPGA_SET_TARGET_DEVICE. When the SC has write protect mode disabled, only the SC has the access to QSPI flash and only SC can write into the flash via SPI (x1 mode). In this case, FPGA can not read from FPGA flash.
For both hyperscaler and OEM customers, by default, SC WP# is enabled (i.e.) FPGA has read-only access. The actual FPGA WP# state (write access) is controlled by command 0x45.
Note: The SC will not store this configuration (from command 0x44) in any persistence memory. A SC reboot or device power cycle results in loss of configuration. During the subsequent boot-up, the SC will restore the default configuration (i.e.) WP enabled.
Table: FPGA_SC_SET_WRITE_ENABLE Server BMC Request
| Server BMC Request | |
|---|---|
| Command code | 0x44 |
| Byte0 | 0x01: FPGA1 primary flash device 0x02: FPGA1 recovery flash device 0x03: FPGA2 primary flash device (if present) 0x04: FPGA2 recovery flash device (if present) |
| Byte1 | 0x01: WP enable 0x02: WP disable |
Table: FPGA_SC_SET_WRITE_ENABLE Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | B0 | See table ‘Flash Operation Return Codes’ for SC’s response |
0x45 - FLASH_SET_WRITE_ENABLE¶
The BMC can send the FLASH_SET_WRITE_ENABLE command to enable/disable write protect the QSPI flash device in either ZYNQ device. This command must follow FPGA_SC_SET_WRITE_ENABLE for any FPGA flash FW upgrade.
For Hyperscaler customers, by default, the FPGA flash devices are write protected and can only run in x2 SPI mode as a preferred/secured mode.
When write protect is disabled, the QSPI flash device can be accessed in x1, x2, or x4 SPI Mode and FPGA has the write access to the QSPI devices. This is the default mode configured for OEM customers. SC FW configures this mode based on FRU parameters written into its EEPROM.
Note: The SC will not store this configuration (from command 0x45) in any persistence memory. A SC reboot or device power cycle results in the loss of configuration. During the subsequent boot-up, the SC will restore the default configuration (i.e.) WP enabled.
Table: FLASH_SET_WRITE_ENABLE Server BMC Request
| Server BMC Request | |
|---|---|
| Command code | 0x45 |
| Byte0 | 0x01: FPGA1 primary flash device 0x02: FPGA1 recovery flash device 0x03: FPGA2 primary flash device (if present) 0x04: FPGA2 recovery flash device (if present) |
| Byte1 | 0x01: WP enable 0x02: WP disable |
Table: FLASH_SET_WRITE_ENABLE Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | B0 | See table ‘Flash Operation Return Codes’ for SC’s response |
0x46 - FLASH_GET_WRITE_PROTECT_STATES¶
The BMC can send the FLASH_GET_WRITE_PROTECT_STATES command to get the write protect state for all the flash devices.
Table: FLASH_GET_WRITE_PROTECT_STATES Server BMC Request
| Server BMC Request | |
|---|---|
| Command code | 0x46 |
| Byte0 | 0x01: FPGA1 primary flash device 0x02: FPGA1 recovery flash device 0x03: FPGA2 primary flash device (if present) 0x04: FPGA2 recovery flash device (if present) |
Table: FLASH_GET_WRITE_PROTECT_STATES Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | Byte0 | Byte 0: SC WP Status 0x01: WP enabled ; 0x02: WP disabled |
| Byte 1 | Byte 1: FPGA WP Status 0x01: WP enabled ; 0x02: WP disabled |
|
0x47 - FLASH_TX_DATA_BLOCK¶
The BMC sends data blocks using the FLASH_TX_DATA_BLOCK command. Data (or payload) is the FW for QSPI flash devices. The SC accumulates 252 byte data from each transaction to build up 64 Kbyte blocks and write into QSPI flash. The sector size for QSPI flash is 64 Kbyte and the SC will cache 1 block of data in the internal SRAM before writing it to QSPI. This command must be preceded by the FPGA_SET_TARGET_DEVICE, FPGA_SC_SET_WRITE_ENABLE, and FLASH_SET_WRITE_ENABLE commands.
Note: The maximum supported size of each I2C transaction is 252 bytes.
Table: FLASH_TX_DATA_BLOCK Server BMC Request
| Server BMC Request | |
|---|---|
| Command code | 0x47 |
| Length byte 0 | Total number of bytes in the current I2C transaction (command) |
| Data bytes | D1, D2, … D252 |
Table: FLASH_TX_DATA_BLOCK Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | B0 | See table ‘Flash Operation Return Codes’ for SC’s response |
0x48 - FLASH_BLOCK_CRC_CHECK¶
The BMC sends the FLASH_BLOCK_CRC_CHECK command to check the CRC from the previous block and tracks the number of bytes sent to the SC. After sending 64K bytes, the BMC sends this command to initiate CRC check. The BMC sends 64-bit CRC (data payload and start address of the block) for the previous 64 Kbyte block along with this command request. This command also indicates the completion of the 64 Kbyte block transfer from the BMC to the SC, which immediately returns the status CRC_CHECK_IN_PROGRESS. This command is a trigger for the SC to start all the flash write and CRC check operations in the background. The BMC can poll the FPGA_GET_FIRMWARE_STATUS command periodically for completion status before moving to next sector. See table ‘Flash Operation Return Codes’ for SC’s response
Table 97: FLASH_BLOCK_CRC_CHECK Server BMC Server
| Server BMC Request | |
|---|---|
| Command code | 0x48 |
| Data bytes | D1 … D8 64-bit CRC data |
Table: FLASH_BLOCK_CRC_CHECK Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | B0 | See table ‘Flash Operation Return Codes’ for SC’s response |
0x49 - FLASH_SECTOR_SET_SEQ_NUM¶
The FPGA flash FW image update needs to be implemented with the retry mechanism in both the server BMC and the Alveo SC.
The BMC sends the FLASH_SECTOR_SET_SEQ_NUM command to set the sequence number (or the sector number) of the sector at which the SC needs to write into the FPGA Flash. This retry mechanism is needed to restart the FPGA FW upgrade process from the point where it was terminated previously. The termination reason could be a reboot of the SC FW, the BMC FW, power loss, or any user triggered event.
When server BMC wants to restart the FPGA FW upgrade process from the middle, the BMC sends a valid 2-byte sequence number. The BMC may keep track of the sequence number and keep updating by 1 for every successful response from the SC for the command FLASH_BLOCK_CRC_CHECK.
The SC will calculate the sector start address based on the sequence number and the fixed start offset (expected to be 0x0) of the FW inside flash. Responsibility is on the BMC to send the correct sequence number and its corresponding payload. Otherwise the SC may write into a non- contiguous flash sector and may end up corrupting the FW.
Note: The SC will not store the sector information in persistence memory. On boot-up, default value 0x00 will be assigned.
Table: FLASH_SECTOR_SET_SEQ_NUM Server BMC Request
| Server BMC Request | |
|---|---|
| Command code | 0x49 |
| Data bytes | B0– Sector number (low byte) B1– Sector number (high byte) |
Table: FLASH_SECTOR_SET_SEQ_NUM Xilinx Alveo Card Response
| Xilinx Alveo Card Response | |
|---|---|
| Data bytes | Byte0 0x01: Request success 0x02: Request failed |
0x4A - FLASH_COPY_FIRMWARE¶
The BMC sends the FLASH_COPY_FIRMWARE command to initiate the copy of FW from one QSPI flash device to another. The BMC sends the source and destination flash devices via this command. Upon receiving this command the SC:
- Copies the sector contents from the source device into 64 Kbyte SRAM within SC FW.
- Checks the CRC.
- Copies the contents into the destination device, checks the CRC, and updates the sector sequence number.
- If failure occurs in the middle of copy, erase all sectors in destination device (no retry support).
- If the CRC is copied successfully, proceed with next sector.
- The BMC can obtain the status of the copy from the FLASH_COPY_FIRMWARE_STATUS command.
Note: If the BMC sends another copy command while the previous copy is in-progress, the SC will ignore the request and respond appropriate error code. The BMC must check the status via the COPY_FIRMWARE_STATUS command and re-trigger. This command is currently supported only for Alveo U30.
Table: FPGA_COPY_FIRMWARE Server BMC Request
| Server BMC Request | |
|---|---|
| Command code | 0x4A |
| Byte0 | B0 – Source flash device: 0x01 - FPGA1 Primary flash device 0x02 - FPGA1 Recovery flash device 0x03 - FPGA2 Primary flash device 0x04 - FPGA2 Recovery flash device |
| Byte1 | B1 – Destination flash device: 0x01 - FPGA1 Primary flash device 0x02 - FPGA1 Recovery flash device 0x03 - FPGA2 Primary flash device 0x04 - FPGA2 Recovery flash device |
Table: FPGA_COPY_FIRMWARE Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | B0 | See table ‘Flash Operation Return Codes’ for SC’s response |
0x4B - FPGA_GET_FIRMWARE_STATUS¶
The BMC sends the FPGA_GET_FIRMWARE_STATUS command to obtain the status of the previously triggered commands like FLASH_BLOCK_CRC_CHECK and/or FLASH_COPY_FIRMWARE commands.
Table: FPGA_GET_FIRMWARE_STATUS Server BMC request
| Server BMC Request | |
|---|---|
| Command code | 0x4B |
| Byte0 | N/A |
Table: FPGA_GET_FIRMWARE_STATUS Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | B0 | See table ‘Flash Operation Return Codes’ for SC’s response |
0x4C - FPGA_SET_KEY_NONCE¶
BMC provides the SC with a randomly generated key and nonce. The SC writes the 16-byte AES128 key and 96-bit (12-byte) nonce value to the Non-volatile memory. The SC provisions storage for one key and one nonce per flash device (non-volatile memory).
SC uses 15-byte nonce, by padding 3 bytes (with 0x00) at the start, to the 12-byte nonce sent by BMC (i.e.) LSB 3 bytes are 0x00. In other words, Bytes[14-3] are the 12-bytes of nonce sent by BMC and Bytes[2-0] are 0x00 0x00 0x00.
BMC must use the exact nonce scheme to calculate the MAC value for comparison. The BMC is expected to select the target flash device.
Table: FPGA_SET_KEY_NONCE Server BMC Request
| Server BMC Request | |
|---|---|
| Command code | 0x4C |
| Byte0 | B0 – Target flash device: 0x01 - FPGA1 Primary flash device 0x02 - FPGA1 Recovery flash device 0x03 - FPGA2 Primary flash device 0x04 - FPGA2 Recovery flash device |
| Byte1 -16 | 16-byte key |
| Byte17 - 28 | 12-byte nonce |
Table: FPGA_SET_KEY_NONCE Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | B0 | See table ‘Flash Operation Return Codes’ for SC’s response |
0x4D - FPGA_CALC_MAC¶
Note: This command is optional. If sent, this command must be preceded by FPGA_SET_KEY_NONCE. This command is only supported in Alveo U30
The BMC is expected to call the FPGA_CALC_MAC command after the entire flash image is written. The BMC is expected to select the target flash device. Upon receiving the command 0x4D, SC increments the stored nonce by 1, calculates the MAC of the entire 128 MByte region of the FPGA flash device, using the existing key and the new nonce. The calculated MAC/HASH value is returned to BMC via the status command 0x4F. SC does not store the MAC/HASH value in Non-volatile memory.
Table: FPGA_CALC_MAC Server BMC Request
| Server BMC Request | |
|---|---|
| Command code | 0x4D |
| Byte0 | B0 – Target flash device: 0x01 - FPGA1 Primary flash device 0x02 - FPGA1 Recovery flash device 0x03 - FPGA2 Primary flash device 0x04 - FPGA2 Recovery flash device |
Table: FPGA_CALC_MAC Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | B0 | See table ‘Flash Operation Return Codes’ for SC’s response |
0x4E - FPGA_VERIFY_MAC¶
NOTE: This command must be preceded by FPGA_SET_KEY_NONCE. This command is supported only in Alveo U30.
The BMC sends the FPGA_VERIFY_MAC command to validate the FPGA flash image. The SC calculates the MAC/HASH of the entire 128 MByte region using the existing key and existing nonce value. The calculated MAC/HASH value is returned to BMC via the status command 0x4F. SC does not store the MAC/HASH value in Non-volatile memory.
Table: FPGA_VERIFY_MAC Server BMC Request
| Server BMC Request | |
|---|---|
| Command code | 0x4E |
| Byte0 | B0 – Target flash device: 0x01 - FPGA1 Primary flash device 0x02 - FPGA1 Recovery flash device 0x03 - FPGA2 Primary flash device 0x04 - FPGA2 Recovery flash device |
Table: FPGA_VERIFY_MAC Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | B0 | See table ‘Flash Operation Return Codes’ for SC’s response |
0x4F - FPGA_GET_MAC_STATUS¶
This command is only supported in Alveo U30 card. The BMC sends the FPGA_GET_MAC_STATUS command to get the status of FPGA_CALC_MAC or FPGA_VERIFY_MAC command. SC responds with the status of MAC/HASH calculation or verification (Byte 0) and 16-byte MAC/HASH value (Bytes 1-17) as response.
Note: Server BMC must use the same key and nonce that the satellite controller used to compute the MAC/HASH value to obtain same results. Refer 0x4C and 0x4D command description for details.
Table: FPGA_GET_MAC_STATUS Server BMC Request
| Server BMC Request | |
|---|---|
| Command code | 0x4F |
| Byte0 | 0x01 - FPGA1 Primary flash device 0x02 - FPGA1 Recovery flash device 0x03 - FPGA2 Primary flash device 0x04 - FPGA2 Recovery flash device |
| Byte1 | 0x01 – Get FPGA_CALC_MAC status 0x02 – Get FPGA_VERIFY_MAC status |
Table: FPGA_GET_MAC_STATUS Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | B0 | See table ‘Flash Operation Return Codes’ for SC’s response |
| Bytes 1 - 16 | 16 Byte MAC value (LSB first) | |
0x50 - FPGA_SET_IMAGE_SIZE¶
Optionally, BMC can send the command 0x50 to notify SC about the size of the FPGA image that it indents to update. The byte-0 selects the target FPGA flash device and the byte1 – byte 4 (4 bytes, unsigned, LSB first) represents the size of flash image in bytes.
NOTE: On boot-up, SC restores the image size to default 128 MBytes to address entire flash memory.
Table: FPGA_SET_IMAGE_SIZE server BMC request
| Server BMC request | |
|---|---|
| Command code | 0x50 |
| Byte0 | 0x01 - FPGA1 Primary flash device 0x02 - FPGA1 Recovery flash device 0x03 - FPGA2 Primary flash device 0x04 - FPGA2 Recovery flash device |
| Byte 1-4 | Size of QSPI image (in bytes) |
Table: FPGA_SET_IMAGE_SIZE Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | B0 | See table ‘Flash Operation Return Codes’ for SC’s response |
0x51 - NOTIFY_WP_TO_FPGA¶
BMC sends this command to request SC to notify the FPGA device about WP status of the flash device. In turn, SC communicates the flash device WP status to FPGA via UART messages. BMC shall send this command prior to initiating the in-band QSPI FW update so that host OS can read the WP status from PCIe BAR config space.
Table: NOTIFY_WP_TO_FPGA server BMC request
| Server BMC request | |
|---|---|
| Command code | 0x51 |
| Byte0 | 0x01 - FPGA1 Primary flash device 0x02 - FPGA1 Recovery flash device 0x03 - FPGA2 Primary flash device 0x04 - FPGA2 Recovery flash device |
Table: NOTIFY_WP_TO_FPGA Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | B0 | 0x01: Operation Success 0x02: Operation failure |
0x52 - FPGA_UART_DEBUG_CONTROL¶
BMC sends this command to FPGA to enable/disable the debug UART. SC communicates this information to the respective ZYNQ/FPGA device. By default, the debug UART is disabled during production settings and it can optionally be enabled for debug purposes.
Table: FPGA_UART_DEBUG_CONTROL server BMC request
| Server BMC request | |
|---|---|
| Command code | 0x52 |
| Byte0 | 0x01 - FPGA1 0x02 - FPGA2 |
Table: FPGA_UART_DEBUG_CONTROL Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | B0 | 0x01: Operation Success 0x02: Operation failure 0x03: Operation unsupported |
0x53 - SET_FPGA_FLASH_READBACK_SECTOR_RANGE¶
BMC sends this command to set the start and end sectors for the FPGA flash content read-back. The range is between sectors 0 and 2047. SC fetches 1 sector at a time from FPGA flash device and transfers it to BMC via command 0x54.
Table: SET_FPGA_FLASH_READBACK_SECTOR_RANGE server BMC request
| Server BMC request | |
|---|---|
| Command code | 0x53 |
| Byte0 | Start sector number (low byte) |
| Byte1 | Start sector number (high byte) |
| Byte2 | End sector number (low byte) |
| Byte3 | End sector number (high byte) |
Table: SET_FPGA_FLASH_READBACK_SECTOR_RANGE Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | B0 | See table ‘Flash Operation Return Codes’ for SC’s response |
0x54 - FPGA_FLASH_READBACK_RX_DATA_BLOCK¶
BMC sends this command in repeated-start mode to read the data from FPGA flash. Before sending the command 0x54, BMC needs to send the following commands in sequence (i.e.) command 0x42 (set target device), command 0x53 (set sectors) and command 0x4B (poll the status).
- Upon requested, SC will read 1 sector (64 KB) from FPGA flash device, compares the CRC before signaling BMC about read-back readiness (via 0x4B command)
- After SC is ready with the payload, BMC can send the read-back command in repeated-start mode to fetch 1 sector of data. It takes 256 transactions to transfer 64 KB payload as SC sends 256 bytes per transaction. NOTE that SC can’t send partial transactions (i.e.) data less than 256 bytes.
- After successfully receiving 64 KB payload, BMC needs to read the sector CRC(via 0x55) before proceeding to read the next sector.
- BMC must poll the status command 0x4B (SC’s readiness for next sector) before proceeding to issue read-back command 0x54 for the next sector.
- No retry is supported in case of any failure/interruption in the middle of sector read-back. But BMC can send the command 0x49 to force set the start sector from which it wants to resume/retry the read-back operation
- Upon receiving the updated start sector number (via 0x49 command), SC starts the read-back process from the beginning of the sector. Once again, BMC needs to follow the same sequence of commands (i.e.) poll the status command 0x4B before issuing 0x54.
Table: FPGA_FLASH_READBACK_RX_DATA_BLOCK server BMC request
| Server BMC request | |
|---|---|
| Command code | 0x54 |
| Byte0 | BMC sends repeated-start I2C command |
Table: FPGA_FLASH_READBACK_RX_DATA_BLOCK Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | D0, D1 … D255 | SC sends 256 data bytes |
0x55 - GET_FPGA_FLASH_READBACK_SECTOR_CRC¶
BMC sends this command to SC to get the CRC for the previously read-back sector (via command 0x54). SC returns the Micron’s CRC-64 back and BMC can use the 8-byte CRC data to compare with the CRC-64 calculated within.
Table: SET_FPGA_FLASH_READBACK_SECTOR_RANGE server BMC request
| Server BMC Request | |
|---|---|
| Command code | 0x55 |
| Byte0 | N/A |
Table: SET_FPGA_FLASH_READBACK_SECTOR_RANGE Xilinx Alveo Card Response
| Xilinx Alveo Card Response | ||
|---|---|---|
| Data bytes | D0, D1, .. D7 | SC sends 8-bytes of Micron CRC-64 |
AMD Support
For support resources such as answers, documentation, downloads, and forums, see the Alveo Accelerator Cards AMD/Xilinx Community Forum.
License
Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License.
You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
All images and documentation, including all debug and support documentation, are licensed under the Creative Commons (CC) Attribution 4.0 International License (the “CC-BY-4.0 License”); you may not use this file except in compliance with the CC-BY-4.0 License.
You may obtain a copy of the CC-BY-4.0 License at https://creativecommons.org/licenses/by/4.0/
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
XD038 | © Copyright 2023, Advanced Micro Devices Inc.