FPGA Flash Update Commands

In the Alveo™ U30 Hyperscaler only SKU, the satellite controller (SC) firmware supports an out-of-band method of FPGA flash image update, read-back and authentication. Server BMC shall initiate and perform all these operations by sending the I2C commands to the SC. FPGA flash operation commands are supported at I2C address 0x65 (0xCA in 8-bit). Currently, Satellite Controller supports only 100 KHz I2C speed for all the FPGA flash update commands mentioned in this section.

Note: The satellite controller will perform I2C clock-stretching, wherever applicable, to perform the requested actions.

Note: It is recommended for the BMC to add 2 seconds as inter-command interval.

In addition to the FPGA flash update via OoB, SC also supports other FPGA flash related operations like:

  • Enable or Disable Write Protect (WP#) settings
  • FPGA flash image read-back
  • Flash image authentication via MAC/HASH calculation
  • Copy of flash image from one flash to another (F2F copy)

The table below lists all the commands supported/needed for FPGA flash operations. Currently, the FPGA flash operation commands are supported only for Alveo™ U30.

Note: MAC in this chapter refers to Message Authentication Code and it can also be referred as HASH. MAC/HASH calculation of the entire or sometimes few select FPGA flash sectors is performed at the request of BMC, to validate the flash contents haven’t been tampered with.

Table: FPGA Flash Upgrade Commands

Command Command Description Server BMC Action SC Action
0x40 FPGA_RESET_DEVICE

B0 from BMC:

0x01: FPGA devices

0x02: SC FW

See table ‘Flash Operation

Return Codes’ for SC’s

response

0x41 FPGA_GET_FW_VER

B0 from BMC:

0x01: FPGA1 Primary

0x02: FPGA1 Recovery

0x03: FPGA2 Primary (if present)

0x04: FPGA2 Recovery(if present)

B0 – Valid Byte

B1 - Minor revision

B2 - Major revision

0x42 FPGA_SET_TARGET_DEVICE

B0 from BMC

0x01: FPGA1 Primary

0x02: FPGA1 Recovery

0x03: FPGA2 Primary (if present)

0x04: FPGA2 Recovery(if present)

See table ‘Flash Operation

Return Codes’ for SC’s

response

0x43 FPGA_SET_BOOT_DEVICE

B0 from BMC:

0x01: FPGA1 Primary

0x02: FPGA1 Recovery

0x03: FPGA2 Primary (if present)

0x04: FPGA2 Recovery(if present)

See table ‘Flash Operation

Return Codes’ for SC’s

response

0x44

FPGA_SC_SET_WRITE

_ENABLE

Provides control to SC

over WP# pins and enables

SC to access QSPI flash

B0 from BMC:

0x01: FPGA1 Primary

0x02: FPGA1 Recovery

0x03: FPGA2 Primary (if present)

0x04: FPGA2 Recovery(if present)

B1 from BMC:

0x01: WP enable

0x02: WP disable

See table ‘Flash Operation

Return Codes’ for SC’s

response

0x45

FLASH_SET_WRITE_ENABLE

Provides control to SC

over WP# pins and enables

FPGA to access QSPI flash

B0 from BMC:

0x01: FPGA1 Primary

0x02: FPGA1 Recovery

0x03: FPGA2 Primary (if present)

0x04: FPGA2 Recovery(if present)

B1 from BMC:

0x01: WP enable

0x02: WP disable

See table ‘Flash Operation

Return Codes’ for SC’s

response

0x46

FLASH_GET_WRITE

_PROTECT_STATES

B0 from BMC:

0x01: FPGA1 Primary

0x02: FPGA1 Recovery

0x03: FPGA2 Primary (if present)

0x04: FPGA2 Recovery(if present)

Byte 0: SC WP Status

0x01: WP enabled

0x02: WP disabled

Byte 1: FPGA WP Status

0x01: WP enabled

0x02: WP disabled

0x47

FLASH_TX_DATA_BLOCK

SC accumulates 252 bytes

from each transaction to

form 64KB sector & writes

it into QSPI flash

BMC sends data bytes

D0, D1 … D251

N/A
0x48

FLASH_BLOCK_CRC_CHECK

SC compares CRC, writes 1

sector to flash, rechecks

CRC with FPGA device

BMC sends 8 byte CRC

B0, B1 … B7

See table ‘Flash Operation

Return Codes’ for SC’s

response

0x49

FLASH_SECTOR_SET_SEQ

_NUM

B0: Sector number (low byte)

B1: Sector number (high byte)

See table ‘Flash Operation

Return Codes’ for SC’s

response

0x4A FLASH_COPY_FIRMWARE

B0: Source flash device

0x01: FPGA1 Primary

0x02: FPGA1 Recovery

0x03: FPGA2 Primary (if present)

0x04: FPGA2 Recovery(if present)

B1: Destination flash device

0x01: FPGA1 Primary

0x02: FPGA1 Recovery

0x03: FPGA2 Primary (if present)

0x04: FPGA2 Recovery(if present)

See table ‘Flash Operation

Return Codes’ for SC’s

response

0x4B

FPGA_GET_FIRMWARE

_STATUS

N/A

See table ‘Flash Operation

Return Codes’ for SC’s

response

0x4C

FPGA_SET_KEY_NONCE

BMC sends key and nonce

that’s randomly generated.

SC stores both in internal

flash (Non-volatile)

B0 from BMC:

0x01: FPGA1 Primary

0x02: FPGA1 Recovery

0x03: FPGA2 Primary (if present)

0x04: FPGA2 Recovery(if present)

B1 - B16: Key

B17 – B28: Nonce

See table ‘Flash Operation

Return Codes’ for SC’s

response

0x4D

FPGA_CALC_MAC

SC increments nonce by 1,

calculates hash using the

existing key & new nonce,

and stores MAC/hash value

in SC’s internal flash

B0 from BMC:

0x01: FPGA1 Primary

0x02: FPGA1 Recovery

0x03: FPGA2 Primary (if present)

0x04: FPGA2 Recovery(if present)

See table ‘Flash Operation

Return Codes’ for SC’s

response

0x4E

FPGA_VERIFY_MAC

SC calculates hash using

existing key & nonce, and

stores MAC/hash value in

SC’s internal flash

B0 from BMC:

0x01: FPGA1 Primary

0x02: FPGA1 Recovery

0x03: FPGA2 Primary (if present)

0x04: FPGA2 Recovery(if present)

See table ‘Flash Operation

Return Codes’ for SC’s

response

0x4F

FPGA_GET_MAC_STATUS

SC responds with status of

MAC/has calculation or

verification along with

the 16-byte MAC value

B0 from BMC:

0x01: FPGA1 Primary

0x02: FPGA1 Recovery

0x03: FPGA2 Primary (if present)

0x04: FPGA2 Recovery(if present)

B1 from BMC:

0x01: Get CALC_MAC status

0x02: Get VERIFY_MAC status

Byte B0 from SC:

See table ‘Flash Operation

Return Codes’ for

response

Bytes B1 - B16 from SC:

16 byte MAC/hash value

0x50 FPGA_SET_IMAGE_SIZE

B0 from BMC:

0x01: FPGA1 Primary

0x02: FPGA1 Recovery

0x03: FPGA2 Primary (if present)

0x04: FPGA2 Recovery(if present)

B1 - B4 from BMC (LSB first):

Size of QSPI image in bytes

N/A
0x51 NOTIFY_WP_TO_FPGA

B0 from BMC:

0x01: FPGA1 Primary

0x02: FPGA1 Recovery

0x03: FPGA2 Primary (if present)

0x04: FPGA2 Recovery(if present)

Byte B0 from SC:

0x01: Operation Success

0x02: Operation failure

0x52 FPGA_UART_DEBUG_CONTROL

B0 from BMC:

0x01: FPGA1

0x02: FPGA2

Byte B0 from SC:

0x01: Operation Success

0x02: Operation failure

0x03: Operation unsupported

0x53

SET_FPGA_FLASH_READBACK

_SECTOR_RANGE

B0: Start sector num (low byte)

B1: Start sector num (high byte)

B2: End sector num (low byte)

B3: End sector num (high byte)

See table ‘Flash Operation

Return Codes’ for SC’s

response

0x54

FPGA_FLASH_READBACK_RX

_DATA_BLOCK

BMC sends repeated-start

I2C command

SC sends 256 data bytes:

D0, D1 … D255

0x55

GET_FPGA_FLASH_READBACK

_SECTOR_CRC

N/A

SC sends 8-bytes of CRC :

D0, D1 … D7

Table: Flash Operation Return Codes

Response Code Description
0x00 Reserved
0x01 Operation success
0x02 Operation failed
0x03 Operation Not Supported
0x04 Flash erase failed. Abort operation, rectify error and re-initiate from start
0x05 Flash write failed. Abort operation, rectify error and re-initiate from start
0x06 Flash read failed. Abort operation, rectify error and re-initiate from start
0x07 Flash CRC failed. Abort operation, rectify error and re-initiate from start
0x08 Invalid Flash Selection
0x09 FPGA_GENERAL_ERROR
0x0A FPGA_MAC_CALCULATION_INVALID
0x0B FPGA_INVALID_IMAGE_LENGTH
0x0C QSPI SC disable WP failed. Abort operation, rectify error and re-initiate from start
0x0D QSPI wrong MCS file format. Abort operation, rectify error and re-initiate from start
0x0E Set the KEY and/or NONCE before proceeding for MAC Calculation
0x0F MAC calculation not performed. Please send command 0x4D before MAC verify
0x10 FLASH_TX_DATA_BLOCK command in-progress
0x11 FPGA1 primary flash update in-progress
0x12 FPGA1 recovery flash update in-progress
0x13 FPGA2 primary flash update in-progress
0x14 FPGA2 recovery flash update in-progress
0x15–0x1F Reserved
0x20 FLASH_BLOCK_CRC_CHECK command in-progress
0x21 FPGA_CRC_CHECK_STATUS_SECTOR_RESEND (Resend last sector (0x47))
0x22 Reserved
0x23 QSPI_SET_UPDATE_DEVICE_NOT_SENT (send command 0x42)
0x24 QSPI_SC_SET_WRITE_NOT_ENABLED (send command 0x44)
0x25–0x2F Reserved
0x30 FLASH_COPY_FIRMWARE command in-progress
0x31 FPGA1 primary to FPGA1 recovery flash copy in-progress
0x32 FPGA1 primary to FPGA2 primary flash copy in-progress
0x33 FPGA1 primary to FPGA2 recovery flash copy in-progress
0x34 FPGA1 recovery to FPGA1 primary flash copy in-progress
0x35 FPGA1 recovery to FPGA2 primary flash copy in-progress
0x36 FPGA1 recovery to FPGA2 recovery flash copy in-progress
0x37 FPGA2 primary to FPGA1 primary flash copy in-progress
0x38 FPGA2 primary to FPGA1 recovery flash copy in-progress
0x39 FPGA2 primary to FPGA2 recovery flash copy in-progress
0x3A FPGA2 recovery to FPGA1 primary flash copy in-progress
0x3B FPGA2 recovery to FPGA1 recovery flash copy in-progress
0x3C FPGA2 recovery to FPGA2 primary flash copy in-progress
0x3D–0x3F Reserved
0x40 FPGA_CALC_MAC command in-progress
0x41 FPGA1 primary MAC calculation in-progress
0x42 FPGA1 recovery MAC calculation in-progress
0x43 FPGA2 primary MAC calculation in-progress
0x44 FPGA2 recovery MAC calculation in-progress
0x45 FPGA_KEY_NONCE update in-progress
0x46–0x4F Reserved
0x50 FPGA_VERIFY_MAC command in-progress
0x51 FPGA1 primary MAC verification in-progress
0x52 FPGA1 recovery MAC verification in-progress
0x53 FPGA2 primary MAC verification in-progress
0x54 FPGA2 recovery MAC verification in-progress
0x55–0x6F Reserved
0x70 MAC verification not performed. Please send command (0x4E) before fetching MAC verification status
0x71 FPGA MAC Calculation/Verification failed. (Abort operation, re-initiate MAC Calculation/Verification)
0x72 EEPROM read failed
0x73 EEPROM write failed
0x74-0x7F Reserved
0x80 FPGA flash data read-back in progress
0x81 FPGA flash data ready for read-back
0x82 Invalid FPGA sector range. Check and resend within the valid range (0 - 2047)
0x83–0xEF Reserved
0xFF FPGA_NO_OPERATION

0x40 - FPGA_RESET_DEVICE

The BMC can send the FPGA_RESET_DEVICE command to reset FPGA device(s) or SC FW. For SC reset (warm reset), the satellite controller firmware responds with the status and reboots itself. In Alveo U30, the reset command will reset both the FPGA devices (ZYNQ1 and ZYNQ2) and internally, both PS (Processing Subsystem) and PL (Programmable Logic) will reload from flash device.

Table: FPGA_RESET_DEVICE Server BMC Request

Server BMC Request
Command code 0x40
Byte0

0x01: FPGA devices

0x02: SC FW

Table: FPGA_RESET_DEVICE Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes B0 See table ‘Flash Operation Return Codes’ for SC’s response

0x41 - FPGA_GET_FW_VER

The BMC sends this command to fetch the FW version running in either FPGA1 or FPGA2 device. This command is currently supported only in the Alveo U30 data accelerator card. The byte 0 (validity byte) in response from SC must be read first.

Table: FPGA_GET_FW_VER Server BMC Request

Server BMC Request
Command code 0x41
Byte0

0x01: FPGA1 primary flash device

0x02: FPGA1 recovery flash device

0x03: FPGA2 primary flash device (if present)

0x04: FPGA2 recovery flash device (if present)

Table: FPGA_GET_FW_VER Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes

B0

B1, B2

B0 - Valid Byte

0x00 - Not supported (B1, B2 - Invalid)

0x01 - Unknown or Reduced Service

0x02 - reserved

0x03 - No error; Valid B1 & B2 bytes

B1 – Minor version; B2 – Major version

0x42 - FPGA_SET_TARGET_DEVICE

BMC sends the FPGA_SET_TARGET_DEVICE command to select the flash device to initiate the FW upgrade.

NOTE: This command is not persistence across SC reboots. On boot-up, SC restores the default configuration (i.e.) Primary flash as teh target device.

Table: FPGA_SET_TARGET_DEVICE Server BMC Request

Server BMC Request
Command code 0x42
Byte0

0x01: FPGA1 primary flash device

0x02: FPGA1 recovery flash device

0x03: FPGA2 primary flash device (if present)

0x04: FPGA2 recovery flash device (if present)

Table: FPGA_SET_TARGET_DEVICE Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes B0 See table ‘Flash Operation Return Codes’ for SC’s response

0x43 - FPGA_SET_BOOT_DEVICE

The BMC sends the FPGA_SET_BOOT_DEVICE command to set the boot device. The SC FW stores this information in the internal flash memory and restores the configuration in case of server cool boot or power cycle.

Note: Primary flash device is selected as the boot device/default configuration in FPGA1 (and FPGA2 if present).

Table: FPGA_SET_BOOT_DEVICE Server BMC Request

Server BMC Request
Command code 0x43
Byte0

0x01– FPGA1 primary flash device

0x02– FPGA1 recovery flash device

0x03– FPGA2 primary flash device (if present)

0x04– FPGA2 recovery flash device (if present)

Table: FPGA_SET_BOOT_DEVICE Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes Byte0

0x01– Request success

0x02– Request failed

0x44 - FPGA_SC_SET_WRITE_ENABLE

The BMC sends the FPGA_SC_SET_WRITE_ENABLE command to enable/disable write protect for SPI mode from the SC point of view. This command must follow FPGA_SET_TARGET_DEVICE. When the SC has write protect mode disabled, only the SC has the access to QSPI flash and only SC can write into the flash via SPI (x1 mode). In this case, FPGA can not read from FPGA flash.

For both hyperscaler and OEM customers, by default, SC WP# is enabled (i.e.) FPGA has read-only access. The actual FPGA WP# state (write access) is controlled by command 0x45.

Note: The SC will not store this configuration (from command 0x44) in any persistence memory. A SC reboot or device power cycle results in loss of configuration. During the subsequent boot-up, the SC will restore the default configuration (i.e.) WP enabled.

Table: FPGA_SC_SET_WRITE_ENABLE Server BMC Request

Server BMC Request
Command code 0x44
Byte0

0x01: FPGA1 primary flash device

0x02: FPGA1 recovery flash device

0x03: FPGA2 primary flash device (if present)

0x04: FPGA2 recovery flash device (if present)

Byte1

0x01: WP enable

0x02: WP disable

Table: FPGA_SC_SET_WRITE_ENABLE Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes B0 See table ‘Flash Operation Return Codes’ for SC’s response

0x45 - FLASH_SET_WRITE_ENABLE

The BMC can send the FLASH_SET_WRITE_ENABLE command to enable/disable write protect the QSPI flash device in either ZYNQ device. This command must follow FPGA_SC_SET_WRITE_ENABLE for any FPGA flash FW upgrade.

For Hyperscaler customers, by default, the FPGA flash devices are write protected and can only run in x2 SPI mode as a preferred/secured mode.

When write protect is disabled, the QSPI flash device can be accessed in x1, x2, or x4 SPI Mode and FPGA has the write access to the QSPI devices. This is the default mode configured for OEM customers. SC FW configures this mode based on FRU parameters written into its EEPROM.

Note: The SC will not store this configuration (from command 0x45) in any persistence memory. A SC reboot or device power cycle results in the loss of configuration. During the subsequent boot-up, the SC will restore the default configuration (i.e.) WP enabled.

Table: FLASH_SET_WRITE_ENABLE Server BMC Request

Server BMC Request
Command code 0x45
Byte0

0x01: FPGA1 primary flash device

0x02: FPGA1 recovery flash device

0x03: FPGA2 primary flash device (if present)

0x04: FPGA2 recovery flash device (if present)

Byte1

0x01: WP enable

0x02: WP disable

Table: FLASH_SET_WRITE_ENABLE Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes B0 See table ‘Flash Operation Return Codes’ for SC’s response

0x46 - FLASH_GET_WRITE_PROTECT_STATES

The BMC can send the FLASH_GET_WRITE_PROTECT_STATES command to get the write protect state for all the flash devices.

Table: FLASH_GET_WRITE_PROTECT_STATES Server BMC Request

Server BMC Request
Command code 0x46
Byte0

0x01: FPGA1 primary flash device

0x02: FPGA1 recovery flash device

0x03: FPGA2 primary flash device (if present)

0x04: FPGA2 recovery flash device (if present)

Table: FLASH_GET_WRITE_PROTECT_STATES Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes Byte0

Byte 0: SC WP Status

0x01: WP enabled ; 0x02: WP disabled

  Byte 1

Byte 1: FPGA WP Status

0x01: WP enabled ; 0x02: WP disabled

0x47 - FLASH_TX_DATA_BLOCK

The BMC sends data blocks using the FLASH_TX_DATA_BLOCK command. Data (or payload) is the FW for QSPI flash devices. The SC accumulates 252 byte data from each transaction to build up 64 Kbyte blocks and write into QSPI flash. The sector size for QSPI flash is 64 Kbyte and the SC will cache 1 block of data in the internal SRAM before writing it to QSPI. This command must be preceded by the FPGA_SET_TARGET_DEVICE, FPGA_SC_SET_WRITE_ENABLE, and FLASH_SET_WRITE_ENABLE commands.

Note: The maximum supported size of each I2C transaction is 252 bytes.

Table: FLASH_TX_DATA_BLOCK Server BMC Request

Server BMC Request
Command code 0x47
Length byte 0 Total number of bytes in the current I2C transaction (command)
Data bytes D1, D2, … D252

Table: FLASH_TX_DATA_BLOCK Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes B0 See table ‘Flash Operation Return Codes’ for SC’s response

0x48 - FLASH_BLOCK_CRC_CHECK

The BMC sends the FLASH_BLOCK_CRC_CHECK command to check the CRC from the previous block and tracks the number of bytes sent to the SC. After sending 64K bytes, the BMC sends this command to initiate CRC check. The BMC sends 64-bit CRC (data payload and start address of the block) for the previous 64 Kbyte block along with this command request. This command also indicates the completion of the 64 Kbyte block transfer from the BMC to the SC, which immediately returns the status CRC_CHECK_IN_PROGRESS. This command is a trigger for the SC to start all the flash write and CRC check operations in the background. The BMC can poll the FPGA_GET_FIRMWARE_STATUS command periodically for completion status before moving to next sector. See table ‘Flash Operation Return Codes’ for SC’s response

Table 97: FLASH_BLOCK_CRC_CHECK Server BMC Server

Server BMC Request
Command code 0x48
Data bytes

D1 … D8

64-bit CRC data

Table: FLASH_BLOCK_CRC_CHECK Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes B0 See table ‘Flash Operation Return Codes’ for SC’s response

0x49 - FLASH_SECTOR_SET_SEQ_NUM

The FPGA flash FW image update needs to be implemented with the retry mechanism in both the server BMC and the Alveo SC.

The BMC sends the FLASH_SECTOR_SET_SEQ_NUM command to set the sequence number (or the sector number) of the sector at which the SC needs to write into the FPGA Flash. This retry mechanism is needed to restart the FPGA FW upgrade process from the point where it was terminated previously. The termination reason could be a reboot of the SC FW, the BMC FW, power loss, or any user triggered event.

When server BMC wants to restart the FPGA FW upgrade process from the middle, the BMC sends a valid 2-byte sequence number. The BMC may keep track of the sequence number and keep updating by 1 for every successful response from the SC for the command FLASH_BLOCK_CRC_CHECK.

The SC will calculate the sector start address based on the sequence number and the fixed start offset (expected to be 0x0) of the FW inside flash. Responsibility is on the BMC to send the correct sequence number and its corresponding payload. Otherwise the SC may write into a non- contiguous flash sector and may end up corrupting the FW.

Note: The SC will not store the sector information in persistence memory. On boot-up, default value 0x00 will be assigned.

Table: FLASH_SECTOR_SET_SEQ_NUM Server BMC Request

Server BMC Request
Command code 0x49
Data bytes

B0– Sector number (low byte)

B1– Sector number (high byte)

Table: FLASH_SECTOR_SET_SEQ_NUM Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes

Byte0

0x01: Request success

0x02: Request failed

0x4A - FLASH_COPY_FIRMWARE

The BMC sends the FLASH_COPY_FIRMWARE command to initiate the copy of FW from one QSPI flash device to another. The BMC sends the source and destination flash devices via this command. Upon receiving this command the SC:

  1. Copies the sector contents from the source device into 64 Kbyte SRAM within SC FW.
  2. Checks the CRC.
  3. Copies the contents into the destination device, checks the CRC, and updates the sector sequence number.
  4. If failure occurs in the middle of copy, erase all sectors in destination device (no retry support).
  5. If the CRC is copied successfully, proceed with next sector.
  6. The BMC can obtain the status of the copy from the FLASH_COPY_FIRMWARE_STATUS command.

Note: If the BMC sends another copy command while the previous copy is in-progress, the SC will ignore the request and respond appropriate error code. The BMC must check the status via the COPY_FIRMWARE_STATUS command and re-trigger. This command is currently supported only for Alveo U30.

Table: FPGA_COPY_FIRMWARE Server BMC Request

Server BMC Request
Command code 0x4A
Byte0

B0 – Source flash device:

0x01 - FPGA1 Primary flash device

0x02 - FPGA1 Recovery flash device

0x03 - FPGA2 Primary flash device

0x04 - FPGA2 Recovery flash device

Byte1

B1 – Destination flash device:

0x01 - FPGA1 Primary flash device

0x02 - FPGA1 Recovery flash device

0x03 - FPGA2 Primary flash device

0x04 - FPGA2 Recovery flash device

Table: FPGA_COPY_FIRMWARE Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes B0 See table ‘Flash Operation Return Codes’ for SC’s response

0x4B - FPGA_GET_FIRMWARE_STATUS

The BMC sends the FPGA_GET_FIRMWARE_STATUS command to obtain the status of the previously triggered commands like FLASH_BLOCK_CRC_CHECK and/or FLASH_COPY_FIRMWARE commands.

Table: FPGA_GET_FIRMWARE_STATUS Server BMC request

Server BMC Request
Command code 0x4B
Byte0 N/A

Table: FPGA_GET_FIRMWARE_STATUS Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes B0 See table ‘Flash Operation Return Codes’ for SC’s response

0x4C - FPGA_SET_KEY_NONCE

BMC provides the SC with a randomly generated key and nonce. The SC writes the 16-byte AES128 key and 96-bit (12-byte) nonce value to the Non-volatile memory. The SC provisions storage for one key and one nonce per flash device (non-volatile memory).

SC uses 15-byte nonce, by padding 3 bytes (with 0x00) at the start, to the 12-byte nonce sent by BMC (i.e.) LSB 3 bytes are 0x00. In other words, Bytes[14-3] are the 12-bytes of nonce sent by BMC and Bytes[2-0] are 0x00 0x00 0x00.

BMC must use the exact nonce scheme to calculate the MAC value for comparison. The BMC is expected to select the target flash device.

Table: FPGA_SET_KEY_NONCE Server BMC Request

Server BMC Request
Command code 0x4C
Byte0

B0 – Target flash device:

0x01 - FPGA1 Primary flash device

0x02 - FPGA1 Recovery flash device

0x03 - FPGA2 Primary flash device

0x04 - FPGA2 Recovery flash device

Byte1 -16 16-byte key
Byte17 - 28 12-byte nonce

Table: FPGA_SET_KEY_NONCE Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes B0 See table ‘Flash Operation Return Codes’ for SC’s response

0x4D - FPGA_CALC_MAC

Note: This command is optional. If sent, this command must be preceded by FPGA_SET_KEY_NONCE. This command is only supported in Alveo U30

The BMC is expected to call the FPGA_CALC_MAC command after the entire flash image is written. The BMC is expected to select the target flash device. Upon receiving the command 0x4D, SC increments the stored nonce by 1, calculates the MAC of the entire 128 MByte region of the FPGA flash device, using the existing key and the new nonce. The calculated MAC/HASH value is returned to BMC via the status command 0x4F. SC does not store the MAC/HASH value in Non-volatile memory.

Table: FPGA_CALC_MAC Server BMC Request

Server BMC Request
Command code 0x4D
Byte0

B0 – Target flash device:

0x01 - FPGA1 Primary flash device

0x02 - FPGA1 Recovery flash device

0x03 - FPGA2 Primary flash device

0x04 - FPGA2 Recovery flash device

Table: FPGA_CALC_MAC Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes B0 See table ‘Flash Operation Return Codes’ for SC’s response

0x4E - FPGA_VERIFY_MAC

NOTE: This command must be preceded by FPGA_SET_KEY_NONCE. This command is supported only in Alveo U30.

The BMC sends the FPGA_VERIFY_MAC command to validate the FPGA flash image. The SC calculates the MAC/HASH of the entire 128 MByte region using the existing key and existing nonce value. The calculated MAC/HASH value is returned to BMC via the status command 0x4F. SC does not store the MAC/HASH value in Non-volatile memory.

Table: FPGA_VERIFY_MAC Server BMC Request

Server BMC Request
Command code 0x4E
Byte0

B0 – Target flash device:

0x01 - FPGA1 Primary flash device

0x02 - FPGA1 Recovery flash device

0x03 - FPGA2 Primary flash device

0x04 - FPGA2 Recovery flash device

Table: FPGA_VERIFY_MAC Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes B0 See table ‘Flash Operation Return Codes’ for SC’s response

0x4F - FPGA_GET_MAC_STATUS

This command is only supported in Alveo U30 card. The BMC sends the FPGA_GET_MAC_STATUS command to get the status of FPGA_CALC_MAC or FPGA_VERIFY_MAC command. SC responds with the status of MAC/HASH calculation or verification (Byte 0) and 16-byte MAC/HASH value (Bytes 1-17) as response.

Note: Server BMC must use the same key and nonce that the satellite controller used to compute the MAC/HASH value to obtain same results. Refer 0x4C and 0x4D command description for details.

Table: FPGA_GET_MAC_STATUS Server BMC Request

Server BMC Request
Command code 0x4F
Byte0

0x01 - FPGA1 Primary flash device

0x02 - FPGA1 Recovery flash device

0x03 - FPGA2 Primary flash device

0x04 - FPGA2 Recovery flash device

Byte1

0x01 – Get FPGA_CALC_MAC status

0x02 – Get FPGA_VERIFY_MAC status

Table: FPGA_GET_MAC_STATUS Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes B0 See table ‘Flash Operation Return Codes’ for SC’s response
  Bytes 1 - 16 16 Byte MAC value (LSB first)

0x50 - FPGA_SET_IMAGE_SIZE

Optionally, BMC can send the command 0x50 to notify SC about the size of the FPGA image that it indents to update. The byte-0 selects the target FPGA flash device and the byte1 – byte 4 (4 bytes, unsigned, LSB first) represents the size of flash image in bytes.

NOTE: On boot-up, SC restores the image size to default 128 MBytes to address entire flash memory.

Table: FPGA_SET_IMAGE_SIZE server BMC request

Server BMC request
Command code 0x50
Byte0

0x01 - FPGA1 Primary flash device

0x02 - FPGA1 Recovery flash device

0x03 - FPGA2 Primary flash device

0x04 - FPGA2 Recovery flash device

Byte 1-4 Size of QSPI image (in bytes)

Table: FPGA_SET_IMAGE_SIZE Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes B0 See table ‘Flash Operation Return Codes’ for SC’s response

0x51 - NOTIFY_WP_TO_FPGA

BMC sends this command to request SC to notify the FPGA device about WP status of the flash device. In turn, SC communicates the flash device WP status to FPGA via UART messages. BMC shall send this command prior to initiating the in-band QSPI FW update so that host OS can read the WP status from PCIe BAR config space.

Table: NOTIFY_WP_TO_FPGA server BMC request

Server BMC request
Command code 0x51
Byte0

0x01 - FPGA1 Primary flash device

0x02 - FPGA1 Recovery flash device

0x03 - FPGA2 Primary flash device

0x04 - FPGA2 Recovery flash device

Table: NOTIFY_WP_TO_FPGA Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes B0

0x01: Operation Success

0x02: Operation failure

0x52 - FPGA_UART_DEBUG_CONTROL

BMC sends this command to FPGA to enable/disable the debug UART. SC communicates this information to the respective ZYNQ/FPGA device. By default, the debug UART is disabled during production settings and it can optionally be enabled for debug purposes.

Table: FPGA_UART_DEBUG_CONTROL server BMC request

Server BMC request
Command code 0x52
Byte0

0x01 - FPGA1

0x02 - FPGA2

Table: FPGA_UART_DEBUG_CONTROL Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes B0

0x01: Operation Success

0x02: Operation failure

0x03: Operation unsupported

0x53 - SET_FPGA_FLASH_READBACK_SECTOR_RANGE

BMC sends this command to set the start and end sectors for the FPGA flash content read-back. The range is between sectors 0 and 2047. SC fetches 1 sector at a time from FPGA flash device and transfers it to BMC via command 0x54.

Table: SET_FPGA_FLASH_READBACK_SECTOR_RANGE server BMC request

Server BMC request
Command code 0x53
Byte0 Start sector number (low byte)
Byte1 Start sector number (high byte)
Byte2 End sector number (low byte)
Byte3 End sector number (high byte)

Table: SET_FPGA_FLASH_READBACK_SECTOR_RANGE Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes B0 See table ‘Flash Operation Return Codes’ for SC’s response

0x54 - FPGA_FLASH_READBACK_RX_DATA_BLOCK

BMC sends this command in repeated-start mode to read the data from FPGA flash. Before sending the command 0x54, BMC needs to send the following commands in sequence (i.e.) command 0x42 (set target device), command 0x53 (set sectors) and command 0x4B (poll the status).

  • Upon requested, SC will read 1 sector (64 KB) from FPGA flash device, compares the CRC before signaling BMC about read-back readiness (via 0x4B command)
  • After SC is ready with the payload, BMC can send the read-back command in repeated-start mode to fetch 1 sector of data. It takes 256 transactions to transfer 64 KB payload as SC sends 256 bytes per transaction. NOTE that SC can’t send partial transactions (i.e.) data less than 256 bytes.
  • After successfully receiving 64 KB payload, BMC needs to read the sector CRC(via 0x55) before proceeding to read the next sector.
  • BMC must poll the status command 0x4B (SC’s readiness for next sector) before proceeding to issue read-back command 0x54 for the next sector.
  • No retry is supported in case of any failure/interruption in the middle of sector read-back. But BMC can send the command 0x49 to force set the start sector from which it wants to resume/retry the read-back operation
  • Upon receiving the updated start sector number (via 0x49 command), SC starts the read-back process from the beginning of the sector. Once again, BMC needs to follow the same sequence of commands (i.e.) poll the status command 0x4B before issuing 0x54.

Table: FPGA_FLASH_READBACK_RX_DATA_BLOCK server BMC request

Server BMC request
Command code 0x54
Byte0 BMC sends repeated-start I2C command

Table: FPGA_FLASH_READBACK_RX_DATA_BLOCK Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes D0, D1 … D255 SC sends 256 data bytes

0x55 - GET_FPGA_FLASH_READBACK_SECTOR_CRC

BMC sends this command to SC to get the CRC for the previously read-back sector (via command 0x54). SC returns the Micron’s CRC-64 back and BMC can use the 8-byte CRC data to compare with the CRC-64 calculated within.

Table: SET_FPGA_FLASH_READBACK_SECTOR_RANGE server BMC request

Server BMC Request
Command code 0x55
Byte0 N/A

Table: SET_FPGA_FLASH_READBACK_SECTOR_RANGE Xilinx Alveo Card Response

Xilinx Alveo Card Response
Data bytes D0, D1, .. D7 SC sends 8-bytes of Micron CRC-64

Xilinx Support

For support resources such as answers, documentation, downloads, and forums, see the Alveo Accelerator Cards Xilinx Community Forum.

License

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License.

You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

All images and documentation, including all debug and support documentation, are licensed under the Creative Commons (CC) Attribution 4.0 International License (the “CC-BY-4.0 License”); you may not use this file except in compliance with the CC-BY-4.0 License.

You may obtain a copy of the CC-BY-4.0 License at https://creativecommons.org/licenses/by/4.0/

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

XD038 | © Copyright 2021 Xilinx, Inc.