How-to install and run a pre-built AVED design on an ALVEO card

This is a getting started guide for the Alveo™ Versal™ Example Design (AVED) to be used with the provided pre-built AVED package file. This guide details installing the package on your server and running key commands to test and evaluate the design example.

Before You Begin

Dependencies

The Example Design has been tested with the operating systems in the table below. AVED may work with other versions of these operating systems, but they have not been tested. All supported operating systems are tested with general access versions (GA). Ubuntu Hardware Enablement (HWE) is not supported. By default, HWE is disabled by Ubuntu Server versions and enabled by Desktop versions.

Operating SystemArchitectureSupported VersionsKernel Version
Ubuntux86_6422.04

5.4.0

5.15.0

RHELx86_648.34.18.0

Note: root/sudo permissions are required.

Tools

  • Vivado™ 2023.2

Preparing for AVED Installation

  1. Confirm the server is running a supported OS. Follow the instructions provided in the Alveo V80 Data Center Accelerator Cards Installation Guide (UG1617) to physically install the card in your server.

  2. Get the latest AVED installation package for your Alveo card.

    1. Refer to the AVED release table to find the latest example design version. The AVED Git Release Tag will direct you to the tagged source code in GitHub. The AVED Deployment Archive is a zipped AVED installation package; it is pre-built from the tagged source code using the Vivado™ tools version noted in the table.

    2. Download the latest AVED Deployment archive to the server that contains your Alveo card.

    3. Unzip the archive, and find the aved_install.sh script, which will be used in the AVED installation. Review to the AVED Deployment Archive for details on the extracted files and directory structure.

  3. Unlike prior Alveo platforms, AVED does not use the Xilinx RunTime Library (XRT). Instead, AVED includes the AVED Management Interface (AMI) as part of the example design. Before AVED installation, ensure that XRT has been removed. It’s not possible to have AMI and XRT running on the same host at the same time.

# --- Ubuntu -----------------------------
# Check if XRT is installed
$ apt list | grep xrt

# Remove XRT if present
$ sudo apt remove xrt

# --- RHEL -------------------------------
# Check if XRT is installed
$ yum list | grep xrt

# Remove XRT if present
$ sudo yum remove xrt

AVED Installation

Installing AVED is a two-step process. First, you will install the pre-built AVED package onto the host. Second, you will program AVED into flash on your Alveo card.

To install the package onto the host, navigate to the directory with the aved_install.sh script, and execute this script.

$ sudo ./aved_install.sh

If successfully installed, the console will display “AVED Installation Complete.” For further information, see Installing AVED Deployment package onto the Host Server for detailed instructions with a complete example output.

To program AVED onto your Alveo card for the first time, update the Flash Partition Table (FPT) image in flash using Vivado HW Manager and JTAG. Refer to AVED Updating FPT Image in Flash for step-by-step instructions. If successful, Vivado HW Manager will report this with a pop-up message: “Flash programming completed successfully.” Cold boot the server to boot AVED from flash.

Testing AVED

Two sets of tools are provided to test the AVED design running in hardware: a suite of AMI commands (ami_tool <command>) and xbtest testcases (xbtest -d <Bus:Device.Function(BDF)> -c <testcase>). A subset of these tools are illustrated below.

ami_tool Overview

The overview command shows basic information including the AMI version, the BDF, the device name, the currently programmed design’s UUID, and the device state. Take note of the BDF of your card (21:00.0 in the example below), which will be used in subsequent commands.

$ ami_tool overview

AMI
-------------------------------------------------------------
Version          | 1.8.0  (0)
Branch
Hash             | e223bc725a7fe2282dcacaa6f7befdd152dbb714
Hash Date        | Thu Nov 30 12:12:38 2023 +0000
Driver Version   | 1.8.0  (0)


BDF       | Device          | UUID                               | AMC          | State
-----------------------------------------------------------------------------------------
21:00.0   | Alveo V80 ES3   | f92f252f3a8a1fbd62569c4a4b46b141   | 1.8.0  (0)   | READY
If the device state is NOT_READY, cold reboot the system.
Confirm the AMI driver version matches the AMC version on the device before continuing. If they do not match, AMI behavior is undefined.

ami_tool mfg_info

The mfg_info command displays the card manufacturing information stored on an EEPROM. The values will vary from card to card.

$ ami_tool mfg_info -d <BDF>

Manufacturing Information
--------------------------------------------------------------------
Eeprom Version              | 4.0
Product Name                | Alveo V80 ES3
Board Revision              | B01
Serial Number               | 51131B139P66
Mac Address 1               | 00:0a:35:18:8e:81
Mac Address N               | 00:0a:35:18:8e:91
Manufacturing Date          | Fri Nov  3 17:08:00 2023
UUID                        | 6a0d7d36-5a56-bb43-b7f2-91ce3318e6a7
Board Part Num              | A-V80-P64G-ES3-G
Mfg Part Num                | 043-05113-01ES

ami_tool sensors

The sensors command displays the available sensors and their values. The values may vary. For removable sensors (such as QSFP devices), the Status may appear as “invalid”; this indicates that no device is inserted. The * displayed after a sensor status indicates that the value retrieved was from a cached store - i.e., the value had been previously updated within the refresh timeframe (1 second).

$ ami_tool sensors -d <BDF>

Name            |      Value | Status
---------------------------------------
1V2_GTXAVTT     |   31.000 C | valid
                |    8.000 A | valid
                |    1.200 V | valid
---------------------------------------
0V88_VCC_CPM5   |   29.000 C | valid*
                |    3.000 A | valid*
                |    0.880 V | valid*
---------------------------------------
PCB             |   28.000 C | valid*
---------------------------------------
Device          |   37.000 C | valid*
---------------------------------------
VCCINT          |   34.000 C | valid*
                |   32.000 A | valid*
                |    0.800 V | valid*
---------------------------------------
Module_0        |   29.000 C | valid*
---------------------------------------
Module_1        |   29.000 C | valid*
---------------------------------------
Module_2        |   27.000 C | valid*
---------------------------------------
Module_3        |   26.000 C | valid*
---------------------------------------
DIMM            |   26.000 C | valid*
---------------------------------------
1V2_VCC_HBM     |   34.000 C | valid*
                |    4.000 A | valid*
                |    1.200 V | valid*
---------------------------------------
Total_Power     |   62.785 W | valid
---------------------------------------
12V_AUX1        |    1.079 A | valid*
                |   12.216 V | valid*
---------------------------------------
12V_AUX2        |    1.400 A | valid*
                |   12.200 V | valid*
---------------------------------------
1V2_VCCO_DIMM   |    1.100 A | valid*
                |    1.200 V | valid*
---------------------------------------
3V3_PEX         |    1.820 A | valid*
                |    3.296 V | valid*
---------------------------------------
12V_PEX         |    2.179 A | valid*
                |   12.192 V | valid*
---------------------------------------
3V3_QSFP        |    0.099 A | valid*
                |    3.304 V | valid*
---------------------------------------
1V5_VCCAUX      |    1.498 V | valid*

xbtest -d <BDF> -c verify

Verify successful AVED installation by running the xbtest verify test, which checks basic communication between the host and xbtest IP’s scratch registers. In the following example, the BDF is 21:00.0; this may differ on your system.

Successful completion of this test is indicated by 0 Errors, 0 Failures, and the message “RESULT: ALL TESTS PASSED”. The Critical Warning ITF_138 “PCIe link speed (4) configuration running on the card does not match expected (5) according to device information” is an indication that the PCIe link speed is limited by the host capability, but is not a failure.

$ xbtest -d 21:00.0 -c verify

INFO      :: GEN_016 :: GENERAL      :: Scanning installed xbtest HW designs...
STATUS    :: MGT_002 :: TIMER        :: Start 1s tick function
INFO      :: GEN_016 :: GENERAL      :: 21:00.0 :: Getting card configuration
INFO      :: GEN_064 :: GENERAL      :: 21:00.0 :: Starting test 1 of 1. Executing: xbtest -d 21:00.0 -j /opt/amd/aved/amd_v80_gen5x8_23.2_exdes_1_xbtest_stress/xbtest/test/verify.json -i /opt/amd/aved/amd_v80_gen5x8_23.2_exdes_1_xbtest_stress/design.pdi -e /opt/amd/aved/amd_v80_gen5x8_23.2_exdes_1_xbtest_stress/xbtest/metadata/xbtest_pfm_def.json
INFO      :: GEN_039 :: GENERAL      :: #####################################################################################################
INFO      :: GEN_039 :: GENERAL      :: xbtest version: 7.0.0
INFO      :: GEN_016 :: GENERAL      ::          - SW Build    : 4064028 on Fri Dec 01 05:09:04 2023
INFO      :: GEN_016 :: GENERAL      ::          - Process ID  : 14941
INFO      :: GEN_016 :: GENERAL      :: #####################################################################################################
INFO      :: GEN_016 :: GENERAL      :: System:
INFO      :: GEN_016 :: GENERAL      ::          - Name          : Linux
INFO      :: GEN_016 :: GENERAL      ::          - Release       : 5.4.0-165-generic
INFO      :: GEN_016 :: GENERAL      ::          - Version       : #182-Ubuntu SMP Mon Oct 2 19:43:28 UTC 2023
INFO      :: GEN_016 :: GENERAL      ::          - Machine       : x86_64
INFO      :: GEN_039 :: GENERAL      ::          - Running with AMI driver 1.8.0
INFO      :: GEN_016 :: GENERAL      :: #####################################################################################################
INFO      :: GEN_039 :: GENERAL      :: Start of xbtest session at: Mon Dec 04 23:09:13 2023 GMT
INFO      :: GEN_039 :: GENERAL      :: #####################################################################################################
INFO      :: ITF_008 :: XBT_SW_CFG   :: Using card: Alveo V80 ES3 (21:00.0)
Starting dynamic display mode...


Repeating last content of dynamic display mode:
  INFO      :: GEN_039 :: GENERAL      :: #####################################################################################################
  INFO      :: GEN_039 :: GENERAL      :: xbtest version: 7.0.0
  INFO      :: GEN_039 :: GENERAL      ::        - Running with AMI driver 1.8.0
  INFO      :: GEN_039 :: GENERAL      :: Start of xbtest session at: Mon Dec 04 23:09:13 2023 GMT
  INFO      :: GEN_039 :: GENERAL      :: #####################################################################################################
  INFO      :: ITF_008 :: XBT_SW_CFG   :: Using card: Alveo V80 ES3 (21:00.0)
  CRIT WARN :: ITF_138 :: DEVICE       :: PCIe link speed (4) configuration running on the card does not match expected (5) according to device information
  WARNING   :: VER_012 :: VERIFY       :: Verify xbtest HW IP has not access to the DNA

  +------------------+ +-----------------------------------------------------------+ +---------------------------------------------------------------------+
  |                  | |                          STATUS                           | |                            ONGOING TEST                             |
  |     TESTCASE     | |-----------------------------------------------------------| |---------------------------------------------------------------------|
  |                  | | Pending | Completed | Passed | Failed | Errors | Warnings | | Remaining time (s) | Parameters                                     |
  |------------------| |-----------------------------------------------------------| |---------------------------------------------------------------------|
  | verify           | |       0 |         3 |      3 |      0 |      0 |        1 | |                n/a | n/a                                            |
  +------------------+ +-----------------------------------------------------------+ +---------------------------------------------------------------------+

  Card status: Power: 64.0 W; Temperature: 35 C; Qty of measurements: 1

  Messages stats: 1 Warnings, 1 Critical Warnings, 27 Passes, 0 Errors, 0 Failures encountered

  Total elapsed: 6 s


INFO      :: GEN_040 :: GENERAL      :: ############################################## SUMMARY ##############################################
INFO      :: GEN_040 :: GENERAL      :: End of xbtest session at: Mon Dec 04 23:09:17 2023 GMT
INFO      :: GEN_040 :: GENERAL      :: 1 Warnings, 1 Critical Warnings, 26 Passes, 0 Errors, 0 Failures encountered
INFO      :: GEN_040 :: GENERAL      :: #####################################################################################################
PASS      :: GEN_024 :: GENERAL      :: RESULT: ALL TESTS PASSED

xbtest -d <BDF> -c memory

Verify the DDR and HBM memory performance by running the xbtest memory test, which checks communication between the xbtest IP and available memories. Read and write bandwidth and latency measurements are logged to .csv files for detailed review. In the following example, the BDF is 21:00.0. This may differ on your system.

Successful completion of this test is indicated by 0 Errors, 0 Failures, and the message “RESULT: ALL TESTS PASSED”. The Critical Warning ITF_138 “PCIe link speed (4) configuration running on the card does not match expected (5) according to device information” is an indication that the PCIe link speed is limited by the host capability, but is not a failure.

$ xbtest -d 21:00.0 -c memory

INFO      :: GEN_016 :: GENERAL      :: Scanning installed xbtest HW designs...
STATUS    :: MGT_002 :: TIMER        :: Start 1s tick function
INFO      :: GEN_016 :: GENERAL      :: 21:00.0 :: Getting card configuration
INFO      :: GEN_064 :: GENERAL      :: 21:00.0 :: Starting test 1 of 1. Executing: xbtest -d 21:00.0 -j /opt/amd/aved/amd_v80_gen5x8_23.2_exdes_1_xbtest_stress/xbtest/test/memory.json -i /opt/amd/aved/amd_v80_gen5x8_23.2_exdes_1_xbtest_stress/design.pdi -e /opt/amd/aved/amd_v80_gen5x8_23.2_exdes_1_xbtest_stress/xbtest/metadata/xbtest_pfm_def.json
INFO      :: GEN_039 :: GENERAL      :: #####################################################################################################
INFO      :: GEN_039 :: GENERAL      :: xbtest version: 7.0.0
INFO      :: GEN_016 :: GENERAL      ::          - SW Build    : 4064028 on Fri Dec 01 05:09:04 2023
INFO      :: GEN_016 :: GENERAL      ::          - Process ID  : 15673
INFO      :: GEN_016 :: GENERAL      :: #####################################################################################################
INFO      :: GEN_016 :: GENERAL      :: System:
INFO      :: GEN_016 :: GENERAL      ::          - Name          : Linux
INFO      :: GEN_016 :: GENERAL      ::          - Release       : 5.4.0-165-generic
INFO      :: GEN_016 :: GENERAL      ::          - Version       : #182-Ubuntu SMP Mon Oct 2 19:43:28 UTC 2023
INFO      :: GEN_016 :: GENERAL      ::          - Machine       : x86_64
INFO      :: GEN_039 :: GENERAL      ::          - Running with AMI driver 1.8.0
INFO      :: GEN_016 :: GENERAL      :: #####################################################################################################
INFO      :: GEN_039 :: GENERAL      :: Start of xbtest session at: Mon Dec 04 23:33:13 2023 GMT
INFO      :: GEN_039 :: GENERAL      :: #####################################################################################################
INFO      :: ITF_008 :: XBT_SW_CFG   :: Using card: Alveo V80 ES3 (21:00.0)
Starting dynamic display mode...


Repeating last content of dynamic display mode:
  INFO      :: GEN_039 :: GENERAL      :: #####################################################################################################
  INFO      :: GEN_039 :: GENERAL      :: xbtest version: 7.0.0
  INFO      :: GEN_039 :: GENERAL      ::        - Running with AMI driver 1.8.0
  INFO      :: GEN_039 :: GENERAL      :: Start of xbtest session at: Mon Dec 04 23:33:13 2023 GMT
  INFO      :: GEN_039 :: GENERAL      :: #####################################################################################################
  INFO      :: ITF_008 :: XBT_SW_CFG   :: Using card: Alveo V80 ES3 (21:00.0)
  CRIT WARN :: ITF_138 :: DEVICE       :: PCIe link speed (4) configuration running on the card does not match expected (5) according to device information
  WARNING   :: VER_012 :: VERIFY       :: Verify xbtest HW IP has not access to the DNA

  +------------------+ +-----------------------------------------------------------+ +---------------------------------------------------------------------+
  |                  | |                          STATUS                           | |                            ONGOING TEST                             |
  |     TESTCASE     | |-----------------------------------------------------------| |---------------------------------------------------------------------|
  |                  | | Pending | Completed | Passed | Failed | Errors | Warnings | | Remaining time (s) | Parameters                                     |
  |------------------| |-----------------------------------------------------------| |---------------------------------------------------------------------|
  | memory                                                                                                                                                 |
  |              DDR | |       0 |         4 |      4 |      0 |      0 |        0 | |                n/a | n/a                                            |
  |              HBM | |       0 |         4 |      4 |      0 |      0 |        0 | |                n/a | n/a                                            |
  |------------------| |-----------------------------------------------------------| |---------------------------------------------------------------------|
  | verify           | |       0 |         3 |      3 |      0 |      0 |        1 | |                n/a | n/a                                            |
  +------------------+ +-----------------------------------------------------------+ +---------------------------------------------------------------------+

  Card status: Power: 64.8 W; Temperature: 38 C; Qty of measurements: 102

  Messages stats: 1 Warnings, 1 Critical Warnings, 67 Passes, 0 Errors, 0 Failures encountered

  Total elapsed: 106 s


INFO      :: GEN_040 :: GENERAL      :: ############################################## SUMMARY ##############################################
INFO      :: GEN_040 :: GENERAL      :: End of xbtest session at: Mon Dec 04 23:34:57 2023 GMT
PASS      :: GEN_022 :: GENERAL      :: memory DDR test passed
PASS      :: GEN_022 :: GENERAL      :: memory HBM test passed
INFO      :: GEN_040 :: GENERAL      :: 1 Warnings, 1 Critical Warnings, 66 Passes, 0 Errors, 0 Failures encountered
INFO      :: GEN_040 :: GENERAL      :: #####################################################################################################
PASS      :: GEN_024 :: GENERAL      :: RESULT: ALL TESTS PASSED

Troubleshooting

For debugging information, see AVED Debug Techniques.

Next Steps

This getting started guide covered AVED installation and basic hardware testing commands. If you would like to explore the full set of hardware test commands available, see:

Notably, some interesting commands to try are:

Note: The gtm_prbs and gtyp_prbs tests are not available for the V80P, which does not have GTs.

For a quick look at how to build AVED, refer to the getting started guide: How-to Rebuild an AVED Design for Yourself.


Page Revision: v. 35