AVED Management Interface userguide (ami_tool)¶
Overview¶
The AVED Management Interface (AMI) provides an interface to an AVED device and control of the AVED management controller (AMC) firmware deployed on it.
AMI is provided as 3 parts - the kernel driver (as a .ko file), the API, and a command line tool - ami_tool.
The ami_tool application provides a command line interface (CLI) to the user.
Dependencies¶
Supported OSs¶
| Linux | Kernel Version | Lib dependencies |
|---|---|---|
| Ubuntu 22.04 | 5.4.0 5.15.0 | libc6 libidn gcc make dkms grep gawk linux-header |
| Redhat 8.3 | 4.18.0 | glibc gccmake kernel-devel kernel-header |
Notes¶
Ensure that the AMI API and AMI Driver versions are the same. Behaviour is undefined when the versions do not match. See overview for more information.
Usage - Options¶
help¶
Description¶
Displays the available ami_tool commands.
When used after a command (e.g. ami_tool sensors –help), it displays the available options and expected usage of that specific command.
Example output¶
~$ ami_tool --help
ami_tool - command line tool for the AMI driver API
Usage:
ami_tool {command} [arguments]
ami_tool {command} -h | --help
ami_tool -h | --help
ami_tool --version
Options:
-h --help Show this screen
--version Show version
Commands:
sensors Show sensor information
cfgmem_program Program a device
cfgmem_fpt Program a device and update the FPT
cfgmem_copy Copy one partition to another
cfgmem_info Show partition information
device_boot Set boot partition
mfg_info View manufacturing information
bar_wr Write to PCI BAR memory
bar_rd Read from PCI BAR memory
overview Show basic AMI/device information
pcieinfo View PCI-related information
reload Reload a device/devices
eeprom_rd Read data from the device EEPROM
eeprom_wr Write data to the device EEPROM
module_byte_rd Read data from a QSFP module
module_byte_wr Write data to a QSFP module
debug_verbosity Set the AMC debug level
~$
Notes¶
For commands which accept a -d (device) option, all parts of the string are optional except the device bus number. The following are all valid inputs and refer to the same device:
0000:21:00.0
0000:21
21:00.0
21:00
21
version¶
Description¶
Displays the version information of the currently installed AMI API and AMI Driver.
Example output¶
~$ ami_tool --version
API Version | 2.2.0 (0)
API Branch | develop
API Hash | 5fac9611caf7982df7a0a66abdb1099bf091d5dd
API Hash Date | 20240307
Driver Version | 2.2.0 (0)
~$
Notes¶
If the API Version and Driver Version do not match, behavior is undefined and may result in unexpected results.
Usage - Commands¶
overview¶
Description¶
Displays an overview of the current AMI and associated devices setup. This includes the same version information as is shown in the ami_tool –version command.
Each detected and supported card will be displayed below.
This output can be logged to a file using the -o option, and the logged format specified by the -f option.
~% ami_tool overview --help
overview - show basic AMI/device information
Usage:
ami_tool overview [options...]
Options:
-h --help Show this screen
-f <table|json> Set the output format
-o <file> Specify output file
-v Print verbose information
~%
Example output¶
~$ ami_tool overview
AMI
-------------------------------------------------------------
Version | 2.2.0 (0)
Branch | develop
Hash | 5fac9611caf7982df7a0a66abdb1099bf091d5dd
Hash Date | 20240307
Driver Version | 2.2.0 (0)
BDF | Device | UUID | AMC | State
-------------------------------------------------------------------------------------
af:00.0 | Alveo V80 ES3 | 194c7d442ca60a32e5c67aceb5e90db9 | 2.2.3 (0) | READY
~$
Notes¶
The version value is the version of the AMI command line tool (and associated API) installed.
The driver version value is the version of the AMI Driver (ami.ko) installed.
In cases where either the AMI tool/API or driver has been replaced with a different version, and the “Version” and “Driver Version” values do not match, behaviour is undefined.
A row is appended to the table for each supported card discovered:
BDF - the ID to use when selecting this device for subsequent commands.
Device - the type of device detected (e.g. V80, etc.).
UUID - the ID of the hardware build loaded on the card.
AMC - the version of AMC running on the card (if applicable).
State - the state of the AMC running on the card (if applicable).
INIT: Device is initialising.
READY: Device is fully initialised and ready to use.
MISSING_INFO: AMC setup and device may be used, but some of the necessary data that AMC sends to AMI during initialisation is missing (e.g. sensors cannot be accessed, the FPT or MFG EEPROM is unavailable or corrupt, etc).
NO_AMC: Device may be used but has no AMC instance.
INIT_ERROR: AMC setup and device may be used but with no data.
SHUTDOWN: All services have been shutdown.
COMPAT: Compatibility mode - most functions unavailable.
When using the verbose options, the following additional fields are appended:
HWMON - the ID of the hwmon directory entry for this device
CDEV - the number of the corresponding /dev/ami character device
pcieinfo¶
Description¶
Displays the PCI information.
This output can be logged to a file using the -o option, and the logged format specified by the -f option.
~% ami_tool pcieinfo --help
pcieinfo - view PCI-related information
Usage:
ami_tool pcieinfo -d <bdf> [options...]
Options:
-h --help Show this screen
-d <b>:[d].[f] Specify the device BDF
-f <table|json> Set the output format
-o <file> Specify output file
~%
Example output¶
~$ ami_tool pcieinfo -d 21:00.0
PCIe Info
----------------------------------
Vendor | 0x10ee
Device | 0x50b4
Link Speed | Gen4 (max Gen5)
Link Width | x8 (max x8)
NUMA Node | 0
CPU Affinity | 0-7,16-23
~$
reload¶
Description¶
Reloads the AMI installation.
~% sudo ami_tool reload --help
reload - reload a device/devices
This command requires root/sudo permissions.
Usage:
ami_tool reload -t <type> [-d <bdf>]
Options:
-h --help Show this screen
-d <b>:[d].[f] Specify the device BDF
-t <type> Specify reload type
Possible values are:
driver=reload the entire driver
pci=force a pci removal and bus rescan
sbr=trigger a secondary bus reset
~%
Example output¶
~% sudo ami_tool reload -d 21:00.0 -t pci
Removing PCI device and rescanning bus...
Done.
~%
Notes¶
This command requires root/sudo privileges.
The ‘pci’ reload type may be used with devices which are not attached to the AMI driver (i.e., devices which do not show up when running the ‘overview’ command). This allows recovery of broken devices.
The ‘driver’ reload type does not accept a device (-d) argument.
The ‘sbr’ reload type will trigger a reset on the device (the image on the device will reload).
sensors¶
Description¶
Retrieves and displays sensor information from the AMC (if available).
This output can be logged to a file using the -o option, and the logged format specified by the -f option.
~% ami_tool sensors --help
sensors - view device sensor information
Usage:
ami_tool sensors [-d <bdf>] [options...]
Options:
-h --help Show this screen.
-d <b>:[d].[f] Specify the device BDF
-f <table|json> Set the output format
-o <file> Specify output file
-n <sensor> Fetch specific sensor
-x <field,...> Print extra fields
Possible values are:
{max, average}
-v Print all extra fields
~%
Example output¶
~$ ami_tool sensors -d 21:00.0
Name | Value | Status
----------------------------------------
1V2_GTXAVTT | 35.000 C | valid
| 1.000 A | valid
| 1.200 V | valid
----------------------------------------
0V88_VCC_CPM5 | 35.000 C | valid*
| 3.000 A | valid*
| 0.880 V | valid*
----------------------------------------
PCB | 35.000 C | valid*
----------------------------------------
Device | 40.000 C | valid*
----------------------------------------
VCCINT | 39.000 C | valid*
| 15.000 A | valid*
| 0.799 V | valid*
----------------------------------------
Module_0 | 0.000 C | invalid
----------------------------------------
Module_1 | 0.000 C | invalid
----------------------------------------
Module_2 | 33.000 C | valid*
----------------------------------------
Module_3 | 0.000 C | invalid
----------------------------------------
DIMM | 35.000 C | valid*
----------------------------------------
1V2_VCC_HBM | 38.000 C | valid*
| 4.000 A | valid*
| 1.199 V | valid*
----------------------------------------
Total_Power | 32.358 W | valid
----------------------------------------
12V_AUX1 | 0.620 A | valid*
| 12.208 V | valid*
----------------------------------------
12V_AUX2 | 0.720 A | valid*
| 12.200 V | valid*
----------------------------------------
1V2_VCCO_DIMM | 1.140 A | valid*
| 1.208 V | valid*
----------------------------------------
3V3_PEX | 0.559 A | valid*
| 3.336 V | valid*
----------------------------------------
12V_PEX | 1.140 A | valid*
| 12.192 V | valid*
----------------------------------------
3V3_QSFP | 0.040 A | valid*
| 3.296 V | valid*
----------------------------------------
1V5_VCCAUX | 1.497 V | valid*
~$
Notes¶
The * displayed after a sensor status indicates that the value retrieved was from a cached store (i.e., the value had been previously updated within the refresh timeframe (1 second)).
A status of invalid indicates that no value could be read for that sensor. For removable sensors (such as QSFP devices), this usually indicates that no devices is inserted.
If the -d option was not selected, AMI will enumerate each available device.
cfgmem_info¶
Description¶
Displays the file partition table (FPT) currently available on the selected card.
On a cold boot, partition 0 is the default boot location.
~% ami_tool cfgmem_info --help
cfgmem_info - get fpt information
Usage:
ami_tool cfgmem_info -d <bdf> -t <type> [options...]
Options:
-h --help Show this screen
-d <b>:[d].[f] Specify the device BDF
-t <type> Specify the boot device type (primary or secondary)
-f <table|json> Set the output format
-o <file> Specify output file
~%
Example output¶
~% ami_tool cfgmem_info -d 21:00.0 -t primary
Primary FPT:
FPT
---------------------
Version | 1
Header Size | 128
Entry Size | 128
Entries | 2
Partition | Type | Address | Size
------------------------------------------------------
0 | PDI_BOOT | 0x00008000 | 0x07e00000
1 | PDI_BOOT | 0x07e08000 | 0x07e00000
~%
cfgmem_program¶
Description¶
Uploads a PDI for AMC to program onto the card.
~% sudo ami_tool cfgmem_program --help
cfgmem_program - program a bitstream onto a device
This command requires root/sudo permissions.
Usage:
ami_tool cfgmem_program -d <bdf> -t <type> -i <path> -p <n>
Options:
-h --help Show this screen
-d <b>:[d].[f] Specify the device BDF
-t <type> Specify the boot device type (primary or secondary)
-i <path> Path to image file
-p <partition> Partition to flash
-y Skip confirmation
-q Quit after programming
~%
Example output¶
~% sudo ami_tool cfgmem_program -d 21:00.0 -i /home/builds/latest/design.pdi -p 1 -t primary
----------------------------------------------
Device | 21:00.0
----------------------------------------------
Current Configuration
----------------------------------------------
UUID | 7574dae0cea793ae3b0503b136a3bfd0
----------------------------------------------
Incoming Configuration
----------------------------------------------
UUID | 7574dae0cea793ae3b0503b136a3bfd0
Path | /home/builds/latest/design.pdi
Partition | 1
----------------------------------------------
Are you sure you wish to proceed? [Y/n]: Y
Updating base flash image...
[####################################################################################################] 100% |
Image programming complete.
Will do a hot reset to boot into partition 1. This may take a minute...
OK. Image has been programmed successfully.
***********************************************
Hot reset has been performed into partition 1.
***********************************************
~%
Notes¶
If this command reports a failure or an error, the card’s storage device is in an unknown state and the card should not be reset or rebooted. It is advised to reattempt this command.
This command requires root/sudo privileges.
If the <type> is specified as primary, and the -q option is NOT used, the card will be automatically rebooted into the selected partition (as shown in the example above).
The path (provided with the -i option) must be the absolute path, not the relative path.
The incoming UUID may be reported as “N/A” if no UUID metadata is provided with the PDI. This is normal behaviour and does not affect the programming.
The provided file will be broken into chunks; each chunk will be submitted individually with a timeout of 30s.
cfgmem_fpt¶
Description¶
Uploads a PDI containing a valid FPT (file partition table) for AMC to program onto the card. The PDI is always programmed to location 0x00 on the flash.
~% sudo ami_tool cfgmem_fpt --help
cfgmem_fpt - program a device and update the fpt
This command requires root/sudo permissions.
Usage:
ami_tool cfgmem_fpt -d <bdf> -i <path>
Options:
-h --help Show this screen
-d <b>:[d].[f] Specify the device BDF
-t <type> Specify the boot device type (primary or secondary)
-i <path> Path to image file
-y Skip confirmation
~%
Example output¶
~% sudo ami_tool cfgmem_fpt -d 21:00.0 -i /home/builds/latest/fpt_setup.pdi -t primary
----------------------------------------------
Device | 21:00.0
----------------------------------------------
Current Configuration
----------------------------------------------
UUID | 7574dae0cea793ae3b0503b136a3bfd0
----------------------------------------------
Incoming Configuration
----------------------------------------------
UUID | 7574dae0cea793ae3b0503b136a3bfd0
Path | /home/builds/latest/fpt_setup.pdi
----------------------------------------------
Will update FPT!
Are you sure you wish to proceed? [Y/n]: Y
Updating FPT...
[####################################################################################################] 100% |
OK. Image has been programmed successfully.
****************************************************
Cold reboot machine to load the new image on device.
****************************************************
~%
Notes¶
If this command reports a failure or an error, the card’s storage device is in an unknown state and the card should not be reset or rebooted. It is advised to reattempt this command.
This command requires root/sudo privileges.
The provided image must contain a valid FPT.
The path (provided with the -i option) must be the absolute path, not the relative path.
The incoming UUID may be reported as “N/A” if no UUID metadata is provided with the PDI. This is normal behaviour and does not affect the programming.
The provided file will be broken into chunks and each chunk will be submitted individually with a timeout of 30s.
cfgmem_copy¶
Description¶
Copies the contents of one partition on the card to another.
~% sudo ami_tool cfgmem_copy --help
cfgmem_copy - copy one partition to another
This command requires root/sudo permissions.
Usage:
ami_tool cfgmem_copy -d <bdf> -i <d:p> -p <d:p>
Options:
-h --help Show this screen
-d <b>:[d].[f] Specify the device BDF
-i <device:partition> Device (primary or secondary):Partition to copy from (source)
-p <device:partition> Device (primary or secondary):Partition to copy to (destination)
~%
Example output¶
~% sudo ami_tool cfgmem_copy -d 21:00.0 -i primary:1 -p primary:0
Copying partition 1 to partition 0...
Estimated time to copy partition: 1234 (seconds) (NOTE: Changing logging levels may increase copy time)
.........................................
Done. Partition copied successfully.
~%
Notes¶
This command requires root/sudo privileges.
The source partition must be smaller than or equal in size to the destination partition.
The timeout value of this command is 45 minutes - this is to allow a maximum-sized FPT partition (126MB) to be copied.
bar_wr¶
Description¶
Writes a value to the PCI BAR memory.
~% sudo ami_tool bar_wr --help
bar_wr - Write to PCI BAR memory
This command requires root/sudo permissions.
Usage:
ami_tool bar_wr -d <bdf> -b <bar> -a <addr> (-i|-I) <input>
Options:
-h --help Show this screen
-d <b>:[d].[f] Specify the device BDF
-b <bar> Specify the BAR to write
-a <addr> Specify the offset to write to
-i <value> Register value to write
-I <file> File to write
~%
Example output¶
~% sudo ami_tool bar_wr -d 21:00.0 -b 0 -a 0x1000 -i 0x22
Writing the following data to device 21:00.0 and BAR 0 at offset 0x0000000000001000
[ 0x0000000000001000 ] 00000022
Are you sure you wish to proceed? [Y/n]: Y
Successfully wrote to 1 register(s)
~%
Notes¶
This command requires root/sudo privileges.
If -i is selected, the following variable is the value you wish to write.
If -I is selected, the following variable is the path to the file you wish to write from.
The file must be of the format of a single 4-byte hexadecimal value per line, e.g.:
0x00112233
0xaabbccdd
bar_rd¶
Description¶
Reads a value from the PCI BAR memory.
~% sudo ami_tool bar_rd --help
bar_rd - Read from PCI BAR memory
This command requires root/sudo permissions.
Usage:
ami_tool bar_rd -d <bdf> -b <bar> -a <addr>
Options:
-h --help Show this screen
-d <b>:[d].[f] Specify the device BDF
-b <bar> Specify the BAR to read
-a <addr> Specify the offset to read from
-l <len> Number of registers to read (default=1)
-o <file> Output file
~%
Example output¶
~% sudo ami_tool bar_rd -d 21:00.0 -b 0 -a 0x1000 -l 10
Reading 10 register(s) from device 21:00.0 and BAR 0 at offset 0x0000000000001000
[ 0x0000000000001000 ] ffffffff ffffffff ffffffff ffffffff
[ 0x0000000000001010 ] ffffffff ffffffff ffffffff ffffffff
[ 0x0000000000001020 ] ffffffff ffffffff
~%
Notes
This command requires root/sudo privileges.
If -o is selected, the data read will be written to the file path provided.
The output file will be of the format of a single 4-byte hexadecimal value per line, e.g.:
0x00112233
0xaabbccdd
device_boot¶
Description¶
Selects the boot partition for the card to reboot from.
~% sudo ami_tool device_boot --help
device_boot - set device boot partition
This command requires root/sudo permissions.
Usage:
ami_tool device_boot -d <bdf> -p <n>
Options:
-h --help Show this screen
-d <b>:[d].[f] Specify the device BDF
-p <partition> Partition to select
~%
Example output¶
~% sudo ami_tool device_boot -d 21:00.0 -p 0
Will do a hot reset to boot into partition 0. This may take a minute...
OK. Partition selected (0) - quitting.
***********************************************
Hot reset has been performed into partition 0.
***********************************************
~%
Notes¶
This command requires root/sudo privileges.
This causes a hot reset of the card from the selected partition.
mfg_info¶
Description¶
Displays the manufacturing information for a selected card.
This output can be logged to a file using the -o option and the logged format specified by the -f option.
~% ami_tool mfg_info --help
mfg_info - view manufacturing information
Options:
-h --help Show this screen
-d <b>:[d].[f] Specify the device BDF
-f <table|json> Set the output format
-o <file> Specify output file
~%
Example output¶
~% ami_tool mfg_info -d 21:00.0
Manufacturing Information
-----------------------------------------------------------------------------------------
Eeprom Version | 3.1
Product Name | V80 ES1
Board Revision | A01
Serial Number | 51131A237M22
Mac Address 1 | 00:0a:35:18:85:41
Mac Address N | 00:0a:35:18:85:81
Memory Size | 16G
FPGA Config Mode | OSPI
Max Power Mode | 300W
Active State | Passive
Manufacturing Date | Sun Sep 3 20:42:00 2023
UUID | d2d6dbed-dada-1044-b364-73f611ce4e42
PCIe Ids(Vendor, Device, SS Vendor, SS Device) | 10ee, 5004, 10ee, 000e
OEM ID | AMD
Capability Word | 00a7
Board Part Num | 05113-01
~% ami_tool mfg_info -d c1:00.0
Manufacturing Information
-----------------------------------------------------------------------------------------
Eeprom Version | 4.0
Product Name | V80 ES1
Board Revision | A01
Serial Number | 51131A136M0Q
Mac Address 1 | 00:0a:35:18:82:3e
Mac Address N | 00:0a:35:18:82:7e
Manufacturing Date | Fri Aug 11 14:16:00 2023
UUID | 36d0316b-9e59-0544-b9f7-1a3604cb1281
Board Part Num | 05113-01
Mfg Part Num | 043-05113-01
~%
eeprom_rd¶
Description¶
Allows performing a raw data read of the device EEPROM.
This output can be logged to a file using the -o option.
~% ami_tool eeprom_rd --help
eeprom_rd - Read the device EEPROM
Usage:
ami_tool eeprom_rd -d <bdf> -a <addr>
Options:
-h --help Show this screen
-d <b>:[d].[f] Specify the device BDF
-a <addr> Specify the offset to read from
-l <len> Number of registers to read (default=1)
-o <file> Output file
~%
Example output¶
~% ami_tool eeprom_rd -d 21:00.0 -a 0 -l 128
Reading 128 byte(s) from device 21:00.0 at offset 0x00
[ 0x0000000000000000 ] 34 2e 30 ff 71 c3 41 6c 76 65 6f 20 56 38 30 20
[ 0x0000000000000010 ] 45 53 33 ff ff ff ff ff ff ff ff ff ff ff ff ff
[ 0x0000000000000020 ] ff ff ff ff ff ff ff ff ff ff 41 2d 56 38 30 2d
[ 0x0000000000000030 ] 50 36 34 47 2d 45 53 33 2d 47 ff ff ff ff ff ff
[ 0x0000000000000040 ] ff ff 30 34 33 2d 30 35 31 31 33 2d 30 31 45 53
[ 0x0000000000000050 ] ff ff 42 30 31 ff ff 35 31 31 33 31 42 31 33 39
[ 0x0000000000000060 ] 50 34 4f ff ff 15 c1 de 00 10 00 0a 35 18 8b 21
[ 0x0000000000000070 ] 19 16 f5 04 5b 73 ff 4b bf 39 e7 20 e9 71 03 97
~%
Notes¶
The -l <len> option specifies the number of bytes to read from the EEPROM.
If -o is selected, the data read will be written to the file path provided.
The output file will be of the format of a single 1-byte hexadecimal value per line, e.g.:
0x00
0x11
eeprom_wr¶
Description¶
Allows performing a raw data write to the device EEPROM.
~% ami_tool eeprom_wr --help
eeprom_wr - Write to the device EEPROM
Usage:
ami_tool eeprom_wr -d <bdf> -a <addr> (-i|-I) <input>
Options:
-h --help Show this screen
-d <b>:[d].[f] Specify the device BDF
-a <addr> Specify the offset to write to
-i <value> Register value to write
-I <file> File to write
~%
Example output¶
~% ami_tool eeprom_wr -d 21 -a 0x80 -i 0x33
Writing the following data to device 21:00.0 at offset 0x80
[ 0x0000000000000080 ] 33
Are you sure you wish to proceed? [Y/n]: Y
Successfully wrote to 1 register(s)
~%
Notes¶
If -i is selected, the following variable is the value you wish to write.
If -I is selected, the following variable is the path to the file you wish to write from.
The file must be of the format of a single 1-byte hexadecimal value per line, e.g.:
0x00
0x11
module_byte_rd¶
Description¶
Allows performing a raw data read of a device QSFP module.
~% ami_tool module_byte_rd --help
module_byte_rd - Read from a QSFP module
Usage:
ami_tool module_byte_rd -d <bdf> -c <n> -p <n> -b <n>
Options:
-h --help Show this screen
-d <b>:[d].[f] Specify the device BDF
-c <cage> Module ID to write to
-p <page> Page number to write
-b <byte> Specify the offset to write to
~%
Example output¶
~% ami_tool module_byte_rd -d 21 -c 1 -p 0 -b 5
Reading byte 0x05 from page 0 (device 21:00.0, cage 1)
Value 0x00
~%
Notes¶
The cage numbers are 1-indexed (i.e. 0 is invalid).
module_byte_wr¶
Description¶
Allows performing a raw data write to a device QSFP module.
~% ami_tool eeprom_wr --help
module_byte_wr - Write to PCI BAR memory
Usage:
ami_tool module_byte_wr -d <bdf> -c <n> -p <n> -b <n> -i <val>
Options:
-h --help Show this screen
-d <b>:[d].[f] Specify the device BDF
-c <cage> Module ID to write to
-p <page> Page number to write
-b <byte> Specify the offset to write to
-i <value> Byte value to write
~%
Example output¶
~% ami_tool module_byte_wr -d 21 -c 1 -p 0 -b 5 -i 0x01
Writing value 0x01 to page 0, byte 0x05 (device 21:00.0, cage 1)
OK - value written successfully
~%
Notes¶
The cage numbers are 1-indexed (i.e. 0 is invalid).
debug_verbosity¶
Description¶
Sets the verbosity level of the AMC output.
~% ami_tool debug_verbosity --help
debug_verbosity - set the AMC debug level
Usage:
ami_tool debug_verbosity -d <bdf> -l <level>
Options:
-h --help Show this screen
-d <b>:[d].[f] Specify the device BDF
-l <level> Debug level
Possible values are:
{log, info, error, warning, debug}
~%
Example output¶
~% ami_tool debug_verbosity -d 21 -l debug
Setting AMC debug level to 'debug'.
OK. debug level updated!
~%
Notes¶
Debug levels are “up to and including”, i.e. a debug level of WARNING will include LOG, INFO, ERROR, and WARNING messages.
The default debug level is LOG.
The output can be viewed in dmesg.
Page Revision: v. 84