Checklist - xilinx-u55c-gen3x16-xdma-3¶

Table of content

Package information
System information
Requirement - Platform high level features
Calibration - Power CU - Results
Calibration - Memory CU power
Calibration - Memory bandwidth and latency
Checklist questionnaire
Pre-canned tests results

Package information ¶

xbtest information
Release Number	3522912
Description (what’s new)	Initial Release

Platform information

Development platform (XSA)

xilinx-u55c-gen3x16-xdma-3-202210-1-dev-1-3514517

Deployment platform (shell, CMC, SC)

xilinx-cmc-u55-1.5.25-3395704

xilinx-sc-fw-u55-7.1.17-1.f7275be

xilinx-u55c-gen3x16-xdma-base-3-3494559

xilinx-u55c-gen3x16-xdma-validate-3-3506150

System information ¶

System information
OS	CentOS / 7.8.2003
Architecture	x86_64
Server type	PowerEdge R740
AUX cable	yes
Card S/N	XFL1BO3WZCH5
XRT version	2.13.449
Measured download time	10.5 seconds

Requirement - Platform high level features ¶

Platform and card requirements¶

Platform and card requirements
FPGA part	xcu55c-fsvh2892-2L-e
PCIe speed and width	gen3x16
DMA type	xdma
DDR: speed, size and quantity	N/A
HBM: size, quantity and power rail	32 Pseudo-Channels; 16,384MB, 12v_pex
Host memory support	yes
P2P support	yes
GTs: type, quad, SLR location	GT[0]: GTY, Quad_X0Y6 (SLR1) GT[1]: GTY, Quad_X0Y7 (SLR1)

Power rails requirements¶

Power rail	Card limit (W)	Note
`12v_pex`	69W	Powers HBM
`3v3_pex`	10.4W
`12v_aux`	78W	User Selectable
`vccint`	N/A	Fatal Limit/Card Shutdown @60A per phase (8 phases with full AUX power)

Calibration - Power CU - Results ¶

Description		Results
Power, temperature & toggle rate graph
Maximum toggle rate reached (%)		84 Board reboot due to 12v_aux threshold. Last measured power at the 12v_aux was :78.568776 W
Power consumption balance achieved: Yes/no and justification (why are some rails not close to their critical current limit?)		No Historically, it is difficult to reach maximum 150W limit for u55c.
`3v3_pex` current (A)	Measurement	1.577
`3v3_pex` current (A)	Critical limit	3.15
`12v_pex` current (A)	Measurement	4.264
`12v_pex` current (A)	Critical limit	5.75
`12v_aux` current (A)	Measurement	6.457
`12v_aux` current (A)	Critical limit	6.5

Calibration - Memory CU power ¶

Memory	Description	`simultaneous_wr_rd`
HBM	Is there enough power for memory?	Yes. The U55C card is capable of supplying full power to the HBM memories.
	Power graph
	BW graph
	Latency graphs

Calibration - Memory bandwidth and latency ¶

Note

N/A: not applicable
1

The outstanding write/read is the maximum number of outstanding reads just before the read pipeline is full, giving the best BW and latency results.

Memory	Description	`simultaneous_wr_rd`	`only_rd`	`only_wr`
HBM	QoS / power reduction rate	Write rate = 33 %; read rate = 33 %	`N/A`	`N/A`
	Best write BW (MBps)	6008	N/A	12255
	Best write latency (ns)	143	N/A	203
	Best read BW (MBps)	6008	12292	N/A
	Best read latency (ns)	681	952	N/A
	Outstanding write/read 1	Write = 2 / read = 21	4	3
	BW/latency graphs (do one for read and one for write)

Checklist questionnaire ¶

Runtime¶

Question

Answer / justification / problem tracker reference number

Have you updated the default download time?

FYI:

u250 xclbin (90Mb) downloads in 3.9 sec (PCIe 3x16).

HBM requires calibration which may take time.

Yes

"download_time": 13

Measured download time is 10.5 sec. Extra padding added to give time for HBM calibration.

Sensors¶

Question

Answer / justification / problem tracker reference number

Do you need to monitor more mechanical sources than the default one (fpga_fan_1)?

Yes, see Fill platform definition JSON

No

Do you need to monitor more thermal sources than the default one (fpga0)?

Yes, see Fill platform definition JSON

No

Do you need to monitor more electrical sources than the default ones (power_consumption, 12v_pex, 12v_aux, 3v3_pex and vccint)?

Yes, see Fill platform definition JSON

No

GT¶

Question

Answer / justification / problem tracker reference number

GT

GT[0]

GT[1]

Have you updated GT default settings to maximize signal integrity?

Run all GT pre-canned tests.

"0": {
    "name": "top",
    "transceiver_settings": {
        "module": {
            "tx_differential_swing_control": 11,
            "tx_pre_emphasis": 0,
            "tx_post_emphasis": 0,
            "rx_equaliser": "DFE"
        },
        "cable": {
            "tx_differential_swing_control": 11,
            "tx_pre_emphasis": 0,
            "tx_post_emphasis": 0,
            "rx_equaliser": "DFE"
        }
    }
}

"1": {
    "name": "bottom",
    "transceiver_settings": {
        "module": {
            "tx_differential_swing_control": 11,
            "tx_pre_emphasis": 0,
            "tx_post_emphasis": 0,
            "rx_equaliser": "DFE"
        },
        "cable": {
            "tx_differential_swing_control": 11,
            "tx_pre_emphasis": 0,
            "tx_post_emphasis": 0,
            "rx_equaliser": "DFE"
        }
    }
}

Memory¶

Question	Answer / justification / problem tracker reference number
Memory	HBM	HOST
Have you defined specific DMA bandwidths (`dma_bw`)? This not applicable for host memory.	No	N/A
Have you defined optimum DMA settings (`buffer_size` and `total_size`) or are default settings enough? Run dma pre-canned test.	No	N/A
Have you defined specific P2P CARD bandwidths (`p2p_card_bw`)? This not applicable for host memory.	No	N/A
Have you defined optimum P2P CARD settings (`buffer_size` and `total_size`) or are default settings enough? Run p2p_card pre-canned test.	No	N/A
Have you defined specific P2P NVME bandwidths (`p2p_nvme_bw`)? This not applicable for host memory.	No	N/A
Have you defined optimum P2P NVME settings (`buffer_size` and `total_size`) or are default settings enough? Run p2p_nvme pre-canned test.	No	N/A
Have you reduced the CU rate to avoid over-power (`cu_rate`)?	No	No
Have you defined `cu_rate` for simultaneous write/read bandwidth QoS?	"cu_rate": { "simul_wr_rd": { "write": { "nominal": 33 }, "read": { "nominal": 33 } } }	"cu_rate": { "only_wr": { "write": { "nominal": 50 } } }
Have you defined `cu_bw`? For each test mode: `only_wr`, `only_rd` and `simul_wr_rd` For host memory these values are depending on the PCIe speed.	"cu_bw": { "only_wr": { "write": { "average": 12255 } }, "only_rd": { "read": { "average": 12292 } }, "simul_wr_rd": { "write": { "average": 6008 }, "read": { "average": 6008 } } }	"cu_bw": { "only_wr": { "write": { "high": 16000, "low": 4000 } }, "only_rd": { "read": { "high": 16000, "low": 4000 } }, "simul_wr_rd": { "write": { "high": 16000, "low": 4000 }, "read": { "high": 16000, "low": 4000 } } }
Have you defined `cu_latency`? For each test mode: `only_wr`, `only_rd` and `simul_wr_rd`	"cu_latency": { "only_wr": { "write": { "high": 508, "low": 141 } }, "only_rd": { "read": { "high": 1239, "low": 666 } }, "simul_wr_rd": { "write": { "high": 232, "low": 100 }, "read": { "high": 1007, "low": 466 } } }	"cu_latency": { "only_wr": { "write": { "high": 6000, "low": 1 } }, "only_rd": { "read": { "high": 6000, "low": 1 } }, "simul_wr_rd": { "write": { "high": 6000, "low": 1 }, "read": { "high": 6000, "low": 1 } } }
Have you defined `cu_outstanding`? For each test mode: `only_wr`, `only_rd` and `simul_wr_rd`	"cu_outstanding": { "only_wr": { "write": { "nominal": 3 } }, "only_rd": { "read": { "nominal": 4 } }, "simul_wr_rd": { "write": { "nominal": 2 }, "read": { "nominal": 21 } } }	No
Have you defined `cu_burst_size`? For each test mode: `only_wr`, `only_rd` and `simul_wr_rd`	No	No

Pre-canned tests¶

Question

Answer / justification / problem tracker reference number

Have you tested all provided pre-canned tests?

If not: list the skipped one and the reason?

Yes

Have you modified the pre-canned test template?

If yes:

Why?

Was it not possible to make a platform definition update (and thus, available for any test as default)?

Update next section with your modifications.

No

Have you saved all pre-canned results?

Update package information section of your checklist.

Yes

Issues tracking¶

Question

Answer / justification / problem tracker reference number

Did you modify the platform definition limits and range after running the pre-canned tests?

If yes:

List the problem tracker reference number

No

Pre-canned tests results ¶

Pre-canned test modifications¶

Test name	Modification from original template	Justifications
/	No modification	/

`dma` pre-canned test¶

Memory	Write BW (MBps)	Read BW (MBps)
HBM	11932	12083

`p2p_card` pre-canned test¶

Memory	Write BW (MBps)	Read BW (MBps)
HBM	3723	1832

`p2p_nvme` pre-canned test¶

Memory	Card mode	Write BW (MBps)	Read BW (MBps)
HBM	source	2181	2855
HBM	target	2968	1950

`memory` pre-canned test¶

Memory	Test mode	Write BW (MBps)	Read BW (MBps)	Write Latency (ns)	Read Latency (ns)
HBM	`alternate_wr_rd`	196608	196160	208	952
	`only_wr`	392172	N/A	208	N/A
	`only_rd`	N/A	393369	N/A	952
	`simultaneous_wr_rd`	192263	192260	144	733

`memory_host` pre-canned test¶

Memory	Test mode	Write BW (MBps)	Read BW (MBps)	Write Latency (ns)	Read Latency (ns)
HOST	`alternate_wr_rd`	5229	5222	829	3835
	`only_wr`	9155	N/A	829	N/A
	`only_rd`	N/A	12178	N/A	3835
	`simultaneous_wr_rd`	9155	12006	829	3890

Checklist - xilinx-u55c-gen3x16-xdma-3¶

Platform and card requirements¶

Power rails requirements¶

Runtime¶

Sensors¶

GT¶

Memory¶

Pre-canned tests¶

Issues tracking¶

Pre-canned test modifications¶

dma pre-canned test¶

p2p_card pre-canned test¶

p2p_nvme pre-canned test¶

memory pre-canned test¶

memory_host pre-canned test¶

`dma` pre-canned test¶

`p2p_card` pre-canned test¶

`p2p_nvme` pre-canned test¶

`memory` pre-canned test¶

`memory_host` pre-canned test¶