AMR RAVE - PCIe Configuration

For common PCIe concepts, see Common Hardware Documentation

This document describes the RAVE (VE2302) PCIe implementation.

Overview

The RAVE Board implements PCIe connectivity using the PCIe Versal IP (xilinx.com:ip:pcie_versal) instantiated in programmable logic. The VE2302 device is a Versal Edge-series device that does not include a CPM (Coherent PCIe Module) hardened block, so PCIe functionality must be implemented using soft logic IP in the PL fabric.

The PCIe IP provides Gen3 x4 connectivity between the VE2302 and the on-board Ryzen processor. This implementation uses GTYP transceivers in Bank 103 and consumes approximately 29,000 LUTs of programmable logic resources. The soft logic approach provides configuration flexibility while requiring careful attention to timing closure and resource budgeting.

PL PCIe Versal IP

Note: Image shows PCIe Versal IP configured for Gen3 x4, GTYP Bank 103, device IDs 0x5700/0x5701.

IP Information

Parameter Value
IP Name PCIe Versal
Vendor xilinx.com:ip:pcie_versal
Implementation Soft logic in PL fabric
Resource Usage ~29K LUTs, ~36 BRAMs, ~26 URAMs

RAVE PCIe Configuration

Setting RAVE Value Notes
Link Speed Gen 3 (8.0 GT/s) VE2302 GTYP limitation
Lane Width x4 Connected to Ryzen via PCIe x4
GT Bank GTYP Bank 103 LPD_GTYP transceivers
Reference Clocks REFP0/REFN0, REFP1/REFN1 External oscillators
Physical Functions 2 (PF0, PF1) Standard AMR pattern
Vendor ID 0x10EE AMD/Xilinx
PF0 Device ID 0x5700 Management function
PF1 Device ID 0x5701 User/DMA function
Subsystem ID 0x000e AMD standard

GT Transceiver Pinout

GTYP Bank 103 (PCIe x4 to Ryzen):

Lane TX Pins RX Pins Purpose
Lane 0 TXP0, TXN0 RXP0, RXN0 PCIe lane 0
Lane 1 TXP1, TXN1 RXP1, RXN1 PCIe lane 1
Lane 2 TXP2, TXN2 RXP2, RXN2 PCIe lane 2
Lane 3 TXP3, TXN3 RXP3, RXN3 PCIe lane 3

Reference Clocks:

  • REFP0/REFN0 - Primary reference clock

  • REFP1/REFN1 - Secondary reference clock

Physical Function Configuration

PF0 (Management Function)

PF0 Configuration:

  • Device ID: 0x5700

  • Base Class: 12 (Processing Accelerators)

  • Sub Class: 00

  • Subsystem ID: 0x000e

PF0 BARs (configurable in PCIe IP):

  • BAR0: Management register space (size TBD)

  • Type: AXI_Bridge_Master

  • Purpose: Firmware management interface

PF1 (User/DMA Function)

PF1 Configuration:

  • Device ID: 0x5701

  • Base Class: 12 (Processing Accelerators)

  • Sub Class: 00

  • Subsystem ID: 0x000e

PF1 BARs (configurable in PCIe IP):

  • BAR0: DMA registers (XDMA IP)

  • BAR2: AXI Bridge (PL access for test/user logic)

  • MSI-X: Configured to align with XDMA IP

Note: MSI-X settings must be configured in PCIe IP to match XDMA capabilities (XDMA cannot reconfigure MSI-X independently).

NoC Connection (RAVE-Specific)

PL PCIe to NoC Routing

The PCIe IP routes traffic to the NoC through programmable logic NMU and NSU interfaces:

Connection Path:

PL PCIe IP
  ↓ (AXI Master/Slave interfaces)
NMU_512 (in AXI NoC IP)
  ↓
NoC Interconnect
  ↓
NSU_512 / Memory Controllers

AXI NoC IP Configuration:

  • Add NMU_512 interface for PCIe master traffic

  • Configure QoS (bandwidth, latency requirements)

  • Route to memory controllers (LPDDR4) and PL slaves

  • Requires manual configuration in AXI NoC IP

PCIe Clocking

Clock Sources:

  • Reference Clocks: External oscillators (REFP/REFN)

  • AXI Clock: From CIPS (pl0_ref_clk or pl2_ref_clk)

  • User Clock: PCIe IP generates user_clk output

Clock Domain Crossings:

  • PCIe IP to AXI NoC: Asynchronous crossing

  • Requires proper clock configuration in NoC IP

  • Multiple proc_sys_reset blocks for synchronization

PCIe Reset Handling

Reset Signals:

Signal Source Purpose
PCIe RST PMC MIO24 PCIe fundamental reset
WAKE_B PMC MIO25 PCIe wake signal
pl0_resetn CIPS PL reset output

Reset Synchronization:

  • PCIe reset synchronized to PCIe clock domain

  • PL reset synchronized to AXI clock domains

  • Multiple proc_sys_reset blocks for different clocks

Pin Constraints (RAVE-Specific)

XDC Constraints Required:

# GTYP Bank 103 - PCIe x4
set_property PACKAGE_PIN <pin> [get_ports {pcie_txp[0]}]
set_property PACKAGE_PIN <pin> [get_ports {pcie_txn[0]}]
# ... (lanes 0-3)

# Reference clocks
set_property PACKAGE_PIN <pin> [get_ports pcie_refclk_p]
set_property PACKAGE_PIN <pin> [get_ports pcie_refclk_n]

See: RAVE XDC files for complete pin assignments.

Performance Characteristics

RAVE PCIe Performance

| Raw Bandwidth | ~4 GB/s | ~32 GB/s | 1:8 | | Link Speed | 8.0 GT/s | 32.0 GT/s | 1:4 | | Lanes | x4 | x8 | 1:2 | | Latency | Higher (PL routing) | Lower (hardened) | - |

  • Sufficient for edge AI inference

  • Adequate for vision processing

  • May limit multi-stream video applications

  • Lower bandwidth balanced by lower cost/power

Design Complexity

Design Complexity

The RAVE design requires:

  1. ✓ PCIe IP instantiation and configuration

  2. ✓ GT transceiver setup and clocking

  3. ✓ NMU_512 NoC interface configuration

  4. ✓ Pin constraints for GT lanes

  5. ✓ Timing closure for PCIe in fabric

  6. ✓ Coordination with XDMA IP for MSI-X

Note: CPM5 handles all of this automatically in hardened logic.

Configuration Files

RAVE Design Files:

  • config_bd.tcl: Includes PCIe IP instantiation

  • impl.pins.xdc: GT lane pin assignments

  • impl.xdc: PCIe timing constraints

The soft logic implementation requires careful floorplanning and timing closure for PCIe operation in the FPGA fabric.

XDMA Configuration

The RAVE Board uses XDMA for DMA operations with the following configuration:

  • Physical Functions: 2 (PF0 for management, PF1 for user/DMA)

  • Device IDs: PF0 (0x5710), PF1 (0x5711)

  • Data Width: 256-bit AXI

  • DMA Channels: Multiple H2C and C2H channels

  • AXI Interfaces: AXI Master, AXI-Lite Master, AXI Bypass

Note: Legacy RAVE designs used QDMA with 512 queues. The board has migrated to XDMA for improved simplicity and compatibility with VE2302 Embedded+ board requirements.

See RAVE XDMA Configuration for detailed XDMA IP setup.

PCIe Debug and Testing

RAVE-Specific Tools:

  • DPDK test application with XDMA

  • PCIe link analyzer

  • Vivado ILA for PCIe signals

Reference: RAVE DPDK and XDMA Debug - Testing documentation

References