User Guide¶
This section describes the details on controlling and configuring the QDMA IP
System Level Configurations¶
QDMA driver provides the sysfs interface to enable to user to perform system level configurations. QDMA PF and VF drivers expose several sysfs nodes under the pci device root node.
Once the qdma module is inserted and until any queue is added into the system and FMAP programming is not done, sysfs provides an interface to configure parameters for the module configuration.
[xilinx@]# lspci | grep -i Xilinx
01:00.0 Memory controller: Xilinx Corporation Device 903f
01:00.1 Memory controller: Xilinx Corporation Device 913f
01:00.2 Memory controller: Xilinx Corporation Device 923f
01:00.3 Memory controller: Xilinx Corporation Device 933f
Based on the above lspci output, traverse to /sys/bus/pci/devices/<device node>/qdma to find the list of configurable parameters specific to PF or VF driver.
1. Instantiates the Virtual Functions¶
QDMA IP supports 252 Virtual Functions(VFs). /sys/bus/pci/devices/<device node> provides two configurable entries
sriov_totalvfs: Indicates the maximum number of VFs supported for PF. This is a read only entry which can be configured during bit stream generation.sriov_numvfs: Enables the user to specify the number of VFs required for a PF
Display the currently supported max VFs:
[xilinx@]# cat /sys/bus/pci/devices/0000:01:00.0/sriov_totalvfs
Instantiate the required number of VFs for a PF:
[xilinx@]# echo 3 > /sys/bus/pci/devices/0000:01:00.0/sriov_numvfs
Once the VFS are instantiated, required number of queues can be allocated the VF using qmax sysfs entry available in VF at
/sys/bus/pci/devices/<VF function number>/qdma/qmax
2. Allocate the Queues to a function¶
QDMA IP supports maximum of 2048 queues. By default there are no queues assigned to the functions.
qmax configuration parameter enables the user to update the number of queues for a PF. This configuration parameter indicates “Maximum number of queues associated for the current pf”.
If the queue allocation needs to be different for any PF, access the qmax sysfs entry and set the required number.
Once the number of queues for any PF is changed from the default value, the remaining set of queues among the 2048 queues are evenly distributed for the remaining PFs.
Display the current value:
[xilinx@]# cat /sys/bus/pci/devices/0000:01:00.0/qdma/qmax
0
Set a new value:
[xilinx@]# echo 1024 > /sys/bus/pci/devices/0000:01:00.0/qdma/qmax
[xilinx@]# dmactl dev list
qdma01000 0000:01:00.0 max QP: 1024, 0~1023
qdma01001 0000:01:00.1 max QP: 0, -~-
qdma01002 0000:01:00.2 max QP: 0, -~-
qdma01003 0000:01:00.3 max QP: 0, -~-
[xilinx@]# echo 1770 > /sys/bus/pci/devices/0000\:01\:00.0/qdma/qmax
[xilinx@]# echo 8 > /sys/bus/pci/devices/0000\:01\:00.1/qdma/qmax
[xilinx@]# echo 8 > /sys/bus/pci/devices/0000\:01\:00.2/qdma/qmax
[xilinx@]# echo 8 > /sys/bus/pci/devices/0000\:01\:00.3/qdma/qmax
[xilinx@]# dmactl dev list
qdma01000 0000:01:00.0 max QP: 1770, 0~1769
qdma01001 0000:01:00.1 max QP: 8, 1770~1777
qdma01002 0000:01:00.2 max QP: 8, 1778~1785
qdma01003 0000:01:00.3 max QP: 8, 1786~1793
qmax configuration parameter is available for virtual functions as well. Once the qmax_vfs is configured, qmax for each VF can be updated from pool of queues assigned for the VFs.
3. Reserve the Queues to VFs¶
QDMA IP supports 2048 queues and from the set of 2048 queues, use the qmax sysfs entry to allocate queues to VFs similar to PFs.
Display the current value:
[xilinx@] #cat /sys/bus/pci/devices/0000:81:00.4/qdma/qmax
0
Set a new value:
[xilinx@] #echo 1024 > /sys/bus/pci/devices/0000:81:00.4/qdma/qmax
4. Set Interrupt Ring Size¶
Interrupt ring size is associated with indirect interrupt mode.
When the module is inserted in indirect interrupt mode, by default the interrupt aggregation ring size is set 0 i.e 512 entries
User can configure he interrupt ring entries in multiples of 512 hence set the intr_rngsz with multiplication factor
Display the current value:
[xilinx@]# cat /sys/bus/pci/devices/0000:81:00.0/qdma/intr_rngsz
0
Set a new value:
[xilinx@]# echo 2 > /sys/bus/pci/devices/0000:81:00.0/qdma/intr_rngsz
5. Set Completion Interval¶
cmpt_intrvl indicated the interval at which completions are generated for an MM or H2C Stream queue running in non-bypass mode.
User can set any of the following list of values for this configuration parameter.
Completion accumulation value is calculated as 2^(register bit [2:0]). Maximum accumulation is 512. Accumulation can be disabled via queue context.
Display the current value:
[xilinx@]# cat /sys/bus/pci/devices/0000:81:00.0/qdma/cmpt_intrvl
0
Set a new value:
[xilinx@]# echo 2 > /sys/bus/pci/devices/0000:81:00.0/qdma/cmpt_intrvl
Queue Management¶
QDMA driver comes with a command-line configuration utility called dmactl to manage the queues in the system.