Concurrent Kernel Execution¶
This example will demonstrate how to use multiple and out of order command queues to simultaneously execute multiple kernels on an FPGA.
KEY CONCEPTS: Concurrent execution, Out of Order Command Queues, Multiple Command Queues
KEYWORDS: CL_QUEUE_OUT_OF_ORDER_EXEC_MODE_ENABLE, setCallback
This example illustrates two ways to implement concurrent kernel execution. 1. Multiple command enqueues 2. Single command queue with out of order execution
Design contains 3 kernels mscale
, madd
and mmult
. madd
needs to wait for mscale
to complete but mmult
can run
independently (no dependency).
multiple_command_queues
function creates two sequential command
queues, one for mscale and madd
and other for mmult
.
cl::CommandQueue ordered_queue1(context, device, CL_QUEUE_PROFILING_ENABLE, &err);
cl::CommandQueue ordered_queue2(context, device, CL_QUEUE_PROFILING_ENABLE, &err);
Both Command queues are created as in-order execution. msacle
and
madd
are enqueued in first command queue one after another. As
Command queue is in-order
, madd
will only start when mscale
will finish. So the order of execution is handled by command queue.
There is another approach of achieve the same using single out-of-order
command queue. out_of_order_queue
function creates a single command
queue with out of order execution enabled which means that enqueues in
the command queue can go out of order (without waiting for a previous
execution to finish).
cl::CommandQueue ooo_queue(context, device,
CL_QUEUE_PROFILING_ENABLE | CL_QUEUE_OUT_OF_ORDER_EXEC_MODE_ENABLE);
However, since we need that madd
kernel only starts executing after
mscale
has finished, cl::event
is used to wait for mscale
to
finish execution.
cl::Event event;
vector<cl::Event> kernel_wait_events
kernel_wait_events.push_back(event);
err = ooo_queue.enqueueNDRangeKernel(kernel_mscale, offset, global, local, nullptr, &event));
err = ooo_queue.enqueueNDRangeKernel(kernel_madd, offset, global, local,
&kernel_wait_events, // Event from previous call
nullptr);
DESIGN FILES¶
Application code is located in the src directory. Accelerator binary files will be compiled to the xclbin directory. The xclbin directory is required by the Makefile and its contents will be filled during compilation. A listing of all the files in this example is shown below
src/host.cpp
src/madd.cpp
src/mmult.cpp
src/mscale.cpp
COMMAND LINE ARGUMENTS¶
Once the environment has been configured, the application can be executed by
./concurrent_execution <matrix_ops XCLBIN>