Tensor Access Patterns (taplib)

taplib provides abstractions for describing how data is tiled and streamed between memory and AIE compute tiles. A Tensor Access Pattern (TAP) describes a multi-dimensional iteration over a buffer, generating the DMA descriptor sequences that the NPU hardware executes.

TensorAccessPattern

TensorAccessPattern(
    tensor_dims: Sequence[int],
    offset: int,
    sizes: Sequence[int],
    strides: Sequence[int],
)

A TensorAccessPattern represents a data access pattern applied to a tensor of a specific dimension.

This is a base class meant to generically represent such as transformation using sizes, strides, and an offset.

An object representing an access pattern applied to a tensor.

Parameters:

Name	Type	Description	Default
tensor_dims	Sequence[int]	Dimensions of the tensor	required
offset	int	Offset into the tensor to begin the transformation	required
sizes	Sequence[int]	Transformation sizes	required
strides	Sequence[int]	Transformation strides	required

Source code in python/helpers/taplib/tap.py

def __init__(
    self,
    tensor_dims: Sequence[int],
    offset: int,
    sizes: Sequence[int],
    strides: Sequence[int],
):
    """An object representing an access pattern applied to a tensor.

    Args:
        tensor_dims (Sequence[int]): Dimensions of the tensor
        offset (int): Offset into the tensor to begin the transformation
        sizes (Sequence[int]): Transformation sizes
        strides (Sequence[int]): Transformation strides
    """  # noqa: D401
    self._tensor_dims = validate_tensor_dims(tensor_dims)
    self._offset = validate_offset(offset, tensor_dims)
    cleaned_sizes, cleaned_strides = validate_and_clean_sizes_strides(
        sizes, strides
    )
    assert cleaned_sizes is not None and cleaned_strides is not None
    self._sizes: Sequence[int] = cleaned_sizes
    self._strides: Sequence[int] = cleaned_strides

tensor_dims property

tensor_dims: Sequence[int]

A copy of the dimensions of the tensor.

Returns:

Type	Description
Sequence[int]	Sequence[int]: Tensor dimensions

offset property

offset: int

Return the offset into the tensor.

Returns:

Name	Type	Description
int	int	offset

sizes property

sizes: Sequence[int]

A copy of the access pattern sizes.

Returns:

Type	Description
Sequence[int]	Sequence[int]: Transformation sizes

strides property

strides: Sequence[int]

A copy of the access pattern strides.

Returns:

Type	Description
Sequence[int]	Sequence[int]: Transformation strides

transformation_dims property

transformation_dims: Sequence[tuple[int, int]]

The access pattern represented as a sequence of (size, stride) tuples.

Returns:

Type	Description
Sequence[tuple[int, int]]	Sequence[tuple[int, int]]: Transformation dimensions

accesses

accesses() -> tuple[ndarray, ndarray]

Return the access_order and access_count arrays.

The access_order ndarray sequentially counts access to elements in the tensor. If an element is accessed more than once, only the last count is reflected.

The access_count ndarray contains the number of times each element is accessed by the tensor access pattern.

Returns:

Type	Description
tuple[ndarray, ndarray]	tuple[np.ndarray, np.ndarray]: access_order, access_count

Source code in python/helpers/taplib/tap.py

def accesses(self) -> tuple[np.ndarray, np.ndarray]:
    """Return the access_order and access_count arrays.

    The access_order ndarray sequentially counts access to elements in the
    tensor. If an element is accessed more than once, only the last count is reflected.

    The access_count ndarray contains the number of times each element is
    accessed by the tensor access pattern.

    Returns:
        tuple[np.ndarray, np.ndarray]: access_order, access_count
    """
    return self._calculate_accesses(calc_order=True, calc_count=True)

access_order

access_order() -> ndarray

Return the access_order ndarray, which sequentially counts access to elements in the tensor.

If an element is accessed more than once, only the last count is reflected.

Returns:

Type	Description
ndarray	np.ndarray: access_order

Source code in python/helpers/taplib/tap.py

def access_order(self) -> np.ndarray:
    """Return the access_order ndarray, which sequentially counts access to elements in the tensor.

    If an element is accessed more than once, only the last count is reflected.

    Returns:
        np.ndarray: access_order
    """
    access_order_tensor, _ = self._calculate_accesses(
        calc_order=True, calc_count=False
    )
    return access_order_tensor

access_count

access_count() -> ndarray

Return the access_count ndarray, which contains the number of times each element is accessed.

Returns:

Type	Description
ndarray	np.ndarray: access_count

Source code in python/helpers/taplib/tap.py

def access_count(self) -> np.ndarray:
    """Return the access_count ndarray, which contains the number of times each element is accessed.

    Returns:
        np.ndarray: access_count
    """
    _, access_count_tensor = self._calculate_accesses(
        calc_order=False, calc_count=True
    )
    return access_count_tensor

access_generator

access_generator() -> Generator[int, None, None]

Return an iterator over the access indices into the flattened tensor that this access pattern represents.

This can be used to calculate the access count or to enumerate accesses.

Yields:

Name	Type	Description
int	int	The next access index

Source code in python/helpers/taplib/tap.py

def access_generator(self) -> Generator[int, None, None]:
    """Return an iterator over the access indices into the flattened tensor that this access pattern represents.

    This can be used to calculate the access count or to enumerate accesses.

    Yields:
        int: The next access index
    """
    total_elems = np.prod(self._tensor_dims)

    # Use itertools.product to collapse len(sizes) nested forloop into one forloop
    for dims in itertools.product(*[range(0, n) for n in self._sizes]):
        yield (
            self._offset + np.sum(np.multiply(dims, self._strides))
        ) % total_elems

compare_access_orders

compare_access_orders(other: TensorAccessPattern) -> bool

Compare access patterns for functional equivalency.

Sometimes access patterns with different sizes/strides are functionally equivalent; to detect functional equivalency, this function uses iterators produced by access_generator() to compare the access patterns. This is more performant than comparing the numpy array access_order or access_count tensors.

Parameters:

Name	Type	Description	Default
other	TensorAccessPattern	The TensorAccessPattern to compare to	required

Raises:

Type	Description
ValueError	other must be of type TensorAccessPattern

Returns:

Name	Type	Description
bool	bool	True if the TensorAccessPatterns are functionally equivalent; false otherwise.

Source code in python/helpers/taplib/tap.py

def compare_access_orders(self, other: TensorAccessPattern) -> bool:
    """Compare access patterns for functional equivalency.

    Sometimes access patterns with different sizes/strides are functionally equivalent;
    to detect functional equivalency, this function uses iterators produced by
    access_generator() to compare the access patterns. This is more performant than
    comparing the numpy array access_order or access_count tensors.

    Args:
        other (TensorAccessPattern): The TensorAccessPattern to compare to

    Raises:
        ValueError: other must be of type TensorAccessPattern

    Returns:
        bool: True if the TensorAccessPatterns are functionally equivalent; false otherwise.
    """
    # This function compares using access generators, which is more performant
    # than actually generating the access order or access count tensors.
    if not isinstance(other, TensorAccessPattern):
        raise ValueError(
            "Can only compare access order against another TensorAccessPattern"
        )
    my_generator = self.access_generator()
    other_generator = other.access_generator()
    return all(
        my_idx == other_idx
        for my_idx, other_idx in itertools.zip_longest(
            my_generator, other_generator
        )
    )

visualize

visualize(
    show_arrows: bool | None = None,
    title: str | None = None,
    file_path: str | None = None,
    show_plot: bool = True,
    plot_access_count: bool = False,
) -> None

Visualize the TensorAccessPattern using a graph.

Parameters:

Name	Type	Description	Default
show_arrows	bool | None	Display arrows between sequentially accessed elements. Defaults to None.	None
title	str | None	Title of the produced graph. Defaults to None.	None
file_path	str | None	Path to save the graph at; if none, it is not saved. Defaults to None.	None
show_plot	bool	Show the plot (this is useful for Jupyter notebooks). Defaults to True.	True
plot_access_count	bool	Plot the access count in addition to the access order. Defaults to False.	False

Raises:

Type	Description
NotImplementedError	This function is not implemented for all dimensions.

Source code in python/helpers/taplib/tap.py

def visualize(
    self,
    show_arrows: bool | None = None,
    title: str | None = None,
    file_path: str | None = None,
    show_plot: bool = True,
    plot_access_count: bool = False,
) -> None:
    """Visualize the TensorAccessPattern using a graph.

    Args:
        show_arrows (bool | None, optional): Display arrows between sequentially accessed elements. Defaults to None.
        title (str | None, optional): Title of the produced graph. Defaults to None.
        file_path (str | None, optional): Path to save the graph at; if none, it is not saved. Defaults to None.
        show_plot (bool, optional): Show the plot (this is useful for Jupyter notebooks). Defaults to True.
        plot_access_count (bool, optional): Plot the access count in addition to the access order. Defaults to False.

    Raises:
        NotImplementedError: This function is not implemented for all dimensions.
    """
    from .visualization2d import visualize_from_accesses

    if len(self._tensor_dims) != 2:
        raise NotImplementedError(
            "Visualization is only currently supported for 1- or 2-dimensional tensors"
        )
    if plot_access_count:
        access_order, access_count = self.accesses()
    else:
        access_count = None
        access_order = self.access_order()
    if title is None:
        title = str(self)
    visualize_from_accesses(
        access_order,
        access_count,
        title=title,
        show_arrows=show_arrows,
        file_path=file_path,
        show_plot=show_plot,
    )

TensorAccessSequence

TensorAccessSequence(
    tensor_dims: Sequence[int],
    num_steps: int,
    offset: int | None = None,
    sizes: Sequence[int] | None = None,
    strides: Sequence[int] | None = None,
    offset_fn: Callable[[int, int], int] | None = None,
    sizes_fn: (
        Callable[[int, Sequence[int]], Sequence[int]] | None
    ) = None,
    strides_fn: (
        Callable[[int, Sequence[int]], Sequence[int]] | None
    ) = None,
)

Bases: MutableSequence, Iterable

TensorAccessSequence is a MutableSequence and an Iterable. Generally, it is a thin wrapper around a list[TensorAccessPattern].

The TensorAccessSequence is useful as a container of TensorAccessPatterns so that a grouping of patterns may be accessed in a particular order, or visualized or animated in sequence.

A TensorAccessSequence is a sequence of TensorAccessPatterns modelled after a list.

The constructor can used given functions to generate a sequence of n steps. Allowed functions are the: * offset_fn(step: int, current_offset: int) -> new_offset: int * sizes_fn(step: int, current_sizes: Sequence[int]) -> new_sizes: Sequence[int] * strides_fn(step: int, currrent_strides: Sequence[int]) -> new_strides: Sequence[int]

In lieu or in addition to a function, a default value for sizes/strides/offsets may also be set.

Parameters:

Name	Type	Description	Default
tensor_dims	Sequence[int]	Dimensions of the tensor. All TensorAccessPatterns in the sequence must share the tensor dimension.	required
num_steps	int	Number of steps (elements) in the sequence.	required
offset	int | None	Offset into the sequence. Defaults to None.	None
sizes	Sequence[int] | None	Sizes for the TensorAccessPatterns Defaults to None.	None
strides	Sequence[int] | None	Strides for the TensorAccessPatterns in the sequence. Defaults to None.	None
offset_fn	Callable[[int, int], int] | None	A function to calculate the offset at each step. Defaults to None.	None
sizes_fn	Callable[[int, Sequence[int]], Sequence[int]] | None	A function to calculate the sizes at each step. Defaults to None.	None
strides_fn	Callable[[int, Sequence[int]], Sequence[int]] | None	A function to calculate the strides at teach step. Defaults to None.	None

Raises:

Type	Description
ValueError	Parameters are validated

Source code in python/helpers/taplib/tas.py

def __init__(
    self,
    tensor_dims: Sequence[int],
    num_steps: int,
    offset: int | None = None,
    sizes: Sequence[int] | None = None,
    strides: Sequence[int] | None = None,
    offset_fn: Callable[[int, int], int] | None = None,
    sizes_fn: Callable[[int, Sequence[int]], Sequence[int]] | None = None,
    strides_fn: Callable[[int, Sequence[int]], Sequence[int]] | None = None,
):
    """A TensorAccessSequence is a sequence of TensorAccessPatterns modelled after a list.

    The constructor can used given functions to generate a sequence of n steps. Allowed functions are the:
    * offset_fn(step: int, current_offset: int) -> new_offset: int
    * sizes_fn(step: int, current_sizes: Sequence[int]) -> new_sizes: Sequence[int]
    * strides_fn(step: int, currrent_strides: Sequence[int]) -> new_strides: Sequence[int]

    In lieu or in addition to a function, a default value for sizes/strides/offsets may also be set.

    Args:
        tensor_dims (Sequence[int]): Dimensions of the tensor. All TensorAccessPatterns in the sequence must share the tensor dimension.
        num_steps (int): Number of steps (elements) in the sequence.
        offset (int | None, optional): Offset into the sequence. Defaults to None.
        sizes (Sequence[int] | None, optional): Sizes for the TensorAccessPatterns Defaults to None.
        strides (Sequence[int] | None, optional): Strides for the TensorAccessPatterns in the sequence. Defaults to None.
        offset_fn (Callable[[int, int], int] | None, optional): A function to calculate the offset at each step. Defaults to None.
        sizes_fn (Callable[[int, Sequence[int]], Sequence[int]] | None, optional): A function to calculate the sizes at each step. Defaults to None.
        strides_fn (Callable[[int, Sequence[int]], Sequence[int]] | None, optional): A function to calculate the strides at teach step. Defaults to None.

    Raises:
        ValueError: Parameters are validated
    """  # noqa: D401
    self._current_step = 0

    # Check tensor dims, offset, sizes, strides
    self._tensor_dims = validate_tensor_dims(tensor_dims)
    if offset is not None:
        offset = validate_offset(offset, self._tensor_dims)
    sizes, strides = validate_and_clean_sizes_strides(
        sizes, strides, allow_none=True
    )

    # Validate and set num steps
    if num_steps < 0:
        raise ValueError(f"Number of steps must be positive (but is {num_steps})")

    if num_steps == 0:
        if (
            offset is not None
            or sizes is not None
            or strides is not None
            or offset_fn is not None
            or sizes_fn is not None
            or strides_fn is not None
        ):
            raise ValueError(
                "If num_steps=0, no sizes/strides/offset information may be specified"
            )
        self._taps = []
    else:
        # Make sure values or not None if iteration functions are None; also set default iter fn
        if offset_fn is not None:
            resolved_offset_fn = offset_fn
        else:
            if offset is None:
                raise ValueError("Offset must be provided if offset_fn is None")
            resolved_offset_fn = _constant_fn(offset)

        if sizes_fn is not None:
            resolved_sizes_fn = sizes_fn
        else:
            if sizes is None:
                raise ValueError("Sizes must be provided if size_fn is None")
            resolved_sizes_fn = _constant_fn(sizes)

        if strides_fn is not None:
            resolved_strides_fn = strides_fn
        else:
            if strides is None:
                raise ValueError("Strides must be provided if stride_fn is None")
            resolved_strides_fn = _constant_fn(strides)

        # Pre-calculate taps, because better for error handling up-front (and for visualizing full iter)
        # This is somewhat against the mentality behind iterations, but should be okay at the scale this
        # class will be used for (e.g., no scalability concerns with keeping all taps in mem)
        self._taps = []
        cur_offset: Any = offset
        cur_sizes: Any = sizes
        cur_strides: Any = strides
        for step in range(num_steps):
            cur_offset = resolved_offset_fn(step, cur_offset)
            cur_sizes = resolved_sizes_fn(step, cur_sizes)
            cur_strides = resolved_strides_fn(step, cur_strides)

            self._taps.append(
                TensorAccessPattern(
                    self._tensor_dims,
                    cur_offset,
                    cur_sizes,
                    cur_strides,
                )
            )

from_taps classmethod

from_taps(
    taps: Sequence[TensorAccessPattern],
) -> TensorAccessSequence

Create a TensorAccessSequence from a sequence of TensorAccessPatterns.

This is an alternative to the traditional constructor, and is useful for patterns that are difficult to express using the sizes/strides/offset functions.

Parameters:

Name	Type	Description	Default
taps	Sequence[TensorAccessPattern]	The sequence of tensor access patterns	required

Raises:

Type	Description
ValueError	At least one TensorAccessPattern must be specified
ValueError	All TensorAccessPatterns in a sequence must share tensor dimensions

Returns:

Name	Type	Description
TensorAccessSequence	TensorAccessSequence	A newly constructor TensorAccessSequence object

Source code in python/helpers/taplib/tas.py

@classmethod
def from_taps(cls, taps: Sequence[TensorAccessPattern]) -> TensorAccessSequence:
    """Create a TensorAccessSequence from a sequence of TensorAccessPatterns.

    This is an alternative to the traditional constructor, and is useful for patterns that are difficult
    to express using the sizes/strides/offset functions.

    Args:
        taps (Sequence[TensorAccessPattern]): The sequence of tensor access patterns

    Raises:
        ValueError: At least one TensorAccessPattern must be specified
        ValueError: All TensorAccessPatterns in a sequence must share tensor dimensions

    Returns:
        TensorAccessSequence: A newly constructor TensorAccessSequence object
    """
    if len(taps) < 1:
        raise ValueError(
            "Received no TensorAccessPatterns; must have at least one TensorAccessPatterns to create a TensorAccessSequence."
        )
    tensor_dims = taps[0].tensor_dims
    for t in taps:
        if t.tensor_dims != tensor_dims:
            raise ValueError(
                f"TensorAccessPatterns have multiple tensor dimensions (found {tensor_dims} and {t.tensor_dims})"
            )
    tas = cls(
        tensor_dims,
        num_steps=1,
        offset=taps[0].offset,
        sizes=taps[0].sizes,
        strides=taps[0].strides,
    )
    for t in taps[1:]:
        tas.append(t)
    return tas

accesses

accesses() -> tuple[ndarray, ndarray]

Return the access_order and access_count arrays of the sequence applied sequentially to the tensor.

The access_order ndarray sequentially counts access to elements in the tensor. If an element is accessed more than once, only the last count is reflected.

The access_count ndarray contains the number of times each element is accessed by the tensor access pattern.

Returns:

Type	Description
tuple[ndarray, ndarray]	tuple[np.ndarray, np.ndarray]: access_order, access_count

Source code in python/helpers/taplib/tas.py

def accesses(self) -> tuple[np.ndarray, np.ndarray]:
    """Return the access_order and access_count arrays of the sequence applied sequentially to the tensor.

    The access_order ndarray sequentially counts access to elements in the
    tensor. If an element is accessed more than once, only the last count is reflected.

    The access_count ndarray contains the number of times each element is
    accessed by the tensor access pattern.

    Returns:
        tuple[np.ndarray, np.ndarray]: access_order, access_count
    """
    return self._calc_accesses(True, True)

access_order

access_order() -> ndarray

Return the access_order ndarray, which sequentially counts access to elements in the tensor.

If an element is accessed more than once, only the last count is reflected. The TensorAccessPatterns in the sequence are applied sequentially.

Returns:

Type	Description
ndarray	np.ndarray: access_order

Source code in python/helpers/taplib/tas.py

def access_order(self) -> np.ndarray:
    """Return the access_order ndarray, which sequentially counts access to elements in the tensor.

    If an element is accessed more than once, only the last count is reflected.
    The TensorAccessPatterns in the sequence are applied sequentially.

    Returns:
        np.ndarray: access_order
    """
    access_order, _ = self._calc_accesses(calc_order=True, calc_count=False)
    return access_order

access_count

access_count() -> ndarray

Return the access_count ndarray, which contains the number of times each element is accessed.

The TensorAccessPatterns in the sequence are applied sequentially.

Returns:

Type	Description
ndarray	np.ndarray: access_count

Source code in python/helpers/taplib/tas.py

def access_count(self) -> np.ndarray:
    """Return the access_count ndarray, which contains the number of times each element is accessed.

    The TensorAccessPatterns in the sequence are applied sequentially.

    Returns:
        np.ndarray: access_count
    """
    _, access_count = self._calc_accesses(calc_order=False, calc_count=True)
    return access_count

animate

animate(
    title: str | None = None,
    animate_access_count: bool = False,
) -> "FuncAnimation"

Create and return a handle to a TensorAccessSequence animation.

Each frame in the animation represents one TensorAccessPattern in the sequence.

Parameters:

Name	Type	Description	Default
title	str | None	The title of the animation. Defaults to None.	None
animate_access_count	bool	Create an animation for the tensor access count, in addition to the tensor access order. Defaults to False.	False

Raises:

Type	Description
NotImplementedError	Not all dimensions of tensor may be visualized by animation at this time.

Returns:

Type	Description
'FuncAnimation'	animation.FuncAnimation: A handle to the animation, produced by the matplotlib.animation module.

Source code in python/helpers/taplib/tas.py

def animate(
    self, title: str | None = None, animate_access_count: bool = False
) -> "FuncAnimation":
    """Create and return a handle to a TensorAccessSequence animation.

    Each frame in the animation represents one TensorAccessPattern in the sequence.

    Args:
        title (str | None, optional): The title of the animation. Defaults to None.
        animate_access_count (bool, optional): Create an animation for the tensor access count, in addition to the tensor access order. Defaults to False.

    Raises:
        NotImplementedError: Not all dimensions of tensor may be visualized by animation at this time.

    Returns:
        animation.FuncAnimation: A handle to the animation, produced by the matplotlib.animation module.
    """
    from .visualization2d import animate_from_accesses

    if len(self._tensor_dims) != 2:
        raise NotImplementedError(
            "Visualization is only currently supported for 1- or 2-dimensional tensors"
        )

    if title is None:
        title = "TensorAccessSequence Animation"
    total_elems = np.prod(self._tensor_dims)

    animate_order_frames = [
        np.full(total_elems, -1, TensorAccessPattern._DTYPE).reshape(
            self._tensor_dims
        )
    ]

    animate_count_frames: list[np.ndarray] | None = None
    if animate_access_count:
        animate_count_frames = [
            np.full(total_elems, 0, TensorAccessPattern._DTYPE).reshape(
                self._tensor_dims
            )
        ]

    for t in self._taps:
        if animate_count_frames is not None:
            t_access_order, t_access_count = t.accesses()
            animate_count_frames.append(t_access_count)
        else:
            t_access_order = t.access_order()
        animate_order_frames.append(t_access_order)

    return animate_from_accesses(
        animate_order_frames,
        animate_count_frames,
        title=title,
    )

visualize

visualize(
    title: str | None = None,
    file_path: str | None = None,
    show_plot: bool = True,
    plot_access_count: bool = False,
) -> None

Provide a visual of the TensorAccessSequence using a graph.

Parameters:

Name	Type	Description	Default
title	str | None	The title of the graph. Defaults to None.	None
file_path	str | None	The path to save the graph at. If None, it is not saved. Defaults to None.	None
show_plot	bool	Show the plot; this is useful when running in a Jupyter notebook. Defaults to True.	True
plot_access_count	bool	Plot the access count in addition to the access order. Defaults to False.	False

Raises:

Type	Description
NotImplementedError	Not all dimensions of tensor may be visualized.

Source code in python/helpers/taplib/tas.py

def visualize(
    self,
    title: str | None = None,
    file_path: str | None = None,
    show_plot: bool = True,
    plot_access_count: bool = False,
) -> None:
    """Provide a visual of the TensorAccessSequence using a graph.

    Args:
        title (str | None, optional): The title of the graph. Defaults to None.
        file_path (str | None, optional): The path to save the graph at. If None, it is not saved. Defaults to None.
        show_plot (bool, optional): Show the plot; this is useful when running in a Jupyter notebook. Defaults to True.
        plot_access_count (bool, optional): Plot the access count in addition to the access order. Defaults to False.

    Raises:
        NotImplementedError: Not all dimensions of tensor may be visualized.
    """
    from .visualization2d import visualize_from_accesses

    if len(self._tensor_dims) != 2:
        raise NotImplementedError(
            "Visualization is only currently supported for 1- or 2-dimensional tensors"
        )

    if title is None:
        title = "TensorAccessSequence"
    if plot_access_count:
        access_order_tensor, access_count_tensor = self.accesses()
    else:
        access_order_tensor = self.access_order()
        access_count_tensor = None

    visualize_from_accesses(
        access_order_tensor,
        access_count_tensor,
        title=title,
        show_arrows=False,
        file_path=file_path,
        show_plot=show_plot,
    )

compare_access_orders

compare_access_orders(other: TensorAccessSequence) -> bool

Compare access pattern sequences for functional equivalency.

Sometimes access patterns with different sizes/strides are functionally equivalent; to detect functional equivalency, this function uses iterators produced by access_generator() to compare the access patterns. This is more performant than comparing the numpy array access_order or access_count tensors, particularly when comparing sequences containing multiple tensor access patterns.

Parameters:

Name	Type	Description	Default
other	TensorAccessSequence	The TensorAccessSequence to compare to	required

Returns:

Name	Type	Description
bool	bool	True is functionally equivalent; False otherwise.

Source code in python/helpers/taplib/tas.py

def compare_access_orders(self, other: TensorAccessSequence) -> bool:
    """Compare access pattern sequences for functional equivalency.

    Sometimes access patterns with different sizes/strides are functionally equivalent;
    to detect functional equivalency, this function uses iterators produced by
    access_generator() to compare the access patterns. This is more performant than
    comparing the numpy array access_order or access_count tensors, particularly
    when comparing sequences containing multiple tensor access patterns.

    Args:
        other (TensorAccessSequence): The TensorAccessSequence to compare to

    Returns:
        bool: True is functionally equivalent; False otherwise.
    """
    if len(self._taps) != len(other._taps):
        return False
    for my_tap, other_tap in zip(self._taps, other._taps):
        if not my_tap.compare_access_orders(other_tap):
            return False
    return True

TensorTiler2D

TensorTiler2D()

A generator (similar to factory pattern) class which produces TensorAccessSequences for common 2-dimensional tiling patterns.

Source code in python/helpers/taplib/tensortiler2d.py

def __init__(self):
    raise Exception(
        f"{self.__class__} cannot be instantiated. Use it as a factory/generator of TensorAccessSequences."
    )

simple_tiler classmethod

simple_tiler(
    tensor_dims: Sequence[int],
    tile_dims: Sequence[int] | None = None,
    tile_col_major: bool = False,
    iter_col_major: bool = False,
    pattern_repeat: int = 1,
    prune_step: bool = True,
) -> TensorAccessSequence

Produce a TensorAccessSequence with one TensorAccessPattern per tile.

The simple_tiler is a special case of the group_tiler.

Parameters:

Name	Type	Description	Default
tensor_dims	Sequence[int]	The dimensions of the tensor to tile.	required
tile_dims	Sequence[int] | None	The dimension of the tile. If None, the tile_dims is set equal to the tensor_dims. Defaults to None.	None
tile_col_major	bool	Iterate column major within each tile. Defaults to False.	False
iter_col_major	bool	Iterate column major over tiles within the TensorAccessSequence. Defaults to False.	False
pattern_repeat	int	Access a tile n times per TensorAccessPattern. Defaults to 1.	1
prune_step	bool	Prune the iteration steps in the tiling process. Defaults to True.	True

Returns:

Name	Type	Description
TensorAccessSequence	TensorAccessSequence	A TensorAccessSequence with one TensorAccessPattern per tile

Source code in python/helpers/taplib/tensortiler2d.py

@classmethod
def simple_tiler(
    cls,
    tensor_dims: Sequence[int],
    tile_dims: Sequence[int] | None = None,
    tile_col_major: bool = False,
    iter_col_major: bool = False,
    pattern_repeat: int = 1,
    prune_step: bool = True,
) -> TensorAccessSequence:
    """Produce a TensorAccessSequence with one TensorAccessPattern per tile.

    The simple_tiler is a special case of the group_tiler.

    Args:
        tensor_dims (Sequence[int]): The dimensions of the tensor to tile.
        tile_dims (Sequence[int] | None, optional): The dimension of the tile. If None, the tile_dims is set equal to the tensor_dims. Defaults to None.
        tile_col_major (bool, optional): Iterate column major within each tile. Defaults to False.
        iter_col_major (bool, optional): Iterate column major over tiles within the TensorAccessSequence. Defaults to False.
        pattern_repeat (int, optional): Access a tile n times per TensorAccessPattern. Defaults to 1.
        prune_step (bool, optional): Prune the iteration steps in the tiling process. Defaults to True.

    Returns:
        TensorAccessSequence: A TensorAccessSequence with one TensorAccessPattern per tile
    """
    if tile_dims is None:
        tile_dims = deepcopy(tensor_dims)
    # Special case of group_tiler
    return cls.group_tiler(
        tensor_dims=tensor_dims,
        tile_dims=tile_dims,
        tile_col_major=tile_col_major,
        iter_col_major=iter_col_major,
        pattern_repeat=pattern_repeat,
        prune_step=prune_step,
    )

group_tiler classmethod

group_tiler(
    tensor_dims: Sequence[int],
    tile_dims: Sequence[int],
    tile_group_dims: Sequence[int] | None = None,
    tile_col_major: bool = False,
    tile_group_col_major: bool = False,
    iter_col_major: bool = False,
    pattern_repeat: int = 1,
    allow_partial: bool = False,
    prune_step: bool = True,
) -> TensorAccessSequence

Produce a TensorAccessSequence with a group of tiles per TensorAccessPattern in the sequence.

The group_tiler is a special case of the step_tiler.

Parameters:

Name	Type	Description	Default
tensor_dims	Sequence[int]	The dimensions of the tensor to tile.	required
tile_dims	Sequence[int]	The dimension of the tile (a contiguous group of elements)	required
tile_group_dims	Sequence[int] | None	Dimensions of the grouping of tiles, specified by number of tiles (not elements). If None, assumed to be (1, 1). Defaults to None.	None
tile_col_major	bool	Iterate column major within each tile. Defaults to False.	False
tile_group_col_major	bool	Iterate column major between tiles in a group within a TensorAccessSequence. Defaults to False.	False
iter_col_major	bool	Iterate column major over tiles within the TensorAccessSequence. Defaults to False.	False
pattern_repeat	int	Apply a pattern n times within a single TensorAccessPattern. Defaults to 1.	1
allow_partial	bool	While a tensor must decompose into tiles easily, a tensor may not decompose into tile groups evenly. If True, uneven groups are allowed. If false, an exception will be thrown. Defaults to False.	False
prune_step	bool	Prune the iteration steps in the tiling process. Defaults to True.	True

Returns:

Name	Type	Description
TensorAccessSequence	TensorAccessSequence	A TensorAccessSequence with one tile grouping per TensorAccessPattern

Source code in python/helpers/taplib/tensortiler2d.py

@classmethod
def group_tiler(
    cls,
    tensor_dims: Sequence[int],
    tile_dims: Sequence[int],
    tile_group_dims: Sequence[int] | None = None,
    tile_col_major: bool = False,
    tile_group_col_major: bool = False,
    iter_col_major: bool = False,
    pattern_repeat: int = 1,
    allow_partial: bool = False,
    prune_step: bool = True,
) -> TensorAccessSequence:
    """Produce a TensorAccessSequence with a group of tiles per TensorAccessPattern in the sequence.

    The group_tiler is a special case of the step_tiler.

    Args:
        tensor_dims (Sequence[int]): The dimensions of the tensor to tile.
        tile_dims (Sequence[int]): The dimension of the tile (a contiguous group of elements)
        tile_group_dims (Sequence[int] | None, optional): Dimensions of the grouping of tiles, specified by number of tiles (not elements).
            If None, assumed to be (1, 1). Defaults to None.
        tile_col_major (bool, optional): Iterate column major within each tile. Defaults to False.
        tile_group_col_major (bool, optional): Iterate column major between tiles in a group within a TensorAccessSequence. Defaults to False.
        iter_col_major (bool, optional): Iterate column major over tiles within the TensorAccessSequence. Defaults to False.
        pattern_repeat (int, optional): Apply a pattern n times within a single TensorAccessPattern. Defaults to 1.
        allow_partial (bool, optional): While a tensor must decompose into tiles easily, a tensor may not decompose into tile groups evenly.
            If True, uneven groups are allowed. If false, an exception will be thrown. Defaults to False.
        prune_step (bool, optional): Prune the iteration steps in the tiling process. Defaults to True.

    Returns:
        TensorAccessSequence: A TensorAccessSequence with one tile grouping per TensorAccessPattern
    """
    if tile_group_dims is None:
        tile_group_dims = (1,) * cls._NUM_DIMS
    # Special case of step_tiler
    return cls.step_tiler(
        tensor_dims=tensor_dims,
        tile_dims=tile_dims,
        tile_group_repeats=tile_group_dims,
        tile_col_major=tile_col_major,
        tile_group_col_major=tile_group_col_major,
        iter_col_major=iter_col_major,
        pattern_repeat=pattern_repeat,
        allow_partial=allow_partial,
        prune_step=prune_step,
    )

step_tiler classmethod

step_tiler(
    tensor_dims: Sequence[int],
    tile_dims: Sequence[int],
    tile_group_repeats: Sequence[int],
    tile_group_steps: Sequence[int] | None = None,
    tile_col_major: bool = False,
    tile_group_col_major: bool = False,
    iter_col_major: bool = False,
    allow_partial: bool = False,
    pattern_repeat: int = 1,
    prune_step: bool = True,
) -> TensorAccessSequence

Build a TensorAccessSequence by tiling a tensor with explicit per-dimension steps.

This is the general-purpose tiler that the simpler simple_tiler / group_tiler factories delegate to. It produces one TensorAccessPattern per tile group, giving full control over tile size, group repeats, per-dimension step, and iteration order.

Parameters:

Name	Type	Description	Default
tensor_dims	Sequence[int]	The dimensions of the tensor to tile.	required
tile_dims	Sequence[int]	The dimension of the tile (a contiguous group of elements)	required
tile_group_repeats	Sequence[int]	Number of times a tile appears in each dimension in each TensorAccessPattern.	required
tile_group_steps	Sequence[int] | None	Space between each tile repeat in each dimension, given in units of tile size. Defaults to None.	None
tile_col_major	bool	Iterate column major within each tile. Defaults to False.	False
tile_group_col_major	bool	Iterate column major between tiles in a group within a TensorAccessSequence. Defaults to False.	False
iter_col_major	bool	Iterate column major over tiles within the TensorAccessSequence. Defaults to False.	False
allow_partial	bool	Whether to allow partial tile groups. A tensor always decomposes into tiles evenly, but it may not decompose into tile groups evenly. When False, a partial tile grouping raises a ValueError; when True, partial groupings at the tensor edges are permitted. Defaults to False.	False
pattern_repeat	int	Apply the access pattern n times within a single TensorAccessPattern. Defaults to 1.	1
prune_step	bool	Prune the iteration steps in the tiling process. Defaults to True.	True

Raises:

Type	Description
ValueError	The parameters are validated
ValueError	Some transformations are not expressible in only 4 dimensions of sizes/strides
ValueError	If allow_partial is False, an error will be thrown if partial TensorAccessPatterns are needed to fully tile the tensor.

Returns:

Name	Type	Description
TensorAccessSequence	TensorAccessSequence	A TensorAccessSequence with one tile grouping per TensorAccessPattern, where the tile grouping may or may not be contiguous.

Source code in python/helpers/taplib/tensortiler2d.py

@classmethod
def step_tiler(
    cls,
    tensor_dims: Sequence[int],
    tile_dims: Sequence[int],
    tile_group_repeats: Sequence[int],
    tile_group_steps: Sequence[int] | None = None,
    tile_col_major: bool = False,
    tile_group_col_major: bool = False,
    iter_col_major: bool = False,
    allow_partial: bool = False,
    pattern_repeat: int = 1,
    prune_step: bool = True,
) -> TensorAccessSequence:
    """Build a TensorAccessSequence by tiling a tensor with explicit per-dimension steps.

    This is the general-purpose tiler that the simpler ``simple_tiler`` /
    ``group_tiler`` factories delegate to. It produces one
    TensorAccessPattern per tile group, giving full control over tile
    size, group repeats, per-dimension step, and iteration order.

    Args:
        tensor_dims (Sequence[int]): The dimensions of the tensor to tile.
        tile_dims (Sequence[int]): The dimension of the tile (a contiguous group of elements)
        tile_group_repeats (Sequence[int]): Number of times a tile appears in each dimension in each TensorAccessPattern.
        tile_group_steps (Sequence[int] | None, optional): Space between each tile repeat in each dimension, given in units of tile size. Defaults to None.
        tile_col_major (bool, optional): Iterate column major within each tile. Defaults to False.
        tile_group_col_major (bool, optional): Iterate column major between tiles in a group within a TensorAccessSequence. Defaults to False.
        iter_col_major (bool, optional): Iterate column major over tiles within the TensorAccessSequence. Defaults to False.
        allow_partial (bool, optional): Whether to allow partial tile groups. A tensor
            always decomposes into tiles evenly, but it may not decompose into tile
            *groups* evenly. When False, a partial tile grouping raises a ValueError;
            when True, partial groupings at the tensor edges are permitted. Defaults
            to False.
        pattern_repeat (int, optional): Apply the access pattern n times within a single TensorAccessPattern. Defaults to 1.
        prune_step (bool, optional): Prune the iteration steps in the tiling process. Defaults to True.

    Raises:
        ValueError: The parameters are validated
        ValueError: Some transformations are not expressible in only 4 dimensions of sizes/strides
        ValueError: If allow_partial is False, an error will be thrown if partial TensorAccessPatterns are needed to fully tile the tensor.

    Returns:
        TensorAccessSequence: A TensorAccessSequence with one tile grouping per TensorAccessPattern,
            where the tile grouping may or may not be contiguous.
    """
    # Validate dimensions
    if tile_group_steps is None:
        tile_group_steps = (1,) * cls._NUM_DIMS
    tensor_dims = validate_tensor_dims(tensor_dims, expected_dims=cls._NUM_DIMS)
    tile_dims = validate_tensor_dims(tile_dims, expected_dims=cls._NUM_DIMS)
    tile_group_repeats = validate_tensor_dims(
        tile_group_repeats, expected_dims=cls._NUM_DIMS
    )
    tile_group_steps = validate_tensor_dims(
        tile_group_steps, expected_dims=cls._NUM_DIMS
    )

    # Check tensor is tileable by tile size
    for i, (tensor_dim, tile_dim) in enumerate(zip(tensor_dims, tile_dims)):
        if tensor_dim % tile_dim != 0:
            raise ValueError(
                f"Tensor dimension {i} ({tensor_dim}) is not divisible by tile dim ({tile_dim})"
            )

    # Validate pattern repeat and check for partial patterns
    if not isinstance(pattern_repeat, int) or pattern_repeat < 1:
        raise ValueError(f"Pattern repeat must be >= 1 but is {pattern_repeat}")
    if not allow_partial:
        for i, (tensor_dim, tile_dim, repeat_dim, step_dim) in enumerate(
            zip(tensor_dims, tile_dims, tile_group_repeats, tile_group_steps)
        ):
            if tensor_dim % (tile_dim * repeat_dim * step_dim) != 0:
                raise ValueError(
                    f"allow_partial={allow_partial} but tensor does not divide evenly into tile groups in dimension {i}"
                )
            if tile_dim * repeat_dim * step_dim > tensor_dim:
                raise ValueError(
                    f"Tile pattern exceeds tensor size in dimension {i} ({tile_dim}x{repeat_dim}x{step_dim} > {tensor_dim})"
                )

    # Prune steps/repeat if steps/repeat larger than initial tensor
    tile_group_steps = list(tile_group_steps)
    tile_group_repeats = list(tile_group_repeats)
    for i, (tensor_dim, tile_dim, tile_step) in enumerate(
        zip(tensor_dims, tile_dims, tile_group_steps)
    ):
        tile_group_repeats[i] = min(
            tile_group_repeats[i], ceildiv(tensor_dim // tile_dim, tile_step)
        )
        if tile_step > tensor_dim // tile_dim:
            tile_group_steps[i] = 1

    # Calculate the number of steps (number of taps) that needs to be in the generated sequence
    steps_per_dim = cls.__get_num_steps(
        tensor_dims=tensor_dims,
        tile_dims=tile_dims,
        step_dims=tile_group_steps,
        repeat_dims=tile_group_repeats,
    )
    num_steps = int(np.prod(steps_per_dim))

    # Define a function to calculate the offset of each tap in the sequence.
    def offset_fn(step_num: int, _prev_offset: int) -> int:
        tile_offsets = cls.__tile_offset_by_step_num(
            step_num,
            tile_group_steps,
            tile_group_repeats,
            steps_per_dim,
            iter_col_major,
        )
        total_offset = 0
        num_dims = len(tile_offsets)
        for dim, (offset, tile_dim) in enumerate(zip(tile_offsets, tile_dims)):
            total_offset += (
                offset
                * tile_dim
                * (
                    int(np.prod(tensor_dims[dim + 1 :]))
                    if dim < num_dims - 1
                    else 1
                )
            )
        return total_offset

    # Define a function that generates either sizes or strides for each tap in the sequence.
    def sizes_or_strides_fn(step_num, _prev_sizes, is_sizes):
        tile_offsets = cls.__tile_offset_by_step_num(
            step_num,
            tile_group_steps,
            tile_group_repeats,
            steps_per_dim,
            iter_col_major,
        )

        iter_sizes, iter_strides = cls.__sizes_strides_for_step_tile_group(
            tensor_dims,
            tile_dims,
            tile_group_steps,
            tile_group_repeats,
            tile_offsets,
            tile_col_major,
            tile_group_col_major,
            pattern_repeat=pattern_repeat,
            prune_step=prune_step,
        )
        if is_sizes:
            return iter_sizes
        else:
            return iter_strides

    sizes_fn = partial(sizes_or_strides_fn, is_sizes=True)
    strides_fn = partial(sizes_or_strides_fn, is_sizes=False)

    # Construct the sequence without defaults, fully relying on the constructor functions.
    return TensorAccessSequence(
        tensor_dims,
        num_steps,
        sizes_fn=sizes_fn,
        strides_fn=strides_fn,
        offset_fn=offset_fn,
    )

Utilities

validate_and_clean_sizes_strides

validate_and_clean_sizes_strides(
    sizes: Sequence[int] | None,
    strides: Sequence[int] | None,
    allow_none: bool = False,
    expected_dims: int | None = None,
) -> tuple[Sequence[int] | None, Sequence[int] | None]

Validate sizes and strides, and remove any unused values from upper dimensions if possible.

Parameters:

Name	Type	Description	Default
sizes	Sequence[int] | None	The transformation strides, or None	required
strides	Sequence[int] | None	The transformation sizes, or None	required
allow_none	bool	Allow sizes and/or strides to be None. Defaults to False.	False
expected_dims	int | None	Number of dimensions expected for both sizes and strides. Defaults to None.	None

Raises:

Type	Description
ValueError	Validate sizes and strides

Returns:

Type	Description
tuple[Sequence[int] | None, Sequence[int] | None]	tuple[Sequence[int] | None, Sequence[int] | None]: The 'cleaned' sizes and strides.

Source code in python/helpers/taplib/utils.py

def validate_and_clean_sizes_strides(
    sizes: Sequence[int] | None,
    strides: Sequence[int] | None,
    allow_none: bool = False,
    expected_dims: int | None = None,
) -> tuple[Sequence[int] | None, Sequence[int] | None]:
    """Validate sizes and strides, and remove any unused values from upper dimensions if possible.

    Args:
        sizes (Sequence[int] | None): The transformation strides, or None
        strides (Sequence[int] | None): The transformation sizes, or None
        allow_none (bool, optional): Allow sizes and/or strides to be None. Defaults to False.
        expected_dims (int | None, optional): Number of dimensions expected for both sizes and strides. Defaults to None.

    Raises:
        ValueError: Validate sizes and strides

    Returns:
        tuple[Sequence[int] | None, Sequence[int] | None]: The 'cleaned' sizes and strides.
    """
    if not allow_none:
        if sizes is None:
            raise ValueError("Sizes is None, but expected Sequence[int]")
        if strides is None:
            raise ValueError("Strides is None, but expected Sequence[int]")
    # After this point can assume None is ok for sizes/strides

    if expected_dims is not None:
        if expected_dims < 1:
            raise ValueError(f"Expected dimensions ({expected_dims}) should be >= 1")

    if sizes is None and strides is None:
        # nothing to do
        return None, None

    # Validate dimensions
    if (sizes is not None) and len(sizes) == 0:
        raise ValueError("len(sizes) must be >0")
    if (strides is not None) and len(strides) == 0:
        raise ValueError("len(strides) must be >0")

    if sizes and strides:
        if expected_dims:
            if len(sizes) != expected_dims:
                raise ValueError(
                    f"Num dimensions of sizes ({sizes}) is not expected number of dimensions ({expected_dims})"
                )
            if len(strides) != expected_dims:
                raise ValueError(
                    f"Num dimensions of strides ({strides}) is not expected number of dimensions ({expected_dims})"
                )
        elif len(strides) != len(sizes):
            raise ValueError(
                f"len(sizes) ({len(sizes)}) != len(strides) ({len(strides)})"
            )
    if strides:
        num_dims = len(strides)
    else:
        assert sizes is not None
        num_dims = len(sizes)

    # Validate sizes/strides values
    if sizes:
        sizes = deepcopy(sizes)
        for s in sizes:
            if s < 1:
                raise ValueError(f"All sizes must be >= 1, but got {sizes}")
    if strides:
        strides = deepcopy(strides)
        for s in strides:
            if s < 0:
                raise ValueError(f"All strides must be >= 0, but got {strides}")

    # Clean (set size=1, stride=0 for as many dims as possible)
    if sizes and strides:
        strides = list(strides)
        # Leave last dimension strides as whatever it happens to be
        for i in range(num_dims - 1):
            if sizes[i] == 1:
                strides[i] = 0
            else:
                break
    return sizes, strides

validate_tensor_dims

validate_tensor_dims(
    tensor_dims: Sequence[int],
    expected_dims: int | None = None,
) -> Sequence[int]

Validate dimensions of tensors by ensuring each dimension is > 0 and the dimensionality is as expected.

Parameters:

Name	Type	Description	Default
tensor_dims	Sequence[int]	Tensor dimensions to check	required
expected_dims	int | None	Expected number of dimensions. Defaults to None.	None

Raises:

Type	Description
ValueError	Validate the tensor dimensions

Returns:

Type	Description
Sequence[int]	Sequence[int]: The validated tensor dimensions.

Source code in python/helpers/taplib/utils.py

def validate_tensor_dims(
    tensor_dims: Sequence[int], expected_dims: int | None = None
) -> Sequence[int]:
    """Validate dimensions of tensors by ensuring each dimension is > 0 and the dimensionality is as expected.

    Args:
        tensor_dims (Sequence[int]): Tensor dimensions to check
        expected_dims (int | None, optional): Expected number of dimensions. Defaults to None.

    Raises:
        ValueError: Validate the tensor dimensions

    Returns:
        Sequence[int]: The validated tensor dimensions.
    """
    if expected_dims is not None:
        if expected_dims < 1:
            raise ValueError(f"Expected dimensions ({expected_dims}) should be >= 1")
    tensor_dims = deepcopy(tensor_dims)

    # Validate tensor dims and offset, then set
    if len(tensor_dims) == 0:
        raise ValueError(
            f"Number of tensor dimensions must be >= 1 (dimensions={tensor_dims})"
        )
    for d in tensor_dims:
        if d <= 0:
            raise ValueError(
                f"Each tensor dimension must be >= 1 (dimensions={tensor_dims})"
            )

    # We can treat a 1-dimensional tensor as a 2-dimensional tensor,
    if len(tensor_dims) == 1:
        tensor_dims = [1, tensor_dims[0]]

    if expected_dims is not None and len(tensor_dims) != expected_dims:
        raise ValueError(
            f"Tensor dimension ({tensor_dims}) does not match expected dimension ({expected_dims})"
        )

    return tensor_dims

validate_offset

validate_offset(
    offset: int, tensor_dims: Sequence[int] | None
) -> int

Validate an offset into the tensor.

Primarily checks to see if the offset is a valid index to the tensor.

Parameters:

Name	Type	Description	Default
offset	int	The offset to check.	required
tensor_dims	Sequence[int] | None	The dimensions of the tensor the offset corresponds to.	required

Raises:

Type	Description
ValueError	Validate the offset.

Returns:

Name	Type	Description
int	int	The validated offset.

Source code in python/helpers/taplib/utils.py

def validate_offset(offset: int, tensor_dims: Sequence[int] | None) -> int:
    """Validate an offset into the tensor.

    Primarily checks to see if the offset is a valid index to the tensor.

    Args:
        offset (int): The offset to check.
        tensor_dims (Sequence[int] | None): The dimensions of the tensor the offset corresponds to.

    Raises:
        ValueError: Validate the offset.

    Returns:
        int: The validated offset.
    """
    if offset < 0:
        raise ValueError(f"Offset must be >= 0 (offset={offset})")
    if tensor_dims:
        if offset >= np.prod(tensor_dims):
            raise ValueError(
                f"Offset too large: {offset}. Max value allowed for tensor: {np.prod(tensor_dims)}"
            )
    return offset