DispatchCreation
-iree-dispatch-creation-annotate-data-tiling-hintslink
Adds data-tiling hint attribute to linalg operations.
The pass does nothing, if any operation already has the data-tiling hint attribute. Otherwise, it aggressively filters linalg ops and adds the data-tiling hint attribute to the operations.
-iree-dispatch-creation-bitcast-unsupported-element-typeslink
Bitcasts tensor element types unsupported by the HAL
-iree-dispatch-creation-bubble-up-expand-shapeslink
Propagate expand_shapes up the program (and collapse_shapes down).
Optionslink
-enable-bubble-up-expand-shapes-across-reduction-ops : Enables propagation of 'expand_shape's through 'linalg.generic's with reductions
-iree-dispatch-creation-clone-producers-into-dispatch-regionslink
Clone producers into dispatch regions to be isolated above.
Pass to clone into dispatch regions producers of values used in the dispatch regions but defined in the above. This prepares the dispatch regions for converting to dispatch workgroups with explicit captures.
Optionslink
-aggressive : Include operations that are cloned only under aggressive fusion mode
-iree-dispatch-creation-collapse-dimensionslink
Collapse dimensions of Linalg Ops on tensor ops.
Collapse dimensions of Linalg Ops on tensor ops inside dispatch.region ops and hoist the reshaping operations out of the dispatch.
Optionslink
-max-iterations : Maximum number of iterations to wait for collapse dimensions to converge
-iree-dispatch-creation-convert-dispatch-regions-to-workgroupslink
Convert dispatch regions to dispatch workgroups.
Pass to convert dispatch regions to dispatch workgroups. This pass is intended to be used after dispatch regions have been formed.
Statisticslink
num-dispatches : Number of dispatches created
-iree-dispatch-creation-convert-encoding-to-flowlink
Convert top-level Encoding ops to Flow ops.
Pass to convert top-level Encoding ops to Flow ops, which only converts the Encoding ops outside flow.dispatch.region to Flow.
-iree-dispatch-creation-convert-tensor-to-flowlink
Convert tensor operations to flow
Pass to convert tensor operations to flow.tensor.* operations.
Statisticslink
num-slow-copy-dispatches : Number of slow copy dispatches (for handling slices) created
-iree-dispatch-creation-elementwise-op-fusionlink
Fuse elementwise operations.
Optionslink
-intra-dispatch : Fuse operations within a dispatch only (default is to fuse only operations outside of a dispatch)
-fuse-multi-reduction : Fuse ops that have multiple reduction iterators
-fuse-truncate-ops : Fuse producer truncate-like operations with consumers
-fuse-broadcast-ops : Fuse broadcast-like ops with producers
-iree-dispatch-creation-fold-unit-extent-dimslink
Fold unit extent dimension of operations.
Imports upstream patterns to fold unit extent dims but with IREE control.
-iree-dispatch-creation-fold-unit-extent-dims-for-funclink
Fold unit extent dimension of operations on a function.
Imports upstream patterns to fold unit extent dims but with IREE control.
-iree-dispatch-creation-form-dispatch-regionslink
Form Dispatch Region Ops from Linalg operations on tensors to form dispatch.regions.
Pass to form dispatch.region ops from Linalg on tensor ops. A dispatch region is created for each tiled loop nest. This pass only moves the root compute op into the dispatch region, allowing producers to be outside.
Optionslink
-aggressive-fusion : Aggressive mode enabling fusions not ready for all backends
-fuse-pad-with-consumers : Enable fusing pad with consumer
-fuse-pad-with-producers : Enable fusion of pad with producers
-iree-dispatch-creation-form-scalar-dispatcheslink
Form Dispatch Regions for scalar computations.
-iree-dispatch-creation-form-split-reduction-dispatcheslink
Partially tile reduction operations and place into dispatches
Optionslink
-split-size : Tile sizes for split reduction (innermost first)
-enable-fuse-pad : Fuse pad into scf.forall
-iree-dispatch-creation-fuse-encoding-ops-into-dispatch-regions-passlink
Fuses set_encoding ops into producer dispatch regions, or forms new dispatches around them.
-iree-dispatch-creation-fuse-horizontal-contractionslink
Fuses horizontal contraction ops
For cases where multiple contractions - that dont have a direct dependence - that have the same LHS operand - all the N dimensions of the RHS operands used are the same Such contractions can be executed as a single contraction, i.e.
A = matmul(lhs, rhs0); B = matmul(lhs, rhs1); C = matmul(lhs, rhs2);
can be combined into result = matmul(lhs, concat_along_N(rhs0, rhs1, rhs2)); A = slice0(result) B = slice1(result) C = slice2(result)
Instead of doing an actual concat of the RHS operands, and extracting slices of the result, the pass generates a single operation with - the lhs operands - all the rhs operands - multiple results representing the individual matmuls
Optionslink
-fusion-limit : Maximum number of contractions fused into one
Statisticslink
num-fusion-groups : Number of fusion groups found
num-size-2-groups : Number of fusion groups of size 2
num-size-3-groups : Number of fusion groups of size 3
-iree-dispatch-creation-fuse-multi-use-elementwise-producerlink
Fuse elementwise linalg operations on tensors when producers have multiple uses.
Optionslink
-intra-dispatch : Fuse operations within a dispatch only (default is to fuse only operations outside of a dispatch)
-num-iterations : Number of iterations to fuse multiuse ops
-iree-dispatch-creation-fusion-preprocessinglink
Run useful preprocessing patterns that help with fusion.
-iree-dispatch-creation-hoist-encoding-opslink
Hoists tensor encoding ops out of flow dispatch regions.
-iree-dispatch-creation-hoist-uniform-scalar-computelink
Hoists scalar (computation) out of dispatch regions.
-iree-dispatch-creation-materialize-default-workgroup-count-regionlink
Canonicalize dispatch workgroups ops.
Apply dispatch workgroups canonicalization patterns.
-iree-dispatch-creation-propagate-encodingslink
Propagate encodings across other operations.
-iree-dispatch-creation-set-encodinglink
Introduces tensor encoding for flow dispatch regions.
Sets the encoding for compute operations to allow execution of the operations in tiled/padded layouts.
Optionslink
-encoding-option : Select the type of encoding options to add.
-iree-dispatch-creation-set-split-reduction-sizeslink
Set 'iree_linalg_ext.split_reduction' attribute on ops
Optionslink
-split-reduction-target-size : Target tile size for split reduction. Inner reduction dimensions are tiled first, with the tile size rounded up until it evenly divides the iteration domain.
-iree-dispatch-creation-sink-reshapeslink
Sink reshapes to allow for compute op -> consumer fusion.
-iree-dispatch-creation-split-reduction-opslink
Split reduction dimension to increase parallelism.
-iree-dispatch-creation-tensor-pad-to-tensor-insert-slicelink
Convert tensor.pad into linalg.fill + tensor.insert_slice.
Optionslink
-skip-one-linalg-use-case : Skip the op that has only one use which is usedby a Linalg op
-iree-dispatch-creation-transpose-generic-opslink
Transpose generic op loops.