[Flow] Fix dispatch naming for dynamic shaped fusions #19439
+50
−1
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently all ops with dynamic shapes are assigned the same estimated cost when naming dispatches. This means that in cases like fused elementwise ops with matmuls, the elementwise and matmuls are assigned the same priority and because of traversal order, the dispatch ends up following the name of the elementwise op. This patch hacks it by treating all dynamic shapes as moderately sized static shapes, but in the future if we have more issues we can look at adding some tensor size range analysis that can give us upper bounds for the dynamic shapes.
qedawkins
requested review from
hanhanW,
MaheshRavishankar and
IanWood1
as code owners
hanhanW
reviewed
Dec 11, 2024
compiler/src/iree/compiler/Dialect/Flow/Transforms/AnnotateDispatches.cpp
Outdated
Show resolved
Hide resolved
IanWood1
force-pushed
the
fix_dispatch_names
branch
from
4e4ff6a to
1bcaf40
Compare
Signed-off-by: Ian Wood <[email protected]>
Signed-off-by: Ian Wood <[email protected]>
Signed-off-by: Ian Wood <[email protected]>
hanhanW
approved these changes
Jan 9, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently all ops with dynamic shapes are assigned the same estimated cost when naming dispatches. This means that in cases like fused elementwise ops with matmuls, the elementwise and matmuls are assigned the same priority and because of traversal order, the dispatch ends up following the name of the elementwise op.
This patch hacks it by treating all dynamic shapes as moderately sized static shapes, but in the future if we have more issues we can look at adding some tensor size range analysis that can give us upper bounds for the dynamic shapes.