[GPU] Enable GEMMs to first attempt LLVMGPUTileAndFuse with intrinsic by default #19520
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nirvedhmeshram
force-pushed
the
enable_tile_and_fuse_matmul
branch
from
38f5a22 to
7d687d7
Compare
nirvedhmeshram
changed the title
nirvedhmeshram
requested review from
MaheshRavishankar,
qedawkins,
kuhar and
Groverkss
as code owners
|
There are compiler failures in the regression suite models, converting to draft while I debug |
nirvedhmeshram
force-pushed
the
enable_tile_and_fuse_matmul
branch
from
7d687d7 to
7e2cdf8
Compare
|
The problem was a missing functionality for GEMMs of the type (f16,f16) ->f16. I filed this issue for it |
nirvedhmeshram
force-pushed
the
enable_tile_and_fuse_matmul
branch
2 times, most recently
from
e6aa895 to
3bc822c
Compare
|
Found another issue with accumulating GEMMs #19546 |
nirvedhmeshram
force-pushed
the
enable_tile_and_fuse_matmul
branch
6 times, most recently
from
2adc85d to
2111358
Compare
|
Also need to disable prefetching when using c promotion due to this issue #19612 |
nirvedhmeshram
force-pushed
the
enable_tile_and_fuse_matmul
branch
from
210ef2a to
017e558
Compare
Signed-off-by: Nirvedh <[email protected]> Signed-off-by: Nirvedh Meshram <[email protected]>
Signed-off-by: Nirvedh Meshram <[email protected]>
nirvedhmeshram
force-pushed
the
enable_tile_and_fuse_matmul
branch
from
017e558 to
982856b
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Based on comparisons with iree-kernel-benchmark here The performance between VectorDistribute vs TileAndFuse when using intrinisics seem comparable. Note that none of the tests in the sheet used the padding extension available in TileAndFuse after, #19484
so its a fair comparison of the pipelines themselves. TileAndFuse in some cases did have a speed up that seems beyond the noise level and overall it averages out to 1.25x faster.
However, we will be looking at LLAMA and SDXL numbers before actually considering this PR for merging,
Fixes : #18858