Adding bf16 data type workaround for max_pool2d op and fixing embedding workaround tests #1657
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sdjordjevicTT
requested review from
svuckovicTT,
mtopalovicTT,
nobradovictt,
jserbedzijaTT,
jnie-TT and
azecevicTT
as code owners
sdjordjevicTT
force-pushed
the
sdjordjevic/max_pool_2d_bf16_workaround
branch
from
599bfb9 to
ce775d3
Compare
sdjordjevicTT
requested review from
kmabeeTT,
AleksKnezevic and
pilkicTT
as code owners
sdjordjevicTT
force-pushed
the
sdjordjevic/max_pool_2d_bf16_workaround
branch
from
ce775d3 to
ee54049
Compare
jnie-TT
reviewed
Dec 23, 2024
sdjordjevicTT
force-pushed
the
sdjordjevic/max_pool_2d_bf16_workaround
branch
from
ee54049 to
f3ec40c
Compare
|
Created the following issue to track the runtime workaround: |
sdjordjevicTT
force-pushed
the
sdjordjevic/max_pool_2d_bf16_workaround
branch
from
f3ec40c to
f88de78
Compare
jnie-TT
reviewed
Dec 23, 2024
| private: | ||
| constexpr Env(bool maxpool2dPreshard, bool swapBinaryOperands, | ||
| bool readUpdateIndexFromDeviceForKVCache) | ||
| bool readUpdateIndexFromDeviceForKVCache, bool typecastOnHost) |
There was a problem hiding this comment.
Maybe name this as toDtypeOnHost or something so that it's explicit that we're using to_dtype when typcasting on host.
Also please add a ttrt command line option to toggle this in runtime/tools/python/ttrt/common/run.py, the procedure will be the same as the other workaround flags.
There was a problem hiding this comment.
Renamed typecastOnHost to toDtypeOnHost. I added an option in run.py; please double-check if I implemented it correctly.
sdjordjevicTT
force-pushed
the
sdjordjevic/max_pool_2d_bf16_workaround
branch
from
f88de78 to
e320d04
Compare
jnie-TT
approved these changes
Dec 23, 2024
mtopalovicTT
approved these changes
Dec 25, 2024
jserbedzijaTT
approved these changes
Dec 25, 2024
sdjordjevicTT
changed the title
tapspatel
approved these changes
Dec 26, 2024
sdjordjevicTT
force-pushed
the
sdjordjevic/max_pool_2d_bf16_workaround
branch
from
e320d04 to
5ba32ac
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Applying bf16 data type workaround on max_pool2d op.
With this change, I rewrote the workaround test, silicon, and compiler test.
Closes #1389