Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ND behavior with test_matmul_1d_2d.py::test_multi_core_matmul_2d #7168

Open
TT-billteng opened this issue Apr 5, 2024 · 7 comments
Open
Assignees
Labels
bug Something isn't working ci-bug bugs found in CI op_cat: mm P3

Comments

@TT-billteng
Copy link
Collaborator

TT-billteng commented Apr 5, 2024

The ND behavior occurs with configuration M=1792, K=2048, N=2048, and isn't specific to N150 or N300. It either hangs or errors out with bad PCC. To reproduce:

pip install pytest-repeat
pytest --count=1000 --repeat-scope=session --timeout=120 -xv tests/tt_eager/python_api_testing/unit_testing/misc/test_matmul_1d_2d.py::test_multi_core_matmul_2d -k '[False-True-True-1792-2048-2048'

Some failing runs in CI:

https://github.com/tenstorrent-metal/tt-metal/actions/runs/8566991717/job/23477858954
https://github.com/tenstorrent-metal/tt-metal/actions/runs/8572995474/job/23497194353
https://github.com/tenstorrent-metal/tt-metal/actions/runs/8572833559/job/23496468681
https://github.com/tenstorrent-metal/tt-metal/actions/runs/8566128457/job/23475414075

@TT-billteng
Copy link
Collaborator Author

@tt-rkim
Copy link
Collaborator

tt-rkim commented Apr 23, 2024

We are skipping the device perf profiling test for this until it is fixed

cc: @yugaoTT @davorchap

@bbradelTT
Copy link
Contributor

@yugaoTT what's the current status of this issue?

@yugaoTT
Copy link
Contributor

yugaoTT commented Jul 10, 2024

@bbradelTT The issue will be there, as long as didt hasn't been fixed

@prajaramanTT
Copy link

@bbradelTT is this issue still there ?

@bbradelTT
Copy link
Contributor

@prajaramanTT This issue may or may not still be there. I still don't see a clear and complete resolution on didt. Therefore we haven't looked into this yet.

@prajaramanTT
Copy link

@TT-billteng @bbradelTT In that case, I'm going to lower the priority of this issue to P3.

@prajaramanTT prajaramanTT added P3 and removed P1 labels Nov 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working ci-bug bugs found in CI op_cat: mm P3
Projects
None yet
Development

No branches or pull requests

6 participants