(TG) TG model perf tests #471
tapspateltg-model-perf-tests.yaml
on: schedule
build-artifact
/
...
/
build-docker-image
23s
Matrix: build-artifact / build-artifact
Matrix: tg-model-perf-tests / tg-model-perf-tests
Annotations
5 errors, 10 warnings, and 7 notices
|
tg-model-perf-tests / TG CNN model perf tests
Process completed with exit code 1.
|
|
pcie-cards-are-being-used-cleanup
Tenstorrent cards seem to be in use. Killing PIDs and exiting unsuccessfully. This can happen if a test hung and is normally an issue with the test, rather than infra.
|
|
tg-model-perf-tests / TG LLM model perf tests
Process completed with exit code 1.
|
|
tg-model-perf-tests / TG LLM model perf tests
Process completed with exit code 2.
|
|
tg-model-perf-tests / TG LLM model perf tests
The action 'Run model perf regression tests' has timed out after 60 minutes.
|
|
tg-model-perf-tests / TG CNN model perf tests
Failed to download action 'https://api.github.com/repos/tenstorrent/tt-metal/tarball/78075c68b6f25399a4c27e5c5082bafc2a350bc1'. Error: Resource temporarily unavailable (api.github.com:443)
|
|
tg-model-perf-tests / TG CNN model perf tests
Back off 26.135 seconds before retry.
|
|
tg-model-perf-tests / TG CNN model perf tests
Failed to download action 'https://api.github.com/repos/actions/download-artifact/tarball/fa0a91b85d4f404e444e00e005971372dc801d16'. Error: Resource temporarily unavailable (codeload.github.com:443)
|
|
tg-model-perf-tests / TG CNN model perf tests
Back off 29.043 seconds before retry.
|
|
tg-model-perf-tests / TG LLM model perf tests
Failed to restore: downloadCacheMetadata failed: getaddrinfo EAI_AGAIN vth0acprodeus2file0.blob.core.windows.net
|
|
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
|
|
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
|
|
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
|
|
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
|
|
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
|
printing-out-smi-info-cleanup
Touching and printing out SMI info
|
|
attempting-reset-cleanup
Attempting to reset card(s).
|
|
successful-reset-cleanup
tt-smi reset was successful
|
|
reset-successful-cleanup
tt-smi reset was successful
|
Artifacts
Produced during runtime
| Name | Size | |
|---|---|---|
|
TTMetal_build_wormhole_b0
|
300 MB |
|
|
perf-report-csv-CNN-wormhole_b0-
|
523 Bytes |
|