Skip to content

(Single-card) Model perf tests #6439

(Single-card) Model perf tests

(Single-card) Model perf tests #6439

Manually triggered November 24, 2024 17:34
Status Failure
Total duration 1h 17m 12s
Artifacts 7

perf-models.yaml

on: workflow_dispatch
build-artifact  /  ...  /  build-docker-image
1m 10s
build-artifact / build-docker-image / build-docker-image
Matrix: build-artifact / build-artifact
Matrix: models-perf / models-perf
Fit to window
Zoom out
Zoom in

Annotations

6 errors, 14 warnings, and 44 notices
models-perf / other N300 WH B0
Process completed with exit code 1.
models-perf / llm_javelin GS
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
models-perf / llm_javelin GS
Process completed with exit code 2.
unsuccessful-reset-cleanup
Unable to reset board successfully, rebooting
models-perf / llm_javelin GS
The operation was canceled.
models-perf / llm_javelin GS
Process completed with exit code 1.
build-artifact / build-docker-image / build-docker-image
Unable to find merge base between e82667eb9bdd42bcd9a0fe256e0081563624772a and 2ba5a2973915aa1c5afbf6f1524e87e0c43bc058
build-artifact / build-docker-image / build-docker-image
Set 'fetch_additional_submodule_history: true' to fetch additional submodule history for: tt_metal/third_party/lfs
build-artifact / build-docker-image / build-docker-image
Unable to find merge base between f0cbccf5c58db19123d01e56351153392e6cd960 and 50796e4f179ed441db96becb59eec17f70a20a7e
build-artifact / build-docker-image / build-docker-image
Set 'fetch_additional_submodule_history: true' to fetch additional submodule history for: tt_metal/third_party/tt_llk_blackhole
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
printing-out-smi-info-cleanup
Touching and printing out SMI info
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 33 %
disk-usage-after-startup
Disk usage is 33 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 33 %
disk-usage-after-startup
Disk usage is 33 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 89 %
disk-usage-after-startup
Disk usage is 89 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 60 %
disk-usage-after-startup
Disk usage is 60 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
attempting-reset-cleanup
Attempting to reset card(s).
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 33 %
disk-usage-after-startup
Disk usage is 33 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 90 %
disk-usage-after-startup
Disk usage is 90 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful

Artifacts

Produced during runtime
Name Size
TTMetal_build_grayskull
300 MB
TTMetal_build_wormhole_b0
300 MB
perf-report-csv-cnn_javelin-grayskull-bare_metal
358 Bytes
perf-report-csv-cnn_javelin-wormhole_b0-bare_metal
510 Bytes
perf-report-csv-llm_javelin-wormhole_b0-bare_metal
1.43 KB
perf-report-csv-other-grayskull-bare_metal
1.14 KB
perf-report-csv-other-wormhole_b0-bare_metal
733 Bytes