Skip to content

(TG) TG model perf tests #538

(TG) TG model perf tests

(TG) TG model perf tests #538

Triggered via schedule December 24, 2024 00:10
Status Failure
Total duration 2h 41m 20s
Artifacts 3
build-artifact-profiler  /  ...  /  build-docker-image
1m 44s
build-artifact-profiler / build-docker-image / build-docker-image
Matrix: build-artifact-profiler / build-artifact
Matrix: tg-model-perf-tests / tg-model-perf-tests
Fit to window
Zoom out
Zoom in

Annotations

4 errors and 11 notices
pcie-cards-are-being-used-cleanup
Tenstorrent cards seem to be in use. Killing PIDs and exiting unsuccessfully. This can happen if a test hung and is normally an issue with the test, rather than infra.
tg-model-perf-tests / TG LLM model perf tests
Process completed with exit code 1.
tg-model-perf-tests / TG LLM model perf tests
Process completed with exit code 1.
tg-model-perf-tests / TG LLM model perf tests
The action 'Run model perf regression tests' has timed out after 60 minutes.
disk-usage-before-startup
Disk usage is 39 %
disk-usage-after-startup
Disk usage is 39 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 39 %
disk-usage-after-startup
Disk usage is 39 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
printing-out-smi-info-cleanup
Touching and printing out SMI info
printing-out-smi-info-cleanup
Touching and printing out SMI info

Artifacts

Produced during runtime
Name Size
TTMetal_build_wormhole_b0_profiler
320 MB
perf-report-csv--wormhole_b0--bare-metal
1.53 KB
perf-report-csv-CNN-wormhole_b0-
529 Bytes