Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create built-in matrix transforms to facilitate splitting tasks by arbitrary values #588

Merged
merged 2 commits into from
Oct 17, 2024

ci: use 'matrix' transforms for test kind

2c19e78
Select commit
Loading
Failed to load commit list.
Merged

Create built-in matrix transforms to facilitate splitting tasks by arbitrary values #588

ci: use 'matrix' transforms for test kind
2c19e78
Select commit
Loading
Failed to load commit list.
firefoxci-taskcluster / codecov-upload succeeded Oct 16, 2024 in 3m 7s

FirefoxCI (pull_request)

Upload coverage.xml to codecov.io (Treeherder push)

Details

View task in Taskcluster | View logs in Taskcluster | View task group in Taskcluster

Task Status

Started: 2024-10-16T18:54:04.340Z
Resolved: 2024-10-16T18:54:17.314Z
Task Execution Time: 12 seconds, 974 milliseconds
Task Status: completed
Reason Resolved: completed
RunId: 0

Artifacts

- public/logs/live_backing.log
- public/logs/live.log


[taskcluster 2024-10-16 18:54:04.382Z] Task ID: Qo5q8MQCQLOil94ZalD1MA
[taskcluster 2024-10-16 18:54:04.382Z] Worker ID: 8584087920560682437
[taskcluster 2024-10-16 18:54:04.382Z] Worker Group: us-west1-a
[taskcluster 2024-10-16 18:54:04.382Z] Worker Node Type: projects/887720501152/machineTypes/c2-standard-4
[taskcluster 2024-10-16 18:54:04.382Z] Worker Pool: taskgraph-t/linux-gcp
[taskcluster 2024-10-16 18:54:04.382Z] Worker Version: 38.0.5
[taskcluster 2024-10-16 18:54:04.382Z] Public IP: 34.168.227.158
[taskcluster 2024-10-16 18:54:04.382Z] Hostname: taskgraph-t-linux-gcp-ieq-cteftokwqwv4upzirq
[taskcluster 2024-10-16 18:54:04.382Z] using cache "taskgraph-level-1-checkouts-v3-0afbf72f72cb5ffedd2f-IG8tGmE7SOS0YlEToMwF6w" -> /builds/worker/checkouts

[taskcluster 2024-10-16 18:54:04.712Z] Image 'public/image.tar.zst' from task 'IG8tGmE7SOS0YlEToMwF6w' loaded.  Using image ID sha256:24bbfe60acdd0f671805a1fa025d1b0e0cd9b216526c2fd873ba195f74587b8f.
[taskcluster 2024-10-16 18:54:04.724Z] === Task Starting ===
[setup 2024-10-16T18:54:05.068Z] run-task started in /
[setup 2024-10-16T18:54:05.069Z] Invoked by command: --ci-checkout=/builds/worker/checkouts/vcs/ --task-cwd /builds/worker/checkouts/vcs -- bash -cx uv run coverage combine --data-file $MOZ_FETCHES_DIR/coverage $MOZ_FETCHES_DIR && uv run coverage xml --data-file $MOZ_FETCHES_DIR/coverage -o $MOZ_FETCHES_DIR/coverage.xml && uv run python taskcluster/scripts/codecov-upload.py
[setup 2024-10-16T18:54:05.069Z] Python version: 3.11.2
[cache 2024-10-16T18:54:05.070Z] cache /builds/worker/checkouts exists; requirements: gid=1000 uid=1000 version=1
[volume 2024-10-16T18:54:05.070Z] changing ownership of volume /builds/worker/.cache to 1000:1000
[volume 2024-10-16T18:54:05.070Z] volume /builds/worker/checkouts is a cache
[setup 2024-10-16T18:54:05.070Z] running as worker:worker
[vcs 2024-10-16T18:54:05.070Z] executing ['git', 'config', '--global', '--add', 'safe.directory', '/builds/worker/checkouts/vcs']
[vcs 2024-10-16T18:54:05.072Z] executing ['git', 'fetch', '--no-tags', 'https://github.com/ahal/taskgraph', 'matrix']
[vcs 2024-10-16T18:54:05.310Z] From https://github.com/ahal/taskgraph
[vcs 2024-10-16T18:54:05.311Z]  * branch              matrix     -> FETCH_HEAD
[vcs 2024-10-16T18:54:05.315Z] executing ['git', 'checkout', '-f', '-B', 'matrix', '2c19e7811bd718e74e5794485471e856d0c5d8af']
[vcs 2024-10-16T18:54:05.318Z] Reset branch 'matrix'
[vcs 2024-10-16T18:54:05.319Z] cleaning git checkout...
[vcs 2024-10-16T18:54:05.319Z] executing ['git', 'clean', '-nxdff']
[vcs 2024-10-16T18:54:05.377Z] removing ['/builds/worker/checkouts/vcs/.pytest_cache/', '/builds/worker/checkouts/vcs/.venv/', '/builds/worker/checkouts/vcs/__pycache__/', '/builds/worker/checkouts/vcs/artifacts/', '/builds/worker/checkouts/vcs/src/taskgraph/__pycache__/', '/builds/worker/checkouts/vcs/src/taskgraph/actions/__pycache__/', '/builds/worker/checkouts/vcs/src/taskgraph/loader/__pycache__/', '/builds/worker/checkouts/vcs/src/taskgraph/optimize/__pycache__/', '/builds/worker/checkouts/vcs/src/taskgraph/run-task/__pycache__/', '/builds/worker/checkouts/vcs/src/taskgraph/transforms/__pycache__/', '/builds/worker/checkouts/vcs/src/taskgraph/transforms/run/__pycache__/', '/builds/worker/checkouts/vcs/src/taskgraph/util/__pycache__/', '/builds/worker/checkouts/vcs/test/__pycache__/', '/builds/worker/checkouts/vcs/test/data/taskcluster/test_taskgraph/transforms/__pycache__/', '/builds/worker/checkouts/vcs/test/data/testmod/__pycache__/', '/builds/worker/checkouts/vcs/test/fixtures/__pycache__/']
[vcs 2024-10-16T18:54:05.596Z] successfully cleaned git checkout!
[vcs 2024-10-16T18:54:05.598Z] TinderboxPrint:<a href='https://github.com/ahal/taskgraph/commit/2c19e7811bd718e74e5794485471e856d0c5d8af' title='Built from taskgraph commit 2c19e7811bd718e74e5794485471e856d0c5d8af'>2c19e7811bd718e74e5794485471e856d0c5d8af</a>
[setup 2024-10-16T18:54:05.598Z] MOZ_FETCHES_DIR is /builds/worker/fetches
[fetches 2024-10-16T18:54:05.598Z] fetching artifacts
[fetches 2024-10-16T18:54:05.598Z] executing ['/usr/bin/python3', '-u', '/usr/local/bin/fetch-content', 'task-artifacts']
attempt 1/5
attempt 1/5Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/OdvqST4gS2W8yzk0OoQSuQ/artifacts/public/codecov to /builds/worker/fetches/codecov

Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/BLhAGCFsTUCOa1TBR-3_RA/artifacts/public/coverage.py310 to /builds/worker/fetches/coverage.py310
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/OdvqST4gS2W8yzk0OoQSuQ/artifacts/public/codecov
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/BLhAGCFsTUCOa1TBR-3_RA/artifacts/public/coverage.py310
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/VlEciHNsSmi2im6Zb0oEHA/artifacts/public/coverage.py311 to /builds/worker/fetches/coverage.py311
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/JMxYSvHRRhec0rN0KsJKQw/artifacts/public/coverage.py312 to /builds/worker/fetches/coverage.py312
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/VlEciHNsSmi2im6Zb0oEHA/artifacts/public/coverage.py311
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/JMxYSvHRRhec0rN0KsJKQw/artifacts/public/coverage.py312
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/VlEciHNsSmi2im6Zb0oEHA/artifacts/public/coverage.py311 resolved to 602112 bytes with sha256 080e9078fe4d0f6395ccdea65e165a9ae8019c4e9371ad0ab3ce4f9c0845908d in 0.301s
Extracting /builds/worker/fetches/coverage.py311 to /builds/worker/fetches
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/IxUFySJxSs2YsZCxA9SXOA/artifacts/public/coverage.py313 to /builds/worker/fetches/coverage.py313
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/IxUFySJxSs2YsZCxA9SXOA/artifacts/public/coverage.py313
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/BLhAGCFsTUCOa1TBR-3_RA/artifacts/public/coverage.py310 resolved to 589824 bytes with sha256 1b546013da0438a39913b21fe957d05dfe807163f3673f95f985eb86b34925a2 in 0.324s
Extracting /builds/worker/fetches/coverage.py310 to /builds/worker/fetches
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/XXXUKVktSy-VSILAP2zSKQ/artifacts/public/coverage.py38 to /builds/worker/fetches/coverage.py38
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/XXXUKVktSy-VSILAP2zSKQ/artifacts/public/coverage.py38
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/JMxYSvHRRhec0rN0KsJKQw/artifacts/public/coverage.py312 resolved to 598016 bytes with sha256 a3a4df4209963627b2d9e8f40ae8ca8794d8a842a1509a509267a5dd85849e30 in 0.413s
Extracting /builds/worker/fetches/coverage.py312 to /builds/worker/fetches
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/bqLUxR0gRbyYm6XSacqfSw/artifacts/public/coverage.py39 to /builds/worker/fetches/coverage.py39
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/bqLUxR0gRbyYm6XSacqfSw/artifacts/public/coverage.py39
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/XXXUKVktSy-VSILAP2zSKQ/artifacts/public/coverage.py38 resolved to 573440 bytes with sha256 cc7ac0cc061cbd8748dc169cc679f313d18a6c13121f6e512c754df4b50d3e27 in 0.166s
Extracting /builds/worker/fetches/coverage.py38 to /builds/worker/fetches
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/IxUFySJxSs2YsZCxA9SXOA/artifacts/public/coverage.py313 resolved to 598016 bytes with sha256 98c7ce2c1e43dc8502e0ab3db828a6e473937a206cad4f2841f99160a93b6632 in 0.199s
Extracting /builds/worker/fetches/coverage.py313 to /builds/worker/fetches
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/bqLUxR0gRbyYm6XSacqfSw/artifacts/public/coverage.py39 resolved to 573440 bytes with sha256 c059a6d027e6310255fc1ca2692cdadaf7541d323df5db688f70218e1e5443b8 in 0.144s
Extracting /builds/worker/fetches/coverage.py39 to /builds/worker/fetches
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/OdvqST4gS2W8yzk0OoQSuQ/artifacts/public/codecov resolved to 28918392 bytes with sha256 209d13481be406d6a2aa9519fa61c84883e3213308b5628c43a5e94cae75b8e6 in 0.889s
Extracting /builds/worker/fetches/codecov to /builds/worker/fetches
PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 0.8925276730000178, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2024-10-16T18:54:06.572Z] finished fetching artifacts
[task 2024-10-16T18:54:06.572Z] executing ['bash', '-cx', 'uv run coverage combine --data-file $MOZ_FETCHES_DIR/coverage $MOZ_FETCHES_DIR && uv run coverage xml --data-file $MOZ_FETCHES_DIR/coverage -o $MOZ_FETCHES_DIR/coverage.xml && uv run python taskcluster/scripts/codecov-upload.py']
[task 2024-10-16T18:54:06.574Z] + uv run coverage combine --data-file /builds/worker/fetches/coverage /builds/worker/fetches
[task 2024-10-16T18:54:06.747Z] Using Python 3.13.0rc2
[task 2024-10-16T18:54:06.747Z] Creating virtualenv at: .venv
[task 2024-10-16T18:54:10.094Z] warning: Failed to hardlink files; falling back to full copy. This may lead to degraded performance.
[task 2024-10-16T18:54:10.094Z]          If the cache and target directories are on different filesystems, hardlinking may not be supported.
[task 2024-10-16T18:54:10.096Z]          If this is intentional, set `export UV_LINK_MODE=copy` or use `--link-mode=copy` to suppress this warning.
[task 2024-10-16T18:54:10.194Z] Installed 73 packages in 99ms
[task 2024-10-16T18:54:11.122Z] Combined data file ../../fetches/coverage.py310
[task 2024-10-16T18:54:11.122Z] Combined data file ../../fetches/coverage.py311
[task 2024-10-16T18:54:11.122Z] Combined data file ../../fetches/coverage.py312
[task 2024-10-16T18:54:11.122Z] Combined data file ../../fetches/coverage.py313
[task 2024-10-16T18:54:11.122Z] Combined data file ../../fetches/coverage.py38
[task 2024-10-16T18:54:11.122Z] Combined data file ../../fetches/coverage.py39
[task 2024-10-16T18:54:11.142Z] + uv run coverage xml --data-file /builds/worker/fetches/coverage -o /builds/worker/fetches/coverage.xml
[task 2024-10-16T18:54:13.128Z] Wrote XML report to /builds/worker/fetches/coverage.xml
[task 2024-10-16T18:54:13.423Z] + uv run python taskcluster/scripts/codecov-upload.py
[task 2024-10-16T18:54:14.950Z] info - 2024-10-16 18:54:14,950 -- ci service found: local
[task 2024-10-16T18:54:14.952Z] debug - 2024-10-16 18:54:14,952 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.GitVersioningSystem'>
[task 2024-10-16T18:54:14.954Z] debug - 2024-10-16 18:54:14,954 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.GitVersioningSystem'>
[task 2024-10-16T18:54:14.956Z] warning - 2024-10-16 18:54:14,956 -- No config file could be found. Ignoring config.
[task 2024-10-16T18:54:14.956Z] debug - 2024-10-16 18:54:14,956 -- No codecov_yaml found
[task 2024-10-16T18:54:14.967Z] debug - 2024-10-16 18:54:14,967 -- Starting upload process --- {"commit_sha": "2c19e7811bd718e74e5794485471e856d0c5d8af", "report_code": "default", "build_code": null, "build_url": null, "job_code": null, "env_vars": {}, "flags": [], "name": null, "network_filter": null, "network_prefix": null, "network_root_folder": "/builds/worker/checkouts/vcs", "files_search_root_folder": "/builds/worker/checkouts/vcs", "files_search_exclude_folders": [], "files_search_explicitly_listed_files": ["/builds/worker/fetches/coverage.xml"], "plugin_names": ["xcode", "gcov", "pycoverage"], "token": "4******************", "branch": "matrix", "slug": "taskcluster/taskgraph", "pull_request_number": null, "git_service": "github", "disable_search": false, "disable_file_fixes": false, "fail_on_error": true, "handle_no_reports_found": false}
[task 2024-10-16T18:54:14.967Z] debug - 2024-10-16 18:54:14,967 -- Starting create commit process --- {"commit_sha": "2c19e7811bd718e74e5794485471e856d0c5d8af", "parent_sha": null, "pr": null, "branch": "matrix", "slug": "taskcluster/taskgraph", "token": "4******************", "service": "github", "enterprise_url": null}
[task 2024-10-16T18:54:15.264Z] info - 2024-10-16 18:54:15,264 -- Process Commit creating complete
[task 2024-10-16T18:54:15.264Z] debug - 2024-10-16 18:54:15,264 -- Commit creating result --- {"result": "RequestResult(error=None, warnings=[], status_code=202, text='{\"status\":\"queued\"}\\n')"}
[task 2024-10-16T18:54:15.265Z] debug - 2024-10-16 18:54:15,264 -- Starting create report process --- {"commit_sha": "2c19e7811bd718e74e5794485471e856d0c5d8af", "code": "default", "slug": "taskcluster/taskgraph", "service": "github", "enterprise_url": null, "token": "4******************"}
[task 2024-10-16T18:54:15.461Z] info - 2024-10-16 18:54:15,460 -- Process Report creating complete
[task 2024-10-16T18:54:15.461Z] debug - 2024-10-16 18:54:15,461 -- Report creating result --- {"result": "RequestResult(error=None, warnings=[], status_code=202, text='{\"status\":\"queued\"}\\n')"}
[task 2024-10-16T18:54:15.461Z] info - 2024-10-16 18:54:15,461 -- Finished creating report successfully --- {"response": "{\"status\":\"queued\"}\n"}
[task 2024-10-16T18:54:15.461Z] debug - 2024-10-16 18:54:15,461 -- Starting upload processing --- {"branch": "matrix", "build_code": null, "build_url": null, "commit_sha": "2c19e7811bd718e74e5794485471e856d0c5d8af", "disable_file_fixes": false, "disable_search": false, "enterprise_url": null, "env_vars": {}, "files_search_exclude_folders": [], "files_search_explicitly_listed_files": ["/builds/worker/fetches/coverage.xml"], "files_search_root_folder": "/builds/worker/checkouts/vcs", "flags": [], "git_service": "github", "handle_no_reports_found": false, "job_code": null, "name": null, "network_filter": null, "network_prefix": null, "network_root_folder": "/builds/worker/checkouts/vcs", "plugin_names": ["xcode", "gcov", "pycoverage"], "pull_request_number": null, "report_code": "default", "slug": "taskcluster/taskgraph", "token": "4******************", "upload_file_type": "coverage"}
[task 2024-10-16T18:54:15.461Z] debug - 2024-10-16 18:54:15,461 -- Selected preparation plugins --- {"selected_plugins": ["<class 'codecov_cli.plugins.xcode.XcodePlugin'>", "<class 'codecov_cli.plugins.gcov.GcovPlugin'>", "<class 'codecov_cli.plugins.pycoverage.Pycoverage'>"]}
[task 2024-10-16T18:54:15.461Z] debug - 2024-10-16 18:54:15,461 -- Running preparation plugin: <class 'codecov_cli.plugins.xcode.XcodePlugin'>
[task 2024-10-16T18:54:15.462Z] debug - 2024-10-16 18:54:15,461 -- Running xcode plugin...
[task 2024-10-16T18:54:15.462Z] warning - 2024-10-16 18:54:15,462 -- xcrun is not installed or can't be found.
[task 2024-10-16T18:54:15.462Z] debug - 2024-10-16 18:54:15,462 -- Running preparation plugin: <class 'codecov_cli.plugins.gcov.GcovPlugin'>
[task 2024-10-16T18:54:15.462Z] debug - 2024-10-16 18:54:15,462 -- Running gcov plugin...
[task 2024-10-16T18:54:15.533Z] warning - 2024-10-16 18:54:15,533 -- No gcov data found.
[task 2024-10-16T18:54:15.533Z] debug - 2024-10-16 18:54:15,533 -- Running preparation plugin: <class 'codecov_cli.plugins.pycoverage.Pycoverage'>
[task 2024-10-16T18:54:15.595Z] warning - 2024-10-16 18:54:15,595 -- No coverage data found to transform
[task 2024-10-16T18:54:15.596Z] debug - 2024-10-16 18:54:15,595 -- Collecting relevant files
[task 2024-10-16T18:54:15.677Z] info - 2024-10-16 18:54:15,677 -- Found 1 coverage files to report
[task 2024-10-16T18:54:15.677Z] info - 2024-10-16 18:54:15,677 -- > /builds/worker/fetches/coverage.xml
[task 2024-10-16T18:54:15.682Z] debug - 2024-10-16 18:54:15,682 -- Selected uploader to use: <class 'codecov_cli.services.upload.upload_sender.UploadSender'>
[task 2024-10-16T18:54:15.689Z] debug - 2024-10-16 18:54:15,688 -- Sending upload request to Codecov
[task 2024-10-16T18:54:15.886Z] info - 2024-10-16 18:54:15,886 -- Your upload is now processing. When finished, results will be available at: https://app.codecov.io/github/taskcluster/taskgraph/commit/2c19e7811bd718e74e5794485471e856d0c5d8af
[task 2024-10-16T18:54:15.886Z] debug - 2024-10-16 18:54:15,886 -- Upload request to Codecov complete. --- {"response": {"raw_upload_location": "https://storage.googleapis.com/codecov/shelter/github/taskcluster%3A%3A%3A%3Ataskgraph/2c19e7811bd718e74e5794485471e856d0c5d8af/8baf659f-5254-43dd-a04b-bc5f71d32d11.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=GOOG1EJWQHUGIBILH4J7Q6ZUSCIFNEOLYSNDS7L3B4N5SIBQ2J4YLYE5CRFCD%2F20241016%2Fus%2Fs3%2Faws4_request&X-Amz-Date=20241016T185415Z&X-Amz-Expires=30&X-Amz-SignedHeaders=host&X-Amz-Signature=c9cd57ca28f694b2e7cf183489789e490d5f359d97a7d67fce2fb78111b04fea", "url": "https://app.codecov.io/github/taskcluster/taskgraph/commit/2c19e7811bd718e74e5794485471e856d0c5d8af"}}
[task 2024-10-16T18:54:15.886Z] debug - 2024-10-16 18:54:15,886 -- Sending upload to storage
[task 2024-10-16T18:54:16.036Z] info - 2024-10-16 18:54:16,036 -- Process Upload complete
[task 2024-10-16T18:54:16.036Z] debug - 2024-10-16 18:54:16,036 -- Upload result --- {"result": "RequestResult(error=None, warnings=[], status_code=200, text='')"}
[fetches 2024-10-16T18:54:16.135Z] removing /builds/worker/fetches
[fetches 2024-10-16T18:54:16.139Z] finished
[taskcluster 2024-10-16 18:54:16.907Z] === Task Finished ===
[taskcluster 2024-10-16 18:54:16.907Z] Successful task run with exit code: 0 completed in 12.526 seconds