Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add kind that demonstrates how to modify the upstream graph in a transform #438

Closed
wants to merge 2 commits into from

Add hacky kind that can modify upstream parts of the graph

0d686a1
Select commit
Loading
Failed to load commit list.
Closed

Add kind that demonstrates how to modify the upstream graph in a transform #438

Add hacky kind that can modify upstream parts of the graph
0d686a1
Select commit
Loading
Failed to load commit list.
firefoxci-taskcluster / split-mono-src-ru succeeded Feb 12, 2024 in 3m 4s

FirefoxCI (pull_request)

split mono for ru

Details

View task in Taskcluster
View logs in Taskcluster


[taskcluster 2024-02-12 20:00:38.554Z] Task ID: ZgfO37FPQeiuQsyBYi_YpA
[taskcluster 2024-02-12 20:00:38.554Z] Worker ID: 524514852579296937
[taskcluster 2024-02-12 20:00:38.554Z] Worker Group: us-central1
[taskcluster 2024-02-12 20:00:38.554Z] Worker Node Type: projects/887720501152/machineTypes/n2-highmem-32
[taskcluster 2024-02-12 20:00:38.554Z] Worker Pool: translations-1/b-linux-large-gcp-300gb
[taskcluster 2024-02-12 20:00:38.554Z] Worker Version: 38.0.5
[taskcluster 2024-02-12 20:00:38.554Z] Public IP: 34.122.73.82
[taskcluster 2024-02-12 20:00:38.554Z] Hostname: translations-1-b-linux-large-gcp-300gb-cus60yzctvmdh9dn2isfsq
[taskcluster 2024-02-12 20:00:38.554Z] using cache "translations-level-1-checkouts-v3-58974d7dcf0417b3fe53-f9v6Z1KkRZuzZXoqK-Q4rg" -> /builds/worker/checkouts

[taskcluster 2024-02-12 20:00:39.063Z] Image 'public/image.tar.zst' from task 'f9v6Z1KkRZuzZXoqK-Q4rg' loaded.  Using image ID sha256:377e5cf268945040e51f09b9261679365b3a09539429ff28b42c40a173e820e8.
[taskcluster 2024-02-12 20:00:39.111Z] === Task Starting ===
[setup 2024-02-12T20:00:39.464Z] run-task started in /builds/worker
[setup 2024-02-12T20:00:39.464Z] Invoked by command: --firefox_translations_training-checkout=/builds/worker/checkouts/vcs/ -- bash -c python3 $VCS_PATH/pipeline/translate/splitter.py --output_dir=/builds/worker/artifacts --num_parts=2 --compression_cmd=zstdmt $MOZ_FETCHES_DIR/mono.$LOCALE.zst
[setup 2024-02-12T20:00:39.464Z] Python version: 3.10.12
[cache 2024-02-12T20:00:39.466Z] cache /builds/worker/checkouts exists; requirements: gid=1000 uid=1000 version=1
[volume 2024-02-12T20:00:39.466Z] volume /builds/worker/checkouts is a cache
[setup 2024-02-12T20:00:39.466Z] running as worker:worker
[vcs 2024-02-12T20:00:39.466Z] executing ['git', 'config', '--global', '--add', 'safe.directory', '/builds/worker/checkouts/vcs']
[vcs 2024-02-12T20:00:39.468Z] executing ['git', 'fetch', '--no-tags', 'https://github.com/bhearsum/firefox-translations-training', 'graph-mod']
[vcs 2024-02-12T20:00:39.673Z] From https://github.com/bhearsum/firefox-translations-training
[vcs 2024-02-12T20:00:39.673Z]  * branch            graph-mod  -> FETCH_HEAD
[vcs 2024-02-12T20:00:39.678Z] executing ['git', 'checkout', '-f', '-B', 'graph-mod', '0d686a1b21f9e8395ead4d22f4e2e0a8bcf5b826']
[vcs 2024-02-12T20:00:39.682Z] Reset branch 'graph-mod'
[vcs 2024-02-12T20:00:39.682Z] executing ['git', 'submodule', 'init']
[vcs 2024-02-12T20:00:39.699Z] executing ['git', 'submodule', 'update', '--force']
[vcs 2024-02-12T20:00:39.731Z] Submodule path '3rd_party/browsermt-marian-dev': checked out '11c6ae7c46be21ef96ed10c60f28022fa968939f'
[vcs 2024-02-12T20:00:39.741Z] Submodule path '3rd_party/extract-lex': checked out '42fa605b53f32eaf6c6e0b5677255c21c91b3d49'
[vcs 2024-02-12T20:00:39.750Z] Submodule path '3rd_party/fast_align': checked out 'cab1e9aac8d3bb02ff5ae58218d8d225a039fa11'
[vcs 2024-02-12T20:00:39.761Z] Submodule path '3rd_party/kenlm': checked out 'bbf4fc511266c5d4515047055d7bdec659a6e158'
[vcs 2024-02-12T20:00:39.775Z] Submodule path '3rd_party/marian-dev': checked out 'e8a1a2530fb84cbff7383302ebca393e5875c441'
[vcs 2024-02-12T20:00:39.785Z] Submodule path '3rd_party/preprocess': checked out '64307314b4d5a9a0bd529b5c1036b0710d995eec'
[vcs 2024-02-12T20:00:39.786Z] cleaning git checkout...
[vcs 2024-02-12T20:00:39.786Z] executing ['git', 'clean', '-nxdff']
[vcs 2024-02-12T20:00:39.788Z] removing ['/builds/worker/checkouts/vcs/pipeline/clean/tools/__pycache__/', '/builds/worker/checkouts/vcs/pipeline/clean/tools/lid.176.bin']
[vcs 2024-02-12T20:00:39.800Z] successfully cleaned git checkout!
[vcs 2024-02-12T20:00:39.801Z] TinderboxPrint:<a href='https://github.com/bhearsum/firefox-translations-training/commit/0d686a1b21f9e8395ead4d22f4e2e0a8bcf5b826' title='Built from firefox-translations-training commit 0d686a1b21f9e8395ead4d22f4e2e0a8bcf5b826'>0d686a1b21f9e8395ead4d22f4e2e0a8bcf5b826</a>
[setup 2024-02-12T20:00:39.801Z] MOZ_FETCHES_DIR is /builds/worker/fetches
[fetches 2024-02-12T20:00:39.801Z] fetching artifacts
[fetches 2024-02-12T20:00:39.801Z] executing ['/usr/bin/python3', '-u', '/usr/local/bin/fetch-content', 'task-artifacts']
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/SIz8Cd8yTQuBYfpdWYo_AA/artifacts/public/build/mono.ru.zst to /builds/worker/fetches/mono.ru.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/SIz8Cd8yTQuBYfpdWYo_AA/artifacts/public/build/mono.ru.zst
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/SIz8Cd8yTQuBYfpdWYo_AA/artifacts/public/build/mono.ru.zst resolved to 574855 bytes with sha256 e369cda2cc774761920983bbe4da89b19bbb91235587c874e224db3b229dd06a in 0.141s
PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 0.14241988400000594, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2024-02-12T20:00:40.020Z] finished fetching artifacts
[task 2024-02-12T20:00:40.020Z] executing ['bash', '-c', 'python3 $VCS_PATH/pipeline/translate/splitter.py --output_dir=/builds/worker/artifacts --num_parts=2 --compression_cmd=zstdmt $MOZ_FETCHES_DIR/mono.$LOCALE.zst']
[task 2024-02-12T20:00:40.064Z] 
[task 2024-02-12T20:00:40.064Z]                                                                                
[task 2024-02-12T20:00:40.071Z] Read :  0 /  0 MB ==> 32%
[task 2024-02-12T20:00:40.071Z]                                                                                
[task 2024-02-12T20:00:40.071Z] /builds/worker/artifacts/file.1 : 28.92%   (1016143 => 293918 bytes, /builds/worker/artifacts/file.1.zst) 
[task 2024-02-12T20:00:40.081Z] 
[task 2024-02-12T20:00:40.081Z]                                                                                
[task 2024-02-12T20:00:40.087Z] Read :  0 /  0 MB ==> 32%
[task 2024-02-12T20:00:40.087Z]                                                                                
[task 2024-02-12T20:00:40.087Z] /builds/worker/artifacts/file.2 : 28.81%   (1008694 => 290589 bytes, /builds/worker/artifacts/file.2.zst) 
[task 2024-02-12T20:00:40.087Z] Splitting /builds/worker/fetches/mono.ru.zst to 2 chunks x 9941 lines
[task 2024-02-12T20:00:40.087Z] A new file /builds/worker/artifacts/file.1 created
[task 2024-02-12T20:00:40.087Z] Compressing /builds/worker/artifacts/file.1 with zstdmt
[task 2024-02-12T20:00:40.087Z] A new file /builds/worker/artifacts/file.2 created
[task 2024-02-12T20:00:40.087Z] Compressing /builds/worker/artifacts/file.2 with zstdmt
[task 2024-02-12T20:00:40.087Z] Done
[fetches 2024-02-12T20:00:40.091Z] removing /builds/worker/fetches
[fetches 2024-02-12T20:00:40.091Z] finished
[taskcluster 2024-02-12 20:00:40.359Z] === Task Finished ===
[taskcluster 2024-02-12 20:00:40.950Z] Successful task run with exit code: 0 completed in 2.397 seconds